Debugging Approaches for Model Transformation

Debugging Approaches for Model Transformation

Exploring Efficient and Scalable Omniscient Debugging for MDE JONATHAN CORLEY CO M MI T TE E : Dr. Jeff Gray (Chair) Dr. Eugene Syriani Dr. Jeffrey Carver Dr. Randy Smith Dr. Susan Vrbsky Background Software Engineering Model-Driven Engineering BACKGROUND

BACKGROUND 4 2 What is a model? Abstraction of a system Blueprints for a house to be built Class Diagram for an existing system BACKGROUND BACKGROUND 3

(Meta-) Meta-Model A model is an instance of a meta-model A meta-model is an instance of a meta-meta-model Potentially infinite layers Common architecture uses 3 or 4 layers Often stops at a self-defining layer BACKGROUND BACKGROUND 8 5 Research Roadmap Collaborative Modeling in

Omniscient Debugging for MTs How Developers Debug Omniscient Debugging for xDSMLs Collaborative Modeling in COLLABORATIVE MODELING 6 Contributions

Primary contributions Designing and Prototyping the AToMPM Architecture to address Collaboration Scenarios Designing and Performing the Evaluation Collaborative contributions Refining the set of Collaboration Scenarios OMNISCIENT DEBUGGING OF XDSMLS 7 Collaborative Modeling Environments Stakeholders should not be forced to work in isolation. COLLABORATIVE MODELING 8

AToMPM Architecture Client independent Modeling-as-a-Service Scalable system load balancing Supercontroller distributes Model Controllers across active nodes add nodes dynamically forwarders avoid dynamic discovery issue Live/Centralized System Model Controllers manage all active models MvK provides long term storage and (currently) CRUD processing COLLABORATIVE MODELING 9 Ensuring Consistency 1. Multi-User Single-View Managed by Model Controller

2. Multi-View Single-Model Managed by Model Controller 3. Multi-View Multi-Model Managed by MvK: no updates Managed by Client: multiple subscribe 4. Single-View Multi-Model Not directly supported Can be refactored to scenario 3 COLLABORATIVE MODELING 10 Study Design Three experiments focused on evaluating scenarios 1-3.

Each User Connects to the Same View of the Same Model Each User Connects to a Distinct View of the Same Model Each User Connects to a Distinct View of a Distinct Model Focus on Create (single element) and Load (multi-element) requests. COLLABORATIVE MODELING 11 Results Overview Create MCtrlr and MvK create bottleneck

1 MCtrlr scales more poorly with many views than all users on same view 2 Load System performs well with low number of users per 3 MCtrlr COLLABORATIVE MODELING 12

Conclusions Supporting collaborative modeling environments cloud architecture enable efficient interactions create scalable environment Summary of evaluation results Controllers support expected load of collaborators per model As expected, MvK needs to be improved to manage distributed load COLLABORATIVE MODELING 13 Omniscient Debugging for MTs OMNISCIENT DEBUGGING OF MTS

14 Model Transformations Input Output 1 1 Non-Determinism Selecting Matches Scheduling Rules 2 BACKGROUND

15 Omniscient Debugging for MT Traverse dynamic execution backwards Is it needed? Fault is located before the visible error Defect is not present in all executions Poorly understood error Nondeterminism Execution of the system is expensive Time to execute Manual input required OMNISCIENT DEBUGGING OF MTS 16

Extending Stepwise Execution Play PlayBack StepIn BackOut StepOver BackOver StepOut BackIn AToMPM Omniscient Debugger (AODB) OMNISCIENT DEBUGGING OF MTS

17 Collecting a History of Execution How to revert execution? o o Collect trace (History) of execution Step contains full state information for elements changed Step 0 Step 1 Step 2 Step 3

Step 4 Glass A Glass B Glass C OMNISCIENT DEBUGGING OF MTS 18 Traversing History Efficiently Two distinct algorithms to build a MacroStep IterateSteps Algorithm: Iterate over each step in history to identify necessary changes. IterateElements Algorithm: Iterate over each element in history to identify necessary changes. Step 0

Step 1 Step 2 Step 3 Step 4 Glass A Glass C OMNISCIENT DEBUGGING OF MTS 19 Recognizing Change Patterns

OMNISCIENT DEBUGGING OF MTS 20 Study Design Compare Existing Stepwise Debugger and AODB Vary Model Size Two Transformation Systems 1) Movie DB Case TTC 2014 Identify pairs of actors starring in shared movies Also, Calculate statistics for the pair Repeated Searches due to updates 2) Sierpinski Triangle TTC 2007 Generate Sierpinski Fractals Single search per generation OMNISCIENT DEBUGGING OF MTS

21 Evaluating Performance Jump Performance Stepwise Duration (ms) JumpForward Step JumpBack Back JumpForward Step JumpBack

Back Omniscient Stepwise IterateSteps IterateElements Duration (ms) Duration (ms) Duration (ms) Omniscient Sierpinski Triangle Algorithm Performance

Duration (ms) Duration (ms) Movie DB Case Execution Performance OMNISCIENT DEBUGGING OF MTS IterateSteps IterateElements 22 Evaluating Scalability Sierpinski Triangle

Movie DB Case Percent of History Steps Executed 15 4 10 3 5 2 1 Percent

of System Steps Executed Steps Executed Steps Executed OMNISCIENT DEBUGGING OF MTS 23 Conclusions Supporting Omniscient Debugging for Model Transformations Focused on two hybrid MTLs supported in AToMPM: T-Core and MoTif Developed minimal history structure to support omniscient traversal Developed algorithm (MacroSteps) to identify and disregard non-essential CRUD operations Utilize a set of 5 change patterns to identify non-essential CRUD operations

Empirical Study Investigating Performance and Scaling of AODB Execution time comparable to existing stepwise debugger Impact on memory usage minor, but notable MacroStep superior depending on transformation process OMNISCIENT DEBUGGING OF MTS 24 Omniscient Debugging for xDSMLs OMNISCIENT DEBUGGING OF XDSMLS 25 Contributions Primary contributions

Designing Omniscient debugging features Designing and Prototyping the visualization of history Collaborative contributions Designing an interface between the generated traces and the omniscient debugging features Designing the empirical evaluation Evaluating the results of the empirical evaluation OMNISCIENT DEBUGGING OF XDSMLS 26 Executable Domain-Specific Modeling Languages Abstract Syntax Net places

* Place +name: string +initialTokens: int Execution Metamodel transitions * input 1..* output 1..* Transition +name: string merges

Place +tokens: int imports run(Net) : while there is an enabled transition, fires it. isEnabled(Transition) : returns true if tokens > 0 for each input Place, false otherwise. fire(Transition) : removes a token from each input Place and adds a token to each output Place. Operational Semantics (summarized) Operational semantics, defined as a set of transformation rules

Abstract syntax properties (e.g., initialTokens) are immutable Execution metamodel properties (e.g., tokens) are mutable BACKGROUND 27 Challenges Generic omniscient debugger supporting any xDSML? Two key problems: 1. 2. Genericity requires additional processing (e.g., introspection) impacting responsiveness Execution states of an xDSML can be arbitrarily complex, impacting understandability OMNISCIENT DEBUGGING OF XDSMLS

28 Efficient Omniscient Debugging OMNISCIENT DEBUGGING OF XDSMLS 29 Multi-Dimensional Omniscient Debugger Visualisation Prototype OMNISCIENT DEBUGGING OF XDSMLS 30 Study Design Research questions

RQ#1: Is our approach more efficient in memory as compared to a weak omniscient debugger and to a clone-based omniscient debugger? RQ#2: Is our approach more efficient in time for omniscient debugging services as compared to a weak omniscient debugger and to a clone-based omniscient debugger? Comparison with Clone-based omniscient debugger: creates a trace by copying a snapshot of the model at each step (uses EMF Compare to restore a state in the model) Weak omniscient debugger: does not create any trace (uses stepwise execution features to restore a state by re-executing from the start) fUML xDSMs Models drawn from the semantic differencing case study of Maoz et al. [1] 40 models with sizes ranging from 36 to 51 objects [1] http://www.se-rwth.de/materials/semdiff/ OMNISCIENT DEBUGGING OF XDSMLS 31

Evaluating Performance and Scalability TIME (MS) 100 1 0.1 1 2 3 4 5

6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 100 TIME (MS) Execution Performance 10 10 1

0.1 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 MultiDimDebugger CloneBasedDebugger WeakDebugger Memory Consumption OMNISCIENT DEBUGGING OF XDSMLS 32 Conclusions Support Omniscient Debugging of xDSMLs Generative/Trace-based approach work of Erwan Bousse Generic omniscient operations

reflection of stepwise operations (e.g., backIn reflects stepIn) multi-dimension operations (e.g, stepByValue) Interactive multi-dimensional timeline view OMNISCIENT DEBUGGING OF XDSMLS 33 How Developers Debug An Exploratory Study of the Formation, Use, and Impact of Queries HOW DEVELOPERS DEBUG 34 Query-Based Debugging Developers pose queries during debugging What MT rule sets model element x?

Why was attribute y set to value z? Existing tools and techniques allow developers to directly query the system either dynamically (live) or statically (forensic) What queries do developers ask? How do these queries fit into the debugging process? How do these queries impact the debugging process? HOW DEVELOPERS DEBUG 35 Participant Demographics Total of 16 participants (11 Grad, 5 Ugrad) HOW DEVELOPERS DEBUG 36

Study Design HOW DEVELOPERS DEBUG 37 Data Collection & Analysis Data Collection Data Collection Merge Forms Types, Factors, Relationships HOW DEVELOPERS DEBUG 38

Do Developers Form Queries? Total Queries Observed Query Total Growth 100 90 80 70 60 50 40 30 20 10 0 0

5 10 15 20 25 30 35 40 45 50

55 60 Elapsed Time (minutes) HOW DEVELOPERS DEBUG 39 What Types of Queries? Language Environment Intent Support (9.3%)

Domain Code Segment Local Variable Attribute Bug Method Feature Behavior (17.7%) State (3.7%)

Class Class Entity QUERY Relevance (16.5%) Responsibility (8.7%) Method Concept Package Location (36.3%) Structure (7.7%) Hierarchy

Contained Entities Control Flow Inheritance Class Reference Method Function Attribute HOW DEVELOPERS DEBUG

Concept Class Definition Method System Text Attribute 40 Why Do Developers Form Queries?

Query Relationship Subquery Follows Result Formed Through Filtering Results No Relation Total Occurrences 288 209 59 50 104 HOW DEVELOPERS DEBUG Percent of All Queries 40.6%

29.4% 8.3% 7% 14.6% 41 Why Do Developers Form Queries? Query Generation Factor Further Investigation Finding A New Path Conceptual Trigger Textual Trigger Related Entity Search Results Total Occurrences 360

113 77 73 45 42 HOW DEVELOPERS DEBUG Percent of All Queries 50.7% 15.9% 10.8% 10.3% 6.3% 5.9% 42 What is the Impact of

Queries? Split developers into success and fail (bug 1 only) HOW DEVELOPERS DEBUG 43 Conclusions Confirmed that developers form queries naturally during the debugging process Identified a set of 29 specific query types Specific query types were categorized into one of 7 fundamental query types Identified 4 types of relationship to prior queries Identified a set of 6 factors for query generation Identified 4 factors significantly different between developers that succeeded and those that failed HOW DEVELOPERS DEBUG

44 Perspectives on Future Work Collaborative Modeling in Omniscient Debugger How Modelers Debug How Developers Debug How Developers Debug Use Tools FUTURE WORK 45 User Study of Omniscient

Debugging for MTs Evaluate AODB in practice Investigate how developers use omniscient features impact of omniscient features on debugging process (success and efficiency) developer perception Study design and Materials are complete Omniscient Debugger FUTURE WORK 46 Exploring Queries for MDE Replicate in an MDE context

How Modelers Debug FUTURE WORK 47 Efficient, Distributed Storage and Processing of Models Identify strategies to efficiently distribute models across a set of storage/processing nodes Explore process to manage security/authentication Collaborative Modeling in FUTURE WORK 48

Exploring the Debugging Process Explore how query usage maps to existing debugging theories (e.g., Information Foraging Theory) Validate existing theories through further analysis of existing data How Developers Debug FUTURE WORK 49 Investigating Tool Support Investigate tool usage by developers during the debugging process Experienced developers vs novice developers Compare development environments Survey existing tool support and map to identified query types

How Developers Debug Use Tools FUTURE WORK 50 Comments & Questions COMMENTS & QUESTIONS

Recently Viewed Presentations

  • Nano-Medicine - Utah State University

    Nano-Medicine - Utah State University

    Nanoshells have a core of silica and an ultra thin metallic outer layer (typically Gold) Gold nanoshells can be made to either preferentially absorb or scatter light by varying the size of the particle relative to the wavelength of the...
  • ECEN667_2017_Lect26

    ECEN667_2017_Lect26

    Doubly fed asynchronous generators (DFAG) are usually a conventional wound rotor induction generator with an ac-dc-ac power converter in the rotor circuit. Power that would have been lost in external rotor resistance is now used.
  • Carleton Benefits Enrollment Carleton benefit enrollment is online

    Carleton Benefits Enrollment Carleton benefit enrollment is online

    Save. To enter a child, click the icon under Children, enter information, and click . Save. Employees with dependents up to the . age of . 26 can be enrolled . in . our medical . and dental plans. There...
  • Chapter 11

    Chapter 11

    Molecular Mechanism of Transformation. Fig. 11-2. Heating S-strain cells killed them but did not completely destroy their DNA. When killed S-strain bacteria were mixed with living R-strain bacteria, fragments of DNA from the dead S-strain cells became incorporated into the...
  • Evaluate the Strengths and Weaknesses of Two Soltuions to The ...

    Evaluate the Strengths and Weaknesses of Two Soltuions to The ...

    Anselm's definition of God was clearly influenced by Christian teachings of an omnipotent, omniscient and omnibenevolent God, a being set apart from humans and perfect in every way. A key weakness of the argument as a whole is its reliance...
  • Bonus Chapter A McGraw-Hill/Irwin McGraw-Hill/Irwin Understanding Understanding Working

    Bonus Chapter A McGraw-Hill/Irwin McGraw-Hill/Irwin Understanding Understanding Working

    Tort -- A wrongful act that causes injury to another person's body, property or reputation. An intentional tort is a willful act that results in injury. Negligence -- An unintentional tort that causes harm or injury that was not intended....
  • Similarity & Trig Review

    Similarity & Trig Review

    A boy who is 5 ft. tall cast a shadow that is 12 ft long. At the same time, a building nearby cast a shadow that is 72 ft long. How tall is the building? Draw a picture! ? 72...
  • Crucible Bellringer #7 8/22/12

    Crucible Bellringer #7 8/22/12

    The Crucible Bellringer #10 8/23/13 . Explain the irony shown in the picture on the bottom right. What was ironic about Alanis Morisette's. song "Ironic"? Today's Target: I can identify examples