SEG3101 (Fall 2018) Requirements Verification and Validation Daniel

SEG3101 (Fall 2018) Requirements Verification and Validation Daniel

SEG3101 (Fall 2018) Requirements Verification and Validation Daniel Amyot, University of Ottawa Based on PowerPoint slides by Gunter Mussbacher with material from: G. Kotonya and I. Sommerville, P. Heymans, K.E. Wiegers, B. Selic, S. Som 2008, D. Amyot 2008-2017, G.v. Bochmann 2010, and Mussbcher 2017 Dilbert and Validation SEG3101. Requirements Verification and Validation 2 Introduction Simple Checks Prototyping Functional Test Design Formal V&V Reviews and Inspections Requirements Verification vs. Validation

Requirements verification: Are we building the product right? Requirements validation: Are we building the right product? SEG3101. Requirements Verification and Validation 3 Introduction Simple Checks Prototyping Functional Test Design Formal V&V Reviews and Inspections Requirements Verification and Validation Requirements Verification Check that product is being built right Ensures that each step followed in the process of building the software is well done

Checks consistency of the software requirements specification artefacts and other software development products (design, implementation, ...) against the specification Requirements Validation Check that the right product is being built Ensures that the software being developed (or changed) will satisfy its stakeholders real needs Checks the software requirements specification against stakeholders goals and requirements Both need to be performed at every stage during the (requirements) process SEG3101. Requirements Verification and Validation 4 Introduction Simple Checks Prototyping Functional Test Design Formal V&V Reviews and Inspections Requirements V&V vs. Requirements Analysis

Both have several activities in common Reading requirements, problem analysis, meetings and discussions... But inputs are different! Analysis works with raw, incomplete requirements as elicited from the system stakeholders Develop a software requirements specification document Emphasis on "we have the right requirements" Requirements V&V works with a software requirements specification and with negotiated and agreed (and presumably complete) domain requirements Check that this these specifications are accurate Emphasis on "we have the right requirements well done" SEG3101. Requirements Verification and Validation 5 Introduction Simple Checks Prototyping Functional Test Design Formal V&V

Reviews and Inspections Various Requirements V&V Techniques Simple checks Prototyping Functional test design Formal V&V Reviews and inspections The software is done. We are just trying to get it to work SEG3101. Requirements Verification and Validation 6 Introduction Simple Checks Prototyping Functional Test Design Formal V&V Reviews and Inspections Simple Checks

Various checks can be done using traceability techniques Given the requirements document, verify that all elicitation notes are covered adequately Often involves developing traceability matrices Tracing between different levels of requirements Checking goals against tasks, features, requirements Completeness and consistency checks Verify that the requirements are well written (according to the criteria already discussed) SEG3101. Requirements Verification and Validation 7 Introduction Simple Checks Prototyping Functional Test Design Formal V&V Reviews and Inspections Prototyping Excellent for validation by users and customers

More accessible than a long, linear specification Demonstrates the requirements and helps stakeholders discover problems Comes in all different shapes and sizes From paper prototype of a computerized system to formal executable models/specifications Important to choose scenarios or use cases for elicitation session Good coverage, not just playing around SEG3101. Requirements Verification and Validation 8 Introduction Simple Checks Prototyping Functional Test Design Formal V&V Reviews and Inspections Functional Test Design

Functional tests at the system level must be developed sooner or later... Tests can (and should) be derived from the requirements specification Designing these tests may reveal errors in the specification (even before designing and building the system)! Missing or ambiguous information in the requirements description may make it difficult to formulate tests Some software development processes (e.g., agile methods) begin with tests before programming Test-Driven Development (TDD) http://en.wikipedia.org/wiki/Test-driven_development SEG3101. Requirements Verification and Validation 9 Introduction Simple Checks Prototyping Functional Test Design Formal V&V Reviews and Inspections

Formal V&V Available V&V techniques vary from one modeling paradigms to another and also depend on the available tools FSM, profiled UML, Petri nets, Z, etc. The following functions may be provided through tools Simulation random or interactive Testing predetermined traces have to be accepted (or rejected) Completeness checking only according to certain syntax rules, templates Consistency checking : given model M, show that M does not imply a contradiction and does not have any other undesirable general property (e.g. deadlock possibility, unhandled execption/message) Refinement checking : given two models M and M, show that the properties of M imply the properties of M. Model checking : given a model M and some properties P, show that any system implementation satisfying M will have the properties P. Theorem proving: prove (through induction or other approach) the correctness and validity of theorems against a model, for all situations. SEG3101. Requirements Verification and Validation 10 Introduction Simple Checks Prototyping Functional Test Design

Formal V&V Reviews and Inspections Formal V&V Available V&V techniques vary from one modeling paradigms to another and also depend on the available tools FSM, profiled UML, Petri nets, Z, etc. The following may be provided for FSMs through tools Simulation random or interactive Testing predetermined traces have to be accepted (or rejected) Completeness checking only according to certain syntax rules, templates Consistency checking : given model M, show that M does not imply a contradiction and does not have any other undesirable general property (e.g. deadlock possibility, unhandled execption/message) Refinement checking : given two models M and M, show that the properties of M imply the properties of M. Model checking : given a model M and some properties P, show that any system implementation satisfying M will have the properties P. Theorem proving: prove (through induction or other approach) the correctness and validity of theorems against a model, for all situations. SEG3101. Requirements Verification and Validation 11 Introduction

Simple Checks Prototyping Functional Test Design Formal V&V Reviews and Inspections Reviews and Inspections (1) A group of people read and analyze requirements, look for potential problems, meet to discuss the problems, and agree on a list of action items needed to address these problems A widely used requirements validation technique Lots of evidence of effectiveness of the technique Can be expensive Careful planning and preparation Pre-review checking Need appropriate checklists (must be developed if necessary and maintained) SEG3101. Requirements Verification and Validation 12 Introduction

Simple Checks Prototyping Functional Test Design Formal V&V Reviews and Inspections Reviews and Inspections (2) A group of people read and analyze requirements, look for potential problems, meet to discuss the problems, and agree on a list of action items needed to address these problems Different types of reviews with varying degrees of formality Reading and approval (sign-off) Encourages the reader to be more careful (and responsible!) Walkthroughs Can be led by author/expert to educate others on his/her work Active reviews Author asks reviewer questions which can only be answered with the help of the document to be reviewed Formal inspections Structured and detailed review, defined roles, preparation, etc E.g., Fagan Inspection

SEG3101. Requirements Verification and Validation 13 Introduction Simple Checks Prototyping Functional Test Design Formal V&V Reviews and Inspections Fagan Inspection (1) Formal and structured inspection process Note: the boss is not involved in the process! SEG3101. Requirements Verification and Validation 14 Introduction Simple Checks

Prototyping Functional Test Design Formal V&V Reviews and Inspections Fagan Inspection (2) Characterized by rules on who should participate, how many reviewers should participate, and what roles they should play Not more than 2 hours at a time, to keep participants focused 3 to 5 reviewers Author serves as the presenter of the document Metrics are collected Important: the authors supervisor does not participate in the inspection and does not have access to data This is not an employee evaluation Moderator is responsible for initiating the inspection, leading the meeting, and ensuring issues found are fixed All reviewers need to prepare themselves using checklists Issues are recorded in special forms SEG3101. Requirements Verification and Validation 15 Introduction

Simple Checks Prototyping Functional Test Design Formal V&V Reviews and Inspections Fagan Inspection (3) The inspection meeting is like a brainstorming session to identify (potential) problems Re-inspection if > 5% of the document change Some variants are less tolerant... too easy to introduce new errors when correcting the previous ones! Do: Critique the artifact Keep review session to less than two hours Plan time for reviews Prioritize reviews of important issues Keep things light Dont: Attack the person Go on for too long without

adequate breaks Skip reviews! Use sarcasm or exaggeration Involve the boss; this is not an employee evaluation SEG3101. Requirements Verification and Validation 16 Introduction Simple Checks Prototyping Functional Test Design Formal V&V Reviews and Inspections Review Typical Problem Categorization Requirements clarification The requirement may be badly expressed or may have accidentally omitted information which has been collected during requirements elicitation Missing information Some information is missing from the requirements document

Requirements conflict There is a significant conflict between requirements The stakeholders involved must negotiate to resolve the conflict Unrealistic requirement The requirement does not appear to be implementable with the technology available or given other constraints on the system Stakeholders must be consulted to decide how to make the requirement more realistic SEG3101. Requirements Verification and Validation 17 Introduction Simple Checks Prototyping Functional Test Design Formal V&V Reviews and Inspections Requirements Review Checklists Essential tool for an effective review process List common problem areas and guide reviewers May include questions on several quality aspects of the document:

comprehensibility, redundancy, completeness, ambiguity, consistency, organization, standards compliance, traceability ... There are general checklists and checklists for particular modeling and specification languages Checklists are supposed to be developed and maintained See example on course website Simple: http://site.uottawa.ca/~damyot/seg3101/notes/Inspection_checklist_Wiegers.pdf Complex (NASA) A few others in the private section of the Web site SEG3101. Requirements Verification and Validation 18 Introduction Simple Checks Prototyping Functional Test Design Formal V&V Reviews and Inspections Comments on Reviews and Inspections

Advantages Effective (even after considering cost) Allow finding sources of errors (not only symptoms) Requirements authors are more attentive when they know their work will be closely reviewed Encourage them to conform to standards Familiarize large groups with the requirements (buy-in) Diffusion of knowledge Risks Reviews can be dull and draining (need to be limited in time) Time consuming and expensive (but usually cheaper than the alternative) Personality problems Office politics SEG3101. Requirements Verification and Validation 19

Recently Viewed Presentations

  • Persian Wars Introduction

    Persian Wars Introduction

    Weapons and Resources . United city-states. Same language . Fighting on their own soil. Banding together against a common enemy. Balance of great Army and Navy. Advantages at the start. Famous Battles of the Persian Wars. ... Persian Wars Introduction
  • Abortion and the Adolescent Patient

    Abortion and the Adolescent Patient

    Is minor mature enough? Is the abortion in their best interest? Guttmacher State Policies. September 2016. 36 states that require parental involvement have an alternative process for minors seeking an abortion. 36 states include a judicial bypass procedure, which allows...
  • The Truth in Modern Society Rev 3:17 Because

    The Truth in Modern Society Rev 3:17 Because

    Equally, is the need for meditation, uninterrupted— Psa 1:1-2 "Blessed is the man that walketh not in the counsel of the ungodly, nor standeth in the way of sinners, nor sitteth in the seat of the scornful.
  • Welcome to James Caldwell High School: A Place

    Welcome to James Caldwell High School: A Place

    Welcome to James Caldwell High School: "A Place of Possibilities" The Class of 2021
  • Ch 21 Infectious Diseases of the Genitourinary SystemGen

    Ch 21 Infectious Diseases of the Genitourinary SystemGen

    Describe the normal microbiota of the urogenital tract. Describe modes of transmission for urinary and reproductive system infections. List the microorganisms that cause cystitis and pyelonephritis. Name the predisposing factors for these diseases. List the causative agents, symptoms, methods of...
  • Arrays and ArrayLists

    Arrays and ArrayLists

    ArrayList. An ArrayList can be thought of as a variable length array. Elements/items can be added and deleted, and inserted at a given position. ... use set (int index, E element) method of java ArrayList to replace any element from...
  • Credit for Prior Learning Middlesex Community College Professor

    Credit for Prior Learning Middlesex Community College Professor

    CLEP. DSST. American Council on Education (ACE) ACE Military Guide. ACE National Guide to College Creditfor Workforce Training. CAEL National Portfolio Evaluation Service:LearningCounts.org (ACE Credit) Online Courses with Study.com (ACE Credit)
  • Advanced Cycles and Replacement Working Fluids in Heat

    Advanced Cycles and Replacement Working Fluids in Heat

    Advanced Cycles and Replacement Working Fluids in Heat Pumps John McMullan Energy Research Centre University of Ulster Thematic Group 3 Contract JOE3-CT95-0009