1.Software EngineeringLecture 1 Introduction to Software Engineering

2. Code of Conduct Software Engineering is a collaborative activity. You are encouraged to work together, but ... Some tasks may require individual work. Always give credit to your sources and collaborators. Good professional practice: To make use of the expertise of others and to build on previous work, with proper attribution. Unethical and academic plagiarism: To use the efforts of others without attribution. 3. Projects Project teams, about 3 to 5 peoples. Select your own project, any branch of software engineering Real project for real client who intends to use the software in production. Feasibility study and plan: during semester Presentations: requirements design final 4. Project Selection Some suggested projects Recitation section to suggest projects Contact potential clients: Gain idea of their expectations Estimate scope and complexity of the project Discuss business decisions Assemble project team Advertise on the web site 5. Previous Experience Your background Biggest program that you have written? Biggest program that you have worked on? Biggest project team that you have been part of? Longest project that you have worked on? Most people who have used your work? Longest that your project has been in production?My background 6. Course Themes 1. Leadership of large software projects Software as a product Clients and their needs Quality Requirements and specification Usability Evolution Project management Personnel management Economic, legal, and social factors 7. Course Themes 2. Large and very large systems Software design Software architecture Object-oriented design Dependable systems Reliability Verification Legacy systems 8. Characteristics of Software Products General characteristics Usability Maintainability Dependability Efficiency Good software products require good programming, but ... Programming quality is the means to the end, not the end itself. 9. Software as a Product Software is expensive!! Every software project has a trade-off between: Functionality Resources (cost) Timeliness Example: Accounting Management System 10. Client (a.k.a Customer) The client provides resources and expects some product in return. Client satisfaction is the primary measurement of success. Question: Who is the client for Microsoft Excel? 11. Variety of Software Products Examples? -Operation System -Database Management System -Embedded System -Games -Application Software - 12. Categories of ProductCategories of client and software product: Generic (e.g., Microsoft Excel) Bespoke (customized) (e.g., IRS internal system) Many systems are customized versions of generic packages (e.g., Cornell's payroll system) 13. Variety of Software Products Software products are very varied --> Client requirements are very different --> There is no standard process for software engineering --> There is no best language, operating system, platform, database system, development environment, etc. A skilled software developer knows about a wide variety of approaches, methods, tools. The craft of software engineering is to select appropriate methods for each project and apply them effectively. 14. Professional Responsibility Organizations put trust in software developers: Competence: Software that does not work effectively can destroy an organization. Confidentiality: Software developers and systems administrators may have access to highly confidential information (e.g., trade secrets, personal data). Legal environment: Software exists in a complex legal environment (e.g., intellectual property, obscenity). Acceptable use and misuse: Computer abuse can paralyze an organization (e.g., the Internet worm). 15. Software EngineeringLecture 2 The Software Process 16. Books Frederick P. Brooks, Jr. The Mythical Man Month. Addison-Wesley, 1972. Ian Sommerville, Software Engineering, 6th edition. Addison-Wesley, 2000. Grady Booch, James Rumbach, Ivar Jacobson, The Unified Modeling Language. Addison-Wesley 1999. 17. Software ProcessFundamental Assumption: Good processes lead to good software Good processes reduce risk 18. Risk Management What can go wrong in a software project? How can the risk be reduced? 19. The Software Process (Simplified) Feasibility and PlanningRequirementsDesignImplementationOperation and Maintenance 20. The Waterfall Model Requirements Definition System and Software design Programming and Unit Testing Integration and System Testing Operation and Maintenance 21. Requirements Analysis and Definition The system's services, constraints and goals are established by consultation with system users. They are then defined in a manner that is understandable by both users and development staff. This phase can be divided into: Feasibility study (often carried out separately) Requirements analysis Requirements definition Requirements specification 22. System and Software DesignSystem design: Partition the requirements to hardware or software systems. Establishes an overall system architecture Software design: Represent the software system functions in a form that can be transformed into one or more executable programs Unified Modeling Language (UML) 23. Programming and Unit TestingThe software design is realized as a set of programs or program units. (Written specifically, acquired from elsewhere, or modified.) Individual components are tested against specifications. 24. Integration and System TestingThe individual program units are: integrated and tested as a complete system tested against the requirements as specified delivered to the client 25. Operation and Maintenance Operation: The system is put into practical use. Maintenance: Errors and problems are identified and fixed. Evolution: The system evolves over time as requirements change, to add new functions or adapt the technical environment. Phase out: The system is withdrawn from service. 26. Discussion of the Waterfall Model Advantages: Process visibility Dependence on individuals Quality control Cost controlDisadvantages: Each stage in the process reveals new understanding of the previous stages, that requires the earlier stages to be revised. 27. Feedback in the Waterfall Model Requirements Definition System and Software design Programming and Unit Testing Integration and System Testing Operation and Maintenance 28. Iterative Refinement (Evolutionary Development) Concept: Initial implementation for user comment, followed by refinement until system is complete. Vaporware: user interface mock-up Throw-away software components Dummy modules Rapid prototyping Successive refinement 29. Iterative Refinement EvaluationRequirementsImplementation (prototype)Design 30. Iterative Refinement Concurrent Activities Requirements Outline DescriptionDesignImplementationInitial Version Intermediate Versions Final Version 31. Iterative Refinement & Software Process Concurrent Activities Outline DescriptionRequirements DesignImplementationFinal Version 32. Iterative Refinement When is iterative refinement appropriate? 33. Iterative Refinement + Waterfall Model: Graphics for Basic Outline Description: Add vector graphics to Dartmouth Basic. Phase 1: Extend current language with a preprocessor and run-time support package. (1976/77) Phase 2: Write new compiler and run-time system incorporating graphics elements. (1978/80) 34. Iterative Refinement + Waterfall Model: Graphics for Basic Design Issues: Pictorial subprograms: coordinate systems, window/viewport User specification of perspective Design Strategy: (Iterative Refinement) Write a series of prototypes with various proposed semantics Evaluate with a set of programming tasks 35. Iterative Refinement + Waterfall Model: Graphics for Basic Phase 1: Implementation (Waterfall) When the final specification was agreed, the entire preprocessor and run-time support were recoded. The system was almost entirely bug-free. Phase 2: New compiler (Waterfall) Phase 1 was used as the requirements definition for the final version. 36. Observations about Software ProcessesCompleted projects should look like the Waterfall Model but ... the development process is always partly evolutionary. Risk is lowered by: Prototyping key components Dividing into phases Following a visible software process Making use of reusable components 37. Software EngineeringLecture 3 (a) Feasibility Study (b) Requirements Definition 38. Feasibility Study Before beginning a project, a short, low-cost study to identify ClientScopePotential benefitsResources needed: staff, time, equipment, etc.Potential obstaclesWhere are the risks? How can they be minimized? 39. Feasibility Study A feasibility study leads to a decision: go ahead do not go ahead think again In production projects, the feasibility study often leads to a budget request. In research, a feasibility study is often in the form of a proposal. 40. CS 501: ClientIn CS 501, you have two clients: The client for the project The professor for the course Can you satisfy them both? 41. Scope What are the boundaries of the project? CS 501 Examples: Static web pages with open access on the Web [Web Profiler] Used by the general public [Digital Collections] Varying data formats [Legal Information] Thousands of sensors [Data mining] Support for Windows, Mac, Unix [SALSA] 42. Potential Benefits Why are you doing this project? Examples Create a marketable product Improve the efficiency of an organization Control a system that is too complex to control manually New or improved service Safety or security Get a good grade on CS 501 43. Resources Examples: CS 501 Staff: 5 to 7 students, with some help. How many hours per week? What skills do people have? Time: Must be completed by end of semester, including operational system, documentation, presentation Equipment and software: What special needs are there? Client: Will the client be sufficiently available and helpful? 44. Obstacles CS 501 projects Start-up time. Creating a team, scheduling meetings, acquiring software, learning new systems, ... Business considerations. Licenses, trade-secrets, ... Too ambitious. Nothing to show at the end of the semester. Changing circumstances. Client leaves the university, ... What else? 45. How to Minimize Risk? CS 501 Projects Several target levels of functionality: required, desirable, optional Visible software process: intermediate deliverables Good communication within team and with Teaching Assistant Good processes lead to good software Good processes reduce risk 46. Feasibility Report A written document For a general audience: client, financial management, technical management, etc. Short enough that everybody reads it Long enough that no important topics are skipped In CS 501, I am looking for a well written, well presented document. 47. Requirements Definition and Analysis Requirements Definition System and Software design Programming and Unit Testing Integration and System Testing Operation and Maintenance 48. Example: Library of Congress (A Partial Failure)Outline Description The Library of Congress requires a repository system to store and make accessible very large amounts of highly varied material over long periods of time. 49. Chronology 1993-94 CNRI carries out research on architectures for digital libraries 1995-97 CNRI implements prototype repository for Library of Congress 1998CNRI and Library of Congress carry out requirements definition 50. The Repository RepositoryUsersIdentification SystemSearch System 51. Storage and Representation of Complex Objects Data Several representations: thumbnail image reference image archival image Metadata Each representation may have its own metadata 52. Repository: Research Achievements 1. CORBA implementation of repository access protocol. 2. Integration of persistent naming through handle system. 3. Use of structural metadata to describe complex objects, elementary typology. 4. Access management framework and implementation. 5. Applet-based middleware for user interfaces. 6. Information visualization program to view the structure of large collections. 53. Good Discoveries During Prototype Structuring complex information in digital libraries Data driven digital library interfaces Comparison of object-oriented, relational, and file based storage systems Naming and identification of library objects Boundaries of required repository system 54. Bad Discoveries During Prototype Resistance to change within Library of Congress Technical weakness of Library of Congress Gaps in CNRI architecture 55. Mistakes Confusion of objectives (research and implementation) Failure to involve all stakeholders Over-ambitious (no proper feasibility study) 56. The Requirements Process Feasibility StudyFeasibility ReportRequirements Analysis Requirements Definition System ModelsRequirements SpecificationDefinition of RequirementsRequirements DocumentSpecification of Requirements 57. Requirements DefinitionHigh-level abstract description of requirements: Specifies external system behavior Comprehensible by customer, management and users Should reflect accurately what the customer wants: Services that the system will provide Constraints under which it will operate 58. Library of Congress Requirements StudyTeam (all experienced): Librarian, Software Engineer (CNRI), Computing Project Leader (Library of Congress), + 2 others Advisors: Mailing list of about 20 knowledgeable stakeholders. Timetable: Preliminary report (2 months). Final report (1 month). 59. Functional RequirementsExample: Library of Congress repository Support for complex digital objects Access management Identification Information hiding Open protocols and formats Integration with other systems (scope) 60. DRAFT OVERVIEW OF ITS SUPPORT FOR NDLP PRODUCTION AND DELIVERY OF AMERICAN MEMORY NDLP collections already releasedCoolidge collection (for repository test)NDLP collections in conversionFuture NDLP collectionsOther applications and materialsNDLP Workflow Tracking SupportCurrent Storage Structure (in Unix files, by aggregate)Object Administration SystemILS Repository Index Generation (including pre-processing)American Memory User Interface (retrieval, navigation, & display)AM user interface plus access management for objects/collectionsOther User Interfaces (e.g. RLG, OCLC, DLF partners)ILS OPAC InterfaceSupporting infrastructure Handle assignment & registration Handle-serverNOWHandle resolutionFUTURE 61. Non-functional Requirements Environment: Estimates of sizes, numbers of users, etc. Reliability and performance measures and targets Preferred: Example: Library of Congress repository Hardware and software systems (e.g., IBM/Unix) Database systems (e.g., Oracle) Programming languages (e.g., C and C++) 62. Evolution of Requirements If the requirements definition is wrong, the system will be a failure. With complex systems, understanding of requirements always continues to improve. Therefore... The requirements definition must evolve. Its documentation must be kept current (but clearly identify versions). 63. Software EngineeringLecture 4 Management I: Project Management 64. The Aim of Project ManagementTo complete a project: On time On budget With required functionality To the satisfaction of the client Without exhausting the team 65. The Project Manager Create and maintain the schedule Should track progress against schedule Keep some slack in the schedule Be continually making adjustments: Start activities before previous activity complete Sub-contract activities Renegotiate deliverables Keep senior management informed 66. Project Planning Methods The Critical Path Method, Gantt charts, Activity bar charts, etc. are roughly equivalent. These methods are best when: Model is updated regularly (e.g., monthly) The structure of the project is well understood The time estimates are reliable Activities do not share resources [Critical Path Method is excellent for large construction projects.] 67. Example: An Open University Course Deliverables: 16 8 8 4 1 4Written texts (bound in pairs) Television programs Radio programs Computer programs Home experimental kit (scientific calculator) Assignments and sample solutions 68. FlexibilitySchedule: Dates for broadcasting TV and radio programs are fixed. Printing and mailings can be accelerated if overtime is used. Functionality: The course team can decide what goes into the components of the course. Resources: The size of the course team can be increased slightly. 69. Scheduling: Critical Path MethodAn activityA dummy activityAn eventA milestone 70. Critical Path Methodother activities STARTRevise Unit 3Edit Unit 3Print Unit 3Mail Unit 3 END 71. Critical Path Methodother activitiesRevise Unit 3Edit Unit 3STARTother activitiesRevise Unit 4Edit Unit 4Typeset Unit 3Print Units 3/4Typeset Unit 4Mail Units 3/4 72. Critical Path Method Script TV 2 STARTEdit Unit 3Make TV 2Edit Unit 4Prototype Computer 1Document Computer 1Mail DeliveryProgram Computer 1 73. Time Estimates for Activities (Weeks)4 16 323 112 121324 413 28 74. Earliest Start Dates1 1 12264 36 15217312 012 1232173 19122123 24 44117825 75. Latest Start Dates11 1 124 36 151217312 012 14 4 13326217 43 2023124 2 171825 76. Critical Path 1/11 12/1226/26 15/1517/17 22/230/0 12/14 4/1317/1719/20 17/1723/2425/25 77. Slack 1/11 10 12/12 00/026/26 103 15/15000 122/2302 12/14 2 17/17 917/174/1319/201 0923/2425/25 11 5 17/170 78. Key PersonnelIn computing, not all people are equal: The best are at least 5 times more productive Some tasks are too difficult for everybody Adding more people adds communications complexity Some activities need a single mind Sometimes, the elapsed time for an activity can not be shortened. What happens to the project if a key person is sick or quits? 79. Key Personnel: Schedule for Editor Earliest Start DateActivityWeeks 15-16 Weeks 17-18 Weeks 19-20 Weeks 21-22Edit Unit 3 Edit Unit 4 Edit Unit 5 Edit Unit 6Week 15 Week 17 Week 19 Week 21Review draft of Unit 7 Review draft of Unit 8 Check proofs of Unit 3 Check proofs of Unit 4Weeks 18-19 Week 22Vacation Out sick 80. Start-up Time On a big project, the start-up time is typically three to six months: Personnel have to complete previous projects (fatigue) or recruited. Hardware and software has to be acquired and installed. Staff have to learn new domain areas and software (slow while learning) Clients may not be ready. 81. Experience with Critical Path Method Administrative computing department at Dartmouth used the Critical Path Method for implementation phase of major projects. Experience: Elapsed time to complete projects was consistently 25% to 40% longer than predicted by model. Analysis: Some tasks not anticipated (incomplete understanding) Some tasks had to be redone (change of requirements, technical changes) Key personnel on many activities (schedule conflicts) System ZZZ (non-billable hours) 82. CS 501: Software EngineeringLecture 5 (a) Documentation (b) Requirements Analysis 83. AssignmentsSeptember 13 October 4 October 16 November 8 Nov 29 - Dec 1 Exam weekFeasibility and plan Requirements Midterm exam Design Project presentations Final examinationDetails are subject to change.Group Group/individual Individual Group/individual Group Individual 84. Assignment 1 Wednesday, September 13: Project plan due -- report. Title of project Client/customer Team members Outline description Current status (e.g., previous work) Plan (e.g., major stages, assignment to tasks, technical environment, schedule, etc.) Any other relevant information 85. Documentation Reasons for documentation: visibility (e.g., project plan, interim report) user support (e.g., user manual) team communication (e.g., interface specifications) maintenance and evolution (e.g., requirements) Characteristics of documentation: accurate and kept current appropriate for audience maintained online (usually) simple but professional in style and appearance Documentation is expensive --> Quality not volume 86. Form of DocumentationExternal Printed Web site Internal Program documentation Program context (e.g., copyright notices) 87. Requirements Definition and Analysis Requirements Definition System and Software design Programming and Unit Testing Integration and System Testing Operation and Maintenance 88. The Requirements Process Feasibility StudyFeasibility ReportRequirements Analysis Requirements Definition System ModelsRequirements SpecificationDefinition of RequirementsRequirements DocumentSpecification of Requirements 89. Requirements Analysis 1. Understand the requirements in depth: Domain understanding Examples: science research, application Stakeholders Example: companies, ministries, Danang City 90. Viewpoint Analysis Example: University Admissions System Applicants University administration Admissions office Financial aid office Special offices (e.g., athletics, development) Computing staff Operations Software development and maintenance Academic departments 91. Interviews with Clients Clients may have only a vague concept of requirements. Prepare before you meet with them Keep full notes If you don't understand, delve further Small group meetings are often most effective Clients often confuse the current system with the underlying requirement. 92. Requirements Analysis2. Organize the requirements: Classification into coherent clusters (e.g., legal requirements) Recognize and resolve conflicts (e.g., functionality v. cost v. timeliness) Example: Dartmouth general ledger system 93. Requirements Analysis3. Model the requirements: Informal Prose Systematic Procedural models Data-centric models Object models Formal models 94. Procedural Models: FlowchartOperation Decision Manual operation Report 95. Flowchart: University AdmissionsForm receivedNew? T Database record Notify studentFUpdate databaseComplete? T F Notify studentEvaluate 96. Procedural Models: Pseudo-code Example: Check project project plan check_plan (report) if report (date_time) > due_date_time then error (too_late) if report (client) = none then error (no_client) if report (team) < min_team or > max_team then error (bad_team) if error() = none then comments = read_report (report) return (comments (text), comments (grade)) else return error() 97. Data-Flow ModelsExternal entities Processing steps Data stores or sources Data flows 98. Example: University AdmissionsRejection Application Completed form Receive application Evaluate application Applicant Offer 99. Example: University Admissions Assemble Application Stage Acknowledgment Application formReceiveApplicantCompleted applicationAcknowledgment ANDInitiate evaluationEvaluation requestANDSupporting information Pending databaseApplicant database 100. Example: University Admissions Process Completed Application Stage Rejection Evaluation requestEvaluationAcceptanceSpecial request Applicant databaseFinancial aidOffer 101. Requirements Analysis v. System Design Dilemma. Requirements analysis should make minimal assumptions about the system design. But the requirements definition must be consistent with computing technology and the resources available. In practice, analysis and design are interwoven. However, do not to allow the analysis tools to prejudge the system design. 102. CS 501: Software EngineeringLecture 6 (a) Requirements Analysis (continued) (b) Requirements Specification 103. The Requirements Process Feasibility StudyFeasibility ReportRequirements Analysis Requirements Definition System ModelsRequirements SpecificationDefinition of RequirementsRequirements DocumentSpecification of Requirements 104. Requirements Analysis Methods for data modeling and design Data flow diagrams Entity-relation diagrams Data dictionaries Object models Many of these methods blur the distinction between analysis and design. 105. Entity-Relation ModelA Design Methodology for Relational Databases A database of entities and relations Tools for displaying and manipulating entityrelation diagrams Tools for manipulating the database (e.g., as input to database design) Warning: There is much confusion about definitions and notation 106. Entity-Relation DiagramAn entity A relation between entities An entity or relation attribute An inheritance relation 107. Example: CS 501 Project MajorClient 1StudentProject 1CS501 Student5 to 7Member of0:n Person0:n0:n Tech contact 108. MARC Format for Monographs (Books) 001 245 260 650 650 70089-16879 r93 Campus strategies for libraries and electronic information {Bedford, Mass.} : Digital Press, c1990. Academic libraries--United States--Automation. Libraries and electronic publishing--United States. Arms, Caroline R. (Caroline Ruth) 109. Entity-Relation Diagram for MARC Book0:n10:nDescribes 0:n Catalog record Short title1:nAuthor of Editor ofIs aboutControl numb0:n Creator0:n 0:nSubject heading 110. Data DictionariesA data dictionary is a list of names used by the system Brief definition (e.g., what is "date") What is it (e.g., number, relation) Where is it used (e.g., source, used by, etc.) May be combined with a glossary As the system is implemented, the data dictionary in the requirements is input to the system data dictionary, which is a formal part of the system specification. 111. A Note on Object ModelsThis course teaches object models as a tool for design. Some people recommend object models for requirements analysis, but it is difficult to use them without constraining the system design. 112. Non-Functional Requirements Product requirements performance, reliability, portability, etc... Organizational requirements delivery, training, standards, etc... External requirements legal, interoperability, etc... 113. Examples of Non-Functional Requirements Privacy (Mercury digital library) Functional requirement: Usage data for management of system Non-functional requirement: Usage data must not identify individuals Minimizing records (NeXT) Functional requirement: Retain all required records Non-functional requirement: Discard all other records 114. Unspoken RequirementsExample: Resistance to change at XXX 115. Requirements Specification What is the purpose of the Requirements Specification? 116. Requirements Specification: Purpose1. It describes the requirements to the stakeholders Expressed in the terms that the stakeholders understand Comprehensible from many viewpoints Reviewed by stakeholders so that they understand implications Must be clear about assumptions (things left out) 117. Requirements Specification: Purpose 2. It describes the requirements to the implementers As precise and specific as possible Expressed in terms that they understand Comprehensible to new team members 118. Requirements Specification: Purpose 3. It records the requirements for the future An essential part of system evolution 4. If may be a contractual document See you in court! 119. Requirements Specification: Approaches Natural language Structured natural language Design description language Requirements specification language Graphical notation Formal specification See Sommerville, Chapter 7. 120. CS 501: Software EngineeringLecture 7 Management II Business and Legal Aspects of Software Engineering 121. Legal Environment Software is developed in a complex legal and economic framework. Changes in laws follow changes in technical world. Jurisdictions: Vietnamese laws International treaties Federal and state statues Precedents Supreme Court Cost of establishing precedent 122. Legal Topics International Intellectual property (copyright, patent, contract) Tort (e.g., liability of Internet service provider) Privacy Free speech and its limitations (government secrets, obscenity, blasphemy, hate) Legal Information Institute: http://www.law.cornell.edu/ 123. Copyright A copyright gives the owner the exclusive right to: reproduce distribute perform display license Gradually extended to cover text, music, photographs, designs, software, ... 124. Copyright Copyright at creation Works for hire Contracts and licenses First sale Fair use Infringement (contamination)International differences Moral rights Copyright registration 125. Software Patents Should be: non-obvious, novel, useful 17 years from award (20 years from application) Poor quality of examining can lead to broad patents for routine computing concepts International differences Copyright applies to the expression of ideas, patents to the ideas themselves. 126. Contracts and Licences Contracts allow intellectual property to be sold or licensed Promise in exchange for adequate consideration Written document with signature Permanent or temporary, whole or part Exclusive or non-exclusive Termination, problems and difficulties Terms and conditions as agreed Enforceable by courts 127. Derivative Works When software is derived from other software: New code is owned by new developer Conditions that apply to old code apply to derived work If you write S, which is derived from A, B, C and D, you can not distribute or licenses S unless you have right to distribute each of A, B, C and D. To create a software product, you must have documented rights to use every component. 128. Privacy Invasions of privacy: intrusion appropriation of name or likeness unreasonable publicity false lightBe very careful about collecting personal data without the knowledge of the individual 129. Software Business Questions You are employed for company X writing software. When you leave, who owns your work? What use can you make of the work? You work free-lance for company X. When you finish, who owns your work? What use can you make of the work? Read the contract! 130. Your Next Job ... Employment contract may restrict your next job (not working for competitors, etc.) Trade-secret information (non-disclosure agreement) Ask when you are interviewed! 131. Trade Secrets and Non-Disclosure Agreements Trade Secret "... information, including a formula, pattern, compilation, program, device, method, technique, or process that derives independent economic value from not being generally known and not being readily ascertainable and is subject to reasonable efforts to maintain secrecy." Uniform Trade Secrets Act Non-Disclosure Agreement Legal agreement not to disclose trade secrets. 132. Some Business Models Software developed in-house Package licensed to customer, binary only (Microsoft model) Package licensed to customer, source code for customer's modifications Bespoke software for customer (may be owned by supplier or customer) Software bundled with hardware product (PalmPilot) 133. Free-Lance Software DevelopmentYou and a few friends create a company to develop software. How much should you charge per hour? You plan to work 40 hours a week for 50 weeks of the year and want to earn $50,000. Hourly rate = $50,000 / (40 x 50) = $25 But ... 134. Free-Lance Software Development Salary Taxes and benefits Rent, equipment, etc. Fees, services, etc. Travel and misc. TOTAL EXPENSE Hours worked less administration less marketing BILLABLE HOURS$50,000 $15,000 $10,000 $15,000 $10,000 $100,000 2,000 400 350 1,250Hourly rate = $100,000 /1,250 = $80 135. Fixed and Variable Cost: Packaged Software Example: The initial development cost of a software product is $10 million. The cost of packaging and distribution of each copy is $5. Technical support costs average $15 per copy. The package sells for $200 per copy. Fixed cost = $10 million Variable cost = $20 136. Fixed and Variable Costs: Profit or Loss $15M$10M$5M2,5005,0007,500Unit sales 137. Community Development Shareware Open source (e.g., Linux, Apache, Perl, etc.) -> Shared development -> Market penetration Example: TCP/IP for Vax/VMS Software may be open source, but packaging and services can be profitable businesses 138. Open Source Free redistribution Source code Derived works Integrity of the author's source code No discrimination against persons or groups 139. Open Source No discrimination against fields of endeavor Distribution of license License must not be specific to a product License must not contaminate other software http://www.opensource.org/osd.html 140. Practical AdviceBe aware of the law, but do not pretend to be a lawyer. Use a professional for: Contracts and licenses Troubles (complaints, injunctions, subpoenas, etc.) Personnel issues When in doubt, ask help! 141. Source Code Management Or Configuration Management: How I learned to Stop Worrying and Hate My Co-workers Less 142. Source Code Management Also known as Configuration Management Source Code Managers are tools that: Archive your development files Serve as a single point of entry/exit when adding or updating development files 143. Why You Want A Source Control System Supports concurrent development Manage diverging source code bases Records file/release versions Easy access to all previous revisions Can record why a revision was made Optimal disk space usage Youll end up doing something equivalent anyway so it may as well be automated 144. Source Code Management Tools Are Not A substitute for project management A replacement for developer communication 145. How They Work Central database of source code, documentation, build tools Each file stored only once - all other versions are diffs of that one copy To Make a Change Check out the latest version of a file Make the changes Update the database 146. What should be in the database Source Code Documentation Build Tools Often need old versions of the tools to build old versions of the software Ensures software is rebuilt exactly as the customer received it Test Suites Anything else you might want later 147. Version Control Companies ship several products from the same source base (i.e. Win NT and Windows 2000 versions of MS Office) When tracking down bugs you want to examine the code as it was when the product shipped 148. Code Sharing Multiple people can work on the same source base without colliding (1) Locks individual files so only one person at a time can modify it *OR* (2) Allows multiple people to modify a source file and the system will automatically merge the changes (usually) 149. Locking Only one person can work on a file at once Works fairly well if developers work on different areas of the project and dont conflict often Problem 1: People forget to unlock files when they are done Problem 2: People work around locking by editing a private copy and checking in when the file is finally unlocked - easy to goof and lose changes 150. Merging Several people can work on a file at once Before committing changes, each user merges their copy with the latest copy in the database This is normally done automatically by the system and usually works, but you should not blindly accept the result of the merge 151. Labeling Label all the files in the source base that make up a product at each milestone Just before and just after a major change (e.g.. changing several interfaces) When a new version ships 152. Version Trees Each file in the database has a version tree Can branch off the version tree to allow separate development paths Typically a main path (trunk) for the next major version and branches off of shipped versions for maintenance 153. Branching When a new version ships, typically create a branch in the version tree for maintenance Double update: fix a defect in the latest version and then merge the changes (often by hand) into the maintenance version Also create personal versions so you can make a change against a stable source base and then merge in the latest version later 154. Examples RCS Solaris: man rcsintro CVS Solaris: man cvs www.cyclic.com/cvs/info.html Visual SourceSafe msdn.microsoft.com/SSAFE ClearCase www.rational.com 155. RCS File management only Transaction model check out and lock edit check in and unlock Little support for binaries 156. CVS Built on top of RCS Therefore little support for binaries Database can be remote No locking: merge before commit Fast Integrates with emacs 157. SourceSafe Microsofts entry into the field Project-based Checkout-edit-checkin model Built-in web site creation tools Integrates with MSDEV 158. Clearcase Clearcase is configuration management on steroids You create a view of the database with a config spec, which describes how to select files from the database. When you set a view, Clearcase creates a virtual filesystem containing only those versions of the files selected by the config spec 159. Clearcase Features Distributed System Several groups at different locations can work on the same database Can install triggers Example: e-mail the author of a file when some one makes a change to it Uses merging model like CVS, but can also lock files 160. More Clearcase Features Integrates with MSDEV Build Management Knows to rebuild out-of-date files even if your makefile doesnt Slow and a bit buggy 161. Helpful Rules for Version Control Bliss Archived Files Should Always Compile Code Review Files Before Check-in Compile and run latest archived files *as a set* before Check-in No Cheating (even simple bug fixes need to undergo this process) 162. Software EngineeringLecture 10 Formal Specification 163. Formal SpecificationWhy? Precise standard to define and validate software Why not? May be time consuming Methods not suitable for all applications 164. Formal Specification Ben Potter, Jane Sinclair, David Till, An Introduction to Formal Specification and Z (Prentice Hall) 1991 Jonathan Jacky The Way of Z (Cambridge University Press) 1997 165. Mathematical Specification Example of specification B1, B2, ... Bk is a sequence of m x m matrices1, 2, ... k is a sequence of m x m elementary matrices B1-1 = 1 B2-1 = 21 Bk-1 = k ... 21 The numerical accuracy must be such that, for all k, BkBk-1 - I < 166. Specification of Programming Languages ::= | ::= {} ::= . {} | . {} E | E ::= | ::= + | Pascal number syntax 167. Formal Specification Using Diagrams unsigned integer digitunsigned number unsigned integer+ .digitunsigned integerE - 168. Two Rules Formal specification does not guarantee correctness Formal specification does not prescribe the implementation 169. Example: Z Specification Language Informal: The function intrt(a) returns the largest integer whose square is less than or equal to a. Formal (Z):intrt: NNa : N intrt(a) * intrt(a) < a < (intrt(a) + 1) * (intrt(a) + 1) 170. Example: Algorithm 1 + 3 + 5 + ... (2n - 1) = n2 171. Example: Program int intrt (int a) /* Calculate integer square root */ { int i, term, sum; term = 1; sum = 1; for (i = 0; sum A sequence diagram emphasizes the time ordering. => A collaboration diagram emphasizes the structural organization of the objects that send and receive messages. 213. Diagrams in UML (continued) Statechart diagram shows a state machine consisting of states, transitions, events, and activities. Activity diagram is a statechart diagram that shows the flow from activity to activity within a system. Component diagram shows the organization and dependencies among a set of components. Deployment diagram shows the configuration of processing nodes and the components that live on them. 214. The HelloWorld Example class nameHelloWorldoperations paint() 215. Abstraction for HelloWorld class nameHelloWorldoperations paint()annotation g.drawString ("HelloWorld", 0, 10)" 216. The "Hello, World" Exampleimport java.awt.Graphics; class HelloWorld extends java.applet.Applet { public void paint (Graphics g) { g.drawString ("Hello, World!", 10, 10); } } Example from: BJR 217. Class DiagramApplet generalizationNote that the Applet and Graphics classes are shown elided.HelloWorldpaint()dependencyGraphics 218. Class Inheritance Diagram Object Panelinterface Component ImageObserverApplet Container HelloWorld 219. Packaging Classesjava HelloWorldappletGraphicsawtlangpackage 220. Notation for Classes and Objects Classes AnyClass attribute1 attribute2 operation1() operation2() or AnyClassObjects anObject:AnyClass or :AnyClass or anObject The names of objects are underlined. 221. Software EngineeringLecture 12 Object-Oriented Design II 222. Requirements: the Long Term Believe that your software will be in use 5 years from now. What happens at end of semester? Packaging and hand-over Client's technical preferences (C++, Java) Some system decisions based on short-term considerations Which formats, protocols, etc. do you think will last? (IIOP, RMI, SNMP, ...) 223. Requirements, Design and Implementation Remember the definitions. Example: Consistency between two players of a board game The requirement is .....The design is .....What is a requirements specification? 224. Modeling Classes Given a real-life system, how do you decide what classes to use? What terms do the users and implementers use to describe the system? They are candidates for classes. Is each candidate class crisply defined? For each class, what is its set of responsibilities? Are the responsibilities evenly balanced among the classes? What attributes and operations does each class need to carry out its responsibilities? 225. Noun Identification: A Library Example The library contains books and journals. It may have several copies of a given book. Some of the books are reserved for short-term loans only. All others may be borrowed by any library member for three weeks. Members of the library can normally borrow up to six items at a time, but members of staff may borrow up to 12 items at one time. Only members of staff may borrow journals. The system must keep track of when books and journals are borrowed and returned and enforce the rules. 226. Noun Identification: A Library Example The library contains books and journals. It may have several copies of a given book. Some of the books are reserved for short-term loans only. All others may be borrowed by any library member for three weeks. Members of the library can normally borrow up to six items at a time, but members of staff may borrow up to 12 items at one time. Only members of staff may borrow journals. The system must keep track of when books and journals are borrowed and returned and enforce the rules. 227. Candidate Classes Library Book Journal Copy ShortTermLoan LibraryMember Week MemberOfLibrary Item Time MemberOfStaff System Rulethe name of the systemevent measure repeat book or journal abstract term general term general term 228. Relations between Classes Book Journal Copy LibraryMember Item MemberOfStaff Is Item needed?is an is an is a copy of aItem Item Bookis aLibraryMember 229. Operations LibraryMemberborrowsCopyLibraryMemberreturnsCopyMemberOfStaffborrowsJournalMemberOfStaffreturnsJournalItem not needed yet. 230. Class Diagram MemberOfStaffLibraryMember11 on loanon loan 0..12 Journal0..* Copyis a copy of 1..*1Book 231. Rough Sketch: Wholesale System A wholesale merchant supplies retail stores from stocks of goods in a warehouse. What classes would you use to model this business? 232. Rough Sketch: Wholesale SystemRetailStore Order Merchant Product Warehouse InvoiceShipment 233. Rough Sketch: Wholesale System RetailStore name address contactInfo financialInfoMerchant Warehouse Order Product ReversalsInvoiceShipmentdamaged() return() wrongItem()Responsibilities -track status of shipped products responsibility (text field) 234. Expanding a Class: Modeling Financial Information RetailStoreassociation 1 * TransactionWhich class is responsible for the financial records for a store?Payment Invoice 235. Modeling Invoice Shipment???RetailStore invoiceRecordgoodsShippedInvoice invoiceNumberadornments +goodsShipped() + public -sendInvoice() - privatePartsList 236. Lessons LearnedDesign is empirical. There is no single correct design. During the design process: Eliding: Elements are hidden to simplify the diagram Incomplete: Elements may be missing. Inconsistency: The model may not be consistent The diagram is not the whole design. Diagrams must be backed up with specifications. 237. Levels of AbstractionThe complexity of a model depends on its level of abstraction: High-levels of abstraction show the overall system. Low-levels of abstraction are needed for implementation. Two approaches: Model entire system at same level of abstraction, but present diagrams with different levels of detail. Model parts of system at different levels of abstraction. 238. Component Diagramhello.hmlexecutable component HelloWorld.classhello.jpghello.java 239. Actor and Use Case DiagramBookBorrowerBorrow book An actor is a user of a system in a particular role. An actor can be human or an external system. A use case is a a task that an actor needs to perform with the help of the system. 240. Use Cases and Actors A scenario is an instance of a use case Actor is role, not an individual (e.g., librarian can have many roles) Actor must be a "beneficiary" of the use case (e.g., not librarian who processes book when borrowed) In UML, the system boundary is the set of use cases. 241. Use Cases for Borrowing BooksBorrow copy of book BookBorrowerExtend loanReturn copy of book Reserve book 242. Relationships Between Use Cases: Extend loan BookBorrowerBorrow copy of bookCheck for reservation 243. Relationships Between Use Cases: BookBorrower Borrow copy of bookRefuse loan 244. Use Cases in the Development Cycle Use cases are a tool in requirements analysis Intuitive -- easy to discuss with clients Use cases are often hard to translate into class models Scenarios are useful to validate design 245. Software EngineeringLecture 13 Object-Oriented Design III 246. Comments on Presentations Presentation Standard of graphics has been high Some text too small (diagrams, screen dumps) Content Level of detail Requirements v. design The client defines the requirements Well done, but time is short. What is your critical path? 247. Modeling Dynamic Aspects of SystemsInteraction diagrams: set of objects and their relationships including messages that may be dispatched among them Sequence diagrams: time ordering of messages Collaboration diagrams: structural organization of objects that send and receive messages Activity diagram: flow chart showing flow of control from activity to activity Statechart diagram: models a state machine 248. Bouncing Ball Diagrams Example: http://www.cs.cornell.edu/ domain name TCP connection HTTP getClientServers 249. Actions on Objects callreturnCopy(c) okToBorrow()return send create destroylocalstatus notifyReturn(b)asynchronous signal stereotypes 250. Links LibraryMember1on loan0..*Copyassociation +borrowCopy() +returnCopy() class libMem:LibraryMember objectmessage borrowCopy(c) linkc:Copy 251. Sequence Diagram: Change in Cornell Program :MEngStudent Cornellian 1 : getName() 1.1 : name 2: new PhDStudent(name):PhDStudent3: sequence numbers added to messages 252. Sequence Diagram: Borrow copy of a Book libMem: LibraryMember BookBorrowertheBook:Book theCopy:Copyborrow(theCopy) okToBorrow borrowborrow 253. Class Inheritance Diagram Object Panelinterface Component ImageObserverApplet Container HelloWorld 254. Sequence Diagram:Painting Mechanism:Thread run:Toolkit run:ComponentPeercallbackLoophandleExpose painttarget:HelloWorld 255. Activity Diagram: Process Modeling Release work order branch [materials not ready] [materials ready] Assign tasksRescheduleguard expression 256. Activity Diagram: Parallel Activities start state Decompress fork Stream video join stop stateStream audio 257. State Diagram returned()not borrowablereturned() borrowable borrowed()[last copy]guard expressionborrowed()[not last copy]State diagram for class Book 258. Implementation Modeling Subsystem A grouping of elements that specifies what a part of a system should do. Component (UML definition) "A distributable piece of implementation of a system, including software code (source, binary, or executable) but also including business documents, etc., in a human system." A component can be thought of as an implementation of a subsystem. 259. Component Diagramhello.hmlexecutable component HelloWorld.classhello.jpghello.java 260. Components and Classesagent.dllAgentActionPatternSearch Policy 261. Components and Classesagent.dll Realizes AgentAction PatternSearch Policy extended component 262. Components and ClassesClasses represent logical abstractions. Components represent physical things. Components may live on nodes. Classes have attributes and operations directly. Components have operations that are reachable only through interfaces. 263. Interfacesrender.javasimulation.exe IRender dependencyrealization interface 264. Application Programming Interface (API) API is an interface that is realized by one or more components. simulation.exe IRenderIModelsILighting 265. Components and ReplaceabilityComponents allow system to be assembled from binary replaceable elements. A component is physical -- bits not concepts A component can be replaced by any other component(s) that conforms to the interfaces. A component is part of a system. A component provides the realization of a set of interfaces. 266. Software EngineeringLecture 14 System Architecture I Data Intensive Systems 267. System ArchitectureThe overall design of a system: Computers and networks (e.g., monolithic, distributed) Interfaces and protocols (e.g., http, CORBA) Databases (e.g., relational, distributed) Security (e.g., smart card authentication, SSL) Operations (e.g., backup, archiving, audit trails) Software environments (e.g., languages, source control tools) 268. Data Intensive SystemsExamples Electricity utility customer billing Telephone company call recording and billing Car rental reservations (e.g., Hertz) Stock market brokerage (e.g., Charles Schwab) Web sales (e.g., Amazon.com) 269. Example 1: Electricity Utility Billing First attempt:TransactionData inputMaster fileEach transaction handled as it arrives.Bill 270. Criticisms of First Attempt Where is this first attempt weak?The requirements have not been specified!!! 271. Transaction Types Create account / close account Meter reading Payment received Other credits / debits Check cleared / check bounced Account query Correction of error etc., etc., etc., 272. Typical Requirements All payments to be credited on day received Customers must be able to query account by telephone Cutting off service for non-payment requires management authorization Data input staff should process n transactions per day per person Error rate must be below 0.01% System available 99.9% of business hours 273. Batch Processing: Validationerrors Edit & validationIncoming transactions Data inputread onlyMaster fileValidated transactions 274. Batch Processing: Master File UpdateValidated transactions in batcheserrorsReportsSort by account Master file updateBillsInstructions 275. Benefits of Batch Updating All transactions for an account are processed together Backup and recovery have fixed checkpoints Better management control of operations Efficient use of staff and hardware 276. Online Inquiry Customer service read onlyTransactionsData inputMaster fileBills 277. Example 2: A Small-town Stockbroker Transactions Received by mail or over telephone For immediate or later action Complex customer inquiries Highly competitive market 278. A Database ArchitectureDatabase(s): Customer and account database Financial products (e.g., account types, pension plans, savings schemes) External databases (e.g., stock markets, mutual funds, insurance companies) 279. Database ArchitectureProducts & services databaseCustomer & account databaseExternal services 280. Real-time Transaction Real-time transactionsProducts & services databaseCustomer & account databaseExternal services 281. Real-time Transactions & Batch Processing Real-time transactionsProducts & services databaseData inputBatch processingCustomer & account databaseExternal services 282. Architectural considerations Real-time service during scheduled hours + batch processing overnight Combine information from several databases Database consistency after any type of failure two-phase commit reload from checkpoint + log detailed audit trail How will transaction errors be avoided? How will transaction errors be corrected? 283. Example: Merger of Two BanksEach bank has a database with its customer accounts. The databases are used by staff at many branches and for back-office processing. The requirement is to integrate the two banks so that they appear to the customers to be a single organization and to provide integrated service from all branches. 284. Merger of Two Banks: OptionsA??????B 285. Merger of Two Banks: Architectural Options I. Convert everything to System A. convert databases retrain staff enhance System A (software and hardware) discard System B II. Build an interface between the databases in System A and System B. III. Extend client software so that it can interact with either System A or System B database. 286. Distributed Computing: General Problem An application that is running on one computer wishes to use data or services provided by another: Network connection private, public, or virtual private network location of firewalls Protocols point-to-point, multicast, broadcast message passing, RPC, distributed objects stateful or stateless Quality of service 287. Network Choices Public Internet: Ubiquitous -- worldwide Low cost Private network: Security Predictable performance Choice of protocols (e.g., IBM's SNA) 288. Quality of Network ServicesPerformance Maximum throughput Variations in throughput Real-time media (e.g., audio) Business Suppliers Trouble shooting and maintenance Upgrades 289. Firewall Public networkPrivate network FirewallA firewall is a computer at the junction of two network segments that: Inspects every packet that attempts to cross the boundary Rejects any packet that does not satisfy certain criteria, e.g., an incoming request to open a TCP connection an unknown packet type 290. Software EngineeringLecture 15 System Architecture II Distributed and Real Time Systems 291. Comments on Requirements ReportAudience Client and design team Will be updated over timeContent Level of detail -- will be used to validate the implementation Requirements, not design Precise, but not legalistic 292. Sequence Diagram: Notation libMem: LibraryMember BookBorrowertheBook:Book theCopy:Copyborrow(theCopy) okToBorrow borrow borrowdotted line shows object lifetime rectangle shows focus of control 293. Sequence Diagram: Branching libMem: LibraryMembertheCopy:CopyBookBorrower 1:borrow(theCopy)[not ok]3:noborrowbranchtheBook:Book2:okToBorrow [ok]3:borrow4:borrow 294. Example: Distributed Databasetwo copies of the same data 295. Distributed Data and Replication Distributed Data Data is held on several computer systems. A transaction may need to assemble data from several sources. Replication Several copies of the data are held in different locations. Mirror: Complete data set is replicated Cache: Dynamic set of data is replicated (e.g., most recently used) With replicated data, the biggest problem is consistency. 296. Example: Broadcast SearchUserUser interface serverDatabases 297. Example: UseNet 298. Stateless Protocol v. StatefulStateless protocol Example: http Open connection Send message Return reply Close connection State in http must be sent with every message (e.g., as parameter string or in a cookie) 299. Stateless Protocol v. Stateful Stateful (session) protocol Example: Z39.50 Open connection Begin session Interactive session End session Close connection Server remembers the results of previous transactions (e.g., authentication, partial results) until session is closed. 300. Firewall Public networkPrivate network FirewallA firewall is a computer at the junction of two network segments that: Inspects every packet that attempts to cross the boundaryRejects any packet that does not satisfy certain criteria, e.g., an incoming request to open a TCP connection an unknown packet type 301. The Domain Name System First attempt to resolve www.cs.cornell.edu.edu server1 2 3cornell.edu server cs.cornell.edu server 302. Discussion of the First Attempt Problems? 303. The Domain Name System Better methodlocal DNS server 1almaden.ibm.com cornell.edu Local ece.cmu.edu cache ibm.com acm.org .edu.edu server 2cornell.edu server 3 cs.cornell.edu server 304. Real Time SystemA real time system is a software system whose correct functioning depends upon the results produced and the time at which they are produced. A soft real time system is degraded if the results are not produced within required time constraints A hard real time system fails if the results are not produced within required time constraints 305. Example: Web Serverhttp message daemon TCP port 80spawned processesThe daemon listens at port 80. When a message arrives it: spawns a processes to handle the message returns to listening at port 80 306. Embedded SystemsSoftware and hardware are combined to provide an integrated unit, usually dedicated to a specific task: Digital telephoneAutomobile engine controlGPSScientific instrumentsThe software may be embedded in the device in a manner that can not be altered after manufacture. 307. Example: Autonomous Land VehicleGPS Steer Sonar Model LaserControl signalsThrottle ControlsSensorsSignal processing 308. Other Applications Response critical Network routerTelephone switchSeat bag controllerShared systems Multi-user data processingTime sharing 309. TechniquesSpecial purpose hardwareMulti-threading and multi-taskingParallel processing => digital signal processingInterrupts => levels and priorities 310. Multi-ThreadingSeveral similar threads operating concurrently: Re-entrant code -- separation of pure code from data for each thread Testing -- single thread and multi threadMay be real time (e.g., telephone switch) or nontime critical 311. Real Time ExecutiveSchedules and dispatches tasks in a real time system Real time clockInterrupt handlerSchedulerResource managerDispatcherMust be extremely reliable 312. Timing Timing mechanisms Synchronous (clocked) -- periodic stimuliAsynchronous -- wait for next signalExample: Communications protocols may be synchronous or asynchronous 313. Hardware v. SoftwareDesign of embedded systems requires close understanding of hardware characteristics Special purpose hardware requires special tools and expertise. Some functions may be implemented in either hardware of software (e.g., floating point unit) Design requires separation of functionsDistinction between hardware and software may be blurred. 314. Example: Dartmouth Time Shared System master processor Communications processor Communications processorI/O MulitplexorCentral processor Central processor Central processor 315. Software Considerations Resource considerations may dictate software design and implementation: Low level language (e.g., C) where programmer has close link to machine Inter-process communication may be too slow (e.g., C fork). May implement special buffering, etc., to control timings 316. Example: CD Controller4 Input block 73 25 6Circular buffer1 Output block 317. Continuous OperationMany systems must operate continuously Software update while operatingHardware monitoring and repairAlternative power supplies, networks, etc.Remote operationThese functions must be designed into the fundamental architecture. 318. Routers and Other Network ComputingInteroperation with third party devicesSupport for several versions of protocolsRestart after total failureDefensive programming -- must survive => erroneous or malicious messages => extreme loadsTime outs, dropped packets, etc.Evolution of network systems 319. Software EngineeringLecture 15 System Architecture II Distributed and Real Time Systems 320. AdministrationAssignment 2: Requirements Grades -- presentation, report, individual Comments at presentation Comments from teaching assistantAssignment 3: Design 321. Comments on Requirements ReportAudience Client and design team Will be updated over timeContent Level of detail -- will be used to validate the implementation Requirements, not design Precise, but not legalistic 322. Sequence Diagram: Notation libMem: LibraryMember BookBorrowertheBook:Book theCopy:Copyborrow(theCopy) okToBorrow borrow borrowdotted line shows object lifetime rectangle shows focus of control 323. Sequence Diagram: Branching libMem: LibraryMembertheCopy:CopyBookBorrower 1:borrow(theCopy)[not ok]3:noborrowbranchtheBook:Book2:okToBorrow [ok]3:borrow4:borrow 324. Example: Distributed Databasetwo copies of the same data 325. Distributed Data and Replication Distributed Data Data is held on several computer systems. A transaction may need to assemble data from several sources. Replication Several copies of the data are held in different locations. Mirror: Complete data set is replicated Cache: Dynamic set of data is replicated (e.g., most recently used) With replicated data, the biggest problem is consistency. 326. Example: Broadcast SearchUserUser interface serverDatabases 327. Example: UseNet 328. Stateless Protocol v. StatefulStateless protocol Example: http Open connection Send message Return reply Close connection State in http must be sent with every message (e.g., as parameter string or in a cookie) 329. Stateless Protocol v. Stateful Stateful (session) protocol Example: Z39.50 Open connection Begin session Interactive session End session Close connection Server remembers the results of previous transactions (e.g., authentication, partial results) until session is closed. 330. Firewall Public networkPrivate network FirewallA firewall is a computer at the junction of two network segments that: Inspects every packet that attempts to cross the boundaryRejects any packet that does not satisfy certain criteria, e.g., an incoming request to open a TCP connection an unknown packet type 331. The Domain Name System First attempt to resolve www.cs.cornell.edu.edu server1 2 3cornell.edu server cs.cornell.edu server 332. Discussion of the First Attempt Problems? 333. The Domain Name System Better methodlocal DNS server 1almaden.ibm.com cornell.edu Local ece.cmu.edu cache ibm.com acm.org .edu.edu server 2cornell.edu server 3 cs.cornell.edu server 334. Real Time SystemA real time system is a software system whose correct functioning depends upon the results produced and the time at which they are produced. A soft real time system is degraded if the results are not produced within required time constraints A hard real time system fails if the results are not produced within required time constraints 335. Example: Web Serverhttp message daemon TCP port 80spawned processesThe daemon listens at port 80. When a message arrives it: spawns a processes to handle the message returns to listening at port 80 336. Embedded SystemsSoftware and hardware are combined to provide an integrated unit, usually dedicated to a specific task: Digital telephoneAutomobile engine controlGPSScientific instrumentsThe software may be embedded in the device in a manner that can not be altered after manufacture. 337. Example: Autonomous Land VehicleGPS Steer Sonar Model LaserControl signalsThrottle ControlsSensorsSignal processing 338. Other Applications Response critical Network routerTelephone switchSeat bag controllerShared systems Multi-user data processingTime sharing 339. TechniquesSpecial purpose hardwareMulti-threading and multi-taskingParallel processing => digital signal processingInterrupts => levels and priorities 340. Multi-ThreadingSeveral similar threads operating concurrently: Re-entrant code -- separation of pure code from data for each thread Testing -- single thread and multi threadMay be real time (e.g., telephone switch) or nontime critical 341. Real Time ExecutiveSchedules and dispatches tasks in a real time system Real time clockInterrupt handlerSchedulerResource managerDispatcherMust be extremely reliable 342. Timing Timing mechanisms Synchronous (clocked) -- periodic stimuliAsynchronous -- wait for next signalExample: Communications protocols may be synchronous or asynchronous 343. Hardware v. SoftwareDesign of embedded systems requires close understanding of hardware characteristics Special purpose hardware requires special tools and expertise. Some functions may be implemented in either hardware of software (e.g., floating point unit) Design requires separation of functionsDistinction between hardware and software may be blurred. 344. Example: Dartmouth Time Shared System master processor Communications processor Communications processorI/O MulitplexorCentral processor Central processor Central processor 345. Software Considerations Resource considerations may dictate software design and implementation: Low level language (e.g., C) where programmer has close link to machine Inter-process communication may be too slow (e.g., C fork). May implement special buffering, etc., to control timings 346. Example: CD Controller4 Input block 73 25 6Circular buffer1 Output block 347. Continuous OperationMany systems must operate continuously Software update while operatingHardware monitoring and repairAlternative power supplies, networks, etc.Remote operationThese functions must be designed into the fundamental architecture. 348. Routers and Other Network ComputingInteroperation with third party devicesSupport for several versions of protocolsRestart after total failureDefensive programming -- must survive => erroneous or malicious messages => extreme loadsTime outs, dropped packets, etc.Evolution of network systems 349. Software EngineeringLecture 16 System Architecture III Distributed Objects 350. Real-Time: Software Considerations Resource considerations may dictate software design and implementation: Low level language (e.g., C) where programmer has close link to machine Inter-process communication may be too slow (e.g., C fork). May implement special buffering, etc., to control timings 351. Buffering Example: CD Controller4 Input block 73 25 6Circular buffer1 Output block 352. Continuous OperationMany systems must operate continuously Software update while operatingHardware monitoring and repairAlternative power supplies, networks, etc.Remote operationThese functions must be designed into the fundamental architecture. 353. Example: Routers and Other Network Computing Interoperation with third party devicesSupport for several versions of protocolsRestart after total failureDefensive programming -- must survive => erroneous or malicious messages => extreme loadsTime outs, dropped packets, etc.Evolution of network systems 354. Example: Transaction MonitormessagesTransaction monitorprocessesA transaction monitor: monitors transactions, routes them across services, balances the load, restarts transactions after failure. 355. Software Reuse: Application Packages Package supports a standard application (e.g., payroll, user interface to Internet information, mathematical algorithms) Functionality can be enhanced by: => configuration parameters (e.g., table driven) => extensibility at defined interfaces => custom written source code extensions 356. Reuse: Object Object Oriented Languages Example: Java is a relatively straightforward language with a very rich set of class hierarchies. Java programs derive much of their functionality from standard classes Learning and understanding the classes is difficult. => Java experts can write complex systems quickly => Inexperienced Java programmers write inelegant and buggy programs 357. Reuse: Objects - Basic Definitions An object is a piece of code that owns attributes and provides services through methods. The methods operate on instance data owned by the object. A class is a collection of like objects. 358. Reuse: Objects - Characteristics Encapsulation. An object has a public interface that defines how other objects or applications can interact with it. methods public instance data Inheritance. Subclasses can be derived from parent classes. They inherit or override the parents' methods and instance data. Polymorphism. The effect of a method can vary depending on the class that implements it (e.g., display_object) 359. Reuse: Objects - Object Binding Binding is the linking of the software interface between two objects. Static binding: The interface is determined at compile or build time. Straightforward Allows type checking Dynamic binding or late binding: The link is established at run time. Flexible and extensible Complex 360. Reuse: Objects - Distributed ObjectsObjects on separate computers interact through method calls and instance data. Major systems: CORBA (Common Object Request Broker Architecture) Microsoft family: OLE, COM, DCOM, Active X ... 361. Desirable Properties of Distributed Objects Different languages and operating environments Reusable code: components Architecture can be extensible Future changes can be localized Standard tools used for client/server interactions 362. Example: Fedora IDLA research project to explore extensibility: -- very simple Interface Definition Language -- powerful tools for extensions -- interoperability, Cornell and CNRI http://www.cs.cornell.edu/cdlrg/fedora.html 363. Object Request Broker (ORB)CC++Cobol ObjectsIDLIDLIDL Interface IDLJavaClientOther IDLServerObject Request Broker 364. Interface Definition Language module { ; ; ;Naming contextinterface [:] { See next slide } interface [:] { ..... } {Define a classDefine a class 365. Interface Definition Language (continued) interface [:] { ; ; ;Define a class[() Define a method [raises exception] [context]; .... [() Define a method [raises exception] [context]; .... } 366. ORB: Programmer's View Client Invoke a on object XServer Invoke a on object YObject XObject YaaObject Request Broker 367. Object Request Broker (ORB) An ORB lets objects make requests to and receive response from other objects located locally or remotely. Static and dynamic method invocations High-level language bindings Self-describing system Local/remote transparency Inter-ORB protocols Internet Inter-ORB Protocol (IIOP) 368. ORB: System ViewInterface repositoryClientDynamic Client IDL ORB invocation stubs interfaceObject Implementation implementation repositoryStatic Dynamic Object skeletons invocation adapterObject Request Broker 369. CORBA Services Naming service Event service Concurrency control service Transaction service Relationship service Externalization service Query service Life cycle service Persistence service Licensing service Properties service Security service Time service 370. Distributed Objects and the System Life-Cycle All large systems change with time. Dynamic binding of objects combined with polymorphism permits the addition of extra object types, incremental changes, etc. to be localized. Development environments change with time. Language bindings and IIOP permit changes. Production environments changes with time. Code can be reused in different environments. 371. Software EngineeringLecture 17 Design for Usability I 372. Q2: Finite State Machine The cruise control system on an automobile is controlled by a master switch and three buttons. Initially, it is turned on by the master switch. The master switch can be turned off at any time. When first turned on, the system enters stand-by mode. When the system is in stand-by mode, the driver of the automobile can press Button A to engage the cruise control at the current speed of the automobile. When the cruise control is engaged, if the driver presses the brake or presses Button B the system will be disengaged and return to stand-by mode. After returning to standby mode, the driver can press Button C to engage the cruise control at the speed that it was set at previously. (After the system is first turned on, Button C has no effect.) When the cruise control is engaged, the driver can press Button A to increase speed by one mile per hour or Button C to decrease speed by one mile per hour. 373. State Transition Diagram ACMS-On OffStandbyAA EngagedC B-BrakeMS-OffStandby1 374. State Transition Table MS onMS offAStandbyOffEngagedEngagedOffEngaged Standby1 EngagedStandby1OffEngagedOffB BrakeCStandbyEngaged 375. Question 4 When software is written, who owns the copyright?How can somebody else be permitted to use the software?How can copyright be transferred to somebody else? 376. Question 4 When software is written, who owns the copyright? The person who writes the software Except work for hire -- the employer How can somebody else be permitted to use the software? By permission from the copyright owner (usually a license) How can copyright be transferred to somebody else? Copyright is property that can be sold or given away (usually a contract) 377. Question 4 You are employed for company X writing software. When you leave, who owns your work?What use can you make of the work? 378. Question 4 You are employed for company X writing software. When you leave, who owns your work? The company (work for hire) What use can you make of the work? None without permission of the copyright owner 379. Question 4 You work free-lance for company X. When you finish, who owns your work?What use can you make of the work? 380. Question 4 You work free-lance for company X. When you finish, who owns your work? It depends on the circumstances Have a written contract What use can you make of the work? If you hold the copyright -- unrestricted Otherwise -- none without agreement 381. Distributed Objects and the System LifeCycle All large systems change with time. Dynamic binding of objects combined with polymorphism permits the addition of extra object types, incremental changes, etc. to be localized. Development environments change with time. Language bindings and IIOP permit changes. Production environments changes with time. Code can be reused in different environments. 382. Design for Usability Usability of a computer system is a combination of factors: User interface design Functionality Performance Help systems and documentation Freedom from errors Anything else? 383. Iterative Design EvaluationRequirementsImplementation (prototype)Design 384. Methods for Evaluation of Usability Observing users (user protocols) Focus groups Measurements effectiveness in carrying out tasks speed Expert review Client's opinions Competitive analysis 385. Levels of Usabilityinterface design conceptual modelfunctional design data and metadata computer systems and networks 386. The Conceptual ModelThe conceptual model is the user's internal model of what the system provides: The desk top metaphor -- files and folders The web model -- click on hyperlinks The library model -- search and retrieve The form filling model -- fill form, submit Example: The Mercury page turner 387. Interface Design The interface design is the appearance on the screen and the actual manipulation by the user Fonts, colors, logos, key board controls, menus, buttons Mouse control or keyboard control? Conventions (e.g., "back", "help") Example: Screen space utilization in the Mercury page turner 388. Principles of Interface Design Interface design is partly an art; there are general principles: Consistency -- in appearance, controls, and function. Feedback -- what is the computer system is doing? why does the user see certain results? Users should be able to interrupt or reverse actions Error handling should be simple and easy to comprehend Skilled users offered shortcuts; beginners have simple, well-defined options The user should feel in control 389. Disabilities What if the user: is visually impaired or color blind? does not speak English? is a poor typist? There is a tradition of blind programmers Navigation of web sites need not be only visual You may have a legal requirement to support people with disabilities 390. Functional Design The functional design, determines the functions that are offered to the user Selection of parts of a digital object Searching a list or sorting the results Help information Manipulation of objects on a screen Pan or zoom 391. Same Functions, Different Interface Example: The desk top metaphor Mouse -- 1 button (Macintosh), 2 button (Windows) or 3 button (Unix) Close button -- left of window (Macintosh) right of window (Windows) 392. Data and Metadata Data and metadata stored by the computer system enable the functions and the interface The desktop metaphor has the concept of associating a file with an application. This requires a file type to be stored with each file: -- extension to filename (Windows and Unix) -- resource fork (Macintosh) Data validation often requires that a user interface has access to a database (e.g., names and addresses) 393. Computer Systems and Networks The performance, reliability and predictability of computer systems and networks is crucial to usability Response time instantaneous for mouse tracking and echo of key stroke 5 seconds for simple transactions Example: Pipelined algorithm for the Mercury page turner Quality of Service for real time information 394. Design Tensions in Networked Systems Client computers and network connections vary greatly in capacity Client software may run on various operating systems; it may be current or an earlier version System designers wish to control clients; users wish to configure their own environments 395. Usability and Cost Performance may be expensive in hardware or special software development User interface development may be a major part of a software development project Costs are multiplied if a user interface has to be used on different computers or migrate to different versions of systems Web browsers provide a general purpose user interface that others maintain 396. Extensibility in Web Browsers Data types: helper applications, plug-ins Protocols HTTP, WAIS, Gopher, FTP, etc. proxies Executable code CGI scripts at server JavaScript at client Java applets Style sheets 397. Software EngineeringLecture 18 Design for Usability II 398. Q5: Object Oriented DesignA system generates weather maps using data collected from unattended weather stations. Each weather station collects meteorological data and produces summaries of the data. On request, it sends the summary information to an area computer. The area computer uses a database of digitized maps to generate a set of local weather maps. 399. Noun Identification: A Library Example The library contains books and journals. It may have several copies of a given book. Some of the books are reserved for short-term loans only. All others may be borrowed by any library member for three weeks. Members of the library can normally borrow up to six items at a time, but members of staff may borrow up to 12 items at one time. Only members of staff may borrow journals. The system must keep track of when books and journals are borrowed and returned and enforce the rules. 400. Q5: Noun IdentificationA system generates weather maps using data collected from unattended weather stations. Each weather station collects meteorological data and produces summaries of the data. On request, it sends the summary information to an area computer. The area computer uses a database of digitized maps to generate a set of local weather maps. 401. Candidate Classes Library Book Journal Copy ShortTermLoan LibraryMember Week MemberOfLibrary Item Time MemberOfStaff System Rulethe name of the systemevent measure repeat book or journal abstract term general term general term 402. Q5: Candidate Classes Systemgeneral termWeatherMap Datasame as MeteorologicalDataWeatherStationis this a general term?MeteorologicalData how does this relate to WeatherStation? DataSummaryhow does this relate to DataSummary?AreaComputerhardwareDatabasegeneral termDigitizedMap 403. Q5: Observations about the Candidate Classes WeatherMapis a DigitizedMap is derived from 1...* DataSummaryWeatherStationhas a set of MeteorologicalDataMeteorologicalData DataSummaryis derived from MeteorologicalDataDigitizedMap Can Meteorological Data be an attribute of WeatherStation? Can DataSummary be combined with WeatherMap? 404. Q5: Attributes and operations WeatherStation location metereologicalData collectData() getSummary() DigitizedMap location geographicData printMap()WeatherMap location date-time geographicData weather gatherData() printMap() Or should MetereologicalData be a separate object? 405. Class Diagram MemberOfStaffLibraryMember11 on loanon loan0..*0..12 JournalCopyis a copy of 1..*1Book 406. Q5: Class Diagram DigitizedMapWeatherStation location metereologicalData collectData() getSummary()WeatherMap 11...* summarylocation date-time geographicData weather gatherData() printMap() 407. Command Line Interfaces User interacts with computer by typing commands Allows complex instructions to be given to computer Facilitates formal methods of specification & implementation Skilled users can input commands quickly Requires learning or training Can be adapted for people with disabilities Can be multi-lingual Suitable for scripting / non-human clients 408. Direct Interaction User interacts with computer by manipulating objects on screen Can be intuitive and easy to learn Users get immediate feedback Not suitable for complex interactions Does not require typing skills Straightforward for casual users, slow for skilled users Icons can be language-independent Difficult to build scripts Only suitable for human users 409. Design for Direct Manipulation Conceptual models, metaphors, icons => there may not be an intuitive model Navigation around large space Conventions are growing over the years => scroll bars, buttons, help systems, sliders => good for users, good for designers 410. Menus Easy for users to learn and use Certain categories of error are avoided Enables context-sensitive help Major difficulty is structure of large menus Scrolling menus (e.g., states of USA) Hierarchical Associated control panels Menus plus command lineUsers prefer broad and shallow to deep menu systems 411. Information Presentation Information to be displayed Presentation softwareDisplay 412. Information Presentation Text precise, unambiguous fast to compute and transmit Graphics simple to comprehend uses of color shows variations 413. Help System Design Help system design is difficult! Must prototype with mixed users Categories of help: => Overview and general information => Specific or context information => Tutorials (general) => Cook books and wizards => Emergency ("I am in trouble ...") Must have many routes to same information Never blame the user! 414. System Considerations of User Interfaces Personal computer cycles are there to be used Any network transfer involves delay Shared systems have unpredictable performance Data validation often requires access to shared data Mobile code poses security risks 415. Usability and Cost Performance may be expensive in hardware or special software development User interface development may be a major part of a software development project Costs are multiplied if a user interface has to be used on different computers or migrate to different versions of systems Web browsers provide a general purpose user interface that others maintain 416. Extensibility in Web Browsers Data types: helper applications, plug-ins Protocols HTTP, WAIS, Gopher, FTP, etc. proxies Executable code CGI scripts at server JavaScript at client Java applets Style sheets 417. Web User Interface: BasicWeb browserWeb servers Static pages from server All interaction requires communication with server 418. Web User Interface: CGI Script User interface tables CGI Scripts Web browserWeb servers Scripts can configure pages Scripts can validate information All interaction requires communication with server 419. Web User Interface: JavaScripthtml Java ScriptUser interface tables CGI ScriptsWeb browserWeb servers JavaScripts can validate information as typed Some interactions are local Server interaction constrained by web protocols 420. Web User Interface: AppletAny server Applets Web browserWeb servers Any executable code can run on client Client can connect to any server 421. Device-Aware User Interfaces Examples of devices: desk-top computer, fast network connection laptop computer, intermittent connectivity PalmPilot, intermittent use of cradle Smart telephone Digital camera, camcorder Device-aware user interfaces are aware of: => Performance of device => Connectivity 422. The Importance of Design Good support for users is more than a cosmetic flourish Elegant design, appropriate functionality, & responsive system: => a measurable difference to their effectiveness A system that is hard to use: => users may fail to find important results, or mis-interpret what they do find => user may give up in disgust A computer system is only as good as the interface it provides to its users 423. Software EngineeringLecture 19 Performance of Computer Systems 424. Moore's Law Original version: The density of transistors in an integrated circuit will double every year. (Gordon Moore, Intel, 1965) Current version: Cost/performance of silicon chips doubles every 18 months. 425. Moore's Law and System DesignDesign system: Production use: Withdrawn from production:2000 2003 2013Processor speeds: Memory sizes: Disk capacity:1 1 11.9 1.9 2.228 28 51System cost:10.40.01 426. Moore's Law: Rules of Thumb Planning assumptions: Every year: cost/performance of silicon chips improves 25% cost/performance of magnetic media improves 30% 10 years = 100:1 20 years = 10,000:1 427. Parkinson's LawOriginal: Work expands to fill the time available. (C. Northcote Parkinson) Planning assumptions: (a) Demand will expand to use all the hardware available. (b) Low prices will create new demands. (c) Your software will be used on equipment that you have not envisioned. 428. False AssumptionsUnix file system will never exceed 2 Gbytes (232 bytes). AppleTalk networks will never have more than 256 hosts (28 bits). GPS software will not last 1024 weeks. Nobody at Dartmouth will ever earn more than $10,000 per month. etc., etc., ..... 429. Moore's Law and the Long Term What level? Within your working life?19652000?When? 430. Predicting System Performance Mathematical models Simulation Direct measurement All require detailed understanding of the interaction between software and systems. 431. Queuesarrivewait in lineserviceSingle server queuedepart 432. Queues service arrivewait in lineMulti-server queuedepart 433. Mathematical Models Queueing theory Good estimates of congestion can be made for singleserver queues with: arrivals that are independent, random events (Poisson process) service times that follow families of distributions (e.g., negative exponential, gamma) Many of the results can be extended to multi-server queues. 434. Utilization: Rule of Thumbmean service time utilization = mean inter-arrival time When the utilization of any system component exceeds 30%, be prepared for congestion. 435. Behavior of Queues: Utilizationmean delay01utilization 436. Simulation Model the system as set of states and events advance simulated time determine which events occurred update state and event list repeat Discrete time simulation: Time is advanced in fixed steps (e.g., 1 millisecond) Next event simulation: Time is advanced to next event Events can be simulated by random variables (e.g., arrival of next customer, completion of disk latency) 437. TimescaleOperations per second CPU instruction:400,000,000Disk latency: read:60 25,000,000 bytesNetwork LAN: dial-up modem:10,000,000 bytes 6,000 bytes 438. Measurements on Operational Systems Benchmarks: Run system on standard problem sets, sample inputs, or a simulated load on the system. Instrumentation: Clock specific events. 439. Serial and Parallel ProcessingSingle thread v. multi-thread e.g., Unix fork Granularity of locks on data e.g., record locking Network congestion e.g., back-off algorithms 440. Example: Performance of Disk Array Each transaction must: wait for specific disk platter wait for I/O channel signal to move heads on disk platter wait for I/O channel pause for disk rotation read data Close agreement between: results from queueing theory, simulation, and direct measurement (within 15%). 441. The Software Process Requirements Definition System and Software design Programming and Unit Testing Integration and System Testing Operation and Maintenance 442. Software EngineeringLecture 20 Coding Standards Tools for Debugging 1 443. Coding Standards Or How to Pound all of your odd-shaped programmers into a one size fits all hole 444. I think there may be a bug in Joes Code - Please Fix func GreenEggsNHam(Not SamIAm, Green EggsNHam) foreach Green TryThem in SamIAm do EatThem(TryThem) = false NotInACarNotOnABus(EggsNHam) func NotInACarNotOnABus(Green EggsNHam) EatThem(EggsNHam) = true NotOnAPlane(EggsNHam) foreach NotLikeThem SamIAm of EggsNHam do if not EatThem(SamIAm) then NotInACarNotOnABus(SamIAm) IDoNotLikeThem(EggsNHam) 445. Joes Code Following a Sane Coding Standard . . . func DepthFirstSearch(graph G, vertex v) foreach vertex w in G do Encountered(w) = false RecursiveDFS(v) func RecursiveDFS(vertex v) Encountered(v) = true PreVisit(v) foreach neighbor w of v do if not Encountered(w) then RecursiveDFS(w) PostVisit(v) 446. What are Coding Standards Coding standards are guidelines for code style and documentation. The dream is that any developer familiar with the guidelines can work on any code that followed them. Standards range from a simple series of statements to involved documents. 447. Areas Typically Covered Program Design Naming Conventions Formatting Conventions Do