1. Scope and RegulationsŸрограмма... · Course «Databases» Syllabus [ 2014-2015 ac. year...

Course «Databases» Syllabus [ 2014-2015 ac. year ] --- page 2

Course Information

Author of the Program: Assoc. Prof., Ph.D. Alexander D. Breyman

Course Title in English: Databases

1. Scope and Regulations

The syllabus is prepared for teachers responsible for the course (or closely related disciplines), teaching assistants, students enrolled on the course as well as experts and statutory bodies carrying out assigned or regular accreditations in accordance with

• educational standards of the State educational budget institution of the Higher Professional Education "State University – The Higher School of Economics" (HSE) categorized as "National Research University",

• B.Sc. curriculum ("Software Engineering", area code 09.03.04) 3rd year, 2014-2015 academic year;

• B.Sc. curriculum ("Software Engineering", area code 09.03.04) 4th year, 2014-2015 academic year;

• Federal state educational standard of higher education in software engineering (Bachelor of Science degree) approved by the Ministry of Education and Science of RF (Russian Federation) Directive N544 on November 9th, 2009 (in Russian).

The course is offered to students of the Bachelor Program «Software Engineering» (code 09.03.04) in the Department of Software Engineering, Faculty of Computer Science of the National Research University - Higher School of Economics (HSE).

This mandatory course belongs to professional cycle curricula unit (Б.3 unit/ Base module [Professional

cycle disciplines 1.3] of 2014-2015 academic year's working syllabus) covered by the list of training courses of bachelor's program (3rd and 4th year of studies).

It is a four module course, which is delivered in modules #3 and #4 of the third academic year and in modules #1 and #2 of the fourth academic year. Number of credits is 8 (4+4). Total course length is 288 academic hours. 3rd year part of course length is 144 academic hours including 60 auditory hours (30 Lecture (L) hours and 30 Practice (P) hours) and 84 Self-study (SS) hours. 4th year part of course length is 144 academic hours including 60 auditory hours (30 Lecture (L) hours and 30 Practice (P) hours) and 84 Self-study (SS) hours. Academic control forms are two home assignments (one for each year; in form of group project), two written tests (one for each year), and two written exams: one after module #4 of 3rd course and one after module #2 of 4th course.

2. Course Objectives

The student should develop skills and understanding in:

the design methodology for databases and verifying their structural correctness;

implementing databases and applications software in the relational model as well as in map/reduce paradigm;

using querying languages, primarily SQL, and other database supporting software;

applying the theory behind various database models and query languages;

implementing security and integrity policies relating to databases;

working in group settings to design and implement database projects. The objective of first part (being taught on 3rd year) of the course is to expose to students topics of (mostly) online transactional processing and relational database theory, database design and implementation, including entity-relationship data modeling, relational model, algebra and calculus, functional dependencies and normalization theory, relational query languages, including SQL. The objective of second part (being taught on 4th year) of the course is to form professional competencies


related to design and implementation of other kinds of databases, including data warehouses, online analytical data processing and big data management tools. Students will get a grasp on strengths and weaknesses of wide spectrum of approaches to data storage, search and retrieval, resulting in informed choice of database model.

This course studies different conceptual database models and their properties. The models that will be discussed are:

Relational model for online transactional processing;

Relational data warehouse;

Multidimensional data warehouse;

Online analytical processing;

Map/reduce massive parallel data processing;

Key/value, document and graph database models;

Semistructured data and XML;

Data stream processing.

For these conceptual models the course will concentrate on the following points: Why was the database model introduced? Which of the shortcomings of other models does it address? What are the most important concepts and notions for the database model? How is the model implemented? Which are the main techniques? The importance of understanding the internals of a particular database model cannot be overemphasized as it is closely connected to its limitations.

3. Learning outcomes

After taking this course the student should have achieved the following objectives:

Knows the data modelling concepts and has an understanding of relational data model. Can compose queries to relational databases using relational algebra, tuple relational calculus and SQL. Knows methods of database design, including entity-relationship approach and normalization-based approach.

Knows and is able to apply database application design and development methods usable for object-oriented program systems, including object-relational mappers, its advantages and disadvantages.

Knows models and methods of internal organization of relational databases including file storage, indexing, query processing and transaction management issues.

Knows the reference architectures of data warehouses and is aware of the basic functionality offered by available commercial and free data warehousing systems. Master methods and tools for creating analytical database solutions. Can choose dimensions for multidimensional database, group them into hierarchies and define aggregates. Knows principles of data integration and extract-transform-load process design and implementation.

Knows the map/reduce approach and its Hadoop implementation. Can create map/reduce-based solutions to large-scale massive data processing problems.

Knows database models and tools of NoSQL class, including key-value stores, document databases and graph databases. Can implement systems and database applications using these tools.

Knows XML document model, DTD and XML Schema. Can write and understand XPath and XQuery expressions.

Knows data stream management systems, its data models and concepts. Can compose queries for executing in these systems.

Students should be able to understand the language of studies models, choose and use appropriate models and programming languages, implement systems using chosen models, methods and tools.


The course contributes to the development of the following competencies:

Research activities.

The ability to formalize in its subject area within the constraints of methods of research (ПК-2).

Preparedness to use research techniques and tools to study objects of professional activity (ПК-3).

Preparedness to justify design decisions taken and perform experiments to verify their accuracy and efficiency;

Analytical activities.

The ability to formalize the subject area of a software project and to develop specifications the software product components (ПК-6).

Project activities.

The ability to apply the fundamentals of computer science and programming to design, construction and testing of software products (ПК-10).

Skills in modeling, analysis and use of formal methods of software design (ПК-12).

The ability to create program interfaces (ПК-14).

Technological activities.

Skills in using the operating systems, network technologies, software interface development tools, languages and methods of formal specifications, database management systems (ПК-15).

Skills in using different software development technologies (ПК-16).

Production activities.

The ability to apply core software development methods and tools (ПК-17);

4. The Course Position in The Curriculum

This mandatory course belongs to professional cycle curricula unit (Б.3 unit/ Base module [Professional

cycle disciplines 1.3] of 2014-2015 academic year's working syllabus) covered by the list of training courses of bachelor's program (3rd and 4th year of studies).

Prerequisite courses: Computer science (Informatics), Mathematical Logic and Theory of Algorithms, Discrete Mathematics, Programming, Software Construction, Algorithms Design and Analysis. The course is based on the knowledge of foundations of discrete mathematics (including logic and set theory), computer science, and computer programming. Students are assumed to understand basic data structures and their uses (lists, arrays), concepts of object-oriented programming (classes, inheritance), modular software design and method linkage, the design, implementation and testing of medium-sized problem solutions, to be familiar with some contexts in which databases are used.

Courses that are based on this course: Information Systems, Program projects management, Group project on Software Engineering, Joint year projects, Final projects.

5. Course Contents (Main Topics)

№ Title of the topic Total Hours Classroom hours

Self-study Lectures Practice

Module #3, 3rd year

1 Introduction. 6 2 2 2

2 Data modeling. 6 2 2 2

3 Relational Model. 6 2 2 2

4 Relational Query Languages. 8 2 2 4


5 SQL. 22 4 2 14

6 Database Design: The E/R and UML

Approaches. 10 2 2 8

7 Relational Database Design. 14 2 2 10

Module #3 totals 72 16 14 42

Module #4, 3rd year 8 Application Design and Development. 20 4 4 12

9 Storage and File Structure. 8 2 2 4

10 Indexing and Hashing. 8 2 2 4

11 Query Processing. 12 2 2 8

12 Transaction Management. 12 2 4 6

13 Distributed and Parallel Databases. 12 2 2 8


Module #1, 4th year

14 Data Warehousing and Big Data

Management. 10 2 2 6

15 Data Warehouse Architectures and

Models. 22 6 6 10

16 Data Cleaning and Integration. 14 2 2 10

17 Map/Reduce and Hadoop. 26 6 4 16


Module #2, 4th year

18 NoSQL Databases. 28 6 8 14

19 Semistructured data and XML. 22 4 4 14

20 Data Stream Management Systems. 22 4 4 14


Total: 288 60 60 168

6. Education control forms / Assessment

6.1 Assessment criteria Attendance record provides data for calculating the ratio of the number of lectures, attended by student, to overall number of lectures and practices. Value of attendance components of final grade formula for two years (A1, A2) is that ratio multiplied by 10 and rounded to the nearest integer value.

Practice activity during practice hours is assessed by evaluating of student involvement into discussions as well as quality of exercise performance during seminars. Value of practice activity components of final grade formula for two years (PA1, PA2) is an integer value from interval [0,10].

Home assignment 1 has to be prepared in modules 3 and 4 of 3rd year by students in groups of 3 to 5 and includes design, implementation and testing of a database and database application for given subject area (chosen by group and approved by instructor or assigned by instructor). Results of group project 1 should be presented in form of report that consists of design document, implementation description, results of testing. Mandatory appendixes are source code for application and database creation script. Report should be submitted to LMS not later that for 7 calendar days before assigned date of its presentation (announced on first lecture). Project should be presented and demonstrated by all group members. Each group member should demonstrate complete understanding of all project details and give correct answers to at least two questions of instructor. Value of group project 1 component of final grade formula (H1) is an integer value from interval [0,10] consists of the common score for the report and presentation (from 0 to 5; same score to all group members) and individual student score for the answers to the questions (from 0 to 5). If a student misses the project presentation because of some valid reason, s/he receives «absence» grade. If a student misses the project presentation because of any other reason, s/he receives grade based on individual score set to 0.

Home assignment 2 has to be prepared in modules 1 and 2 of 4th year by students in groups of 3 to 5. Students should identify a Big Data problem that they would like to work on and specify a difficult question to data and identify data sources that might help answering the question. A data warehouse


including OLAP options and a Hadoop (or NoSQL) solution both aimed at answering the question should be built. Built solutions should be compared, their relative strengths and weaknesses are revealed and described. Chosen Big Data problem should be approved by instructor. Results of group project 2 should be presented in form of report that consists of design document, implementation description, results of testing and comparing of solutions. Mandatory appendixes are source code for both applications and data managements scripts. Report should be submitted to LMS not later that for 7 calendar days before assigned date of its presentation (announced on first lecture of second year). Project should be presented and demonstrated by all group members. Each group member should demonstrate complete understanding of all project details and give correct answers to at least two questions of instructor. Value of group project 2 component of final grade formula (H2) is an integer value from interval [0,10] consists of the common score for the report and presentation (from 0 to 5; same score to all group members) and individual student score for the answers to the questions (from 0 to 5). If a student misses the project presentation because of some valid reason, s/he receives «absence» grade. If a student misses the project presentation because of any other reason, s/he receives grade based on individual score set to 0.

Written test at the end of the first module (module 3 of 3rd year of study) implies arrangement of the written test (in lecture room) for all students enrolled to the course. Topics covered by the test embraces all course material of first module. If a student misses the written test because of some valid reason, s/he receives «absence» grade. The written test 1 (T1) is assessed on usual ten-point scale.

Written test at the end of the third module (module 1 of 4rd year of study) implies arrangement of the written test (in lecture room) for all students enrolled to the course. Topics covered by the test embraces all course material for third module. If a student misses the exam because of some valid reason, s/he receives «absence» grade. The written test 2 (T2) is assessed on usual ten-point scale.

Written exam at the end of the second module (module 4 of 3rd year of study) implies arrangement of the written test (in lecture room) for all students enrolled to the course. Topics covered by the test embraces all course material of first two modules. If a student misses the written test because of some valid reason, s/he receives «absence» grade. The written exam 1 (E1) is assessed on usual ten-point scale.

Written exam at the end of the fourth module (module 2 of 4rd year of study) implies arrangement of the written test (in lecture room) followed by oral conversation for all students enrolled to the course. Topics covered by the test embraces all course material. If a student misses the exam because of some valid reason, s/he receives «absence» grade. The written exam 2 (E2) is assessed on usual ten-point scale.

6.2 Grade calculation rules The overall and accumulated course grades for first part (3rd year of study) Go1 and Ga1 (10-point scale each) are calculated as follows:

Ga1 = 0.15A1 + 0.15PA1 + 0.3H1 + 0.4T1;

Go1 = 0.5Ga1 + 0.5E1.

The overall and accumulated course grades for second part (4th year of study) Go2 and Ga2 (10-point scale each) and final grade (FG) are calculated as follows:

Ga2 = 0.15A2 + 0.15PA2 + 0.3H2 + 0.4T2;

Go2 = 0.5Ga2 + 0.5E2.

FG = 0.5Go1 + 0.5Go2.

The overall and accumulated course grades Go1, Go2, Ga1 and Ga2 (10-point scale each) include results achieved by students in their attendance records A1 and A2, practice activities PA1 and PA2, home assignments H1 and H2, written tests T1 and T2 and exams E1 and E2; it is rounded up to an integer number of points. The rounding procedure accounts for students' practice activities during seminars. Intermediate assessment retakes are not allowed.

Conversion of the concluding rounded grade (FG) to five-point scale grade is done in accordance with the following table:


Summary Table : Correspondence of ten-point (10) to five-point (5) system's marks

Ten-point scale [10] Five-point scale [5]

1 - unsatisfactory

2 - very bad

3 - bad

Unsatisfactory (UnS) - 2

4 - satisfactory 5 - quite satisfactory

Satisfactory (S) - 3

6 - good 7 - very good

Good (G) - 4

8 - nearly excellent 9 - excellent 10 - brilliantly

Excellent (E) - 5

7. Detailed Course Contents

Тopic 1: Introduction. ♦ Outline:

Course overview and logistics. History of data management approaches. Basic database system concepts. Database environment. Database users. Database development process. Database planning.

♦ Core books (sources of information)

Silberschatz A., Korth H.F., Sudarshan S. Database System Concepts, 6th ed, McGraw-Hill, 2010. — 1376pp. [Ch.1]

Garcia-Molina H., Ullman J., Widom J. Database Systems: The Complete Book, 2nd Edition, Prentice Hall, 2009. — 1248pp. [Ch.1]

Elmasri R., Navathe S.B. Fundamentals of Database Systems, 6th ed., Addison Wesley, 2010. — 1200 pp.

Тopic 2: Data Modeling. ♦ Outline:

Three level database architecture. Data model. Data independence. Inverted files. Early data models: hierarchical and network. Basic relational model concepts. Object-oriented model. Object-relational model. Semi-structured model. Semantic data models.




Garcia-Molina H., Ullman J., Widom J. Database Systems: The Complete Book, 2nd Edition, Prentice Hall, 2009. — 1248pp. [Ch.1, 2.1]


Тopic 3: Relational Model. ♦ Outline:

History of relational model. Advantages of relational model. Basic relational data structures. Mathematical and database relations. Relation schema. Relational database. Integrity constraints. Relation keys. Key constraint. Foreign key constraint.



Garcia-Molina H., Ullman J., Widom J. Database Systems: The Complete Book, 2nd Edition, Prentice Hall, 2009. — 1248pp. [Ch.2.2]


Тopic 4: Relational Query Languages. ♦ Outline:

Relational algebra. Relational algebra operations. Selection. Projection. Set operations. Renaming. Join, equijoin, antijoin, theta-join. Natural join. Division. Tuple relational calculus: atoms, formulas, queries. Domain relational calculus.






Тopic 5: Structured Query Language. ♦ Outline:

SQL language history. SQL standards. Data definition and data manipulation (sub)languages. SQL data types. Table declaration. Primary keys, unique constraints, default values, nullable

attributes. Check constraints. Foreign key constraints. Handling foreign key violations. Indexes. Database schema modifications. SQL query sublanguage: SELECT. Single-table queries. Filtering conditions. Logical operations IN, ALL, EXISTS. Join queries. Join types: cross, natural, inner, outer, self. Duplicates elimination. Set operations. Nested queries. Correlated nested queries. Aggregate functions. Grouping and group filtering. Query result sorting. INSERT. UPDATE . DELETE. Views: creation, use and updating. Triggers: creation, activation, execution. Multiple triggers. View materialization. SQL procedural extensions and dialects: T-SQL, PL/SQL, PgSQL. Stored procedures.


Silberschatz A., Korth H.F., Sudarshan S. Database System Concepts, 6th ed, McGraw-Hill, 2010. — 1376pp. [Ch.3,4,5]

Garcia-Molina H., Ullman J., Widom J. Database Systems: The Complete Book, 2nd Edition, Prentice Hall, 2009. — 1248pp. [Ch.6,7,8]


Тopic 6: Database Design: E/R and UML Approaches. ♦ Outline:

Entity-relationship model. Entities and attributes. Entity types. Keys. E/R diagram.


Relationships. Attributes and roles. Relationship type, degree and cardinality. Relationship participation constraints. Strong and weak entities. Entity type hierarchies. Specialization and generalization. Total and partial unions. Unified modeling language class diagram. Association and aggregation in UML. Generalization hierarchies in UML. Multiplicity indicators in UML.





Тopic 7: Relational Database Design. ♦ Outline:

Objectives of normalization. Limitations of E/R design. Redundancy. Anomalies: insertion, deletion, update. Decomposition. Informal guidelines for relation design. Functional dependencies. Axioms of functional dependencies. Closure. Minimal cover of a set of dependencies. Desirable properties of decompositions: attributes preservation, dependency

preservation, lossless join. First normal form. Full functional dependencies. Second normal form. Transitive dependency. Third normal form Boyce/Codd normal form (BCNF). Multivalued dependencies. Fourth normal form. Fifth normal form. Domain/Key normal form (DKNF). BCNF decomposition algorithm and its properties. Normalization drawbacks. Denormalization.






Тopic 8: Application Design and Development. ♦ Outline:

Database access from programming languages. ODBC architecture. ODBC Drivers. ODBC connection strings. JDBC architecture. Connecting to DBMS. Preparing and executing queries. Using result sets and cursors. Handling exceptions. Transactions in JDBC. Object-relational mapping. Design patterns for data persistence. Active Record pattern. Data Mapper pattern. Hybernate.


Silberschatz A., Korth H.F., Sudarshan S. Database System Concepts, 6th ed, McGraw-Hill,

2010. — 1376pp. [Ch.9]



Тopic 9: Storage and File Structure. ♦ Outline:

Memory hierarchy. Disk storage: physical disk structure, pages and blocks I/O time: seek latency, transfer time. Disk cache. RAID. SSD. SAN and NAS. Datafiles: blocks and extents. Block structure. Fixed and variable record formats. Large objects (LOBs).






Тopic 10: Indexing and Hashing. ♦ Outline:

Indexing concepts. B-tree index. B-tree insertions and deletions. Hash index. Hash functions. Extendible hashing. Bitmap index. Join index. GiST.





Тopic 11: Query processing. ♦ Outline:

Query processing overview. Relational algebra translation. Query tree. Relational algebra equivalences. Heuristics for optimization. Cost-based optimization. Cost factors and estimation. External merge sort. Duplicate elimination. Implementing set operations. Sort-based and hash-based projection. Computing selection without indexes. Computing selection with clustered index. Computing selection with b-tree index. Computing selection with hash index. Computing joins: nested loops. Computing joins: block nested loops. Computing joins: sort-merge join. Computing joins: hash-join.



Silberschatz A., Korth H.F., Sudarshan S. Database System Concepts, 6th ed, McGraw-Hill, 2010. — 1376pp. [Ch.12,13]

Garcia-Molina H., Ullman J., Widom J. Database Systems: The Complete Book, 2nd Edition, Prentice Hall, 2009. — 1248pp. [Ch.15,16]


Тopic 12:Transaction management. ♦ Outline:

Single-user and multi-user systems. Basic concepts of transactions. ACID properties. Isolation. Serial and interleaved execution. Schedules: serial, serializable. Methods to ensure serializability. Concurrency control. Optimistic and pessimistic concurrency. Two-phase locking. Locking and deadlocks. Implementing isolation levels with locks. Snapshot isolation. Timestamping. Atomicity and Durability. Write-ahead log. Redo and undo records. Recovery from crash. Checkpoints. Distributed transactions. Two-phase commit protocol. Replication.



Garcia-Molina H., Ullman J., Widom J. Database Systems: The Complete Book, 2nd Edition, Prentice Hall, 2009. — 1248pp. [Ch.17,18,19]


Тopic 13: Distributed and parallel databases. ♦ Outline:

Distributed database systems: advantages and drawbacks. Distributed database architectures. Software components and functions of distributed DBMS. Data placement. Transaction management for distributed DBMS. Locking protocols.


Timestamping protocols. Commit protocols. Distributed recovery from failures. Distributed query processing. Data integration. Parallel database systems.





Тopic 14: Data Warehousing and Big Data Management. ♦ Outline:

Role and purpose of a data warehouse. Components of a data warehouse. Data warehousing and OLAP. Semistructured data and XML. Big data concepts and notions.



Franks B. Taming The Big Data Tidal Wave: Finding Opportunities in Huge Data Streams with Advanced Analytics (Wiley and SAS Business Series), Wiley, 2012. — 336pp.

Тopic 15: Data Warehouse Architecture and Models. ♦ Outline:

Requirements to data management from decision support systems. Historical, summarized, integrated data. Statistical and analytical queries. Business intelligence applications. Three-tier architecture. Data warehouse, data mart. Online analytical processing (OLAP). Conceptual models for decision support. Multidimensional view on the data. Cross-tabulation. Data cubes. Operations with data cubes: roll-up, drill-down, pivot, slice & dice, select. Attribute hierarchies. Types of hierarchies. Query languages for supporting OLAP. SQL extensions: Group by cube, group by rollup. Analytic functions. Rank functions, window functions, lag/lead functions. Multidimensional expressions (MDX).


Relational OLAP (ROLAP): Star schema, snowflake schema, snowflake constellation.

Multi-dimensional OLAP (MOLAP): multicubes and hypercubes, sparse and dense dimensions.

Indexing of dimensions: b-tree, bitmap, join indexes. Hybrid OLAP (HOLAP). Slowly changing dimensions. Temporal databases. Valid time, transaction time. Bitemporal data model.


Silberschatz A., Korth H.F., Sudarshan S. Database System Concepts, 6th ed, McGraw-Hill, 2010. — 1376pp. [Ch. 20]

Golfarelli M., Rizzi S. Data Warehouse Design: Modern Principles and Methodologies, McGraw-Hill Osborne Media, 2009. — 480pp.

Celko J. Joe Celko's Analytics and OLAP in SQL, Morgan Kaufmann, 2006. — 208pp.

Smith B.C., Clay C.R. Microsoft SQL Server 2008 MDX Step by Step, Microsoft Press, 2009. — 400pp.

Chaudhury S., Dayal U. An overview of data warehousing and OLAP technology. // SIGMOD Record, v.26 n.2, pp.507-508, 1997.

Harinarayan V., Rajaraman A., and Ullman J. Implementing Data Cubes Efficiently. // In Proceedings of the 1996 ACM SIGMOD international conference on Management of data (SIGMOD '96), pp. 205-216, 1996.

Jensen C., Pedersen T., Thomsen C. Multidimensional Databases and Data Warehousing, Morgan & Claypool Publishers, 2010. — 111p.

Garcia-Molina H., Ullman J., Widom J. Database Systems: The Complete Book, 2nd Edition, Prentice Hall, 2009. — 1248pp. [Ch.10.6,10.7]

Rainardi V. Building a Data Warehouse: With Examples in SQL Server. Apress, 2008. — 541pp.

Inmon W. H., Krishnan K. Building the Unstructured Data Warehouse: Architecture, Analysis, and Design, Technics Publications, 2011. — 216pp.

Kimball R., Ross M. The Data Warehouse Toolkit, Wiley, 2002. — 447pp.

Тopic 16: Data Cleaning and Integration. ♦ Outline:

Data integration issues and stages. ETL process design and implementation. Error handling in ETL process. Metadata and ontologies in data integration. Data structures for ETL. Extracting data from external sources. Data profiling. Data cleaning. Data quality dimensions, issues and constraints. Typical cleaning checks. Conforming data. Data enrichment.


Entity resolution. Data transformation. Loading data into warehouse. Bulk load.



Kimball R., Caserta J. The Data Warehouse ETL Toolkit, Wiley, 2004. — 491pp.


Rodrigues F., Coles M., Dye D. Pro SQL Server 2012 Integration Services, Apress, 2012. — 636 pp.

Тopic 17: Map/Reduce and Hadoop. ♦ Outline:

Large-scale massive data processing. Map/reduce paradigm. Hadoop map/reduce implementation. Hadoop distributed filesystem HDFS. Hadoop I/O. Developing hadoop applications. Managing job configuration. Mapper, Reducer, Combiner. Running job locally and on a cluster. Map/reduce design patterns. Pig. Pig Latin. Hive. HiveQL. HBase BigTable implementation.


White T. Hadoop: The Definitive Guide, 3rd edition, O’Reilly, 2012. — 657pp.

Holmes A. Hadoop In Practice, Manning Publications, 2012. — 511pp.

Miner D., Shook A. MapReduce Design Patterns, O’Reilly, 2012. — 232pp.

Gates A. Programming Pig, O’Reilly, 2011. — 203pp.

Тopic 18: NoSQL Databases. ♦ Outline:

Hashing for distributed data storage. Consistent hashing. Dynamic hash tables (DHT). Vector clocks and conflict detection. Gossip protocol and hinted handoff. Merkle trees. Key/value store examples: Project Voldemort, Membase, Riak, Cassandra. Extended key/value (data structures) store: Redis. Document databases: MongoDB. Graph databases: Neo4j.



Tiwari S. Professional NoSQL, Wrox, 2011. — 384pp.

Redmond E. Seven Databases in Seven Weeks: A Guide to Modern Databases and the NoSQL Movement, Pragmatic Bookshelf, 2012. — 352pp.

Robinson I. Graph Databases, O’Reilly, 2013. — 224pp.

Тopic 19: Semistructured Data and XML. ♦ Outline:

Semistructured data for human and machine consumption. XML: tags, elements, attributes, values. Well-formed and valid documents. Namespaces. XPath: axes, node-tests, predicates. XQuery: expressions, functions. FLWOR expressions. XQuery data model. DTD and XML Schema. Simple and complex types of XML Schema.



Melton J., Buxton S. Querying XML: XQuery, XPath, and SQL/XML in context, Morgann Kaufmann, 2006. 848pp.

Ben-Gan I. et al. Inside Microsoft SQL Server 2008: T-SQL Programming, Microsoft Press, 2009. — 832 pp.

Garcia-Molina H., Ullman J., Widom J. Database Systems: The Complete Book, 2nd Edition, Prentice Hall, 2009. — 1248pp.


Тopic 20: Data Stream Management Systems. ♦ Outline:

Data stream example applications. Data stream systems: STREAM, Aurora. Data stream models: relational and XML. Window-based processing. STREAM CQL. Query semantics. Mapping Streams to Relations and vice versa. Aurora SQuAL. Operators. Query processing. STREAM: architecture, query plan, optimizations. Aurora: architecture, optimizations. Distributed stream processing: Borealis.


Golab L., Oszu M. T. Data Stream Management (Synthesis Lectures on Data Management), Morgan & Claypool Publishers, 2010. — 80 pp.

Arasu A., Babu S., Widom J. The CQL Continuous Query Language: Semantic


Foundations and Query Execution, VLDB Journal, 15(2), 2006. pp.121-142.

Arasu A., Babcock B., Babu S., Cieslewicz J., Datar M., Ito K., Motwani R., Srivastava U., Widom J. STREAM: The Stanford Data Stream Management System. // M. Garofalakis, J. Gehrke, and R. Rastogi, editors, Data Stream Management: Processing High-Speed Data Streams, Springer, 2009.

Cherniack M., Zdonik S. Stream-Oriented Query Languages and Operators, Encyclopedia of Database Systems, Springer, 2009. pp 2848-2854.

8. Educational Methods and Technologies Course studies are organized in the form of lectures and practical studies. Besides traditional forms, some active and interactive forms are provided: discussion of real industry case studies; proposing and discussing group projects topics and its planned outcomes, using interactive simulators for database languages.

9 Assessment tools for students current evaluation and attestation

9.1 Topics for assignments The course includes two group projects (one in each year of study), compulsory to all students. Students will work in groups of 3 to 5 students on one of the suggested topics. First project: Build an relational database application using the techniques studied in the course. Second project: Student should identify a Big Data problem that s/he would like to work on and specify the following:

Ask a difficult question. Identify data sources that you believe might help answering the question. Build a data warehouse aimed at answering the question. Build OLAP solution aimed at answering the question. Build Hadoop (or NoSQL) solution aimed at answering the question. Compare built solutions, describe relative strengths and weaknesses.

9.2 Topics for course final quality assessment Basic database system concepts.

Database environment.

Database planning.

Three level database architecture.

Basic relational model concepts.

Object-oriented model.

Object-relational model.

Semi-structured model.

Mathematical and database relations. Relation schema. Relational database.

Integrity constraints.

Relation keys. Key constraint. Foreign key constraint.

Relational algebra operations. Selection. Projection.

Relational algebra operations. Set operations.

Relational algebra operations. Join, equijoin, antijoin, theta-join. Natural join.

Relational algebra operations. Division.

Tuple relational calculus: atoms, formulas, queries.

SQL data types.

SQL. Table declaration. Primary keys, unique constraints, default values, nullable attributes.


SQL. Check constraints.

SQL. Foreign key constraints. Handling foreign key violations.

SQL. Database schema modifications.

SQL. Single-table queries. Filtering conditions. Logical operations IN, ALL, EXISTS.

SQL. Join queries. Join types: cross, natural, inner, outer, self.

SQL. Duplicates elimination.

SQL. Set operations.

SQL. Nested queries. Correlated nested queries.

SQL. Aggregate functions.

SQL. Grouping and group filtering.

SQL. Query result sorting.

SQL. INSERT.

SQL. UPDATE .

SQL. DELETE.

SQL. Views: creation, use and updating.

SQL. Triggers: creation, activation, execution. Multiple triggers.

SQL. View materialization.

Stored procedures.

Entity-relationship model. Entities and attributes. Entity types. Keys.

Entity-relationship model. Relationships. Attributes and roles. Relationship type, degree and cardinality. Relationship participation constraints.

Entity type hierarchies. Specialization and generalization. Total and partial unions.

Unified modeling language class diagram. Association and aggregation in UML. Generalization hierarchies in UML. Multiplicity indicators in UML.

Objectives of normalization. Limitations of E/R design. Data redundancy.

Anomalies: insertion, deletion, update.

Functional dependencies. Axioms of functional dependencies.

Closure.Minimal cover of a set of dependencies.

Desirable properties of decompositions: attributes preservation, dependency preservation, lossless join.

First normal form.

Full functional dependencies. Second normal form.

Transitive dependency. Third normal form. Boyce/Codd normal form (BCNF).

Multivalued dependencies. Fourth normal form.

Fifth normal form.

Domain/Key normal form (DKNF).

BCNF decomposition algorithm and its properties.

Normalization drawbacks.

JDBC architecture, connecting to DBMS, preparing and executing queries, using result sets and cursors.

Handling exceptions in JDBC.

Transactions in JDBC.

Object-relational mapping.

Design patterns for data persistence. Active Record pattern.

Design patterns for data persistence. Data Mapper pattern.


Hibernate.

Datafiles: blocks and extents. Block structure. Fixed and variable record formats. Large objects (LOBs).

B-tree index. B-tree insertions and deletions.

Hash index. Hash functions.Extendible hashing.

Bitmap index.

Join index.

GiST.

Relational algebra translation. Query tree.

Relational algebra equivalences.

Cost-based optimization. Cost factors and estimation.

External merge sort.

Duplicate elimination.

Implementing set operations.

Sort-based and hash-based projection.

Computing selection without indexes.

Computing selection with clustered index.

Computing selection with b-tree index.

Computing selection with hash index.

Computing joins: nested loops.

Computing joins: block nested loops.

Computing joins: sort-merge join.

Computing joins: hash-join.

Basic concepts of transactions. ACID properties.

Schedules: serial, serializable. Methods to ensure serializability.

Optimistic and pessimistic concurrency.

Two-phase locking. Locking and deadlocks. Implementing isolation levels with locks.

Snapshot isolation.

Write-ahead log. Redo and undo records. Recovery from crash.

Distributed transactions. Two-phase commit protocol.

Distributed database architectures.

Software components and functions of distributed DBMS.

Transaction management for distributed DBMS.

Distributed recovery from failures.

Distributed query processing.

Data integration.

Parallel database systems.

Object-relational mapper.

Requirements to data management from decision support systems.

Extract-transform-load process.

Conceptual models for decision support.

Multidimensional view on the data.

Operations with data cubes: roll-up, drill-down, pivot, slice & dice, select.

Query languages for supporting OLAP.


SQL extensions: Group by cube, group by rollup.

Multidimensional expressions (MDX).

View materialization: optimal set of views, partial order on views, cost model, greedy algorithm.

Relational OLAP (ROLAP): Star schema, snowflake schema, snowflake constellation.

Multi-dimensional OLAP (MOLAP): multicubes and hypercubes, sparse and dense dimensions.

Indexing of dimensions: bitmap indexes.

Indexing of dimensions: join indexes.

Hybrid OLAP (HOLAP).

XML: tags, elements, attributes, values.

Well-formed and valid documents.

XPath: axes, node-tests, predicates.

XQuery expressions.

XQuery functions.

FLWOR expressions.

XQuery data model.

DTD.

XML Schema: simple and complex types.

ETL process design and implementation.

Data cleaning. Data quality dimensions, issues and constraints. Typical cleaning checks.

Entity resolution.

Data transformation.

Loading data into warehouse. Bulk load.

Map/reduce paradigm.

Hadoop map/reduce implementation.

Hadoop distributed filesystem HDFS.

Map/reduce design patterns.

Pig. Pig Latin.

Hive. HiveQL.

HBase BigTable implementation.

Dynamic hash tables (DHT).

Vector clocks and conflict detection.

Gossip protocol and hinted handoff.

Key/value stores: properties and usage.

Extended key/value (data structures) store Redis: properties and usage.

Document database MongoDB: properties and usage.

Graph database Neo4j: properties and usage.

Data stream models: relational and XML.

Window-based processing.

STREAM CQL. Query semantics. Mapping Streams to Relations and vice versa.

Aurora SQuAL. Operators.

STREAM: architecture, query plan, optimizations.

Aurora: architecture, optimizations.

Distributed stream processing: Borealis.


10. Learning resources

10.1 Core Textbooks

Silberschatz A., Korth H.F., Sudarshan S. Database System Concepts, 6th ed, McGraw-Hill, 2010. — 1376pp.

Garcia-Molina H., Ullman J., Widom J. Database Systems: The Complete Book, 2nd Edition, Prentice Hall, 2009. — 1248pp.


10.2 Recommended books

Blaha M. Patterns of Data Modeling (Emerging Directions in Database Systems and Applications), CRC Press, 2010. — 261pp.

Kuate P.H. et al. NHibernate in Action, Manning Publications, 2009. — 400pp.


Celko J. Joe Celko's Analytics and OLAP in SQL, Morgan Kaufmann, 2006. — 208pp.

Kimball R., Ross M. The Data Warehouse Toolkit, Wiley, 2002. — 447pp.

Kimball R., Caserta J. The Data Warehouse ETL Toolkit, Wiley, 2004. — 491pp.

Smith B.C., Clay C.R. Microsoft SQL Server 2008 MDX Step by Step, Microsoft Press, 2009. — 400pp.

Rodrigues F., Coles M., Dye D. Pro SQL Server 2012 Integration Services, Apress, 2012. —

636 pp.

Ben-Gan I. et al. Inside Microsoft SQL Server 2008: T-SQL Programming, Microsoft Press, 2009. — 832 pp.

Melton J., Buxton S. Querying XML: XQuery, XPath, and SQL/XML in context, Morgann Kaufmann, 2006. 848pp.

Franks B. Taming The Big Data Tidal Wave: Finding Opportunities in Huge Data Streams

with Advanced Analytics (Wiley and SAS Business Series), Wiley, 2012. — 336pp.

White T. Hadoop: The Definitive Guide, 3rd edition, O’Reilly, 2012. — 657pp.

Redmond E. Seven Databases in Seven Weeks: A Guide to Modern Databases and the NoSQL Movement, Pragmatic Bookshelf, 2012. — 352pp.

Robinson I. Graph Databases, O’Reilly, 2013. — 224pp.

Golab L., Oszu M. T. Data Stream Management (Synthesis Lectures on Data Management), Morgan & Claypool Publishers, 2010. — 80 pp.

10.3 Additional books

Chaudhury S., Dayal U. An overview of data warehousing and OLAP technology. // SIGMOD Record, v.26 n.2, pp.507-508, 1997.

Harinarayan V., Rajaraman A., and Ullman J. Implementing Data Cubes Efficiently. // In Proceedings of the 1996 ACM SIGMOD international conference on Management of data (SIGMOD '96), pp. 205-216, 1996.


Jensen C., Pedersen T., Thomsen C. Multidimensional Databases and Data Warehousing, Morgan & Claypool Publishers, 2010. — 111p.

Rainardi V. Building a Data Warehouse: With Examples in SQL Server. Apress, 2008. — 541pp.

Inmon W. H., Krishnan K. Building the Unstructured Data Warehouse: Architecture, Analysis, and Design, Technics Publications, 2011. — 216pp.

Holmes A. Hadoop In Practice, Manning Publications, 2012. — 511pp.

Miner D., Shook A. MapReduce Design Patterns, O’Reilly, 2012. — 232pp.

Gates A. Programming Pig, O’Reilly, 2011. — 203pp.

Arasu A., Babu S., Widom J. The CQL Continuous Query Language: Semantic Foundations and Query Execution, VLDB Journal, 15(2), 2006. pp.121-142.

Arasu A., Babcock B., Babu S., Cieslewicz J., Datar M., Ito K., Motwani R., Srivastava U., Widom J. STREAM: The Stanford Data Stream Management System. // M. Garofalakis, J. Gehrke, and R. Rastogi, editors, Data Stream Management: Processing High-Speed Data Streams, Springer, 2009.

Cherniack M., Zdonik S. Stream-Oriented Query Languages and Operators, Encyclopedia of Database Systems, Springer, 2009. pp 2848-2854.

Tiwari S. Professional NoSQL, Wrox, 2011. — 384pp.

10.4 Reference materials, dictionaries, encyclopedias

MSDN. Available at: http://msdn.microsoft.com

RSDN. Databases. Available at: http://rsdn.ru/summary/248.xml

CITForum. Available at: http://www.citforum.ru/database

SQL.RU Available at: http://sql.ru

Liu L., Özsu M.T. Encyclopedia of Database Systems. Springer, 2009. — 748 pp.

10.5 Software Microsoft SQL Server 2008 R2 (or later)

Microsoft SQL Server Analysis Services 2008 R2 (or later)

Microsoft Visual Studio 2008-2010 (or later)

Apache Hadoop

Apache Pig

10.6 Remote support LMS is used for remote course support.

11 Material and technical resources Projector for lectures and practical studies.

Computer classes having Microsoft Visual Studio 2010 (or later) and Microsoft SQL Server 2008 (or later) Management Studio installed.

The author of program: _____________________ A.D.Breyman

1. Scope and RegulationsŸрограмма... · Course «Databases» Syllabus [ 2014-2015 ac. year...

Documents

Transcript of 1. Scope and RegulationsŸрограмма... · Course «Databases» Syllabus [ 2014-2015 ac. year...