Module03

Module 3: File and database organizationOverview

This module introduces the basic concepts of files and databases, their components, and organization.Database characteristics, advantages, and disadvantages will be reviewed, followed by a comparison ofhierarchical, network, and relational databases. You will also study database management systems and newdevelopments.

Test your knowledge

Begin your work on this module with a set of test-your-knowledge questions designed to help you gauge thedepth of study required.

Topic outline and learning objectives

3.1 Data organization andinformation

Describe how fields, records, files, and databases are organized within a datahierarchy. (Level 1)

3.2 Database organization

methodsDescribe database organization and database components. (Level 1)

3.3 Database management

systemsDescribe a database management system and explain why it is needed.(Level 1)

3.4 Database storage and

analysis Describe database storage techniques. (Level 2)

3.5 Database developments Describe database developments, including data warehousing, data marts, and

data mining. (Level 2)

Module summary

Print this module

Course Schedule Course Modules Review and Practice Exam Preparation Resources

Module 3: Test your knowledge

1. Multiple choice

a. Which of the following is the lowest level in the hierarchy of data?

1. Entity2. Field3. File4. Record

b. What is a data definition language?

1. A language used by Java to define data on the Web2. A language used in data communication to route data packets3. A language used to define and describe data and data relationships in

a database4. A language used in decision support systems to define data

c. What is the most important characteristic of a primary key field?

1. It is short.2. It is a file.3. It represents an entity.4. It is unique.

d. Which of the following is a problem of traditional file environment?

1. Requires administrator for data maintenance2. Requires fourth-generation language to program3. Program data dependence4. Storage capacity

e. Which of the following file or database models has a many-to-many relationship?

1. Indexed2. Flat file3. Hierarchical4. Relational

Solution

2. Chapter 5, Review question 2, page 210

Solution

3. Chapter 5, Review question 13, page 210

Solution


3.1 Data organization and information

Learning objective

Describe how fields, records, files, and databases are organized within a data hierarchy. (Level 1)

Required reading

Chapter 5, pages 174-177

LEVEL 1

Databases can be used for business intelligence purposes such as obtaining product profitability, customerprofiles, and targeting promotions. In the opening case of Chapter 5, the Valero Energy company uses afully-integrated enterprise business intelligence system, called WebFocus, to make meaningful data availableand accessible throughout the organization.

Basic terms

Data is generally organized in a hierarchy that starts with a character and progresses into a database. Forillustrative purposes, let's look at the components of a student database that holds the students' names,courses enrolled, and the students' grades. A character may be alphabetic, numeric, or a symbol, and eachcharacter occupies a single position in a field. Each letter in a student's name is a character.

A field is a group of related characters and it is the smallest piece of information in a record. For example, in astudent file, one field could hold the first name of each student; in an accounts receivable file, one field couldhold the invoice number. A field can also hold graphical, video, or sound images. More than one field makes upa record.

A record is a collection of related data fields. It holds all the information about an entity in the file. All therecords in a file must have the same fields.

A file is a collection of related records. Each file has a unique structure. For example, a paper-based file isidentified by a folder and all the pages it holds, organized in some fashion, perhaps with a table of contents. Anelectronic file on a computer is identified by a filename, and holds all the records stored under the filename.

An entity is a generalized class of people, places, or things (objects) for which data is collected, stored, andmaintained. For example, in a student database, one entity could be a student. In general, each entity has atleast one record associated with it.

An attribute is a characteristic of an entity. In the above example, the student has a student number, name,date of birth, and so on. Attributes are contained in the fields that are grouped by entities. Not only must eachrecord in a file contain the same fields, each field must hold the same type of information and have the sameattributes. An example of an attribute defined for the NAME field of a personnel file could be:

Field description: NAMEField type: Character fieldField width: 30 charactersField structure: Last name, followed by a blank, then first name, followed

by a blank, then initial. The first character in the last andfirst names must be in upper case, subsequent charactersto be in lower case, unless specified otherwise. The initialis always in upper case. If a name contains more than one


initial, use the first initial only. If the name is too long tofit in the field, drop the initial, then truncate (shorten) thefirst name as needed.

Each record can be seen as a row in a table and each field can be seen as a column. A database is anorganized collection of records in one or multiple tables.

All databases require that every record contain at least one key. A key is a field or set of fields that identifiesthe record. A primary key is a field or set of fields that uniquely identifies each record in the table. In casethe primary key is not unique, a secondary key can be used. For example, in a file containing a studentdirectory, the key field could be the name, and the secondary field could be the address, so that in case ofidentical names, the secondary field can be used for sorting.

3.2 Database organization methods

Learning objective

Describe database organization and database components. (Level 1)

Required reading


LEVEL 1

Database approach

As computer applications became more complex and required the use of several related files, databasetechniques were developed to meet these needs. The Data Base Task Group of the Conference on DataSystems Languages (CODASYL) published the first formal documentation of the key features of databases in1971. This publication, which has been updated several times, has become the model that many softwaredevelopers use to develop databases.

Unlike the file approach, the database approach allows different applications (for example, accounting,personnel, and payroll) to access the same database. Instead of organizing the data to meet the needs of aparticular application (for example, payroll), the database approach requires the organization to analyze itsoverall information requirements, and then design a common database to meet the needs of multipleapplications.

Database systems provide a centralized repository of information that is not application-specific. The data in thedatabase is managed centrally regarding the data integrity, primary and secondary key management, andindexing. Various applications access the database to update information. Because the information is no longerorganized in application-specific files, it is much easier to update or change software applications as long as theinformation is used as structured in the database.

The database approach requires the use of database management systems (DBMS).

Data modelling

Logical design describes logical relationships among data and groups them in a logical order, whereasphysical design takes the logical design and structures it for efficiency and effectiveness. For example, itmight be more effective to create summary totals as data are entered, rather than calculate them each timethey are required, or some data attributes could be carried in more than one entity. These are examples ofplanned data redundancy, with the goal of improving system performance to meet user needs.

An important tool for database designers is a data model, which is used to show relationships betweenentities. If this is done at the highest level for the organization, it is known as enterprise modelling. Acommonly used tool for modellers is an entity-relationship(ER) diagram. By using these tools, designerscan ensure that relationships are logically structured so that when databases and application programs aredeveloped, they will in fact meet the needs of the system's users.

Database models

The data in a database can be interrelated in many ways. Historically, databases were organized in ahierarchical or network structure. Today, the most popular structure is a relational database. Do not be overly


concerned with the mechanics of these structures. Instead, focus on the essential differences between thedatabase types, and the general organization of the data.

Hierarchical database

A hierarchical database organizes information in a tree-like structure in which data elements are related toeach other in a parent (superior) to child (subordinate) relationship. A data element can be a data field, arecord, or a database file.

The hierarchical database provides a one-to-many relationship, in a top-down manner. To access the employeeof any department, you must specify the department because department is the parent of employee. If youhave no information on the parent, it is impossible to retrieve the item because you must access the itemthrough its parent.

A hierarchical structure is particularly useful for databases containing structured information where access toinformation is keyed to the structure, that is, the logical access is in the same hierarchy as the physical layoutof the database. The rigid structure of a hierarchical database enables it to be updated efficiently. Typically, itis used in applications such as inventory management, where a large number (hundreds of thousands ormillions) of records are in the system.

Network database

A network database is similar to a hierarchical database except that a child in the system can have morethan one parent. Thus, because more than one path to a particular data element exists, the database structureis many-to-many. Network databases are particularly efficient for looking up information because they permitaccess from more than one starting point. Unlike a hierarchical database, the process of querying a networkdatabase is less restrictive. A network database is appropriate for situations where queries of the database maynot follow a predetermined pattern. An example is a database of students and their course enrolment, where astudent can be enrolled in multiple courses. The relationship between students and courses is thus many-to-many, and a hierarchical database is inappropriate.

Relational database

A relational database uses two-dimensional tables called relations to store data. In the relational model,each row of a table represents an entity, with the columns representing attributes. Each attribute can haveonly certain predefined values, and these allowable values are called the domain. This provides automaticerror-checking features to all applications using the table.

The relational database is particularly easy to manage for answering user questions and producing reports.Basic data manipulation includes selecting (eliminates rows), projecting (eliminates columns), and joiningand linking (creates a new table).

One distinctive feature of a relational database is that you can combine any number of tables as long as thereare common fields. You can combine (join) two tables to form a third, provided there is a common column. Aslong as the tables share at least one common attribute, they can be linked to answer queries or producereports. What is especially important is that data from multiple tables can be linked to answer queries. Using arelational database, you can answer a complex query with a few simple commands, whereas the traditionalfile-based approach would require several programs to be written and run against the various files containingthe required data, and then creating a new file after several operations.

A relational database has properties beyond two-dimensional tables. For example, there is no need for order orsequence in a table, and the relation is a logical structure, thus users need not be concerned with physicalstorage details.

Because of the flexibility provided by relational databases, they are becoming the design of choice for computerprofessionals. Relational databases reduce data redundancy (facilitated by the database joining capability) andallow data tables to be added with relative ease. With relational databases, it is relatively easy to perform

queries on the data without being constrained by the actual structure of the data. Microsoft Access is arelational database program.

Example 3.1

Choosing a database model

Francine Ong has been assigned to design a database for a new inventory control system. The following is apartial description of the data items and their relationship:

Product items are organized by product lines, and each product can only belong to one product line. Eachsalesperson is assigned one or more product lines. A product line can have more than one salespersonassigned. Each salesperson is assigned a sales territory. For large territories, more than one salesperson can beassigned.

Q: Of the three database models (hierarchical, network, and relational), which model is suitable for theinformation described?

Solution

Exhibit 3-1 graphically represents the network model for the inventory database, while Exhibit 3-2 depicts therelational model. Exhibit 3-3 is a short-form notation of the relational model.

Exhibit 3-1

Network model

Exhibit 3-2

Relational model

Sales territory (partial table)

Territory code Territory name Territory manager G/L profit centre1001 Northern B.C. J. Chrieten 9010011002 North Vancouver Island B. Beverly 9010021003 South Vancouver Island C. Cleverly 9010031004 Lower Mainland K.C. Leung 901004

Salesperson (partial table)

Salesperson ID Salesperson name Territory code Quota for the year810 Kelvin Longile 1001 500,000811 Rowanda Dhaliwal 1002 450,000812 James Jones 1003 600,000813 Mathew Mah 1004 800,000814 Lucien Chong 1004 800,000

Product line (partial list)

Product line code Description

E01 Electrical partsE02 Plumbing supplies

Product assignment (partial list)

Product line code Salesperson IDE01 810E01 811E01 812E02 813E02 814

Product (partial table)

Productcode

Product name Manufacturer Productline code

Unitcost

C1023 Centronics plug Acme Manufacturing E01 10.57C0143 Triplex plug Acme Manufacturing E01 13.98C1045 Universal plug TDK Manufacturing E01 23.52P4106 Peerless faucet LEW Piping Supplies E02 65.23P4107 Kitchen Sink

Stainless Kitchen AidManufacturing E02

105.21

Instead of drawing the tables as shown in Exhibit 3-2, another common practice is to list the contents of eachtable in short-form notation, marking the key field with an asterisk, as shown in Exhibit 3-3.

Exhibit 3-3

Short-form notation

Tables: Sales territorySalespersonProduct lineProduct assignmentProduct

Sales territory: Territory code*Territory nameTerritory managerG/L profit centre

Salesperson:

Salesperson ID*Salesperson nameTerritory codeQuota for the year

Product line: Product line code*Description

Product assignment: Product line code*Salesperson ID*

Product:

Product code*Product nameManufacturerProduct line codeUnit cost

Many computer database programs, such as Access, FileMaker Pro, and Paradox, provide relational capabilities.Oracle and Microsoft SQL are examples of fully relational databases.

Any database is only as useful as the data it contains. Data should be accurate, complete, economical, flexible,reliable, relevant, simple, timely, verifiable, accessible, and secure. The purpose of data cleanup is to developprocesses to ensure those characteristics. Data cleanup is particularly important when moving from a file-basedsystem to a database or migrating from one database to another one.

3.3 Database management systems

Learning objective

Describe a database management system and explain why it is needed. (Level 1)

Required reading


LEVEL 1

The goals and activities of a business should be supported by the appropriate database structure. To create,implement, and use a database, a database management system (DBMS) is required. A DBMS is a groupof programs used as an interface between the database and either the application programs or a user. Usersinclude end-users, who use the information from the database or enter data into the database; programmers,who develop applications for the database; and database administrators (DBA), who create and manage thedatabase. All DBMSs have certain common functions, but are classified by the type of database they support.

Providing a user view

The first step in creating a database is to define the business objective or goal of the database in a measurablemanner. The next step is providing the DBMS with information about the physical structure and logicalrelationships among the data to be contained in the database. This description is called a schema orschematic. Subschemas, which defines a set of data that users can view or modify, or do both, are used togive users access to only a portion of the entire database that they need based on business rules and their rolein the organization. For example, the subschema for the accounts payable clerks should only allow them tohave access to the accounts payable-related information and not payroll information. The use of subschemas isnot only efficient but also ensures data security.

Creating and modifying the database

A data definition language (DDL) is used to define and describe data and data relationships in a database.The schema and subschema are applied using a DDL. When creating or modifying a database, it is also criticalto establish a data dictionary that contains a complete description of all data in the database, includingnomenclature, attributes, users, and applications. Typical uses of a data dictionary are to

provide a standard definition of terms and data elementsassist programmers in designing and writing programssimplify database modification

A data dictionary helps achieve the advantages of the database approach by

reducing data redundancyincreasing data reliabilityspeeding up program developmentfacilitating modification of data and information

Storing and retrieving data

Potential problems arise if more than one user or program attempts to access the same record in the samedatabase at the same time, and so there is a need for concurrency control. Data access control functions


within the DBMS ensure that two users cannot modify the same field at the same time.

Manipulating data and generating reports

When the DBMS is operational, a variety of programming languages can be used by different users to createapplications that will access the data from the database. Data manipulation language (DML) arecommands that are part of the DBMS package. Structured query language (SQL) is a popular DML toolthat can be used across a wide range of hardware platforms.

The personal computer environment is significantly different from the corporate mainframe or networkedenvironment. Typical database software for personal computers, such as Microsoft Access, MySQL andFileMakerPro, allows the user to interact directly with the database without needing to know or understand thedifferent components such as the DDL and DML.

3.4 Database storage and analysis

Learning objective

Describe database storage techniques. (Level 2)

Required reading


LEVEL 2

Database storage techniques

For any database, a number of database storage techniques can be used to store and manage it. Mostdatabases are stored in a central location. Mainframe computers and personal computers use a centralizeddatabase storage technique. However, distributed database storage is growing in popularity.

Distributed databases

Distributed databases are technically quite complicated to implement and administer. Distributed databasestorage involves storing an organization's data in several different servers that are connected viatelecommunication equipment. It is sufficient to know that such a technology exists, and that one form ofimplementation is a replicated database. For the purpose of this course, the description in the text on page203 is adequate.


3.5 Database developments

Learning objective

Describe database developments, including data warehousing, data marts, and data mining.(Level 2)

Required reading

Chapter 5, pages 196-202 (up to "Distributed databases"), pages 204-207

LEVEL 2

Data warehouses, data marts, and data mining

The value of data ultimately lies in the decisions it enables. Companies have started developing datawarehouses and data marts to collect business information from the multiple sources within an organizationwith the objective of making better business decisions. Data mining and online analytical processing (OLAP) areinformation-analysis tools that help automate the identification of patterns, trends, or relationships in a datawarehouse to support decision making.

A data warehouse enables an organization to consolidate massive amounts of information extracted fromoperational and production systems for analysis. Data warehousing techniques are becoming increasinglypopular with large organizations that have amassed trillions of bytes of data. Ordinary database analysistechniques do not work well with such massive amounts of data.

A well-designed and properly built data warehouse

delivers a good return on investmentimproves the company's competitive advantage by linking both internal and external informationstores data extracted from the production databases and conventional files in one placehas directories that show users what is in the database and how to access itprovides information that meets the organization's need for business intelligence

Building a data warehouse is a very time-consuming process because it is difficult to define what data arenecessary and what level of consolidation is desired. Many organizations now start with a smaller version ofdata warehouse called a data mart for departmental use. Data marts are also used by small andmedium-sized businesses. Departmental data marts can be used for online analytical processing (OLAP) withindepartments and form the basis of the data warehouse for the organization.

Data mining is an information analysis tool that involves the automated discovery of patterns andrelationships in a data warehouse. Business intelligence has stimulated the interest in and the use of datamining because of the enormous amounts of data being collected. Because of the rapid growth and potentialfor data mining, the traditional DBMS vendors are incorporating data mining tools into their products.

While both online analytical processing (OLAP) and data mining support data analysis and decision making, adata-mining tool generally does the work for the user and presents results, while OLAP requires the user to bemore knowledgeable about the data and their business context to gain insight from the data. OLAP is nowbeing used to store and deliver vast amounts of data warehouse information efficiently.

Business intelligence


Business intelligence (BI) is the process of getting enough of the right information in a timely manner andusable form to support the business strategy, tactics, or operations.

Competitive intelligence is the continuous legal and ethical collection and analysis of information aboutcompetitors for comparison purposes.

Counterintelligence is what a firm does to protect its information from the competition.

Knowledge management is a collection of techniques that captures and manages structured andunstructured information to improve the ability of the organization to make timely and good business decisions.

Open database connectivity (ODBC) is a set of standards that helps database integration and has theability to share information between databases. Software developed according to these standards can be usedwith any ODBC-compliant database. This is extremely important to organizations that use a variety of levels ofdatabase applications. ODBC is frequently a standard requirement when organizations select software.

Object-oriented and object-relational database management systems

Instead of storing individual records, an object-oriented database management system (OODBMS)stores objects which, unlike records, may not be uniform in shape and size and may exist in a variety of formsincluding audio, video, and graphical data. An object-relational database management system(ORDBMS) allows third parties to add new data types and operations to the database. The growth ofe-commerce, web-based applications, and web servers has created increasing demands for ORDBMS.

Virtual or hypermedia databases contain linked nodes of data. A web page containing hypertext links canbe viewed as a form of hypermedia database of information. On a web page, a user does not need to navigatethrough the information in a sequential manner. Instead, hypertext links can be used to explore other parts ofthe database. The advantage hypertext has is that, unlike traditional database manipulation languages, userscan search for and manipulate alphanumeric data in an unstructured form. Hypermedia databases are anextension of hypertext that store and access graphics, sound, and video, as well as alphanumeric data.

One other database system of increasing importance is spatial data technology, also known as geographicinformation systems (GIS). The global positioning system (GPS) is one of the applications that providedata input to the GIS. The databases store spatial location data. In the case of NASA and Canadian satellites,over a terabyte of data is stored every day. The cumulative data is nearing the petabyte (1,000 terabytes)mark. For such large databases, special tools are being developed to handle the data.

Module 3 summary

File and database organization

This module introduces the basic concepts of files and databases, their components, and organization.Database characteristics, advantages, and disadvantages will be reviewed, followed by a comparison ofhierarchical, network, and relational databases.

Describe how fields, records, files, and databases are organized within adata hierarchy.

Data must be organized and structured so that they can be used effectively.

Data hierarchy (from largest to smallest element):1. Database — a group of files holding related information2. File — a collection of related information called records3. Record — a collection of attributes of an entity in a file. For example, in a personnel

file, an employee is an entity. Attributes of an employee include employee number,date of birth, and start date.

4. Field:A field is the smallest piece of information in a record, correspondingto one attribute of an entity.A primary key field is a field that uniquely identifies a record in a filefor quicker access of data and sorting.A secondary key field is sometimes used for access and sorting but itdoes not uniquely identify a record.

5. Entity — people, places, or objects for which data is collected, stored, andmaintained

6. Attribute — a characteristic of an entity7. Character — a letter, number, or symbol

Describe database organization and database components.

A database is a collection of data organized so that they can be accessed and used by manydifferent applications. Data is stored and managed centrally.

Logical and physical view of data:logical view — presents what end-users seephysical view — reflects the way data is actually organized and structured onphysical storage media

Some advantages of using a database approach:data independent of application programreduction of data redundacy and inconsistencyelimination of data confusionconsolidation of data managementease of information access and use

Disadvantages of database approach:Organization is more vulnerable in the event of system failures because data iscentralized.Software and hardware requirements are higher.Because data is centralized, errors that do enter the database may have a


widespread effect.A database administrator (DBA) is required to manage the DBMS.

Three principal database models are:hierarchical model — organizes information in a tree-like structurenetwork model — the database structure is many-to-manyrelational model — uses two-dimensional tables called relations to store data

Which model to use depends onthe nature of the data relationshipsthe need for flexibilitythe volume of requests or changes to the database to be processedthe ease of use for end-users

Describe a database management system and explain why it is needed.

Database management system (DBMS) is the software that serves as an interface between acommon database and various application programs.

Three components of a DBMS are:data definition languagedata manipulation languagedata dictionary

A schema describes physical structure and logical relationships of data.A subschema provides a specific user view.

A data definition language (DDL) is used to define and describe data and data relationships in adatabase.

A data dictionary contains a complete description of all data in the database.A data dictionary reduces data redundancy, increases data reliability, and facilitatesdevelopment and modification of the database.

Data manipulation language (DML) — commands are part of a DBMS package, and are used tomanipulate the data and generate reports.

Structured query language (SQL) is a tool to be used across a wide range ofhardware platforms.

Describe database storage techniques and services.

Most databases are stored in a central location. Mainframe computers, personal computers, aswell as LANs, use a centralized database storage technique.

Distributed databases are technically quite complicated to implement and administer.

A replicated database holds a duplicate set of frequently-used data at different locations and isone type of distributed database.

Describe database developments, including data warehousing, data marts,and data mining.

Data warehouse

Data warehouse consolidates data from various operational systems and external data.

It enables online analytical processing (OLAP) to provide information that meets the organization’sinformation needs.

It is difficult and costly to build; however, it provides a good return on investment if properlydesigned.

Data marts

Smaller versions of data warehouse, called data marts, may be built first. These data marts canbe used for departmental OLAP and form the basis of data warehouse for the organization.

Data mining

Data mining consolidates data from various operational systems and external data.

It enables online analytical processing (OLAP) to provide information that meets the organization’sinformation needs.

Solution 1

a. 2) Text, page 177b. 3) Module Notes, Topic 3.3c. 4) Text, page 179d. 3) Text, page 179e. 4) Text, page 182


Solution 2

A database is a collection of integrated and related files. A database management system is the software usedto manipulate the database and provide an interface between the database and the user or applicationprograms. A database management system is systems software that helps organize data for effective accessand storage by multiple applications. A DBMS provides different users with different views of the data(subschemas), avoids redundancy, encourages program independence, offers flexible access, and providescentralized control.


Solution 3

Data mining is the automated discovery of patterns and relationships in data warehouses. OLAP tools can tellusers what happened in their business. Data mining searches the data for statistical "whys" by seeking patternsin the data and then developing hypotheses to predict future behaviour. Online analytical processing (OLAP)programs are used to store and deliver data warehouse information. The OLAP allows users to explorecorporate data in new and innovative ways using multiple dimensions such as products, salespeople, or time.OLAP programs include spreadsheets, reporting and analysis tools, and custom applications.


The hierarchical model is inappropriate in this case because of the many-to-many relationships betweensalespersons, product lines, sales territories, and inventory items. The network and relational models, however,are both suitable. The preferred model is a relational database due to its flexibility to associate or link differenttypes of data.


Module03

Engineering

Transcript of Module03