Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web...

74
Data Warehousing ITEM TITLE 1) Data Warehousing Introduction 2) Data Warehouses At a Glance 3) Data Warehouse Overview 4) Data Warehouse Tools 5) Data Warehousing Methods 6) Data Warehouse Design Strategies 7) How To Assess Your Data Warehouse 8) How To Create a Data Warehouse Structure 9) How To Protect Your Data Warehouse 10) How To Manage Current and Historical Information Within Your Data Warehouse 11) How To Use Data Warehouses Strategically 12) Maintaining Records Within a Data Warehouse 13) Advantages and Disadvantages to Using a Data Warehouse 14) The Difference Between Data Mart and Data Warehouse 15) How To Properly Manage a Data Warehouse 16) How To Manage Meta Data Within a Data Warehouse 17) Federated Data Warehouse Architecture 18) Historical Information About Data Warehouses 19) Data Warehouse Business Principles 20) Understanding The Data Warehouse 21) The Benefits of Data Warehouses 22) The Disadvantages of a Data Warehouse 23) Rules to Use With Your Data Warehouse 24) Data Warehouse Issues 25) Crucial Requirements For Successful Data Warehouses 26) Why Data Warehouses Can Be Useful 27) Fundamental Themes For Your Data Warehouse 28) What You Should Know About Building a Data Warehouse 29) How To Rate Your Data Warehouse

Transcript of Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web...

Page 1: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

Data Warehousing

ITEM TITLE

1) Data Warehousing Introduction 2) Data Warehouses At a Glance 3) Data Warehouse Overview 4) Data Warehouse Tools 5) Data Warehousing Methods 6) Data Warehouse Design Strategies 7) How To Assess Your Data Warehouse 8) How To Create a Data Warehouse Structure 9) How To Protect Your Data Warehouse 10) How To Manage Current and Historical Information Within Your Data

Warehouse 11) How To Use Data Warehouses Strategically 12) Maintaining Records Within a Data Warehouse 13) Advantages and Disadvantages to Using a Data Warehouse 14) The Difference Between Data Mart and Data Warehouse 15) How To Properly Manage a Data Warehouse 16) How To Manage Meta Data Within a Data Warehouse 17) Federated Data Warehouse Architecture 18) Historical Information About Data Warehouses 19) Data Warehouse Business Principles 20) Understanding The Data Warehouse 21) The Benefits of Data Warehouses 22) The Disadvantages of a Data Warehouse 23) Rules to Use With Your Data Warehouse 24) Data Warehouse Issues 25) Crucial Requirements For Successful Data Warehouses 26) Why Data Warehouses Can Be Useful 27) Fundamental Themes For Your Data Warehouse 28) What You Should Know About Building a Data Warehouse 29) How To Rate Your Data Warehouse 30) How Data Is Stored Within a Data Warehouse 31) How Does a Data Warehouse Differ From a Database 32) Creating an Efficient Process for Data Warehouses 33) Understanding Quality Management For Data Warehouses 34) How To Evaluate The Software For your Data Warehouse 35) Understanding The Challenges of Using Data Warehouses

 

Page 2: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

Data Warehousing Introduction

A data warehouse is a type of computer database that is responsible for collecting and storing the information of a particular organization. The goal of using a data warehouse is to have an efficient way of managing information and analyzing data.

Despite the fact that data warehouses can be designed in a number of different ways, they all share a number of important characteristics. Most data warehouses are subject oriented. This means that the information that is in the data warehouse is stored in a way that allows it to be connected to objects or events which occur in reality.

Another characteristic that is frequently seen in data warehouses is called a time variant. A time variant will allow changes in the information to be monitored and recorded over time. The information that exists in data warehouses is non-volatile. This means that it cannot be deleted, and must be held to be analyzed in the future. All of the programs that are used by a particular institution will be stored in the data warehouse, and it will be integrated together. The first data warehouses were developed in the 1980s. As societies entered the information age, there was a large demand for efficient methods of storing information.

Many of the systems that existed in the 1980s were not powerful enough to store and manage large amounts of data. There were a number of reasons for this. The systems that existed at the time took too long to report and process information. Many of these systems were not designed to analyze or report information. In addition to this, the computer programs that were necessary for reporting information were both costly and slow. To solve these problems, companies begin designing computer databases that placed an emphasis on managing and analyzing information. These were the first data warehouses, and they could obtain data from a variety of different sources, and some of these include personal computers and mainframes.

Spreadsheet programs have also played an important role in the development of data warehouses. By the end of the 1990s, the technology had greatly advanced, and was much lower in cost. The technology has continued to evolve to meet the demands of those who are looking for more functions and speed. There are four advances in data warehouse technology that has allowed it to evolve. These advances are offline operational databases, real time data warehouses, offline data warehouses, and the integrated data warehouses.

The offline operational database is a system in which the information within the database of an operational system is copied to a server that is offline. When this is done, the operational system will perform at a much higher level. As the name implies, a real time data warehouse system will be updated every time an event occurs. For example, if a customer orders a product, a real time data warehouse will automatically update the information in real time. The offline data warehouse is a database that is updated on a regular from an operational system.

Page 3: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

Wih the integrated data warehouse, transactions will be transferred back to the operational systems each day, and this will allow the data to easily be analyzed by companies and organizations. There are a number of devices that will be present in the typical data warehouse. Some of these devices are the source data layer, reporting layer, data warehouse layer, and transformation layer. There are a number different data sources for data warehouses. Some popular forms of data sources are Teradata, Oracle database, or Microsoft SQL Server.

Another important concept that is related to data warehouses is called data transformation. As the name suggests, data transformation is a process in which information transferred from specific sources is cleaned and loaded into a repository.

Data transformation can either be a manual or automated process. Code can be manually generated, or an ETL tool can be utilized. The device that is responsible for transforming the data will compare it to other systems. It will also placed the data in a specific standard. In addition to this, it will often be linked to other systems which can assist it. The goal of using a data warehouse is to store and monitor information in a way that allows it to easily tbe analyzed. The data held in the warehouse will typically remain on file for a year.

Data Warehouses At a Glance

Data warehouses have played an important role in information technology since the 1990s. They are tools that allow organizations to use relevant information to make important business decisions.

While data warehouses were originally only used by large companies, the decreasing cost of computing has allowed them to be adapted by smaller companies. To understand data warehouses, it is important to learn about the data warehouse architecture.

Data warehouses are computerized systems that store information. They information will almost always come from another source. These sources could be programs or applications. Not only is the data warehouse able to store information, it can also analyze that information as well. This is what separates a data warehouse from being a mere computer storage device. Managers can search through the warehouse looking for specific information. Data mining programs can be initiated, and they will look through the data warehouse for patterns or relationships that can help companies make important marketing decisions.

A data warehouse is basically a database that can answer certain questions. They are subject oriented, and they will analyze information and can help managers solve problems. There are a number of steps that are involved with building a data warehouse. These procedures are similar to those that would be used to build other computer programs. The users of the data warehouse must play a role in its construction. The user is important because they are the people who will be using the tools. Each data

Page 4: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

warehouse is different, and it must be designed in a way that will allow it to meet the needs of those who use it.

The users will decide what type of information will be placed in the warehouse. Once the requirements for the data warehouse have been developed, the elements must be placed in a conceptual model. This will act as a diagram that will be used to build the actual database. At this point, there are a large number of decisions which need to be made about the design and implementation of the warehouse. Once the warehouse has been built, the data must be acquired and stored. It is up to the data warehouse managers to decide what information must be stored in the database. Much of this data will be related to the organization that owns the data warehouse.

However, some of the data may be taken from other sources. An extraction application must be created in order to pull data from other sources to be placed in the data warehouse. The sources must be identified, and some of these will be file, legacy systems, or other databases. The information will be copied into what is called a staging area. Once the data is placed in the staging area, it will need to be cleaned. Once it is clean and free of errors, it will then be copied into the warehouse. It is crucially important to make sure the data is moved into the warehouse correctly. It is not, the project will not be successful.

Metadata is another important concept that is connected to data warehouses. In fact, high quality meta data is important for the function of the database. Metadata is information about information. It is used in the information collection process, and it is also used when the data is accessed or transformed. In the acquisition phase, the information will be mapped and transferred from the operational system. I will provide a large amount of information about the data, and some of this includes updates or algorithms. Data marts are also important. While managers will want to keep them updated, they don't need to be updated in real time. Data marts are small in comparison to data warehouses and are only hold information about departments that exist within an organization.

Many companies have begin combining a number of small data marts in order create a data warehouse. However, this has led to controversy. Some feel that data marts where never designed to function as data warehouses, and they should not be used for this purpose. It is best to use data marts as a component to a data warehouse instead of a standalone entity. Security is an important issue with data warehouses, and the information must be protected.

Data Warehouse Overview

The word data warehouse was first developed by Bill Inmon in the early 1990s. He referred to it as being a integrated collection of information that could help companies and organizations make better decisions.

To be effective, a data warehouse had to be integrated, subject oriented, non-volatile, and time variant. In this article, I will go over all these factors in detail. If you

Page 5: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

are building a data warehouse, it is important for you to understand why they are important.

Being subject oriented means that the data will provide information about a specific subject rather than the information about the functions of a company. Because a data warehouse is subject oriented, it will allow you to analyze information that is connected to a specific subject. Being integrated means that the data that is collected within the data warehouse can come from different sources, but can be combined into one unit that is relevant and logical. Having a time-variant means that all the information within the data warehouse can be found with a given period of time.

It is important that the information contained within a data warehouse is stable. While data can be added, it should never be deleted. This property is referred to as being non-volatile. When a company uses a data warehouse that is stable, this will allow them to get a better understanding of the operations within their company. Despite the fact that these terms were first coined in the the 1990s, they are still highly accurate today. However, it should be noted that some data warehouses are volatile. The reason for this is because many modern data warehouses deal with terabytes of data.

Because they must store terabytes of data, many companies are forced to delete some of their information after a certain period of time. For instance, some companies will systematically delete data that has reached three years of age. Before a data warehouse can be built, the correct data must be located. Generally, the information that will be added to the warehouse will come from daily information or historical information. The historical information may be stored in a legacy system, and is challenging to extract.

The design of the data warehouse is important as well. It is important for designers to make sure the design is consistent with the queries that will be conducted within the warehouse. To do this successfully, it is important for designers to understand the database schema. It is crucial to make sure the data warehouse is designed correctly, as it is difficult to recreate some forms of data. Another important aspect of data warehouses is data acquisition. Data acquisition can be defined as transferring data from a source to the warehouse. Data acquisition is one of the most expensive parts of building a data warehouse. This process will often be conducted with an ETL tool.

As of this time, there are just over 50 ETL tools being sold. It may cost a company millions of dollars in order to transfer data from sources to the warehouse. Once the initial data has been transferred to the data warehouse, the process must be repeated consistently. Data acquisition is a continous process, and the goal of a company is to make sure the warehouse is updated on a regular basis. When the warehouse is updated, it is often hard to determine which information in the source has changed since the previous update. The process of dealing with this issue is called changed data capture. This process has become a separate field, and there are a number of products currently be sold to deal with it.

It is important for data to be cleaned before it can be placed in the warehouse. The data

Page 6: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

cleansing process is usually done during the data acquisition phase. Any data that is placed in a warehouse before being clean will pose a danger to the system, and it cannot be used.

The reason for this is because the data may not be correct if it is not cleaned, and a company may make incorrect decisions based on it. This could lead to a number of problems. For example, all the information within a data warehouse that means the same thing must be stored in the same form. If there is information that reads "MS" and "Microsoft," even though they mean the same thing, only one of them can be used to recognize the element within the data warehouse.

Data Warehouse Tools

There are a number of important tools which are connected to data warehouses, and one of these is data aggregation. A data warehouse can be designed to store information based on a certain level of detail.

For example, you can store data based on each transaction, or you can store it based on a summary. These are examples of data aggregation. When data is summarized, the queries will move at a much faster rate. However, some of the information may be lost during a query, and this information may be important for solving a certain problem.

Before you decide which one you will use, it is important to weigh your options carefully. Once you have carried out an operation, you will need to rebuild the warehouse in order to undo it. The best way to handle this situation is to make sure the data warehouse is constructed with a large amount of detail. However, the cost for this can be huge depending on the storage options you choose. Once you have filled your data warehouse with important information, you will want to use this data to help you make smart investment decisions. The tools that can allow you to do this will fall under a topic that is called business intelligence.

Business intelligence is a field which is very diverse. It is comprised of things such as Executive Information Systems, Decision Support Systems, and On-Line Analytical Processing. Business intelligence can further be broken down into a field that is called multi-dimensional analysis tools. These are tools that will allow a user to view data from a wide variety of angles. A query tool will allow a user to send SQL queries within a warehouse to look for results. Data mining is also a field that falls under business intelligence, and will allow you to look for patterns and relationships within a data warehouse.

Another tool that is connected to data warehouses is data visualization. The tools that are used for data visualization will present visual models of data. This data could come in the form of intricate 3D images. The goal of data visualization is to allow the user to view trends in a method which is easier to understand than complicated models that are based off statistics. One tool that is allowing this field to advance is VRML, or Virtual Reality Modeling language. In order for data warehouses to function properly, it is also important

Page 7: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

to place an emphasis on metadata management. Meta data can be described as being "information about information."

Meta data must be managed when data is acquired or analyzed. Meta data will be held in a repository, and can give you important information about many of the data warehouse tools. The process of properly managing meta data has become a science within itself. If it is done properly, the company can greatly benefit. The reason why it is important is because it can allow organizations to analyze the changes that occur within database tables. This is a tool that plays an important part of the construction of a data warehouse.

Data warehousing is a field which is somewhat complicated. There are many vendors who are attempting to advertise the tools, but the cost and complexity involved with the products has not allowed them to be used by a large number of companies. Any company that is thinking of using data warehouses must make sure they have taken the time to review and understand the technology. It can only be useful if you know how to use it. Once you understand and acquire the technology, it is possible for you to gain a powerful advantage over your competitors. This has made data warehouses attractive to many companies.

One of the biggest advantages to data warehouses is that they allow you to store information that you can use to improve the marketing strategies of your company. Not only can you improve the marketing strategies, but you will also be able to make strategic decisions based on the information you have compiled and organized. With techniques such as data mining and data visualization, you will be able to discover important patterns that you didn't know existed. The patterns that you discover can allow your company to earn large profits.

Data Warehousing Methods

Most organizations agree that data warehouses are a useful tool. They benefit from the ability to store and analyze data, and this can allow them to make sound business decisions. It is also important for them to make sure the correct information is published, and it should be easy to access by the people who are responsible for making decisions.

There are two elements that make up the data warehouse environment, and these are presentation and staging. The staging could also be known as the acquisition area. It is composed of ETL operations, and once the data has been prepared, it will be sent to the presentation area.

When the data is placed within the presentation area, a number of programs will analyze and review it. While many organizations agree on the overall goal of data warehouses, the approaches to building them may differ. Attempting to use data marts alone is not a good approach, because they are geared towards departments. In addition to this, attempting to use data marts alone will be inefficient, and you will run into a number of long term problems. There are two techniques for building data warehouses that have become very popular. These are the Kimball Bus Architecture and the Corporate

Page 8: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

Information Factory.

With the Kimball technique, the rough data will be transformed and refined within the staging area. It is important to make sure the data is properly handled during this step. During the staging process, the rough data will be pulled from the source systems. While some of the staging processes may be centralized, others will be distributed. The presentation area will have a dimensional structure, and this model will hold the same information as a standard model. However, it will be easier to use, and it will display information that is summarized.

A dimensional model will be created by a business operation. Departments within the organization do not play a role in this. The data will be populated once it is placed within the dimensional warehouse, and is not dependent on the various departments that may compose an organization. When business processes have been developed within the warehouse, the system will become highly efficient. The next popular data warehouse approach that you will want to become familiar with is the Corporate Information Factory. Another name for this technique is the EDW approach. The data that is extracted from the source will be coordinated.

Within the CIF, a standard data warehouse is used to hold data repositories, and it may also have specific data warehouses which are designed for data mining. The data marts may be designed for specific departments, and they may have summary data which is in the form of a dimensional structure. The atomic data may be obtained from the standard data warehouse. While there are some similarities between these to techniques, there are some notable differences as well.

One of the primary differences between these two techniques is the normalized data foundation. With the Kimball approach, the data structures that must be obtained before the dimensional presentation will be dependent on the source data and transformation. In most cases, the duplicate storage of data is not required in both dimensional and normalized foundations. Many of the people who choose to use a normalized data structure believe that it is faster than the dimensional structure, but they often fail to take ETL into consideration.

Another thing that separates the two data warehouse approaches is the management of atomic data. With the CIF, atomic data will be stored within a normalized data warehouse. In contrast, the Kimball method states that the atomic data should be placed within a dimensional structure. When the data is placed within a dimensional structure, it can be summarized in a wide variety of different ways.

It is important to make sure the information you have is detailed so that users will be able to ask relevant questions. While most users will not place an emphasis on the details of one atomic transaction, they may want a summary of a large number of transactions. It is important for them to have the details so that they will be able to answer important questions. The approach that you choose should be the one which best serves the needs of your company.

Page 9: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

Data Warehouse Design Strategies

To build an effective data warehouse, it is important for you to understand data warehouse design principles. If your data warehouse is not built correctly, you can run into a number of different problems.

The proper methods for building a powerful data warehouse are based on information technology tactics. First off, it is important that you and your organization understand the importance of having a data warehouse. If workers feel that a data warehouse is unnecessary, they may not use it, and this could cause conflicts. Everyone in your organization should understand the importance of using the system.

After you have got your colleagues behind the concept of using a data warehouse, you will want to next focus on data integrity. You will want to avoid designing a data warehouse that will load data that is not consistent. It is also important to avoid creating a database that will replicate data. The goal of your organization should be to integrate data and create standards that will be used and followed. After data integrity, you will next want to look at implementation efficiency. This basically means that you will want to design at system that is simple to use. It doesn't matter how well designed your data warehouse is if your workers have a hard time using it.

If your workers have a hard time using the data warehouse, it will slow down the speed and productivity of your operation. When it comes to creating a data warehouse, you will want to make it as simple as possible. All of your workers should be able to use it without problems. Implementation efficiency is a principle that naturally leads to the next topic you will want to focus on, and this is user friendliness. This is a concept that is an important part of your business. The reason for this is because end users will not utilize a program that is too difficult to use. It is important for you to keep them in mind. Use a design which is friendly and easy to learn.

Once you have designed a data warehouse that is user friendly, you will next want to look at operational efficiency. Once the data warehouse has been created, it should be able to carry out operations quickly. In addition to this, it should not have errors or other technical problems. When errors or technical problems do occur, they should be simple to fix. Another thing you will want to look at is the cost involved with supporting the system. You will want to keep these costs low as much as possible.

The design principles that have been discussed in this article so far are more related to business than information technology. However, there are a number of IT design principles that you will want to follow. One of these is scalability. This is a problem that many data warehouse designers run into. The best way to deal with this issue is to create a data warehouse that is scalable from the beginning. Design it in a way which will allow it to support expansions or upgrades. You should be able to adapt it to a number of different business situations. The best data warehouses are those which are scalable.

The data warehouse that you design should fall under the guidelines of information

Page 10: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

technology standards. Every tool that you use to build your data warehouse should work well with IT standards. You will want to make sure it is designed in a way that makes it easier for your workers to use. While following the guidelines in this article won't allow you to always be successful, it will greatly tip the odds in your favor. You should be wary of companies that promise you perfect results if you use their design methods. No matter how well designed your data warehouse is, you will always run into problems. However, following the right principles will make the problems easier to recognize and solve.

When it comes to using a data warehouse, it is not a matter of "if" you will run into problems. It is matter of "how" and "when." When your data warehouse is well designed, you will be better equipped to solve any problems you encounter.

How To Assess Your Data Warehouse

While many large companies now use data warehouses, the concept has not yet become fully mature. The principles and methods which are used to manage data warehouses have not been developed.

One reason for this is the difficulty that is often involved with data warehouses. There are a number of techniques that must be used in order to identify and extract data, and the tools have continued to change on a consistent basis. Because of this, it often requires a large amount of technical skill in order to manage data warehouses. Many of these complications have caused a number of data warehouse programs to fail.

Despite these problems, there is a large demand for information management systems. Many companies use data warehouses because they are faced with powerful competition, and must be able to record, monitor, and analyze information in order to make strategic decisions. However, it will be difficult for companies to meet these challenges if they are not capable of properly using their data warehouses. The first step in properly using your data warehouse is to develop powerful business processes and methods. It is not simply enough to acquire a data warehouse. Any company with sufficient resources can do this.

The success of your company lies in its ability to produce powerful processes which can be used to achieve the best results. Data warehouses are tools, and how you use them will play a powerful role in whether you succeed or fail. No matter which process you develop for your data warehouse, there are a number of things you will want to keep in mind. First, you will want to avoid making the same mistakes over again. Second, you will want to review and find warehouse processes that were successful and use them to your advantage. It is these two issues that companies will want to pay attention to.

This is where assessing your data warehouse is so important. You will be able to find mistakes that you can avoid in the future, and you will also be able to find successful methods that can be used again. The terms that you will need to deal with when you assess your data warehouse is "how," "why," and "what." The goal of looking at these terms is to find the best processes and methods which will allow you and your company to prosper. But before you can begin assessing your data warehouse, you will need to

Page 11: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

know when you should assess it. Time is money, and you don't want to waste time assessing the warehouse if it is not necessary.

If you are about to use your data warehouse for the first time, this is an example of a time when you will want to assess it. The information that you gain from an assessment will allow you to make better decisions about how the data warehouse should be used. You should know the needs of your business, and you should also know how your data warehouse can help you care for these needs. You should also determine if your organization is ready to use the data warehouse after they have build it. However, it is not enough to assess the data warehouse once. As your company continues to grow, your needs will change, and the data warehouse will need to be reassessed. The best time to assess your data warehouse is when you are not certain which direction your company should go in.

Another time to assess your data warehouse is when your company is running into problems. It should also be assessed if you notice that it is lagging in certain areas. As technology continues to advance, you will want to assess the data warehouse periodically to find out which areas need to be upgraded.

In fact, you or your company may decide that the data warehouse should become the central point in your operation, and you may decide to place an emphasis on knowledge management. Assessing your warehouse is not something that can only be done once. It must be done whenever it is necessary. When you are able to properly assess your data warehouse, you will be able to make good decisions that can allow you company to succeed.

How To Create a Data Warehouse Structure

A data warehouse structure can be defined as the elements and components which make up the database. The structure will show how these components work together, and it may also show you how the database will grow over a given period of time.

While every data warehouse will have a structure, some are highly organized, while others or not. As you can imagine, it is the data warehouses with the best structures that are the most likely to succeed.

When your data warehouse does not have an organized structure, it will not be as flexible as it should be. If it doesn't have a structure, there will be no connections, and the database will be difficult to maintain. If you want your company to compete successfully in your industry, you will need to design a data warehouse structure that is highly organized and efficient. As an example, how well do you think a house would be built of the architects failed to use blue prints, maps, or specifications? Would you want to buy such a house? It is likely that you would not want to purchase such a home. Not only would it be uncomfortable, but it could also be dangerous.

The same principle can be applied to designing a data warehouse. You will need to use a

Page 12: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

"blue print" to make sure it is designed correctly. All of the components should be designed, and you should know how the system will grow as it is used. Because the design of a data warehouses is a new concept, there are no standards that have been developed. In fact, the terminology which is related data warehouses is still being developed. It is these things which currently make data warehouses challenging to build. However, it is crucial that you build them with the right structure. You cannot afford to build a database that has a structure which is disorganized.

The first important thing to realize is that the structure of your data warehouse should be directly connected to your business. For example, if you need to design a data warehouse that will update each night, this will need to be built into the structure. To build this function into the structure, you will need to have a technical understanding of the system. Some of the things that you may want to build within your data warehouse structure are global availability, customer analysis, data sources, dependability, and daily updates. It is also important to look at the components that will make up the structure.

The two primary components that will make up your data warehouse structure are technical elements and data elements. You will also want to look at business operations, hardware, networking, and operating systems. When it comes to the data, you may need to create a structure that provides information that is related to shipping or billing. The data that exists within the data warehouse must have the same structure. They should also be maintained in the same way. A common issue that is raised among organizations is the decision of whether they show data as dimensional or entity/relationship.

The one you choose is dependent on how your organization operates. Once you have made your decision, you will next want to look at infrastructure architecture. The most important factors that are connected to infrastructure architecture are flexibility, size, and scalability. If you do your research, you shouldn't have a hard time building this. When it comes to the network, you will want to pay attention to the data sources. There should be a sufficient amount of bandwidth available to transfer information. The desktop computers that you use should be powerful enough to run the necessary software programs. In addition to this, the software you use should be easy to transfer between the machines.

An emphasis should also be placed on the technical architecture. It should use a process which is based off a meta data catalog. The data warehouse should be able to pull data from numerous sources, and this data may need to be encrypted or compressed. You will also want to make sure the data is transformed properly, It should be cleaned, integrated, and audited. It is also important to make sure the loading is conducted on numerous targets.

How To Protect Your Data Warehouse

Page 13: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

While many data warehouses are used to access and analyze information, some companies wish to place limits on the type of information that a worker can access. While there are advantages to doing this, there are also some disadvantages as well.

Generally, a company will not place an emphasis on security until after the data warehouse has been built. Before you set up a security system for your data warehouse, it is first important to understand what function your data warehouse is being designed for.

If most of the people who use the data warehouse will only be looking at basic reports, you will want to set up a security system that can accomodate it. While you will want to take some security measures, it is important to make sure you don't add too much. When you set up a security system for your data warehouse, there are three areas you will want to pay attention to, and these are analytical, consolidation, and standardized reporting.

The analytical aspect of data warehouses is the thing you will hear about the most. It is especially important during the planning stages. Despite this, over 70 percent of data warehouses are built with standardized reporting in mind. There will only be a handful of people in most companies that will know how to make plans based on the information they obtained in a analytical warehouse. While these people have a tremendous amount of skill and knowledge, they only comprise a small part of those working with data warehouses.

Trying to use advanced security measures in an analytical data warehouse is generally worthless. However, standardized reporting is a different issue. With standardized reporting, a security system is not an option. The reason for this is because it is this area of the warehouse that will have the most activity, and it is the most vulnerable to performance problems. A data warehouse that places an emphasis on consolidation seeks to integrate the information that it contains. Despite this, some companies may choose to merge the data into a single source. When multiple sources of information are brought together, security will become a complex issue.

As you can imagine, the financial information within a data warehouse will need to have a different level of security than the information that is related to inventory. In addition to this, different departments within a company may have their own levels of security. In order for you to secure your data warehouse, it is important to make sure that each form of information has its own security system. If you are thinking of adding security to your data warehouse, you will need to decide where the security system should be added. The two places which are commonly chosen are the database level and application.

Placing a security system within an application is efficient because it can be connected to the data that is being processed by the application. In addition to this, the actual functions of the program can be secured as well. After the application, the next best place to add security is the data warehouse itself. When security is added to the data warehouse, all the computer programs will be secure. The one that you choose will be dependent on a number of factors. If you are using more than one program within the database, it may be best to use a database level security program. You may also want to use a database level

Page 14: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

security system if more than 100 users will be accessing the data warehouse.

Another thing that you will want to become familiar with is the security table. The security table will contain characteristics that are secured along with the identification of the user. The table will hold values which are related to the information that the user is allowed to access.

You may find that the security table can become the largest table within the warehouse. However, the security table can play an important role in making sure your information is secure. When you secure your data warehouse, it is important to make sure the right levels of security are set up. Each type of information within the database will need to be secured in a different way. The speed of your data warehouse may also be slowed.

How To Manage Current and Historical Information Within Your Data Warehouse

In order for a company to use a data warehouse successfully, it must be designed so that users are able to analyze historical and current information. There are a number of things that will result from this technique, and they will have an effect on the data model design and ETL functions.

As an example, a finance company may need to analyze their profits for the last three years. Looking at the data from the last three years will allow the user to view the transformations the company has gone through. Unfortunately, only having a current view will not allow the user to get the information they need.

One of the biggest challenges that data warehouse managers face today is the issue of how to manage dimensional tables over a given period of time. Not only must they be able to do this, but they must also be able to manage this information with current data. There are a number of basic modeling techniques that are used, and one example of this is slowly changing dimensions, which can come in type 2 or 3. The SCD 2 technique can be utilized to display a dimensional table when a change within similar columns needs to be analyzed over a given period of time. The SCD 2 method will use keys within the table that will not change.

Whenever a change occurs, a new key will be added to the table. If meta data columns were added to the structure of the table, the new keys can be tagged as current while the records which were loaded in the past can be tagged as being historical. The SCD 2 is a technique which will solve part of the problem, because it will add historical information into the dimension. The queries that are run within the table will generate the correct historical views, and can use the keys from both tables. The second common element that is used to generate historical information is called SQL, or Structured query language.

The goal of using SQL is to produce facts which are grouped together over time. The information is grouped based on iterations and the keys that the dimensions have moved

Page 15: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

through. Using SQL may not be helpful in a situation where the goal is to overlook historical changes which have been made to a table. It may also not be useful when all the facts that must be displayed must be connected to current information. The meta data can find the rows in the table and ignore certain types of data. The fact table will be connected to the historical keys. If the current dimension is constrained, and the fact table data is retrieved, the reporting results may not be correct.

Intricate SQL techniques can be utilized in order to gain the data on the current row. As the same time, you will still be able gather the fact table rows which are connected to the production key. The success of SCD 2 is dependent on abilities of the tool. The use of the meta data will also play a role in its success. Another method that is used to balance historical and current data is SCD 3. Specifically, SCD 3 can model a dimension table to grab the current and historical changes which have been made on a key. It will use two columns within the table, and while one of them will be for current data, the other will be used for historical data.

The fact tables are connected to a single row, and this solves the problem of using multiple keys which is found when using SCD 2. A database management system can be utilized to identify either the historical or current columns.

It may also be possible for other tools to use alternative techniques. Another technique that is used to manage historical and current information is SCD 2+. With this technique, you will simply manage two sets of tables, and one will deal with current facts while the other will deal with historical facts. If you want to use SCD 2+, you will need a tool that can use the dimension tables that will be used to issue reports.

How To Use Data Warehouses Strategically

It is not enough for a company to simply acquire a data warehouse. Many of them have done this, and they have still not been able to use it successfully. Once you purchase or build the data warehouse, you must learn how to use it strategically.

There are a number of ways you can use a data warehouse in order to make important decisions. The data warehouse is merely a tool. It will not be useful to you and your company if you don't know how to use it. One of the most important aspects of data warehouses is time. A company who knows how to use their data warehouse will be able to compete over time.

To successfully use your data warehouse over time, it is important to understand the past, present, and future. A company can analyze the information within their data warehouse to learn about the mistakes they made in the past, and they can find ways to avoid repeating the same mistake in the future. When it comes to the present, companies must be able to analyze information related to current issues so that they will be able to deal with them. When it comes to a future, a company must be able to study the data of the present to make smart investment decisions which will allow them to profit tomorrow.

Page 16: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

Companies who use data warehouses will also want to place an empasis on learning. This is process that is ongoing, and it should never stop. While some mistakes may be made, they should not be repeated. It is also important for a company to maneuver quickly to find gaps in the armor of their competitors. Companies should use data warehouses to figure out the best ways to market their products. Employees should be given all the necessary tools and training to allow them to use the data warehouse. If they don't like it, or they find it too complicated to use, they won't use it. If this happens, the company has wasted money on a tool that their workers don't want to use.

When it comes to strategic thinking, time plays an extremely important role. Data warehouses are directly connected to time. A company will study patterns, connections, and changes over a given period of time in order to make important decisions that can allow their company to succeed. A data warehouse is a tool that can allow companies to measure their success and failures. They can measure marketing strategies and other important information. While data warehouses are important tools to have in the information age, they are not the cure to all the problems a company or organization will face. Many organizations are under the false impression that data warehouses will automatically allow them to gain an edge on their competitors.

In reality, the data warehouse will only give your company an edge if you know how to use it. It is likely that your competitors are using data warehouses as well. The company that uses their warehouse with the most efficiency will be able to dominate in the market place. A data warehouse alone isn't enough. It is simply a place where you store information. A data warehouse can become a powerful weapon when it is combined with tools such as data mining. A company that uses a data warehouse will need to learn how to analyze information. Once they've done this, they will need to make decisions based on the information they've studied.

Another issue that companies will want to look at is security. There are two ways that you can set up security for your warehouse. You can create a security system that covers all the data, or you can create different security systems for different types of data.

The latter is generally the best, because it is not always good to have too much security on certain types of data. For instance, you will want to place a higher level of security on your financial information than on marketing plans. However, the option that you choose should be dependent on your needs. It is also important for a company to decide how the data warehouse will be designed. There are multiple design approaches, and you will want to choose one that is useful or your organization.

The Difference Between Data Mart and Data Warehouse

The biggest decision facing most IT managers today is whether or not they should construct the data mart before the data warehouse. Many vendors will tell you that data warehouses are hard to build, as well as expensive.

Page 17: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

If you listen to some vendors, you may be left thinking that building data warehouses is a waste of time. However, this view is inaccurate, and any data mart vendor that tells you this are looking out for their own best interests.

The problem with many data mart vendors is that they see data warehouses as barriers which stop them from earning a profit. It is natural that they would tell you about all the disadvantages you will encounter when trying to build a data warehouse. Some vendors will even tell you that you can create a data warehouse by simply building a few data marts and allowing them to grow. However, there are a number of problems you will run into by using this method. When data warehouses were first advertised, data mart companies tried to tout their products as being the same product.

Unfortunately, many people were confused by this, especially those that went to trade shows. Many of these customers purchased data marts and begin constructing them without data warehouses. As they continued constructing data marts, they begin to realize that the architectural structure was flawed. There are a number of reasons why you will want to build data warehouse. If you only use data marts, the information between data marts will become redundant. The information that is presented from each data mart will be different, and this will demonstrate inconsistency.

People who use data marts will have a hard time managing the interface between them. Because of this, trying to use data marts in place of a data warehouse will not give your the results you are looking for. Despite these problems, many data mart companies are not willing to admit they made mistakes. Instead, they are now trying to say that data warehouses are merely a collection of data marts. Again, this is not correct, and will cause confusion among customers. There is not way you can purchase a collection of data marts and grow them into data warehouses.

It is also important to realize that data warehouses and data marts are not the same thing. There are some notable differences between the two. A data warehouse has a structure which is separate from a data mart, even though they may look similar. A data mart is a group of subjects that are organized in a way that allows them to assist departments in making specific decisions. For example, the advertising department will have its own data mart, while the finance department will have a data mart that is separate from it. In addition to this, each department will have full ownership of the software, hardware, and other components that make up their data mart.

Because of this, it is difficult to coordinate the data across multiple deparments. Each department will have its own view of how a data mart should look. The data mart that they use will be specific to them. In contrast, a data warehouse is designed around the organization as a whole. Instead of being owned by one department, a data warehouse will generally be owned by the entire company. While the data contained in data warehouses are granular, the information contained in data marts are not very granular at all.

Another thing that separates data warehouses from data marts is that data warehouses

Page 18: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

contain larger amounts of information. The information that is held by data marts are often summarized. Data warehouses will not contain information that is biased on the part of the department. Instead, it will demonstrate the information that is analyzed and processed by the organization.

Much of the information that is held in data warehouses is historical in nature, and they are designed to process this information. As you can see, there are many differences between data marts and data warehouses. It is important to make sure you're not confused. Many people purchase data marts thinking that they are data warehouses, but this is not correct.

How To Properly Manage a Data Warehouse

When you manage your data warehouse, you will want to do more than simply place an emphasis on maintaining the data. You will want to maintain the data warehouse in a way which is directly related to caring for your customers.

When you maintain your data warehouse, it is important to place an emphasis on measurements. If you don't take the time to make measurments, your information and views will be subjective. Measuring your data warehouse will allow you to determine if you are improving as a company or organization. However, there are specific areas of a data warehouse that need to be measured.

One example of this is processes. You will want to spend time measuring the activities that you are carrying out. It is also important for you to measure these activities over a given period of time. When it comes to data warehouses, measuring a process is distinct from measuring a product. It will cost your company less to develop a measurement program than you will have to pay if you measure data improperly. It is also important to place an emphasis on planning. Your company must be able to deal with the many problems it will run into. To successfully maintain your data warehouse, it is important to understand where the true value lies.

The purpose of having a data warehouse is to allow you and your company to analyze the data you've collected and make decisions which are based on it. If the data is not organized in a way that makes it easy to analyze, your company won't be as competitive as it could be. A data warehouse is not a panacea, and it is not the solution to all of your problems. It is a database that contains large amounts of information. It will evolve as you continue to use it over time. To successfully manage your data warehouse, you must take the time to understand any mistakes that have been made.

To do this, you will need to study the way you measure quality. When you measure your warehouse properly, it will be high in quality, and it will make your operation run smoothly. There are a number of different ways you can measure your data warehouse. However, there are three common methods that are used, and these are economic success, technical success, and political success. As the name implies, economic success is a method you will use to measure the financial impact that the data warehouse is having on

Page 19: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

your company. It will allow you to find out if your company is gaining or losing profits.

Political success is a measure of how much people like the data warehouse. If they never use it, this is a sign that your data warehouse is not politically successful. Technical success is the method of measurement that is the easiest to perform. Despite this, it is important to make sure you don't use too much technology. You should be practical with the technology you use, and you should be able to apply it to a given situation. Your data warehouse will have an impact on the quality of your business. Every company that wants to succeed should have goals. In order to make sure that these goals are being accomplished, they should look at their data warehouses to determine if they are on such a path.

The data warehouse that you use should support your business plans. It should be able to effectively help you develop tactics that can be used to increase your profits. What is the quality of the information that is provided by your data warehouse?

No matter how much information you have, it is of little consequence if it cannot be used. Because of this, it is important for you to make sure you are using the information that is held in your data warehouse. The quality of the data is not important unless it can help you improve the quality of your business. The information you store will play a role in the political success of the data warehouse. If the workers to do not know how to utilize the information that is placed in the data warehouse, it is not likely they will use it.

How To Manage Meta Data Within a Data Warehouse

Many large companies are now implementing data warehouses. They may use them to analyze specific areas such as customer service or financial issues. Many companies will create multiple goals that they will work towards by using data warehouses.

While this approach has worked well for some companies, this tactic is beginning to show a number of notable weaknesses. Both data and meta data are used with numerous data warehouses, and many of the technicians who manage these warehouses are trying to figure how to process and organize the meta data.

One of the main problems with contemporary data warehouse management strategies is that information changes rapidly. Because of this, it is difficult to be consistent when managing data warehouses. For example, what will a data warehouse manager do when the developer of an application decides to change definitions within the application? The impact that can be created by this and other issues has caused many organizations to rethink their data warehouse management strategies. Many of them are not placing an emphasis on the management of meta data. Being able to process meta data across a wide variety of different warehouses will make things easier.

One tool that can allow data warehouse managers to deal with meta data is called a repository. By using a repository, the meta data can be coordinated among different

Page 20: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

warehouses. By doing this, all the members of the organization would be able to share data structures and data definitions. The repository could act as a platform that would be capable of handling information from a number of different sources. This information could include programs from companies like Microsoft, IBM, or Oracle. In addition to this, the repository could also handle a wide variety of different tools.

One of the best advantages of using a repository is the consistency that will exist within the system. It will create a standard that can be understood among a number of different departments. If a new definition is created for a data mart implementation, a repository can support the change. A number of different departments would be able to share this information. In a nutshell, the transfer of information will become easier. It is important for the repository to function well over the lifecycle of the data warehouse. In order for this to occur, it is important to make sure the database and information is documented. A legacy model can help you in this area, and it can assist you in building a powerful data warehouse.

A repository can help data warehouse managers in a number of different ways. It can help you in the development phase, and it can also help lower the cost of maintenance. There are a number of things you will want to add to the repository to make it operate smoothly. One of the things you will want to add is a database management system. Using an industry standards tends to be better than a DBMS that was created by a vendor. You will have a number of advanced tools that can assist your in managing your database. In addition to this, it can also help you in the reporting process.

When you use an industry standard database management system, you can place an emphasis on the repository. If you decide to use a repository, it is important to make sure it is based on an entity/relationship structure. When you use an industry standard, it will be easier for your to customize information. There are a number of meta data extensions that should be supported by your repository. Some of these are adding an entity type, modifying an entity type, or modifying relationship characteristics. It will also be helpful if the vendor works with the Microsoft Open Information Model. Another thing that you can combine with a repository is API or application programming interface.

When an API is used, it will be easier for a company to custom build a meta data management system. The meta data will be separated from other tools, and this will make it easier to modify. Even if the data changes, there is no need to change the tools. By using these tools and strategies, a company can produce a data warehouse that is much more powerful and efficient.

Federated Data Warehouse Architecture

Page 21: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

Federated data warehouse architecture is a system that works with numerous data mart systems, analytical applications, and operational data stores. A federated DW architecture is a system that is composed of multiple architectures.

Many experts will tell you that there are a number of advantages to using a centralized data warehouse system. A federated data warehouse architecture will share information among a number of different systems. Critical master files will be shared, and the other system will be able to use this information. The federated data warehouse architecture will work with an ETL tool. The ETL tool will host a meta data repository.

When many people first hear about a federated DW architecture, they will often wander if it is the same as a bottom-up dimenison approach. Though the transfer of common data is the same, the federated DW architecture will have unique components that will hold feeds for numerous types of data. These feeds will not be shared by other components. The federated DW architecture has a data sharing system which is not as clean as the bottom-up approach. In a federated DW architecture, the shared information may be taken from the mid-section of the system instead of the source. This architecture was designed to be an orthodoxed solution.

There are a number of ways that a federated DW architecture can be built. The first thing you can do is document the data warehouse system that you're already using with an enterprise data warehouse architecture. At the zenith of this system is a diagram that will show you the numerous systems and meta data that is exchanged between them. After you have done this, you will next need to document your current data warehouse system based on the data flow. The data flow will come from multiple sources, and it may also come from transformations or meta data repositories. Each element will need to be rated based on the quality and accessibility.

Once this has been done, you will need to figure out which data is useful for numerous systems. An example of this would be adding financial information to advertising data. This will allow your company to better market its products to customers. Now that you've done this, you will need to gather the candidates from the third step and study them to determine how important they are. There are a number of build assessments that are available online. Generally, it is best to choose the candidate that has an excellent balance between risk and impact. While you don't want too much risk, it is also important to make sure you don't have too large of an impact. There are a large number of architectures that can be used with a data warehouse, and each will have its own advantages and disadvantages.

The last thing you will need to do is create an iteration which is connected to the federated DW architecture. It will need to be reserved for the best candidate. Despite many of the disadvantages that are associated with this system, it is one of the best for those who want a powerful data warehouse. A data warehouse is an important tool that can allow a company to profit. The information can be stored in a central location, and it can be reviewed and analyzed. Once the information is analyzed, an organization or company can make important business decisions that can allow them to compete. Data

Page 22: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

warehouses are extremely advanced, because a large amount of space is necessary to store the data that must be reviewed.

A data warehouse is an indispensable tool for large companies and organizations. While information can be useful, it means little if it cannot be stored and analyzed in a useful way. If the information can be organized, it can be studied, and this can allow a company to develop a number of strategies that can be used to solve problems.

A company can also learn more about their customers, transactions, and profits. A company can study this items to find patterns which can allow them to improve the quality of their business. In addition to this, the company can learn how to avoid costly mistakes. Information can provide the knowledge that companies need.

Historical Information About Data Warehouses

Data warehouses have become an important part of information technology since the 1990s, and it could be said that the existence of data warehouses are a result of the Information Age. As the global competition between corporations continued to grow in the 1980s, it became crucially important for large corporations to process data and use it for the purpose of making strategic business decisions.

At the most basic level, data warehouses are somewhat similar to databases. They are used to store information that a company can use for making important decisions. However, while databases are typically used for one domain, data warehouses can be used to process multiple domains.

Data warehouses are designed to provide information about the company as a whole. They can provide executives with crucial information that will allow them to compete on a global scale, and the data warehouses can also be used to find trends and relationships between various entities. The data warehouse is proficient when it comes to dealing with database queries. They can pull information from various databases, and this can be done in very unique ways. Whether you are the owner of a small, medium, or large business, there can be no doubt that you have questions about your company that you would like answered. Many of these questions, especially those that deal with trends and business processes, are difficult to answer if you do not utilize computerized tools.

It has been said that information is the ultimate weapon, and it is hard to argue with this assumption. Throughout most of history, getting a hold of useful information was the biggest challenge to companies and organizations. However, the advent of the Information Age has changed this. While information has become much more easy to acquire, processing this information and using it in strategic ways has become the fundamental challenge. It is no longer simply enough to store information in a database. While this may work for some small businesses, it is futile for multinational corporations, entities that process millions of transactions within the course of a single day.

Because the competition between these companies has become so fierce, it has become

Page 23: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

imperative for these companies to spend a great deal of time analyzing their data. By finding various relationships between their customers, transactions, trends, and business processes, this allows the corporation to make strategic decisions that can enhance their profitability. One of the most powerful benefits of data warehouses if the ability of company managers to rapidly pull information from them within a short period of time. In most cases, data warehouses will be read only, and they are designed to answer specific questions. In this light, data warehouses can be seen as a type of digital business advisor.

While databases are used primarily to store information, data warehouses will analyze information, and they can deal with specific subjects. While data warehouses are very powerful if they are utilized properly, they can cause problems for a company if they are not implemented properly. Many of these problems will be financial in nature. Simply put, a failure to properly implement a data warehouse can cause a company to lose tens of millions of dollars. During the 1990s, many large companies learned the hard ways of implementing their data warehouses. Another problem that plagued the industry during this time was the cost. Like all forms of new technology, data warehouses where expensive when they were first introduced.

While they are still quite costly today, there were not in the price range of most small to medium sized businesses during the 1990s. Only Fortune 500 or 1000 companies could afford them, and it is to these companies that vendors tailored their products.

However, since many large companies have now implemented a data warehouse, a market has been created for small to medium sized businesses, and the costs involving with implementing a data warehouse have dropped substantially. While it was not possible to implement a data warehouse for $100,000 during the 1990s, this price range is now available today.

Whether you are running a small or large business, there are a number of reasons why you may want to consider using a data warehouse. They have played a fundamental role in the advancement of technology, and they will change the face of global competition for many

Data Warehouse Business Principles

While data warehousing is a promising technology, it can become problematic for companies that fail to use core principles. In addition to having a proper design, data warehouses must be properly maintained and implemented.

To do this successfully, there are a number of important principles that companies will want to follow. The first thing that a company will want to have is a consensus within organization. The organization should be guided through the process of setting up the data warehouse, and the employees and managers should be able to understand the purpose in using it. A failure to create a consensus within the organization is one of the key reason why many data warehouse projects fail.

Page 24: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

The next business principle you will want to focus on is data integrity. It could be argued that this is one of the most important aspects of data warehousing, and of business intelligence in general. If a company wants to succeed with their data warehousing project, the data must have a high level of integrity. The process of data integrity begins when the data warehouse is constructed. It should be constructed in a way that reduces the chances of the data being duplicated or inconsistent. The data should also be highly integrated. Many companies will create a methodology for data integrity, and this is fine so long as it is done with the end result.

Another principle that companies will want to pay attention to is the efficiency of the implementation. When this is done properly, the costs involved with setting of the data warehouse will be much lower, and the needs of the company can be met early in the process. The design for the data warehouse should be simple to implement. The implementation is one of the most fundamental aspects of creating a data warehouse. Even if a company has a data warehouse that showcases an impressive design, the data warehouse project will become tedious if this design is not easy to comprehend. Most importantly, a project like this can become costly.

When a data warehouse is designed, it is important for a company to focus on simplicity. Everything should be done from a practical standpoint, and this will allow the company to stay within its budget while helping it achieve its needs. Another pivotal aspect of a successful data warehouse is the user friendliness. Countless data warehouse projects fail because the data warehouse is not use friendly, and workers and managers do not see the need of using it in place of their traditional systems. Many vendors focus too much on the technicalities of the data warehouse, and fail to take the end user into consideration.

The best way to make the data warehouse user friendly is to create a standard front end that is used throughout the company. This front end should be based around security and the roles of the users. At the very least, the system should have a minimum learning curve. While this will be difficult to achieve given the complexity of the data warehouse, it is generally best for it to be within the range of users of have the least technical skill. After user friendliness, another core data warehouse principle is operational efficiency. As the name suggests, this is the efficiency of the data warehouse itself.

The operational efficiency of a data warehouse is closely related to its implementation. Once the data warehouse has been implemented, it should be much easier to support. Any business request that are made should easily be taken care of, and errors shouldn't be of major concern.

The implementation of a data warehouse plays an important role in its operational efficiency, because if the implementation is done correctly in the early stages, the operational efficiency of the data warehouse will be much more efficient over the long term. There are certain IT principles that a company will also want to consider, and the most important is scalability.

Page 25: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

Many companies run into problems when they try to add scalability to the design of their data warehouses. The best way to deal with the scalability issue is to build it into the system from the very beginning. This will alleviate any problems that may be caused by design inconsistencies.

Understanding The Data Warehouse

To understand the data warehouse, it is important for you to realize that it is not a single object. It is more of a strategy or a process, an integration of various support systems and programs that are knowledge based. The goal of using a data warehouse is to allow businesses and organizations to make strategic decisions.

The data warehouse is a tool that is designed for the long term. It takes the data from the organization as a whole, and presents it in a way that allows users to use it for a number of different purposes. In most cases, the information obtained from a data warehouse will be used for strategic analysis. Some of the things which it is commonly used for include market research, forecasting, and the identifications of trends.

Data warehouses are specifically designed for business executives and anyone who has the responsibility of making strategic decisions. By using a data warehouse, a company will have access to information that is detailed and consolidated. This information will be taken from various sources, and while some of this information is external, other parts of it are internal. Despite this, many people make the mistake of believing that a data warehouse is merely a tool that is used for collecting data and making reports on it. To run a data warehouse efficiently, the users are expected to have a great deal of technical skill. In addition to this, having business skills are useful as well.

To use a data warehouse, the user must be able to correctly identify the business information that is comprised in it, and they must also be able to set priorities for the information that is stored in the data warehouse. The data within the data warehouse will be broken down into subject areas, and the user must be able handle each of these areas efficiently. It is also important for the data warehouse to be scalable. This scalable structure will play an important role in the foundation of the system, and the proper hardware and software must be implemented. In addition to the data warehouse itself, the maintenance of the data is equally important.

While operational databases played an important role in the past, they are not used directly for information processing within modern data warehouses. In most cases, they will merely act as a repository for data, and they will also play a fundamental role in information processing. There are a number of reasons why a company should want to separate operation data bases from those that are information based. One of the most important reasons for this is because the users of both forms of data are different. While analysts will often be responsible for working with informational data, administrative employees will spend more time working with operational data.

The data must be cleaned and transformed in a way that allows it to remain accurate. The

Page 26: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

consistency of data within the data warehouse is extremely important. Summarization also plays an important role in the function of the data warehouse, and it is important for the user to make sure the correct level is created, thus allowing the organization to make important business decisions. Another issue with many data warehouses is user friendliness. If the employees have a had time utilizing the capabilities of the data warehouse, this could limit the success of the company. In some cases, data warehouse projects have failed because the tool was not user friendly, and millions of dollars were lost.

Because of the complexities surrounding the data warehouse, the user must be educated in how to efficiently use it. A help desk will generally be of great use, and designing tutorial for the user can be helpful as well. The success of a data warehouse is not just dependent on the tool itself, but it is also dependent on the implementation and how the company educates the employees in using it. Before the introduction of data warehouses, most companies used various databases to store information that was related to transactions, reporting, or other business processing. In addition to this, the technology for both database types are inherently different. The information data is more closely related to historical trends, and this is not a necessity for operational databases.

The Benefits of Data Warehouses

There are a number of reasons why many large corporations have spent large amounts of money implementing data warehouses. The most fundamental benefit of using data warehouses is that they store and present information in such a way that it allows business executives to make important decisions.

Instead of looking at an organization in terms of the departments that it comprises, data warehouses allow business executives to look at the company as a whole. Another benefit of data warehouses is their ability to handle server tasks connected to querying which is not used by most transaction systems. The vast majority of companies wish to set up transaction systems so there is a good chance that these transactions will be completed within a desirable time frame. The biggest problem with reports and queries is that these entities can reduce the chances of a transaction being made within a good time frame. It should also be emphasized that running reports on a server via transaction systems can be quite challenging. Because of these challenges, many companies seek to alleviate the problem by implementing a data warehouse system. Another powerful benefit of data warehouses is that they allow compnies to use data models for querying tasks that are quite difficult for transaction processing.

There are a number of ways that data can be modeled, and the goal of modeling it is generally to speed of reporting. This will often be done via a star schema, and it is generally not recommended for transaction processing systems. The reason for this is

Page 27: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

because certain modeling methods can slow down transaction processing systems. At the same time, the server units may speed up the transaction process, but they will slow down the querying process. Perhaps one of the most important benefits of data warehouses is that they set the stage for an environment where a small amount of technical knowledge about databases can be used to write queries and speed of the maintenance of these queries.

Simplicity plays an important role in the success of a data warehouse, and this is something that companies will want to pay attention to early on. Most data warehouses can be set up in such a way that simple queries can be written by workers who do not have a lot of technical skill. Even then, workers who do not have a lot of technical skill will often run into problems when trying to perform certain tasks. Data warehouses are unique in the fact that they can act as a repository, a repository for transaction processing systems that have been cleaned. The data can be reported against them, and it may not require the transaction process systems to be fixed of calibrated.

Data warehouses can be highly efficient because they will allow the user to make queries of data on a regular basis. This can be done from numerous transaction systems, and it can also be done from outside sources. Before the advent of data warehouses, companies that wanted reports from numerous systems had to produce data extracts and run special logic programs to combine this data. In most cases, this strategy worked fine. Despite this, companies that had large amounts of data may have had problems if they wanted to sort through it frequently. While there are a number of challenges to these scenarios, a company can handle them if they take the time to establish the right procedures.

In older systems, data that was considered to be old would often be removed from transaction processing systems. This was done for the purpose of making the response time easier to maintain. For tasks that required querying, the older data and the recent data may be stored in the data warehouse in a way that gives the user control over the response time. Workers may run into some challenges depending on the information they need. When data warehouses are implemented and designed properly, they can bring a large number of advantages to the companies that use them. The can give the company a forecast on how the company is performing as a whole, and it can allow the executives and managers to make crucial decisions that can help a company succeed.

The Disadvantages of a Data Warehouse

Many vendors will spend a great deal of time talking about the advantages of data warehouses, and why companies need them if they wish to survive in the global market. While there is a degree of truth to the statements that are made by many vendors, it is important for companies to realize that data warehouses are not a panacea, a solution to all the problems a company will face.

Being able to maximize the efficiency of a data warehouse requires the company to look at it from multiple views, and this includes those which are both positive and negative.

Page 28: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

The ultimate goal of a data warehouse system is to store historical information about a company's transactions, and present this informatin in a way that will allow business executives to make important decisions. However, this data may make up only a small part of the information that a company needs to operate, and its value may even be limited. In some situations, the end user will not have a strong interest in older processing data, and much of this data is made available in basic reports. Many of the markets that businesses operate in today are in constant transition. Depending on the conditions, it may not be necessary to use a historical system.

In other words, data warehouses may be too much for most businesses. This is especially true for many small to medium sized businesses that are analyze their transactions with needing expensive programs. One of the criticisms which are commonly made of data warehouses is their complexity. The implementation of a data warehouse can be so complex that it can make the business processes harder to deal with. Some experts have even said that these complications can eventually strangle the business. If a company is not capable of placing a lower emphasis on some processes, the data warehouse can cause the business environment to become much more cluttered.

The second problem that has become quite common with data warehouses is their cost. Like all advanced technology, when data warehouses were first introduced, only the truly wealthy companies could afford them. Even today, most data warehouses are outside the price range of companies that don't fall under the Fortune 500 or 1000 category. While vendors in recent years have begun tailoring their products towards small to medium sized businesses, many of these companies may not see the need of using a sysem that is overly complex. Because of the speed of the business world, many companies aren't patient enough to wait for the implementation of a data warehouse.

In the past, it wasn't uncommon for a data warehouse project to take many months for implementation. In one case, it took a company 18 months to fully implement the system. Most firms today want results, and they want them fast. They don't see the need for waiting months on a system that is unproven, and could inevitably become a failure. The ROI for data warehouse projects will typically be much lower than vendors promise, and it will take time before a company begins seeing a return on their investment. Many firms simply don't have the patience to wait for these returns.

It is also important for companies to pay attention to the data aspect of the warehouse. Because of the complexity of these systems, it could be said that data warehouses can take on a life of their own. It must be emphasized that placing data in a data warehouse for the sake of adding for no specific purpose can reduce its value.

When you combine this with the fact that the costs involved with maintaining the data warehouse can become larger, it is easy to see why companies should be careful in their decision to use it.

Data warehouses are a technology that has brought a great deal of success to many

Page 29: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

companies. However, many vendors paint a rosy picture and fail to talk about the challenges that a company will face. This is done because of the fact that the vendor is interested in selling the product. Companies must analyze the organization carefully to decide if a data warehouse is conducive to their needs. Only after they've done this analysis can they decide if a data

Rules to Use With Your Data Warehouse

Once a company has successfully implemented their data warehouse, it is important for them to establish rules and regulations with which to use it. While different companies will have different rules when it comes to handling their data warehouses, their are some general principles that you will want to pay attention to.

These principles will not only make using your data warehouse easier, but it will also allow the organization to use it much more efficiently. The very first rule of thumb is to realize that data warehouses are challenging to use. Many experts say that at least 30% of the info they give out may not be consistent.

One of the most problematic things about this is the company may not notice the error if they are dealing with an operational unit that is transaction based. Despite this, this error percentage should not be allowed in the data warehouse. When you consider the fact that many large scale data warehouses can cost millions of dollars to purchase and implement, and error percentage of 30% is not acceptable. To solve this problem, it is important for companies to analyze their data carefully before making decisions that are based on it. It is unwise to just accept data as it is, without carefully looking for errors or other problems.

The second rule of data warehouses is to understand the data that is stored. It has been said that knowledge is power, but this is only a half truth. Knowledge that is stored and unused is potential power. Companies will want to perform an analysis each day of the databases that are connected to the data warehouse. To understand the data, the analysts must be able to find relationships among numerous systems. Once these relationships are found, they must be maintained when the data is moved within the data warehouse. The implementation of a data warehouse will often require the user to make some modifications to the schema of the database.

If the user does not understand the various relationships among the systems, they may be prone to generating errors that could compromise the accuracy and efficiency of the system. Another important rule of thumb is to learn how to find entities that are equivalent or equal to each other. One of the most common problems that can occur in a data warehouse is when the same pieces of data appear in various parts of the system with different names. For instance, two departments within the organization may be helping one customer, but the name of the department may be placed in the system twice under different names.

One name could be spelled out, and the other name could be an abbreviation. This can

Page 30: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

create serious problems in the system if it is not corrected, and the best way to solve this problem is to use a data transformation tool. Because many large companies and organizations are comprised of many different departments, serious problems can arise when each them decides to store information in a different way. One of the situations where this occurs frequently is during mergers. To avoid this problem, companies will want to establish a standard database structure. This will make mergers much easier when they occur.

Perhaps one of the most important principles of data warehousing is to use metadata in a way that supports the quality of the data within the data warehouse. Metadata can be defined as "the data about data." It is the data which describes the data within the data warehouse.

One of the biggest challenge that companies will face is trying to harmonize the metadata across multiple vendor tools. To deal with this issue, companies will want to make sure they generate the metadata and use it for interfaces or other products. Look for vendors who are able to integrate metadata from numerous sources that are disparate.

It is also important for companies to make sure they choose the right data transformation products. A data transformation product is a device that extracts, cleans, and loads data into the data warehouse. It will also record a history of this process. The data transformation product is crucially important, and companies must choose the product carefully.

Data Warehouse Issues

There are certain issues surrounding data warehouses that companies need to be prepared for. A failure to prepare for these issues is one of the key reasons why many data warehouse projects are unsuccessful. One of the first issues companies need to confront is that they are going to spend a great deal of time loading and cleaning data.

Some experts have said that the typical data warehouse project will require companies to spend 80% of their time doing this. While the percentage may or may not be as high as 80%, one thing that you must realize is most vendors will understate the amount of time you will have to spend doing it. While cleaning the data can be complicated, extracting it can be even more challenging.

Not matter how well a company prepares for the project management, they must face the fact that the scope of the project will probably be longer then they estimate. While most projects will begin with specific requirements, they will conclude with data. Once the end users see what they can do with the data warehouse once its completed, it is very likely that they will place higher demands on it. While there is nothing wrong with this, it is best to find out what the users of the data warehouse need next rather than what they want right now. Another issue that companies will have to face is having problems with their systems placing information in the data warehouse.

Page 31: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

When a company enters this stage for the first time, they will find that problems that have been hidden for years will suddenly appear. Once this happens, the business managers will have to make the decision of whether or not the problem can be fixed via the transaction processing system or a data warehouse that is read only. It should also be noted that a company will often be responsible for storing data that has not be collected by the existing systems they have. This can be a headache for developers who run into the problem, and the only way to solve it is by storing data into the system. Many companies will also find that some of their data is not being validated via the transaction processing programs.

In a situation like this, the data will need to be validated. When data is placed in a warehouse, there will be a number of inconsistencies that will occur within fields. Many of these fields will have information that is descriptive. When of the most common issues is when controls are not placed under the names of customers. This will cause headaches for the warehouse user that will want the data warehouse to carry out an ad hoc query for selecting the name of a specific customer. The developer of the data warehouse may find themselves having to alter the transaction processing systems. In addition to this, they may also be required to purchase certain forms of technology.

One of the most critical problems a company may face is a transaction processing system that feeds info into the data warehouse with little detail. This may occur frequently in a data warehouse that is tailored towards products or customers. Some developers may refer to this as being a granular issue. Regardless, it is a problem you will want to avoid at all costs. It is important to make sure that the information that is placed in the data warehouse is rich in detail.

Many companies also make the mistake of not budgeting high enough for the resources that are connected to the feeder system structure. To deal with this, companies will want to construct a portion of the cleaning logic for the feeder system platform.

This is especially important if the platform happens to be a mainframe. During the cleaning process, you will be expected to do a great deal of sorting. The good news about this is that the mainframe utilities are often proficient in this area. Some users chosoe to construct aggregates within the mainframe since aggregation will also require a lot of sorting. It should also be noted that many end user will not use the training that they receive for using the data warehouse. However, it is important that the be taught the fundamentals of using it, especially if the company wants them to use the data warehouse frequently.

Crucial Requirements For Successful Data Warehouses

There are certain requirements that companies need to meet if they wish to use their data warehouses effectively. When data warehouses were first introduced in the 1990s, many companies placed an emphasis on defining the data warehouse as a system that was distinct from a standard operational system.

Page 32: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

This view was shared by many companies, and the data warehouse was also seen as being a centralized copy of data that is operational. However, over the last decade, many companies have been to change their perspectives on how they see data warehouses. The 1990s were a decade of trial and error. While there were many successes, there were many more failures.

One of the things that has improved the data warehouse industry is the increasing computer processing power. Technology has also advanced to the point where OLAP engines can focus on pulling out the data rather than placing it within the data warehouse. It should also be noted that the field of dimensional modeling has greatly improved over the last decade. To succeed in the current market, companies need to understand the requirements they must meet if they want their data warehouses to be successful. The first thing companies will want to do is go from a centralized development strategy to one that is decentralized. In addition to this, the development should also be incremental.

One thing that companies must realize is that it is inevitable that smaller departments will create their own small warehouses. Because this practice cannot be stopped, it is important for companies to create a framework which allows these departments to share their information with the rest of the company. Remember, the goal of a data warehouse is to give a view of the company as a whole. Even though individual departments will need their own small warehouses to answer crucial questions, this information should be made available to the rest of the company. Despite this, a department must be able to design their data marts in a unique way.

The second requirement that companies will want to meet is the ability to deal with changes when they occur. The only thing that remains constant is change, and a company must prepare for this. The data warehouse should be constructed in a way that allows it to evolve. It will be frustrating and tedious to have to change the schemas every time the company needs to adjust to a new change. A company must be able to add additional information to their data warehouse without having to modify any of its components. Once this is done, the company can add new information to their system without having to make tedious changes, and they can focus on more important issues.

The third requirement that companies will want to meet is rapid implementation. This follows closely to building a system that is decentralized rather than centralized. In the past, it took companies months and sometimes years to build a data warehouse that was centralized. This greatly increased the costs involved with building the system, and the company wasted a great deal of time. By using rapid deployment, the data warehouse can be constructed in pieces, and it can be done much faster with a high level of efficiency. To do this rapidly, all the parts of the data warehouse should use the same structure.

Once this is done, it will be much easier for the company to construct the parts and index them. Querying the parts would also become much easier. The fourth requirement that companies will need to have is the ability to easily drill to the most basic form of atomic data.

Page 33: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

The vast majority of data marts in the company will need to use atomic data, and it is important for departments to access this information without having to give their employees a great deal of training. Another requirement that a company must have are data marts that when combined can create the totality of the data warehouse. The data marts should be comprised of the fundamental atomic data, because it is inefficient to replicate the data measurements throughout the company.

It is also important for companies to make sure they data warehouses are available 24 hours a day. In the past, data warehouses would be down for certain periods of time, and this led to a lack of efficiency. Having the data warehouses online 24 hours a day allows the company to be highly efficient.

Why Data Warehouses Can Be Useful

A data warehouse is a tool that is constructed to give a specific view of data that an organization or company can gather during the course of carrying out various processes. Data warehouses are useful because they can allow a company to give managers and executives crucial information that will allow them to make better decisions.

In a day and age when the decision of one executive can make or break a company, this is crucially important. Every successful business gathers and records information that is related to their customers and various transactions. Many of these businesses will use an OLTP, or online transaction processing tool.

In the past, it was very difficult for managers or executives to get information about their company as a whole. This is still challenging today for companies that don't use data warehouses. When a company uses a number of different systems, the information they retrieve can be inconsistent. Data warehouses are useful because they collect data and remodel it. The information is placed in a single unit, and the company can get a clear picture of how their company is performing. Most importantly, they will be able to make decisions with a great deal of confidence. Data will be stored in the warehouse from multiple sources. Once the data is stored, it must be cleaned and transformed.

The process of cleaning and transforming data is known as ETL, or Extraction, Transformation, and Loading. Properly caring for the data is an important part of maintaining a successful data warehouse. Most companies store data for the long term, and they follow set rules and procedures. The data warehouse is specifically designed to give managers information about the company as a single entity. Data will be placed in the warehouse periodically, and it will be done in batches. In most cases, the data will be stored at times when the company isn't extremely busy. The data is considered to be non-volatile. One of the most powerful benefits of a data warehouse is the fact that operational forms of data can be optimized for a certain level of efficiency.

One concept that you will want to become familiar with is metadata. Metadata can be defined as the information on the data that is stored in the warehouse. In other words, its data about data. Metadata can be broken down into three categories, and this is

Page 34: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

operational, business, and administrative. The administrative is related to the columns and tables of the warehouse, and it also deals with the rules by which the data is maintained. As the second name implies, business metadata deals with various business terms. This data is especially important to those who will be making the key decisions. The operational metadata deals with the errors, history, and usage. As the name suggests, it deals with the operational issues surrounding the data.

Because many managers in the company will have different needs for the data, many of them will construct smaller data warehouses that are tailored towards certain subjects. These small data warehouses are referred to as being data marts. The data mart will get its information from the central data warehouse that is being used by the company. The last part of the data warehouse is the decision support program. These programs will get their information from the data warehouse, as well as the data marts. They will take the information they are given, and they will use it for querying purposes.

The decision support programs will fall under one of three categories, and these are data mining, SQL, or OLAP. These applications are designed in a way that will allow managers and executives to get important answers to their questions. These answers can assist them in the decision making process. The data can be presented in such a way, that it allows the decision makers to look at summarized data before looking for information that is much more specific. Data mining is quite powerful because it allows an AI or neural network to sift through the dat looking for important trends or relationships, connections that are impossible for humans to find within a short time period. Data mining will typically use logistic regression or specific algorithms.

Fundamental Themes For Your Data Warehouse

While each data warehouse may differ in their size, scope, or complexity, there are certain fundamental themes that they all share. The three important themes that all data warehouses share are processing time, drilling down, and drilling up. If a system has all of these factors, and it runs with a high level of efficiency, it can be truly called a data warehouse.

Despite the fact that all data warehouses are comprised of these three elements, they all lay the foundation for structures that are truly powerful. In this article, I will describe the fundamental themes that make up all data warehouses, and I will explain why they are so important.

The term "drilling down" is used to describe the addition of a row header, particularly within a relational database. In most cases, the row header will be added to a "select" statement. When a user studies the sales of a product at a specific level, such as from the manufacturer, the query will present them with information that is related to the sum, manufacturer, and sales. In addition to this, the query will also have information that is related to time or other units. If you want to drill down further to the brand that the manufacturers sale, you will want to add the appropriate row headers. Once this is done, the data warehouse will show numerous rows which list the brands which are sold.

Page 35: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

A row header will often be referred to as being a grouping column. The reason for this is because all the items that are not connected to an operator such as SUM will need to be identified in the SGQ group with a certain clause. It could basically be said that grouping columns and row headers are identical. If the information contained in the example above is located in a dimensional star schema, the brand unit and the manufacturer unit can generally be found in the same dimension table. Once a user runs a query at the level of the manufacturers, they can see a collection of characteristics for the dimensions of the products. It is also possible for the user to place the attributes of a brand within the query.

Once this is done, the user can run a query again, and they can drill down in a certain manner. If the attributes for the manufacturer and brand are contained within the same dimensional table, then this reduces the number of adjustments that need to be made to the SQL. In other words, this will make the query much more simple. It is crucial for the data warehouse to support drilling down, especially at the user interface level. It is best to use a large amount of atomic data during this stage, and the reason for this is because the atomic data is much more dimensional than other forms of data. It should also be noted that the atomic data is more expressive as well.

When two points are combined together, the atomic data must be comprised of the same schema. The atomic data should be easily accessible, and this will make it easier when the drill down procedure is used. When a company fails to do this, it is one of the leading causes of having an architecture that is a strange structure. The atomic data may be hidden, and it may only be accessed after a user has used the drilling through process.

There are actual some people which support this structure, but most of them can never explain how this happens. These are generally people who have never used query tool that was designed by a commercial company.

There are a number of things a company should do if they wish to build a system for drilling down. They will first need to acquire and use query tools that are ad hoc. These should be tools that will showcase the drill down options without the need for distinct schema programming. Many experts have said that distinct schema programming is the bane of many data warehouses. The reason for this is because each schema will need to have an application that is custom built. This problem was quite prevalent during the 1990s, but this doesn't mean that a company can't fall victim to it today.

What You Should Know About Building a Data Warehouse

As we move further into the Information Age, the global competition among companies has become more fierce, and many of them are relying more on data warehouses to help them make critical decisions. Before a company can use a data warehouse to achieve their own goals, they must first understand how it is built.

Page 36: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

Some of the greatest challenges involving a data warehouse will be seen when it is implemented by the company for the first time. The quality of the data within the warehouse is very important. It must be accurate if the company wants to make good decisions based on it. Some of the things that the company will want to look at is the source of the data, and whether or not it comes from a program that is operational.

If the data does come from an operational program, a company will need to analyze the rules of this data before they allow it to be placed within a program that is operational. It should also be noted that it is critically important for companies to understand the rewards that surround placing data in the warehouse properly. When data is pulled from the Internet, this is of great importance. While some areas of the Internet like e-commerce may be highly accurate, other parts of the Internet may be highly inaccurate. Because of this, one important aspect of running a successful data warehouse is making sure the data has been cleaned.

Once the data has been cleaned, it can be analyzed properly. The maintainence of data plays an important role in the construction of the data warehouse. Dealing with personal names can be very challenging because many people may go by numerous names. Some of these may be legal names or nicknames. The same problem may also occur when dealing with addresses. If a person lives in an apartment, they can enter the information in a number of different ways, and this can cause inconsistencies in the system. To solve these problems, it is important for companies to construct programs that are capable of making correlatoins between data that is similar.

Every company that decides to use a data warehouse must figure out how they will store the data within it. A company must understand that there is a big difference between moving the data into the warehouse versus changing the operational systems in a way that makes them more friendly to the data warehouse. The recent advent of software that can automate the data cleaning process and combine this process with the transport of data via an operational system is useful, and can play an important role in the business decisions that are made by the company. In addition to placing data within the data warehouse, it is also important to make sure that data is defined.

Many of the users may not have a technical knowledge of the warehouse, and it is important for companies to make sure a definition of the data is made. Analysts will want to look at the potential queries that will commonly be made by the user, and make preparations for them. The number of possible queries for a data warheouse are quite numerous, and this is why it is so crucial for the analysts and developers to make sure the data is defined.

Queries which are highly intelligent will make the information that is found that much more valuable. The warehouse schema should be set up in such a way that it allows the largest number of questions to be asked and answered.

This schema should only be limited by the database management system itself. Another issue that companies will want to consider is how updated the data warehouse should be.

Page 37: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

The maintenance of the warehouse should be done based on the volatility of the data within it. For instance, an address that was entered for a customer 18 months ago may not be current, and the customer may no longer live at that address. The ultimate goal of a company should be to design a schema that is close to real time. A company can't afford to make decisions on data that is old or obsolete. When the data warehouse is being updated in near real-time, this will allow the company to make decisions which are highly accurate.

How To Rate Your Data Warehouse

Data warehouses have greatly evolved over the last 10 years. They now have their own transaction systems as well as their own design structures. The dimensional design has become the most prominent method of construction for data warehouses in the 21st century. Despite this, it is important for companies to take the time to rate their data warehouses.

By doing this, the company will have a good idea of the efficiency of their systems. One of the first things a company will want to pay attention to is the ETL, which stands for extract, transformation, and load. It is important for a company to make sure the data warehouse is comprised of parallelism, since this will speed up the ETL process.

Most companies place an emphasis on the data they obtain within the company rather than data that is obtained from external sources. In my opinion, this can be a costly mistake, because external pieces of information can be just as valuable as the information that is gained within the company. When a database is designed, it is important for a company to place an emphasis on external data. When data is brought into the data warehouse, it is crucial for it to be properly maintained. The process of transforming data can be very costly, and processing extensive amounts of data will generally need to be done via VLDBs, or very large databases.

One way to rate your data warehouse is to determine if you've laid down strict rules for the maintainence of data. Data consistency is one of the most important factors in the success or failure of a data warehouse, and it is something that a company will not want to take lightly. It should also be noted that various decisions will be made based on differing views on the information that is stored within the data warehouse. The input data should be consistent with the decisions and the goals of the organization as a whole. When you analyze the various input systems, it is important for you to come up with criteria to determine how much transformation each piece will need.

Another important part of the data warehouse that you will want to pay attention to is the structure of the data, as well as partitioning. Structuring the data properly will be crucial for the total performance of the data warehouse. In some cases, it may be possible to partition certain elements within the data warehouse by breaking data into dual columns. Each column will act as a key, and it can be partitioned to the data that has been split. A partial index key is powerful because it will allow you to either combine or split the data throughout the database. A partial key can be created by simply taking one column and

Page 38: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

breaking it into two columns. However, this must be done carefully so that the end user access is not harmed.

It should also be noted that it is possible to create keys from an algorithm. This is a very effective method when you want to randomize information throughout the database. The second column can be used for the purpose of data partitioning, and it may be possible to spread the data across dozens of segments within the database. The numbers could be evenly distributed in numerous places within the database. The ability for a company to rate its database is critically important. If the data is not managed properly, it could easily lead to disaster. It is important for companies to realize that they will make crucial decisions based on the data that is contained within their warehouse.

If the data is inconsistent, or it hasn't been cared for properly, this can lead to situations where poor decisions are made based on the information that is presented. This could hurt the productivity of the company, and this will inevitably lead to a loss in profits. Being able to effectively implement a data warehouse is not enough. The data within the warehouse must be properly maintained. Having said that, it could be said that a data warehouse can be rated based on the procedures that a company uses when dealing with it. Learning how to rate your data warehouse is very important.

How Data Is Stored Within a Data Warehouse

The data that is stored in the data warehouse is just as important as the data warehouse itself. Having a fundamental understanding of how this data is stored can be useful in the successful implementation and utilization of a data warehouse. One term that you will want to become familiar with is OLTP, or online transaction processing systems.

The OLTP uses the field of data modeling to utilize the Codd laws of normalizing data in order to create a high level of integrity with the data. By using the Cood rules, elaborate information can be split into a structure which is simple and basic. The most basic structure for information is a table.

The Codd laws define 5 important guidelines that must be used if data is to be stored with a 3rd normalization level. When the database uses a design that is highly normalized, the data from a single transaction will often be stored in a large number of tables. Managers who specialize in relational databases are particularly skilled in handling the relationships within the tables, and this allows the data warehouses to perform at a very high level of efficiency. The reason for this is because only a small amount of the data is effected when a transaction is made. The biggest challenge that companies will face is when they need to assemble the various bits of data into a record that can be used for analysis.

OLTP databases are useful because they deal with the information that is specifically related to a single transaction. When a company is ready to analyze its data, they may need to assemble data that comes from hundreds of millions of transactions. As you can imagine, this is a tremendous workload. If enough time is allowed, the program will be able to give the company the results it needs, but it is generally best to keep it

Page 39: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

disconnected from the database that is OLTP based. One reason for this is because the reduced performance of the system when the two are connected. It is also important for the data to be structured and reformatted on a regular basis.

When this is done, the program can be easily used by uses who are novices. One of the most impressive aspects of OLTP databases is they are designed to provide a high level of performance by handling the applications in certain ways. The programmers who design these applications will often be adept in understanding the limitations of the technology. Even if the data warehouse has a high level of efficiency, it will be useless if the wokers are not trained in how to use it. It is also important to realize that the data warehouse must be capable of handling large amounts of data that is collected over a period of time. Because advanced queries will be used, the data warehouse must support formats from various systems.

To understand how data is stored within a warehouse, you must understand the purpose of using the data warehouse. The purpose of using a data warehouse is to bring in data from a number of different databases with the purpose of analyzing it. This analysis will be used for reporting and management. It is best to store the data in its most basic form, because this provides a high level of flexibility in the reporting process. However, it should be noted that there may be times where an emphasis is placed in different requirements. There are a wide variety of methods that can be used for the implementation of a data warehouse. All these approaches can be broken down into two categories, and these are the normalized approach and the dimensional approach.

With the normalized approach, the data within the data warehouse will be held in a third normal form. Once this has been done, the tables will be collected together via subject areas, and these areas will define the data. One of the most powerful advantages of this approach is that it is relatively simple to add new data within the database. With the dimensional approach, the data will be broken down into facts or dimensions. The facts will be number based, and they will define certain values. The dimension will hold reference information.

How Does a Data Warehouse Differ From a Database

There are a number of fundamental differences which separate a data warehouse from a database. The biggest difference between the two is that most databases place an emphasis on a single application, and this application will generally be one that is based on transactions. If the data is analyzed, it will be done within a single domain, but multiple domains are not uncommon.

Some of the separate units that may be comprised within a database include payroll or inventory. Each system will place an emphasis on one subject, and it will not deal with other areas. In contrast, data warehouses deal with multiple domains simultaneously.

Page 40: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

Because it deals with multiple subject areas, the data warehouse finds connections between them. This allows the data warehouse to show how the company is performing as a whole, rather than in individual areas. Another powerful aspect of data warehouses is their ability to support the analysis of trends. They are not volatile, and the information stored in them doesn't change as much as it would in a common database. The two types of data that you will want to become familiar with is operational data and decision support data. The purpose, format, and structure of these two data types are quite different. In most cases, the operational data will be placed in a relational database.

In the relational database, tables are frequently used, and they may be normalized. The operational data will be calibrated in a way that allows it to deal with transactions that are made on a daily basis. Every time an item is sold to a customer by the company, a record must be made of it. As can be expected, this data will be updated on a frequent basis. To ensure the efficiency of the system, the data must be placed in a certain number of tables, and the tables must have fields. Because of this, a single transaction may be comprised of at least five fields. While this system may be highly efficient in an operational database, it is not conducive to queries. In this situation, decision support data is often useful, and it offers support for things that are not readily used by operational data.

If you wish to take out a single invoice, you will often be required to join multiple tables. While operational data will deal mostly with transactions that are made daily, decision support data will give meaning to the data that is operational. The differences between decision support data and operational data can be split into three categories, and these are dimensionality, timespan, and granularity. Dimensionality is a concept which shows that the data is connected in various ways. The data that is stored in a data warehouse will often be multidimensional, and it is much different than the simple view that is often seen with operational data. Many data analysts are concerned with the many dimensional aspects of data.

The timespan deals with transactions that are atomic, or current. These transactions will deal with things such as the inventory movement, or the purchase of an order. Generally, operational data will deal with a short time frame. However, decision support data tends to deal with long time frames. Many company managers are interested in transactions that occured over a certain time period. Instead of dealing with the purchase of one customer, managers are often more interested in the buying patterns of a group of customers. If a sale has just been made, it will not be found in a decision support data warehouse.

Granularity is the third concept that separates operational data from decision support data. Operational data will deal with transactions that have occured within a certain period of time. However, the decision support data must be broken down into different parts of aggregation. While it may be summarized, it may also be more current. The managers within an organization will need information that is summarized at various degrees. Data warehouses have become more important in the Information Age, and they are a necessity for many large corporations, as well as some medium sized businesses.

Page 41: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

They are much more elaborate than a mere database, and they can find connections in data that cannot be readily found within most databases.

Creating an Efficient Process for Data Warehouses

Since data warehouses were first introduced during the 1990s, a large number of companies have failed when attempting to implement and use them. Many of these failures are not a result of the data warehouse itself, but rather the policies and the processes that the company used when trying to implement and utilize it.

It could be said that the early years of data warehousing was filled with trial and error. Today, a number of approaches have been devised which make using the data warehouse much easier and efficient. One approach is called the MDMP, or Management Decision Making Process system. The goal of the MDMP is to simplify the task of dealing with the data warehouse while reaching core goals.

The MDMP is a spherical model that is based on cause and effect. It can be fully broken down into four fundamental parts. The MDMP is excellent for those who approach data warehouses from a business stand point. The first thing that an executive will want to decide with this model is how their business is doing. This is an important question, because it will give you a good idea of how you should approach a data warehousing project. In other words, you should be able to determine if your business is performing good or bad. Once you have established this, you will next want to decide why the business is performing this way.

If your business is doing well, you should know about the factors that have caused this. If your business is not doing so well, you should know about the factors that have caused this as well. The information should be detailed., and you will need to have the data summarized so that you can analyze it with simple tools. The tool that you use should allow you to get to the detailed data that will allow you to see what caused the success or failure. Once you have established why your business is performing the way it is, you will next want to deal with the "what if?" This is important, because it can set the stage which will allow you to make fundamental changes to your business.

If the business is conducted differently, any problems you're experiencing at this time can be solved. One of the most important characteristics of a data warehouse is its ability to give executives a type of forecast on how certain business moves will pan out. One of the best ways to approach this issue with your data warehouse is to set up a low, medium, and high classification for various investments dependent on their risk. Once you have completed this stage, you will now want to invest. You have the idea that can allow the company to improve, and the business manager can be given the necessary capital for the investment.

Once a company or organization has followed all the steps above, they will want to again ask themselves how their organization is performing. Within the data warehouse, it is possible to generate a memory of various initiatives that you have tried, along with the

Page 42: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

results of these initiatives. As you can imagine, this can become very beneficial for the company over the long term. While the basic premise of data warehousing may sound simple, the technical details of it can become exceptionally complicated. Many companies lose sight of the simple premise, and this is precisely why they fail. Remember, the goal of a data warehouse is to help management make better business decisions. You cannot lose sight of this fact.

Many people have asked themselves why data warehouses are so complicated. One of the reasons for this is because data warehouse processes have not been well defined up until today. This products were produced by technical individuals who paid more attention to function than the actual goals of the product. By using the simple model that is discussed in this article, a company can save itself a great deal of headaches and financial losses. In other words, keep it simple. Don't lose sight of the primary goals of the data warehouse, and look past all the bells and whistles. This is what will allow a company to succeed withn implementing this tool.

Understanding Quality Management For Data Warehouses

Quality is an important concept when it comes to data warehouses, as well as their environment. Quality should not be defined in terms of data, even though having quality data is important. When I talk about quality in this article, I'm talking about the big picture.

I'am referring to the success rate of the data warehouse in conjunction with its ability to help the company achieve its goals. In addition to this, it is also important for companies to learn when quality needs to be emphasized before the actual data warehouse is built. A company that wants to succeed must measure what they've already done, along with making the necessary adjustments for the actual measurement of the data.

Quality can simply be defined as reaching the expectations of your customers without going above them. The reason for this is because getting higher levels of quality is costly, and even if you surpass the expectations of your customers, there is no guarantee that you will have a higher rate of return. Some business executive may want to know why taking measurements is so important. If a company doesn't take measurements, everything they perceive will be highly subjective. In other words, a company won't know if they are continuing to improve over time. This is why data warehouses are referred to as a process rather than just a technology or a product.

Because data warehouses are a process, it is process based measurments that should be used. Companies will want to measure things such as "activities" and "lengths." Measuring a process is much different than measuring a product, and a data warehouse much be approached from a process oriented perspective. With a product measure, you will measure things such as the volume of the data you have. While getting quality in your data warehouse will not be free, the costs will be much lower than having a data warehouse with poor quality. The costs that you will have to pay for quality will come in

Page 43: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

the form of re-planning, implementation, and measurments.

It could be argued that re-planning is the most important factor in data warehouse quality. Once the problems of today are solved, and company must be prepared to deal with the problems that will occur tomorrow. It is also important to analyze the value of data warehousing from the business perspective. For business people, the purpose of using a data warehouse is clear: to gain a powerful insight into decisions they can make to help their company become more productive. Based on this, the true measurement of a data warehouse is whether or not the data warehouse can help the business succeed. Over time, the upper management in the company must be able to see progress. If they cannot, the data warehouse project will be considered a failure.

Many companies make the mistake of believing that a data warehouse is silver bullet. They think that by simply using the most cutting edge technology, they will automatically be given an edge in the marketplace. It is attitudes that like that often cause data warehouse projects to become failures. A data warehouse is not one technology. It is multiple technologies combined, and once a company purchases it, it will need to be customized. Most importantly, the company will need to establish guidelines for operating the data warehouse if they wish to run the program efficiently.

It is also important to realize that data warehouses are tools that must evolve. This is precisely why they are often built in an incremental format. Some experts feel that data warehouses is a process of evolution, and they also feel that companies need large scale projects that can be built in three months rather than three years.

Companies that want to produce quality management for their data warehouses must know what they have done right, as well as what they have done wrong. This is where metadata becomes so useful. Metadata can play an important role in the measurement and quality of your data warehouse.

There are three types of success that companies must aim for, and this is political, economic, and technical success. When the data warehouse increases the bottom line, a company has succeeded economically. When the company is using the data warehouse daily, it has succeeded politically. When the right tools have been chosen for the right tasks, the company has succeeded technologically.

How To Evaluate The Software For your Data Warehouse

When a company evaluates software to use in conjunction with their data warehouse, they goal should be purchase software which falls under the best of breed category. The first step in successfully evaluating software for your data warehouse is to do the analysis yourself. You should never rely on someone who is not a part of your organization..

Every technology that you come across will need to be evaluated carefully in order to be of great benefit to you and your organization. You have a greater knowledge of the needs of your company, and this knowledge is superior to an entity that lies outside your

Page 44: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

organization.

Another problem with dealing with outsiders is that they may have biases. If they have a vested (financial) interest in a specific product, they may urge you to purchase it without carefully evaluating rather or not it is truly the product your company needs. The only thing that outsiders should be used for is their knowledge of the things that you can use to evaluate the product. Another important tip of finding good software for your data warehouse is to decide whether or not you have technology within the company that can perform the task. This is crucially important, because you don't want to waste money on something you don't need.

A number of high quality data warehouses systems can be successfully built without having to purchase outside tools. Any time your purchase new technology, it will place a type of burden on your workers in which they are forced to deal with it. If possible, it is much better to use existing technology rather than spend money on new software, software that you workers will need to learn how to use. Because of the costs involved with data warehouse software, you can afford to place a high emphasis on the information provided by the software vendor. You should also get references from reputable sources that can give you an idea of the quality of the product.

Getting references is important, because it allows you to get practical information about the software you're evaluating. Many business executives find that there are a number of operational issues that they find when they begin seeking references. Many of the reference sources you will encounter have spent a great deal of time evaluating the software, and they will tell you about problems that the vendor would "forget" to mention. This information can allow you to make intelligent decisions on whether or not a software product is worth your money. When you talk to the reference sources, you will want to ask them for websites or other places where you can find useful information on the product.

When you want to see the data warehouse software in action, the vendor will often require you to look at a demo. One could way to evaluate this software is to create a test case, and have each vendor follow it. This is very important, because it will allow you to make comparisons between the software products. Making comparisons between them will allow you to find advantages and disadvantages among them. It should also be noted that most vendors will not want to do this unless you are interested in spending a great deal of money.

When you read some technology magazines, you will often here of some pundits talking about the benefits of using certain data warehouse products. My advice to you is to be cautious in their endorsements, as many of these individuals are paid by the software vendors to praise the benefits of using their software packages. One good way to evaluate a software program is to analyze the stock of the company, as well as the industry as a whole.

Page 45: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

This will give you an unbiased view of the company's performance. The software maintenance is also very important. Many data warehousing tools will work with a device called a data dictionary. Find out what will happen if the fields in the data dictionary are changed.

One thing that should be emphasized is that there is no such thing as a perfect software product. Each one will have its own trade offs, and your company must be prepared. Any company who thinks they can "have their cake and eat it to" are in for a rude awakening.

Understanding The Challenges of Using Data Warehouses

While data warehouses can be greatly beneficial to the companies that use them, there are many challenges that a company will face in their implementation and utilization. Some experts have even said that data warehouses are one of most overrated tools in the computer industry.

Many companies decide to use data warehouses beause they simple think that it is the "next big thing," and they don't take the time to think about the requirements they will need to meet in order to use this tools.

Being able to afford a data warehouse is just one of the many requirements a company will want to look at when implementing them. Many of the companies that implement data warehouses are disappointed with its performance.

A large number of these companies fall victim to vendors that promise to help them easily implement and use the product. However, many of these vendors are more concerned with turning a profit than helping the company succeed. To face the challenge of implementing a data warehouse, it is first important for you to understand why they can be challenging. First, constructing a data warehouse is much different than constructing an OLTP system. Many companies compare data warehouses to standard OLTP systems, and this is a grave mistake. Data warehouses are much more complex than OLTP systems. There are a large number of tools in the standard warehouse, and these tools are further broken down into many categories.

It is also important for companies to realize that data warehouses are not core business tools. What this means is that a data warehouse is much more vulernable to the politics that may occur within a company or organization. If the data warehouse does not have the support of the employees, it will fail. Many employees have a hard time using data warehouses because of their complexity, and the companies they work for will often make the situation worse by failing to educate them. It is also challenging for companies to keep their data warehouses in tune with their production units. To make matters worse, many of their developers are not trained in calibrating them.

It should be noted that data warehouse projects fail frequently. While they are great for the companies that properly implement them, they are disasters for companies who are not prepared. Much of the literature that is written on data warehouses is to positive

Page 46: Data Warehousing - dbmanagement.infodbmanagement.info/Books/MIX/Data_Warehousing.doc  · Web viewData Warehousing Introduction . ... Some popular forms of data sources are Teradata,

toward the topic, and they don't spend enough time talking about the negatives of using a data warehouses. The biggest mistake that a company can make is not properly analyzing a data warehouse project before paying for it. By not doing a detailed requirements analysis, a company sets themselves up for failure. They will waste time, money, and the careers of some of their employees may be ruined.

This article is not meant to scare a company away from implementing a data warehouse. However, a company must be aware of the pitfalls involved with using a data warehouse. Don't rely on the vendors to tell you about this, because they will be too busy trying to sell your company the product. Unless they are a good company, your success implementation of the product is of little interest to them. If a company wants its workers to use the data warehouse, they must become familiar with fundamental SQL, since it plays an important role in the construction of many data warehouses. If the user does not understand basic SQL, it will be difficult for them to use the product efficiently.

It is also important for the company to pay attention to the vendor who is selling them the product. Is the vendor reputable?

Do they have a history of helping companies successfully implement their data warehouses?

How long have they been in the business?

It is amazing to see so many companies fail at their data warehouse implementation because they were distracted by the bells and whistles of a product. It is best to avoid doing business with vendors that don't have a proven history of success in implementing a data warehouse. It is also important to make sure your employees accept and understand the use of the data warehouse.

They need to be given an elaborate education on using the program, and this should play an important role in the implementation of the data warehouse.