DATA WAREHOUSING AND MINIG ENGINEERING LECTURE NOTES--Building a Data warehouse

Building a Data warehouse:

 

The entire mindset of the classical operational system and application developer is dominated by the need for perfection of design and the need to gather all requirements before design and development ensues. Such an attitude may well be proper for the operational application environment where processes are run repetitively, and where requirements can be divined before a system is built. But the data warehouse environment is one where many requirements cannot be discerned until the data warehouse is built and the data in the warehouse is available for analysis. Simply stated, only some of the informational requirements of a corporation can be known before the data warehouse is built.

 

For this reason, the data warehouse is built iteratively, in small, fast bursts of development. And the first small, fast burst of development is not achieved by massively studying and gathering large amounts of requirements before the firs t of development starts. It is imperative that the data warehouse developer have the attitude of moving quickly through the steps of design and development, even though it is known that the processing and data requirements are not complete at the outset of development.

 

The data warehouse environment is built in an entirely different way than the classical operational environment has been built. The classical operational environment has been built on the system development life cycle - the SDLC - that requires that requirements be identified, that analysis proceed, followed by design and programming. Next testing occurs, and is completed by implementation. The SDLC is fine where requirements can be known in advance. But the DSS analyst - who is the ultimate beneficiary of the data warehouse - does not and cannot know his/her requirements. DSS analysts' requirements are discovered in a trial and error manner - in a mode of discovery. There is an entirely different mode of development that is required for data warehouse development.

 

Creating a data warehouse is a significant project with a number of steps. The topics in this section address these steps.

Topic	Description
Designing a Data Warehouse	Describes considerations specific to designing data warehouses and the use of dimensional modeling.
Creating the Data Preparation Area	Describes the creation of the relational database used to prepare data for the data warehouse.
Creating the Data Warehouse Database	Describes the creation of the relational database that holds the data warehouse data.
Extracting Data from Operational Systems	Describes the process of extracting data from operational systems into the data preparation area.
Cleansing and Transforming Data	Describes the process of cleansing and transforming data in the data preparation area before loading the data into the data warehouse.
Loading Data into the Data Warehouse Database	Describes the process of loading data into the data warehouse database from the data preparation area.
Preparing Presentation Information	Describes the process of preparing data in the data warehouse for presentation to users.
Distributing Data to Data Marts	Describes the process of distributing data from the data warehouse to data marts.