Building Data Warehouses with Agile not Waterfall

Sometime in the recent past some of the processing associated with data warehousing was renamed to something called business intelligence, and the reason was because data warehousing had come of age in that its ownership and purpose was being highlighted as being a business oriented process. So it is now a recognized fact that the business side rather than the operational side of a company has more of a vested interest in guiding what data warehouses deliver to its customers, in other words the business side of a company. The reason why is because a data warehouse can be used to forecast future patterns based on past trends and thus what the data warehouse delivers can have a major effect on how the business is run. One of the biggest development problems in recent years is how to build data warehouses, given that business objectives are often unknown at the start, and often much larger at the end.

The Waterfall approach of functional specifications defines a software product from start through to completion, in a lean step-by-step process that is broken down into parts that can only be developed one after the other. But flexibility is needed because a business can change requirements anywhere during the development process. So the Agile approach allows some ownership by business users as opposed to software developers. The idea is to allow for the business users to change their minds during the development process so as to allow for a product that matches business requirements rather than a technical specification that could be out of date the day it’s printed.