The use of agile techniques in DW/BI projects

The companies are increasingly more interested in DW/BI solutions that can be implemented and that shows results quickly, that is why the use of agile techniques in DW/BI projects are becoming important nowadays.

The Modern software development processes are iterative and incremental, it means evolutionary.

The concept of agile software development started in the 90's and emerged in 2001 when some software engineers came together to discuss ways to creating better software. They created the agile manifesto and defined some principles.

The Principles behind the Agile Manifesto are:
- Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
- Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.
- Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
- Business people and developers must work together daily throughout the project.
- Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
- The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
- Working software is the primary measure of progress.
- Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
- Continuous attention to technical excellence and good design enhances agility.
- Simplicity-the art of maximizing the amount of work not done--is essential.
- The best architectures, requirements, and designs emerge from self-organizing teams.
- At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

You can use agile techniques in DW/BI

Although the approach to develop DW/BI projects be different of software development, the most of principles of agile can be apply throughout DW/BI lifecycle.

Some important steps that you should to use during the DW/BI lifecycle development are:
- Stakeholder involvement - The participation of stakeholder during all the project is crucial.
- Use an evolutionary approach - Envision the requirements and architecture at a high-level to start, and define the details during the process when they are required, using iterations. You can work from a broad requirement, with capabilities that are identified and determined through a prototyping process.
- Usage-centered approach - use cases and/or usage scenarios and/or use stories.
- Prove the DW/BI architecture works early - you need to prove that the ETL strategy works, you are accessing the major data sources and your front-end tools can access the DW.
- Use requirements to organize your work, not technical issues.
- Deliver some product working in each iteration.
- Check the data quality problems early.
- Test throughout the DW/BI lifecycle - The database regression test is necessary to assure your data, using a test suite to validate your data.
- DW/BI development is complex, you need to use good tools to data modeling, data quality, ETL process and front-end.