Smarter ideas worth writing about.

The Challenges and Successes of Running BI Projects within an Agile Framework

Tags: Agile Enablement


Often BI projects are painful because they can take months, even years to see meaningful data that the business can actually use.  Business Intelligence delivery doesn't have to be a repetitive, vicious cycle of delivering minimal value over long stretches of time.  There is an easier way to pave the road to business satisfaction, but in all fairness, it does come with its own unique challenges.

While it might seem complicated to those familiar with Waterfall managed BI projects, the use of an agile framework for BI delivery works well and offers many benefits to both IT and the business alike.  Introducing agility into your BI projects can bring value and realized ROI to the forefront much sooner, even when change is introduced often throughout the development cycles. 

Running an Agile BI project sets the stage to achieve project success, but having an understanding and preparing for any challenges that might be faced pre-project or throughout any of the iterations, is a key component to the overall success of any project.

Here are some challenges that one might face when running a BI project using an agile framework, in addition to, recommendations for those challenges to prevent such interruptions:

Agile What?

"We don't see how Agile will work for us."

The problem is not that Agile won't work, it's that anything different is considered risky and the predisposition associated with risk is always perceived as negative in general.  Listen to what the concerns are about the delivery approach and educate all direct and indirect project contributors on how those concerns will be addressed through the agile framework being used.  The problem is lack of education and understanding, which can be easily resolved through upfront training and on-going support.  The value will soon be realized within the first few sprints, supporting the fundamentals learned.

The Right Foundation

"We already have a data warehouse in place and there is no need to change the current model."

While many organizations already have systems in place for delivering business intelligence and reporting capabilities, a lot of times those systems may not be utilized best for delivering useful business intelligence to the end user.  The single biggest factor in being able to determine how long and how much effort delivering BI value can take may lie in the type of source systems and the amount of bad data being handled in those source systems.  Once you have a solid understanding of the business and the systems that help you answer your business questions, this allows for quicker delivery with agile development.  The structure needs to align with the long term vision of the business and allow for flexibilities and scalability.  If your current foundation doesn't support this type of flexibility, this could impact your entire reporting solution (data model changes, ETL re-work, reporting layer changes); ultimately redoing everything already done in the past just to support a more flexible reporting solution, so you don't have to deal with it in the future.  Flexible/Scalable Structure + High Performing Quality Data + Agile Development = Win/Win...but remember, the right foundation is always the best foundation.

Offsite Engagement

"We don't see how the business can be fully engaged if they are not co-located with the team."

Not having your product and business owners co-located with the team can pose certain communication challenges for the team, but that doesn't mean communication and engagement is shut off.  Communication is essential to agile development and in setting the right expectations, understanding what the team is supposed to be working on from the prioritized backlog, mitigating risk, understanding business requests for change, etc., which is why daily standups, planning and review meetings, with the use of online tools that can help facilitate communication and progress, are key to keeping a good engagement and communication cadence.  Any and all of these meetings can be hosted online (i.e. Lync, WebEx, Live Meeting, etc. using a cloud based management application such as TFS) in combination with different tool offerings that can help facilitate conversations.  Engagement and communication are at the forefront of agile practices.

An Indecisive Business

 "We're not sure we know what we want."

A business that doesn't know what it wants is a problem regardless of what type of project or development methodology is being used.  Most often this indecisiveness stems from lack of knowledge, unknown options, and unknown impacts of business decisions.  The business needs to be educated, recommendations or solutions need to be presented to help facilitate a decision, and the business must always understand any impacts associated with any decision they make (or lack of decision(s)).  The business is going to make decisions based on what they believe is the best overall direction/option, so feed the right information back to them so they can make intelligent project decisions.  Empower your Product/Business Owner(s) to keep blocked development at bay.

The challenges listed above are just few of the most common challenges that teams might face when delivering an agile BI project.  Challenges can always be overcome by taking proactive steps towards actions that are contributors to success. 

The reason why agility is so important to Business Intelligence projects is because the benefits far outweigh any of the challenges that might be experienced through agile development and definitely outweigh the standard Waterfall approach, which most are accustom to through past BI experiences.

The business and IT are contributors to the success of any project, but this is especially prevalent through agile delivery.  Introducing agility in Business Intelligence is a game changer because the business will constantly see the value and realized ROI through shorter development cycles and there is also a higher satisfaction rate that comes with predictable delivery.  Agile BI successes lie in the details of agile delivery:

Adaptability and Business Value

Change is expected and accepted; those two horrible words "scope creep" don't exist anymore.  An agile framework provides the flexibility to remain aligned with the ever-changing needs of the business, in turn the team delivers constant business value within shorter development cycles.

Risk & Visibility

Agile practices promote open communication through a continuous planning and feedback loop.  The information or needs that business feeds, IT digests and responds, and everyone is driving towards common delivery goals.  There are no development and/or project surprises because everyone is engaged and transparency is at the forefront of every conversation.    Additionally, team collaboration leads to better development which lends itself to less associated project risk.

Predictable Delivery & Customer Satisfaction

Agility can be measured through customer satisfaction.  However, agile development practices promotes adaptability, realized business value, less risk, and visibility, which offers up predictable delivery that the business can come to expect.  

Just remember that there is no secret formula to BI agility, but a Flexible/Scalable Structure + High Performing Quality Data + Agile Development does = Win/Win.  As long as there is constant incremental output of value which is fully aligned with the ever-changing and evolving needs of the business, this is the formula for BI success.


About The Author

Senior Project Manager

Kelley is a senior Project Manager in Cardinal's Tampa office and is passionate about her pursuit to continuously deliver successful projects.