Endorse Maximize your ROI, minimize your project risk Consulting
Data Warehouse Project Management
Why is project management for a data warehouse different than most other applications? The answer is that a data warehouse is never really a completed project. Although there are phases with start and end dates, the data warehouse itself never reaches an end state until it is terminated.
Data warehouses are a different breed. They are organic, and need to be nurtured to grow. Being organic, they are ever changing; dynamic in other words. This is what makes project management for a data warehouse so unique and challenging.
There are 2 paths to project managing a data warehouse. The first is to approach the project strictly from a project management perspective by managing the project scope, timeline, etc. The 2nd route is to follow through with the traditional responsibilities of project management such as the project scope, timeline, etc. and at the same time, do a deep dive into the inner workings of the data warehouse.
Understanding in detail what is “under the hood” will allow you to speak with authority on the capabilities and issues of the data warehouse. Intimate knowledge of the data and its relationships is the distinction between a generic project manager and data warehouse project manager.
I’ll also call out that a technical project manager is not the same as a data warehouse project manager because a data warehouse does not function like any other system. Only first hand knowledge of what makes a data warehouse hum can brand someone as a data warehouse project manager.
This is not to be disrespectful in anyway to all the good project managers out there, but the reality is, project managing a data warehouse is very different than other applications. If you have a peer that is a data warehouse project manager, take them to lunch and pick their brain to see for yourself.
Whether you consider yourself a data warehouse project manager or not, the following content is relevant to you as the project manager. You still need a team, solid requirements, and a road map to deliver a quality product on time and within budget.
Project Team Members As a project manager you sometimes have the luxury of putting your team together. Often, your team appears because the prior project has completed and these people are available. Keep in mind what was previously mentioned about the data warehouse being a never ending project. When formulating your team, try to recruit people that you feel will be around for the long haul, not just a quick stint.
Ideally you would like all of your people to have had experience in data warehousing but that is not always the case. At a minimum, you would like to seed your group with key people that have had the desired experience and could spread the knowledge.
Project Manager Expertise Since the project manager is the single point of contact and the visionary of the team, being knowledgeable in data warehousing would be very advantageous. However, as a competent project manager with no data warehousing experience, it is still possible to deliver a data warehouse as planned but this requires strong reliance on the project team for subject matter expertise.
You may be asking the question of “well I am the project manager and I do not need to know the details”. This is true, but to ensure a quality product and to be in a position to speak from authority about the data and workings of the data warehouse, you will need to get your hands dirty. If you are not intimate with data warehousing, at least ensure that a senior member of your team is.
Data Warehouse Business Requirements As with any system, requirements are a good indicator of success (or failure). As the project manager who is overseeing the requirements gathering and analysis, prior knowledge of data warehousing is again a major plus. Knowing what questions to ask other than what you received as requirements will not be apparent. There are significant unspoken words that go into the design of a data warehouse that only prior experience or a true visionary will bring to light. If members of the project team have prior experience in data warehousing, make sure they participate in the requirements review.
To flush out these hidden details, you will need to gather the requirements from your users in bite size chunks based on their attention span and availability. This will be an iterative process because you too will have work to do when gathering the business requirements. As the requirement sessions progress, you should build a mock-up of the application based on what you have to date. This will become your platform for discussion explaining how you view what the customer has requested. This mock-up will help your customer make the leap from concept to reality by allowing them to visualize their ideas.
Your platform mock up does not have to be a working model at this point. A simple slide presentation or static web pages will suffice. Don’t be surprised if your business requirements change drastically at this point because your customers may view what they were asking for in a whole different light now. Before you call the business requirements a wrap, have a walk through of the document to ensure all the content is still required and that no ambiguity exists.
A tip for who should be part of the audience when gathering your business requirements besides your users is representation from development and test. You may not need/want them initially, but you certainly want them involved once you get to the mock up phase. Getting early buy in from all participants is of the utmost importance for any project to succeed.
Data Warehouse Technical Requirements Now that you have a robust set of business requirements, it is time to begin developing the technical requirements. Your technical requirements document should maintain the same structure as the business requirements wherever possible. If you need to deviate, ensure there is traceability back to the business requirements.
One approach that can be used to tie the different documents together is to derive a numbering system that can be inherited by the subsequent documents as a reference as shown in Figure 2. By coming up with a logical structuring of the business document, the task of reconciling business and technical requirements becomes much simpler and more effective. You can also almost guarantee yourself full test coverage if your test case development follows the same methodology.
Figure 1– Requirements Traceability Outline • Business Requirement 1
• Technical Requirement 1 (refers to Business Requirement 1) Detailed Technical Requirements 1.1 Detailed Technical Requirements 1.2
• Test Suite 1 (refers to Technical Requirement 1) Test Scenario 1.1 (refers to Technical Requirements 1.1) Test Case 1.1.1 Test Case 1.1.2 Test Case 1.1.3 Test Case 1.1.4 Test Scenario 1.2 (refers back to Technical Requirements 1.2) Test Case 1.2.1 Test Case 1.2.2 Test Case 1.2.3 Test Case 1.2.4
Once you have the technical requirements to the point where the system flow is well defined, a working prototype should be developed. The prototype does not need to do more than convey the screen logic flow and basic functionality. The intent of presenting the prototype is to ensure the technical requirements accurately reflect the business requirements. The prototype will allow everyone on the development side of the house to gain an understanding of what is to be built. You most likely will be peppered with some tough questions from the crowd but this is good constructive criticism. If you follow this advice, fewer issues are likely to surface during the design phase.
Data Warehouse Implementation You’ve probably have heard this before that you should not try to implement the entire data warehouse all at once. This is so true and besides, no one would want to wait a year or so before they see any results. The project plan should break up the functionality to be delivered in phases of 2 - 3 months or whatever works best for you.
Delivering in phases can be viewed as a Rapid Application Development (RAD) approach which is very effective. The diagram below depicts a single cycle using RAD for delivering your data warehouse. Each phase therefore would be a new cycle.
Figure 2 – SDLC
By delivering in phases you not only deliver something tangible for your users, but you may also flush out issues that can be quickly corrected. Issues that would normally have a downstream impact can be addressed at this time before the added functionality is developed.
Project Manager’s Litmus Test for Success Now that you have rolled your data warehouse into production, how do you know how good a job you’ve done? Finishing on time and on budget is good, but that alone doesn’t guarantee the project’s success.
You know when you have delivered a quality product that produces actionable results for the business when
• Users are knocking on your door constantly for more information than what the data warehouse currently has. This tells you the thirst for knowledge is starting to spread and people have faith in the data warehouse. • The buzz in the hallways has mention of the data warehouse or meetings make reference to it as the source of data. • The data warehouse becomes the heartbeat of the business where decisions are made from the data intelligence it provides.
You will want to build on this success by continuing to nurture the data warehouse. If you emphasized data integrity and flexibility in the design, momentum will keep rolling along as new data and functionality are added to the data warehouse.
Ken Pohl is the president and founder of Endorse Consulting which specializes in minimizing the risk to a project’s ROI by mitigating potential problems before they arise. His many years of developing and implementing data warehousing, business intelligence and CRM systems has provided a wealth of knowledge as proven techniques. These techniques translate into real world not from a book “best practices”.
