In the Field with our Experts

Succeed Despite Failing

By Stephen Putman, Senior Consultant

I just finished reading a post on the Netflix blog - 5 Lessons We've Learned Using Amazon Web Services (AWS). Even though this article is specific to a high-traffic cloud-based technology platform, I think that it holds a great lesson for the optimization of any computer system, and especially a system that relies on outside sources such as a business intelligence system. 

Netflix develops their systems with the attitude that anything can fail at any point in the technology stack, and their systems should respond in as graceful a way as possible. This is a wonderful attitude to have for any system, and their lessons can be applied to a BI system just as easily:

1. You must unlearn what you have learned. Many people who develop and maintain BI systems come from the transactional application world, and apply their experience to a BI system, which is fundamentally different in several ways - for example, the optimization goal of a transactional system is the individual transaction, while the optimization point of a BI system is the retrieval and manipulation of often huge data sets. Managers and developers that do not realize these differences are doomed to failure with their systems, while people who successfully make the transition meet organizational goals much more easily.

2. Co-tenancy is hard. The BI system must manage many different types of loads and requests on a daily basis while simultaneously appearing to be as responsive to the user as all other software used. The system administrator must balance data loads, operational reporting requests, and the construction and manipulation of analysis data sets, often at the same time. This is the same sort of paradigm shift as in lesson 1 - people who do not realize the complications of this environment are doomed to failure since the success of a BI system is directly proportional to the frequency of use, and an inefficient system quickly becomes unused.

3. The best way to avoid failure is to fail constantly. This lesson seems counter-intuitive, but I've seen a lot of failed systems that always assumed that things would work perfectly - source feeds would always have valid data, in the same place, at the same time, always - that this philosophy gains more credence daily. Systems should always be tested for outages at any step of the process, and coded so that the response is graceful and as invisible to end-users as possible. If you don't rehearse this in development, you will fail in production - take that to the bank.

4. Learn with real scale, not toy models. It would seem that proper performance testing on systems equivalent to production hardware and networking with full data sets would be self-evident, but many development shops look at this as an unnecessary expense that adds little to the finished product. But, as in lesson 3 above, if you do not rehearse the operation of your system on the same size of system as your production environment, you have no way of knowing how the system will respond in real-world situations, and are effectively gambling with your career. The smart manager avoids this sort of gamble.

5. Commit yourself. This message surfaces in many different discussions, but it should be re-emphasized frequently - a system as important as your enterprise business intelligence system should have strong and unwavering commitment from all levels of your organization to survive the inevitable struggles that occur in the implementation of such a large computer system.

It is sometimes surprising to realize that even though technology continues to become more complex and distributed, the same simple lessons can be learned from every system and applied to new systems. These lessons should be reviewed frequently in your quest to implement successful data processing systems.

photo by PseudoGil via Flickr (Creative Commons License)

---

Stephen Putman has over 20 years experience supporting client/server and internet-based operations from small offices to major corporations.  He has extensive experience in a variety of front-end development tools, as well as relational database design and administration, and is extremely effective in project management and leadership roles. He is the co-author of The Data Governance eBook, available at information-management.com.

New Years’ Resolutions: Assess and Revise Your BI Strategy

By Dick Voorhees, Senior Consultant

The New Year is upon us. And for many, the coming of the New Year involves making new resolutions, or reaffirming old ones. This resolution-making process includes corporations and organizations, not just individuals. In terms of personal resolutions, some undertake this process in earnest, but many seem to deal with resolutions superficially, or at least not very effectively. The same is frequently true for organizations as well.

So how then should an organization go about deciding which “resolutions” to pursue in the New Year, which goals and objectives are both worthy and achievable? Often there are no "good" or "bad" opportunities, a priori, but some are more likely to result in a successful outcome and/or have more significant payoff than others.

1. Take stock of the opportunities, and develop a list of key potential initiatives (or review the existing list, if one exists). Consider recent or imminent changes in the marketplace, competitors’ actions, and governmental regulations. Which of these initiatives offers the possibility of consolidating/increasing market share, improving customer service, or represents necessary future investment (in the case of regulations)? And which best supports the existing goals and objectives of the organization?
2. Assess the capabilities and readiness of the organization to act on these initiatives. An opportunity might be a significant one, but if the organization can’t respond effectively and in a timely manner, then the opportunity will be lost, and the organization might better focus its attention and resources on another opportunity with lesser potential payback, but that has a much greater chance of success.
3. Develop a roadmap, a tactical plan, for addressing the opportunity. Determine which resources are required – hardware, software, capital, and most importantly people – what policies and procedures must be defined or changed, etc…

Then be prepared to act! Sometimes the best intentions for the New Year fail not for lack of thought or foresight, but for lack of effective follow through. Develop the proper oversight/governance mechanisms, put the plan into action, and then make sure to monitor progress on a regular basis.

These are not difficult steps to follow, but organizations sometimes need help doing so. We’ve found that clients who call us have learned the hard way – either directly or through stories they’ve heard in their industries – that some careful planning, deliberate program design, and – if necessary – some skill assessment and training can take them a long way in their resolutions for success in 2011. Good luck!

photo by L.C.Nøttaasen via Flickr (Creative Commons)

 

---

Dick Voorhees is a seasoned technology professional with more than 25 years of experience in information technology, data integration, and business analytic systems. He is highly skilled at working with and leading mixed teams of business stakeholders and technologists on data enabling projects.

Do You Know What Your Reports Are Doing?

By Stephen Putman, Senior Consultant

The implementation of a new business intelligence system often requires the replication of existing reports in the new environment. In the process of designing, implementing and testing the new system, issues of data elements not matching existing output invariably come up. Many times, these discrepancies arise from data elements that are extrapolated from seemingly unrelated sources or calculations that are embedded in the reports themselves that often pre-date the tenure of the project team implementing the changes. How can you mitigate these issues in future implementations?

Issues of post-report data manipulation can range from simple - lack of documentation of the existing system - to complex and insidious - "spreadmarts" and stand-alone desktop databases that use the enterprise system for a data source, for example. It is also possible that source systems make changes to existing data and feeds that are not documented or researched by the project team. The result is the same - frustration from the business users and IT group in defining these outliers, not to mention the risk absorbed by the enterprise in using unmanaged data in reports that drive business decisions.

 The actions taken to correct the simple documentation issues center around organizational discipline:

• Establish (or follow) a documentation standard for the entire organization, and stick to it!
• Implement gateways in development of applications and reports that ensure that undocumented objects are not released to production
• Perform periodic audits to ensure compliance

Reining in the other sources of undocumented data is a more complicated task. The data management organization has to walk a fine line between control of the data produced by the organization and curtailing the freedom of end users to respond to changing data requirements in their everyday jobs. The key is communication - the business users need to be encouraged to communicate data requirements into an easy-to-use system and understand the importance of sharing this information with the entire organization. If there is even a hint of disdain or punitive action regarding this communication, it will stop immediately, and these new derivations will remain a mystery until anther system is designed.

The modern information management environment is heading more and more towards transparency and accountability, which is being demanded by both internal and external constituencies. The well-documented reporting system supports this change in attitude to reduce risk in external reporting and increase confidence in the veracity of internal reports, allowing all involved to make better decisions and drive profitability of the business. It is a change whose time has come.

photo by r h via Flickr (Creative Commons License)

---

Stephen Putman has over 20 years experience supporting client/server and internet-based operations from small offices to major corporations.  He has extensive experience in a variety of front-end development tools, as well as relational database design and administration, and is extremely effective in project management and leadership roles. He is the co-author of The Data Governance eBook, available at information-management.com.

Understanding Where Our Work Comes From

By Mary Anne Hopper, Senior Consultant

I’ve written quite a bit about the importance of establishing rigor around the process of project intake and prioritization.  If you’re sitting there wondering how to even get started, I believe it is important to understand where it is these different work requests because unlike application development projects, BI projects tend to have touch points across the organization.  I tend to break the sources into three main categories—stand-alone, upstream applications, and enhancements.

Stand-alone BI projects are those that are not driven by new source system development. Project types can include as new data marts, reporting applications, or even re-architecting legacy reporting environments. Application projects are driven by changes in any of the upstream source systems we utilize in the BI environment including new application development and changes to existing applications. Always remember that the smallest of changes in a source system can have the largest of impacts to the downstream BI application environment. The enhancements category is the catch-all for low risk development that can be accomplished in a short amount of time.

Just as important in understanding where work requests come from is prioritizing those work requests.   The three need to be considered in the same prioritization queue—this is a step that challenges a lot of the clients I work with.  So, why is it so important to prioritize work together?  The first reason is resource availability.  Resource impact points include project resources (everyone from the analysts to the developers to the testers to the business customers), environment availability and capacity (development and test), and release schedules.  And, most importantly—prioritizing all work together ensures the business is getting their highest value projects completed first.

 

---

Mary Anne has 15 years of experience as a data management professional in all aspects of successful delivery of data solutions to support business needs.  She has worked in the capacity of both project manager and business analyst to lead business and technical project teams through data warehouse/data mart implementation, data integration, tool selection and implementation, and process automation projects.

The Price of a Tube of Toothpaste

By Mary Anne Hopper, Senior Consultant

As you can imagine, I travel quite a bit as a consultant for Baseline.  Over my tenure, I have developed a standard routine for getting through the airport.  More often than not, things have gone pretty smoothly for me.  Until this week – my bag was pushed into the extra screening area where it turned out there was an over-sized tube of toothpaste that had to be thrown away.   How did this happen when week in and week out, I use the same bag for my stuff and always get through without a hitch?  Well, I deviated from my process.

You see, the prior week I actually checked a bag and was able to throw a full tube of toothpaste in the ditty bag and I never checked when I was packing for this week’s trip.  I deviated from my standard process.  If you’ve ever implemented a “small” or “low impact” change that has blown up an ETL job, changed the meaning of a field, or caused a report to return improper results, you know where I’m going with this.

Process is important.  Discipline in implementing to that process is even more important.  Am I proposing that every small change go through an entire full-blown project lifecycle?  Absolutely not.  But, there should be a reasonable life cycle for everything that goes into a production quality environment.  Taking consistent steps in delivery helps to ensure that even the smallest of changes do not result in high impact outages.  This can be achieved by taking the time to analyze, develop, and then test changes prior to implementation.  What that right level of rigor is depends on the impact of the environment being unavailable or incorrect.

So, what did I learn from my experience with the tooth paste?  My deviation only cost me about $3.50, some embarrassment in the TSA line, and an unplanned trip to CVS.  I learned I will no longer change my travel packing plans (whether or not I check luggage).  What can you learn?  There is a cost in time and/or dollars if you don’t follow a set process.  The best starting place is to work with your business and/or IT partners to reach consensus on that right level of rigor – and stick with it.

Photo provided by CogDogBlog via Flickr (Creative Commons License).

---

Mary Anne has 15 years of experience as a data management professional in all aspects of successful delivery of data solutions to support business needs.  She has worked in the capacity of both project manager and business analyst to lead business and technical project teams through data warehouse/data mart implementation, data integration, tool selection and implementation, and process automation projects.

Governance at the Turkey Fry

By Mary Anne Hopper, Senior Consultant

There is a long-standing tradition in our area for the annual turkey fry and it is the competition for 'Best Dish.' The first year we made awesome grilled artichokes and lost to boxed lemon squares (no, I’m not bitter). The second year, we deep-fried artichokes, wrapped them in bacon, and put them on a stick and lost to some chocolate cake monstrosity. The third year I walked in the door and was appointed to the judging team. How exciting—I would finally figure out how to win the contest. Well, guess what?  The host said there were no rules and to just pick something, and keep in mind that desserts always win.

Sound familiar? It probably does because this is how a lot of you determine what BI projects you’re going to work on. Here is a sampling of techniques I’ve heard from some of you:

• We pick what looks most interesting.
• The queue is prioritized based on the level of the requestor.
• [Insert name here] from the PMO decides.
• Prioritization?  Everything is the most important so we work on it all.
• If it’s not breaking, we don’t touch it.
• I don’t know how stuff gets done but my request always seems to be at the bottom.

I’m going to suggest to you this isn’t the most effective or collaborative way to manage your portfolio of projects not to mention that the business isn’t getting as much value as they could out of you. So how do you fix it? A good place to start is by developing a standardized and transparent project intake process for requests and prioritization. Build the process with your key business stakeholders and then stick to it—the first exception is the one that begins to derail your process. 

As for the ‘Best Dish’ award – not being a dessert person, the potato casserole won.

Photo provided by Pedula Man via Flickr (Creative Commons License).

---

Mary Anne has 15 years of experience as a data management professional in all aspects of successful delivery of data solutions to support business needs.  She has worked in the capacity of both project manager and business analyst to lead business and technical project teams through data warehouse/data mart implementation, data integration, tool selection and implementation, and process automation projects.

Repeating the Basics

By Mary Anne Hopper, Senior Consultant

It’s not like I spend my weekend and down-time thinking about work but some of the lessons I learn over the weekend apply to Monday thru Friday.  Case in point, my kids recently attended a racing seminar with an Olympic sailing hopeful.   He worked with them for three solid days focusing on boat handling drills.  Some of the kids wanted to know why.  His response – because you have to be able to repeat the basics the same way every time so you can deal with all the things that will be different every time (like wind velocity, wind shifts, waves, other boats, etc).

How does that relate to BI projects?   In order to deliver value to our business partners in the timeframe they expect, we have to be able to execute our projects repeating the basics every time so we can deal with the things that will be different every time.   I hope that sounds familiar.

Two great starting points are the project intake and requirements processes.  I use the phrase ‘starting point’ for a reason.  The intake process defines what work your BI team will be working on and in what order.  After those requests become defined projects, the requirements process (business, data, functional/application) then define what is going to be delivered to the business.  No matter how well defined the design and delivery processes are, the beginning of the cycle is imperative to success.  The table shows some examples of what process components need to be consistent and repeatable and what types of things are likely to change on you.

The Basics	What's the Same	What's Different
Project Intake	Request cataloging Prioritization	Business users Business priorities Resource availability
Requirements	Business requirements supported by data and functional/application requirements Artifacts Acceptance criteria Sign-off	Data availability Data quality Business users

This is a short example list and by no means all inclusive – there are numerous other examples.  The take-away is to understand that ongoing delivery success starts is dependent on the basics.  And nailing the basics with consistency will allow for more easily handling all the things that continue to change.

Photo provided by John Hopper.

---

Mary Anne has 15 years of experience as a data management professional in all aspects of successful delivery of data solutions to support business needs.  She has worked in the capacity of both project manager and business analyst to lead business and technical project teams through data warehouse/data mart implementation, data integration, tool selection and implementation, and process automation projects.

I’m All Ears

By Caryn Maresic, Senior Consultant

Most Data Warehouse designs include constructs for Address, Phone, and/or Email for Customers.  Len Silverston came up with what he calls a Universal Data Model that does a very good job of abstracting address, email and phone number data.  I have seen clients use the Contact Point portion of his model as-is and with a few simplifications with great success.  That being said, in the area of Marketing and Sales, the manner in which we reach out to our customers and prospects gets more diverse every day.  Disneyland has just partnered with Verizon so that park guests can get real time information about the park and play Disney games on their phones….and, of course, Disney gets access to more information about its customers!

How does this new and ever changing world of communication change the way we think about and model contact points?  What would my “address” look like if I were near the Haunted Mansion looking for a lunch spot?  Would it be different than if I were at Downtown Disney looking for a cup of coffee?  On Main Street looking for Winnie the Pooh?  In all instances I would be using the same phone, possibly the same IP address, but I would be in different locations which would be important to the marketeers at Disney.

As time goes by (and cell phone GPS systems become more accurate) I suspect that the way we run marketing campaigns to smart phones will be similar to the way in which we use billboards today.  Where the customer is physically located at any given time will be as important as the phone number and/or IP address, thus creating a two dimensional contact point.

Have you come across this issue in your organization?  Have you changed your data model to include two dimensional contact points?  If not, has the use of smart phones changed your data model in other ways?

photo by wrayckage via Flickr (Creative Commons license)

---

Caryn has over 20 years experience in providing high-quality data solutions to clients in the areas of Business Intelligence, Data Warehousing and System Integration.  Caryn has expertise in across industries with an emphasis in Pharmaceutical, Manufacturing, and Insurance.  Prior to joining to Baseline, she ran her own consulting company.

A Data Governance Primer (Part 1 of 3)

By Carol Newcomb, Senior Consultant

They say that Data Governance is about People, Process and Organization.  Much of the design work in planning for data governance is around people’s roles and responsibilities, then designing the organizational structure that will provide authority for decisions to be made and enforced.  The processes, however, are not new.  They are probably already being practiced within your organization, just in a decentralized, informal way.  In this blog series, I discuss the processes for 1) investigating and isolating the data quality issues—Root Cause Analysis—, 2) starting to collect complete Metadata Definitions, and 3) performing Data Quality Analysis.  Only when your governance group has worked through each step, in order, will you be more likely to design the appropriate solution.

Root Cause Analysis

The process of data governance is fundamentally very simple.

1. Identify the data quality issues to address
2. Prioritize the portfolio of issues to isolate/tackle the most important
3. Perform Root Cause Analysis to determine the true source of the data issue
4. Design the corrective action
5. Formalize the correction through consideration & approval by the Data Governance organization
6. Implement the fix
7. Monitor the results

It seems like when we start to map out the discrete steps involved in the data governance process, much of the work is already being done in informal ways throughout the organization.  What some folks don’t realize is that data governance is often nothing more than formalizing a whole bunch of informal processes that either don’t get communicated, or aren’t accepted as a data standard.

Root Cause Analysis is the process of identifying probable causes of a data issue, and isolating the contributing factors.  In order to resolve any particular issue, root cause analysis involves fact-finding, drilling into details of the problem, talking to the right people, and separating out other associated (but not contributing) factors.

A standard tool for supporting the detailed findings is the Ishikawa Diagram, below.  

To conduct a thorough Root Cause Analysis, use the following checklist:

• Diagnose the problem as if you are a physician or a detective. Consider all possible sources of the symptom. Don’t rule anything out yet!
• Boil the ocean—be exhaustive and creative.
• Don't practice problem solving before collecting all possible causes.
• Practice the “5 Why’s”—don’t stop asking “Why” until you have exhausted every conceivable potential reason.
• Rank the factors if possible.  Identify the Primary causes versus the Secondary or associated factors.
• Rule out each possible factor one at a time.  Justify why (you may need to come back to this later).
• Find all potential business process and data owners to involve them in your understanding of the possible sources of the problem.
• Share the findings with everyone involved in troubleshooting. They could rule out certain factors with their knowledge.
• Test your hypotheses with actual data.   
• Fix the problem and test again.
• Publish/share your findings and fixes.  Communicating your findings may reveal additional factors you hadn’t considered.

After a thorough Root Cause Analysis has been completed, Data Stewards should proceed to Metadata Analysis and Data Quality Analysis.  These two techniques will be discussed in my next blogs.

---

Carol Newcomb is a Senior Consultant with Baseline Consulting. She specializes in developing BI and data governance programs to drive competitive advantage and fact-based decision making. Carol has consulted for a variety of health care organizations, including Rush Health Associates, Kaiser Permanente, OSF Healthcare, the Blue Cross Blue Shield Association and more. While working at the Joint Commission and Northwestern Memorial Hospital, she designed and conducted scientific research projects and contributed to statistical analyses.

Agile Analytics: Put Your SaaS in the Sand

By Rob Paller, Consultant

Recently at a client, the data warehouse administrator was asked to define a sandbox environment in the production data warehouse for  analysts and developers working on a small project. The idea behind this sandbox was to allow the team a working area for collaboration and intermediate storage of results while working with the data in a purely ad hoc capacity. Instantly it was recognized this could be the start of something bigger within the organization—something that could not currently be provided by the incumbent business intelligence tools. The response had to be formulated quickly in order to avoid stifling the creativity of the analysts—or worse, the progress of the project—but care had to be taken as well; if managed incorrectly it could get out of hand and become a waste of system resources and a drain on human resources that had already been spread thin.  The business unit in question is looking to move from the confines the current business intelligence environment and push the edges.

This was a group of analysts that wanted to get their hands dirty and weren’t afraid to fail. They wanted to mash data together that previously could not be done by the business intelligence tools in their controlled ad hoc environments. This was data mining for the next set of KPIs that would shape the way business moves forward.

The concept of agile analytics is not new, eBay presented on and blogged about this concept in 2008. The idea at this client was simple. By leveraging the existing enterprise data warehouse system to house their sandbox environment the duplication of data is all but eliminated. Groups interested in sharing data between their sandbox environments are strongly discouraged until the data has been properly integrated into the production environment. The sandbox environments would also be given a short life expectancy at their inception to prevent the prototypes from becoming production and data ending up in a wasteland. This all sounded great on paper.

In the midst of a development architecture overview, a brief conversation among a few enterprise architects uncovered the potential Screw-Me Scenario that could bring the concept of agile analytics to an untimely demise. “The users of the data warehouse are not permitted to write ad hoc queries outside of a controlled business intelligence tool. They might write a bad query.” Thanks for the warning, we’ll be sure to refine our pitch to the enterprise architects to diffuse this scenario before it turns ugly.

In Oliver Ratzesberger’s presentation for eBay’s Analytics as a Service, he acknowledges that the metrics we already know are cheap and the unknown metrics are expensive. But the known metrics are not pushing the edges. Known metrics are found in the middle of the box. Agile analytics is about pushing the edges about how your enterprise data warehouse is used to improve response to the needs of the business. It is about the evolution of the user community from one who plays in controlled ad hoc environments to encouraging them to experiment with new ideas and not to fear failing along the way. Agile analytics is about encouraging your users reach out for the edges and P U S H. Only once the edges are stretched can the middle of the box redefined.

photo by edenpictures via Flickr (Creative Commons License)

---

Rob Paller is an expert at business analytics and database administration. Since joining Baseline, Rob has been responsible for developing a case analysis system to streamline the oversight of food assistance benefits, implementing a common citizen data model, and assisting in the rollout of a new public assistance data model integrating data from over 10 years of legacy with a new benefit eligibility determination system.