In the Field with our Experts

The Dreaded Stairs

By Stephen Putman, Senior Consultant

Recently, a friend of mine posted a link on Facebook that reinforced a philosophy that I have had for a long time that applies to all activities in life that are not duty-bound:

The Dreaded Stairs (part of The Fun Theory project)

I have long felt that humans do things for two reasons:

A) They're fun

B) They're lucrative

This applies to the field of Data Governance and Quality as it does everything else. One of the reasons data governance and quality initiatives are not more widely adopted and followed is that the work is not terribly fun - data owners must be identified, policies and processes must be adopted, and the entire process must be monitored and attended once it is in place. It's also not seen as lucrative in a direct sense - the act of cleansing the data in a transaction usually doesn't provide immediate financial reward, and while the implementation of governance and quality initiatives can affect the company's bottom line, the benefits are very difficult to quantify in a traditional sense.

Phil Simon has produced a terrific series for The Data Roundtable on incentive ideas for data quality programs, so I will not address these here - he says it much better than I can. I am concerned with "fun." The video above demonstrated an innovative idea to make a mundane but healthy activity (climbing stairs) into a joyful experience. What sort of innovative programs can be created to make managing high-quality data fun?

"Fun" is a difficult concept because it means something different to everyone. One way to find out what is "fun" to your employees is by conducting surveys or workshops to ask them directly. Another possibility could be to have a "company carnival" in your parking lot, and award employees who identify quality issues with raffle tickets or a "boss' dunk tank." The White House holds a yearly contest with government employees for the best quality improvement or cost-savings idea (this is more of an incentive, but some people also consider contests like this fun).

These are just a few ideas off the top of the head - do you have creative people who can come up with other ideas? If it is indeed true that fun makes unpleasant activities more palatable, this would be time well spent to reinforce data governance and quality in your organization.

photo by Robin Fensom via Flickr (Creative Commons license)

---

Stephen Putman has over 20 years experience supporting client/server and internet-based operations from small offices to major corporations.  He has extensive experience in a variety of front-end development tools, as well as relational database design and administration, and is extremely effective in project management and leadership roles. He is the co-author of The Data Governance eBook, available at baseline-consulting.com/ebooks.

New Years’ Resolutions: Assess and Revise Your BI Strategy

By Dick Voorhees, Senior Consultant

The New Year is upon us. And for many, the coming of the New Year involves making new resolutions, or reaffirming old ones. This resolution-making process includes corporations and organizations, not just individuals. In terms of personal resolutions, some undertake this process in earnest, but many seem to deal with resolutions superficially, or at least not very effectively. The same is frequently true for organizations as well.

So how then should an organization go about deciding which “resolutions” to pursue in the New Year, which goals and objectives are both worthy and achievable? Often there are no "good" or "bad" opportunities, a priori, but some are more likely to result in a successful outcome and/or have more significant payoff than others.

1. Take stock of the opportunities, and develop a list of key potential initiatives (or review the existing list, if one exists). Consider recent or imminent changes in the marketplace, competitors’ actions, and governmental regulations. Which of these initiatives offers the possibility of consolidating/increasing market share, improving customer service, or represents necessary future investment (in the case of regulations)? And which best supports the existing goals and objectives of the organization?
2. Assess the capabilities and readiness of the organization to act on these initiatives. An opportunity might be a significant one, but if the organization can’t respond effectively and in a timely manner, then the opportunity will be lost, and the organization might better focus its attention and resources on another opportunity with lesser potential payback, but that has a much greater chance of success.
3. Develop a roadmap, a tactical plan, for addressing the opportunity. Determine which resources are required – hardware, software, capital, and most importantly people – what policies and procedures must be defined or changed, etc…

Then be prepared to act! Sometimes the best intentions for the New Year fail not for lack of thought or foresight, but for lack of effective follow through. Develop the proper oversight/governance mechanisms, put the plan into action, and then make sure to monitor progress on a regular basis.

These are not difficult steps to follow, but organizations sometimes need help doing so. We’ve found that clients who call us have learned the hard way – either directly or through stories they’ve heard in their industries – that some careful planning, deliberate program design, and – if necessary – some skill assessment and training can take them a long way in their resolutions for success in 2011. Good luck!

photo by L.C.Nøttaasen via Flickr (Creative Commons)

 

---

Dick Voorhees is a seasoned technology professional with more than 25 years of experience in information technology, data integration, and business analytic systems. He is highly skilled at working with and leading mixed teams of business stakeholders and technologists on data enabling projects.

Understanding Where Our Work Comes From

By Mary Anne Hopper, Senior Consultant

I’ve written quite a bit about the importance of establishing rigor around the process of project intake and prioritization.  If you’re sitting there wondering how to even get started, I believe it is important to understand where it is these different work requests because unlike application development projects, BI projects tend to have touch points across the organization.  I tend to break the sources into three main categories—stand-alone, upstream applications, and enhancements.

Stand-alone BI projects are those that are not driven by new source system development. Project types can include as new data marts, reporting applications, or even re-architecting legacy reporting environments. Application projects are driven by changes in any of the upstream source systems we utilize in the BI environment including new application development and changes to existing applications. Always remember that the smallest of changes in a source system can have the largest of impacts to the downstream BI application environment. The enhancements category is the catch-all for low risk development that can be accomplished in a short amount of time.

Just as important in understanding where work requests come from is prioritizing those work requests.   The three need to be considered in the same prioritization queue—this is a step that challenges a lot of the clients I work with.  So, why is it so important to prioritize work together?  The first reason is resource availability.  Resource impact points include project resources (everyone from the analysts to the developers to the testers to the business customers), environment availability and capacity (development and test), and release schedules.  And, most importantly—prioritizing all work together ensures the business is getting their highest value projects completed first.

 

---

Mary Anne has 15 years of experience as a data management professional in all aspects of successful delivery of data solutions to support business needs.  She has worked in the capacity of both project manager and business analyst to lead business and technical project teams through data warehouse/data mart implementation, data integration, tool selection and implementation, and process automation projects.

The Price of a Tube of Toothpaste

By Mary Anne Hopper, Senior Consultant

As you can imagine, I travel quite a bit as a consultant for Baseline.  Over my tenure, I have developed a standard routine for getting through the airport.  More often than not, things have gone pretty smoothly for me.  Until this week – my bag was pushed into the extra screening area where it turned out there was an over-sized tube of toothpaste that had to be thrown away.   How did this happen when week in and week out, I use the same bag for my stuff and always get through without a hitch?  Well, I deviated from my process.

You see, the prior week I actually checked a bag and was able to throw a full tube of toothpaste in the ditty bag and I never checked when I was packing for this week’s trip.  I deviated from my standard process.  If you’ve ever implemented a “small” or “low impact” change that has blown up an ETL job, changed the meaning of a field, or caused a report to return improper results, you know where I’m going with this.

Process is important.  Discipline in implementing to that process is even more important.  Am I proposing that every small change go through an entire full-blown project lifecycle?  Absolutely not.  But, there should be a reasonable life cycle for everything that goes into a production quality environment.  Taking consistent steps in delivery helps to ensure that even the smallest of changes do not result in high impact outages.  This can be achieved by taking the time to analyze, develop, and then test changes prior to implementation.  What that right level of rigor is depends on the impact of the environment being unavailable or incorrect.

So, what did I learn from my experience with the tooth paste?  My deviation only cost me about $3.50, some embarrassment in the TSA line, and an unplanned trip to CVS.  I learned I will no longer change my travel packing plans (whether or not I check luggage).  What can you learn?  There is a cost in time and/or dollars if you don’t follow a set process.  The best starting place is to work with your business and/or IT partners to reach consensus on that right level of rigor – and stick with it.

Photo provided by CogDogBlog via Flickr (Creative Commons License).

---

Mary Anne has 15 years of experience as a data management professional in all aspects of successful delivery of data solutions to support business needs.  She has worked in the capacity of both project manager and business analyst to lead business and technical project teams through data warehouse/data mart implementation, data integration, tool selection and implementation, and process automation projects.

Governance at the Turkey Fry

By Mary Anne Hopper, Senior Consultant

There is a long-standing tradition in our area for the annual turkey fry and it is the competition for 'Best Dish.' The first year we made awesome grilled artichokes and lost to boxed lemon squares (no, I’m not bitter). The second year, we deep-fried artichokes, wrapped them in bacon, and put them on a stick and lost to some chocolate cake monstrosity. The third year I walked in the door and was appointed to the judging team. How exciting—I would finally figure out how to win the contest. Well, guess what?  The host said there were no rules and to just pick something, and keep in mind that desserts always win.

Sound familiar? It probably does because this is how a lot of you determine what BI projects you’re going to work on. Here is a sampling of techniques I’ve heard from some of you:

• We pick what looks most interesting.
• The queue is prioritized based on the level of the requestor.
• [Insert name here] from the PMO decides.
• Prioritization?  Everything is the most important so we work on it all.
• If it’s not breaking, we don’t touch it.
• I don’t know how stuff gets done but my request always seems to be at the bottom.

I’m going to suggest to you this isn’t the most effective or collaborative way to manage your portfolio of projects not to mention that the business isn’t getting as much value as they could out of you. So how do you fix it? A good place to start is by developing a standardized and transparent project intake process for requests and prioritization. Build the process with your key business stakeholders and then stick to it—the first exception is the one that begins to derail your process. 

As for the ‘Best Dish’ award – not being a dessert person, the potato casserole won.

Photo provided by Pedula Man via Flickr (Creative Commons License).

---

Mary Anne has 15 years of experience as a data management professional in all aspects of successful delivery of data solutions to support business needs.  She has worked in the capacity of both project manager and business analyst to lead business and technical project teams through data warehouse/data mart implementation, data integration, tool selection and implementation, and process automation projects.

The Data Governance eBook: Maps, Mechanics and Morals

Recently, we launched our newest e-book, The Data Quality eBook. We ran an excerpt from that book last week. But did you know we have another e-book out there? Earlier this year, we published The Data Governance eBook to rousing success. If you haven't had a chance to download and read this book by Kimberly Nevala and Stephen Putman, don't worry—it's still available online.

Here's a description:

When the fictitious business team at the SpectraDynamo corporation launches a new data governance initiative, the trials and tribulations begin. The Data Governance eBook, authored by the experts at Baseline Consulting, is an entertaining business parable with a message aimed at helping you launch data governance the right way. This bold new look at data governance is more than just a cautionary tale. It's what Baseline partner Jill Dyché calls "a prescription for getting it right." The Data Governance eBook includes all the nuts and bolts necessary to transcend the requisite decision-by-committee and role confusion so endemic to companies’ information management efforts.

Here's a brief excerpt that showcases what you'll find in our first e-book:

The moral of the story: data governance can be tough. By definition, innovation is disruptive and antithetical to established paradigms. Applying rigor in the form of governance and oversight to information is no exception to this rule. Mix the laundry list of competing information needs and issues in the average enterprise with the inevitable confusion regarding ‘what does it mean’ and the data governance proposition quickly becomes overwhelming. The multitude of data governance definitions and methodologies parlayed by both industry analysts and vendors—is it: data quality? information policy? a dashboard? a council?—certainly doesn’t help.

The fact is: there is no standard playbook for data governance. The structure of your program is dependent on the specific problems governance needs to address and the organization’s incumbent organization and culture.  Some companies—like SpectroDynamo—find a bottom-up approach centered around core data management functions provides the greatest initial leverage. Organizations that are more hierarchical or consensus driven in nature may require formal enlistment of executive sponsorship and deliberate assignment of decision rights for governance to take root. And yes, you may start somewhere in the middle...

What is consistent? The fundamental roles and responsibilities found in functioning data governance programs. In this section, we define those core functions: executive sponsorship, data governance (policy making and decision rights), data management and data stewardship. These functions align with and support everything from strategy alignment and organizational buy-in to tactical enablement and operations.

When evaluating these capabilities, remember the single most important success factor for entrenching data governance as a core operating principle: continuous improvement. Rather than boiling the ocean, identify key business pain points and the core capabilities required to address them. Prioritize and pick a pilot project. Finally, prototype data governance-related processes and don’t be afraid to refine them as your capabilities and reach grow.

Interested in knowing more? Download The Data Governance eBook today!

The Data Quality eBook: When Bad Data Happens to Good Companies.

You may have noticed we've slowed down our "In The Field" blog entries, but it's for a good reason. Last week, Baseline launched its latest e-book, co-authored by Baseline consultants and frequent bloggers, Carol Newcomb and Caryn Maresic.

The Data Quality eBook is both a cautionary tale and a nuts-and-bolts toolkit for bringing a set of formalized data quality processes to your company. When the Central Health Alliance discovers just how costly bad data can be, the health care provider launches a data quality program that not only improves services—it can actually save lives. This e-book looks at data issues faced by companies across industries, and shows you how to apply a step-by-step process to prevent over-investment in untrustworthy data and drive business value in the bargain.

The book is currently available for download at Information-management.com.

And now, a brief excerpt from the book:

At Central Health Alliance—as with many companies—protracted explanations and guesswork cede to manual effort.  If there is a problem hidden in the data, an analyst will surely find it. The question is: how long will that take?

The problem with manual data exploration is that you’ve got a lot of data—probably a lot more than you know.  Data is captured, copied and transformed—it is everywhere in all shapes and forms.  When digging through the data, where do you start?  More importantly, where do you stop?  Unfocused and manual data  profiling might lead to interesting discoveries, but won’t get you a cohesive roadmap to better data quality. Moreover, it’s hardly scalable.

The right way to improve data quality is by focusing on four incremental steps:

Identify the Business Issue – Defining the business issue and its impact on business operations, strategic goals, or decision making maintains focus for the remainder of this process. The scope of the business issue should be well understood. You might identify several related business issues that have bad data as their core. Or you might have a number of overarching issues, as Central Health Alliance does.

Assess Conformance to Requirements – After your business issue is well understood, it is time to do a data quality assessment.  The assessment is a focused effort to determine where in the data lifecycle things go wrong.  Central Health Alliance knows its business issues and they are poised to kick off the data assessment.

Discover the Root Causes – After you’ve assessed your data quality issues, it is time to discover why these problems are occurring. What are the root causes? Is there a lack of consistent training for the people who key in data?  Is there some buggy code that is moving data around behind the scenes?  Maybe there is some confusion about what the data actually means?

Formalize Improvements – Once you know the “what” and the “why,” it is time for action.  Improving data quality is often a two-pronged effort—you’ve got to fix what’s wrong and you’ve got to put a monitoring system in place so that you will know when something goes awry in the future. By fixing the data problem at its source, you can not only prevent it from recurring, you can improve the quality of the data in upstream systems as well.

What are you waiting for? Go download the entire e-book today!

Data Architect – Who is right for the job?

by Caryn Maresic, Senior Consultant

The Data Architect is the core of any BI team.  It is important to choose a person with the right skill set.  As I tried to put together a list of skills I looked to IT Toolbox and Database Answers for help, but my mind wandered a bit.  System Construction. Data Architect. Data Warehouse. Software Factory.  We like to portray what we do in terms of construction and/or manufacturing.  A recent client bemoaned her departments inability to move from “building custom cars” to “an assembly line”.  Comparing ourselves to these burly industries might make us feel strong, but it does it accurately represent what we aspire to be?

What is a Data Architect?  What should they know how to do?   I borrowed the following description from this article. Before you click, read on and see if you can guess what this is really describing.  I think it is a great description for a Data Architect:

A Data Architect is qualified by education, experience, and imagination to enhance the function and quality of systems. The purpose of this pursuit is to improve the quality of life, increase productivity, and protect the health, security, and welfare of the business.

The best Data Architects are capable of analyzing a client's needs, goals, safety and business requirements and integrating this information into a design that is both pleasing to the eye and functional. They will work with the client closely to develop preliminary design concepts that meet their aesthetic, functional, and economic needs while maintaining adherence to standards.

In essence, the best Data Architects are part detective, part artist, and part psychologist and they use these skill sets to create systems that fit a client's tastes and needs with their budget in mind.

Doesn’t that sound like a great job?  Sign me up!  What this is actually describing is an interior designer.  While I doubt that HGTV has any plans to showcase the next dashboard you build, we are indeed closer to Designing Women than Rosie the Riveter!  Stay tuned for future posts on the talents of a good Data Design Star.

photo by Annahape Gallery via Flickr (Creative Commons License)

---

Caryn has over 20 years experience in providing high-quality data solutions to clients in the areas of Business Intelligence, Data Warehousing and System Integration.  Caryn has expertise in across industries with an emphasis in Pharmaceutical, Manufacturing, and Insurance.  Prior to joining to Baseline, she ran her own consulting company.

A Data Governance Primer (Part 3 of 3)

By Carol Newcomb, Senior Consultant

Diamond in the Rough: Data Quality

The third part of my summertime primer addresses Data Quality Analysis.  Don’t even start a data quality analysis until you have completed the first two steps of your Root Cause Analysis--investigate & prioritize any potential causative factors, and start your metadata assessment.  Otherwise, you may be misled by your findings.

Data quality is defined as complete and accurate data that is ready for business consumption.  Sources of poor data quality may include lack of data entry rules, unclear data element definitions, inconsistent metadata definitions for field type, format or intent, or breakdowns in data transformation processes as data flow between systems or applications.  Poor data quality results in bad business decisions; it contributes to major problems in using data effectively, and costs companies millions of dollars/year in terms of rework and inefficiency.  Data quality, in combination with robust metadata definitions, is part of the foundation of good data governance.

Data Quality Analysis

A Data Quality Management process should be designed to enable an area to start with a simple approach and over time to mature to one that is more proactive and comprehensive.  Initially, investigation may be focused on single data elements or events.  As patterns, data commonalities and other relationships appear, the data quality management process will grow to support complete business processes.   A mature data quality management process will not just resolve individual issues; it will also track relationships between data elements, ensure that business rules are consistent and generate statistical analyses to monitor previously addressed issues to ensure that data quality is stable and that an early warning system is in place as part of the data governance program.  The goal is to design a data quality management lifecycle, as shown in this diagram:

Initial Data Quality Analysis Process

I. Define data scope

• Determine data elements that are associated with or are direct results of the reported issue
• Check that all metadata definitions are present and current
• Enlist the involvement of the Data SME or Data Stewards
• Identify all source systems where the data originates, is  entered or derived

II. Extract and profile the data

• Extract the relevant data from all key source systems.
• Design the profile.  A profile will consist, at a minimum, of total record counts, min/max values, frequency of unique values, and frequency of invalid values (if defined) for each data element profiled. 
• Profile the data to determine key characteristics that are contributing to the issue, such as:
• 1. Wrong values
  2. Missing values
  3. Corrupt transformation processes
  4. Incorrect business rules
  5. Incorrect usage rules

III. Analyze Data Profile Results

• Summarize the key findings from the profile detail
• Determine what key drivers are contributing to the impact
• Determine accountability for the data quality issue
• Involve other Data Stewards in troubleshooting and designing the data quality solution

IV. Design the Corrective Action Plan

Two types of plans should be developed to address known data quality issues: a corrective action plan to fix the immediate source of the problem identified, and an ongoing monitoring plan, where thresholds have been determined and metrics are routinely collected and reported to data stakeholders.  This monitoring process should be scalable based on the number of data elements being tracked.

1. Corrective Action Plan
2. • Does scope of problem warrant change in metadata definitions, business practices or data entry rules?
   • Does scope of problem warrant a data governance standard?
   • Does the corrective action plan include details on how to fix the source of the problem as well as ways to correct historical data in the system?
3. Preventive Action Plan
4. • This plan will be designed to minimize the probability of data quality issues from recurring
   • Determine ‘early warning triggers’ based on designated thresholds.  These thresholds should reflect the business tolerance for inaccurate data (is 95% acceptable?)
   • If data latency is the source of a data quality issue, then latency thresholds should be included in the monitoring plan
   • Determine how frequently results of the monitoring plan will be reported to data stakeholders or governance oversight committees

So, now that summer is officially here, this wraps up my Data Governance Primer series.  Time for some iced tea and my favorite beach towel.  Come August, these little refreshers might be just the thing!

photo by Swamibu via Flickr (Creative Commons License)

---

Carol Newcomb is a Senior Consultant with Baseline Consulting. She specializes in developing BI and data governance programs to drive competitive advantage and fact-based decision making. Carol has consulted for a variety of health care organizations, including Rush Health Associates, Kaiser Permanente, OSF Healthcare, the Blue Cross Blue Shield Association and more. While working at the Joint Commission and Northwestern Memorial Hospital, she designed and conducted scientific research projects and contributed to statistical analyses.

A Data Governance Primer (Part 2 of 3)

By Carol Newcomb, Senior Consultant

Minding Your Metadata

The second part of my summertime primer addresses ‘Minding your Metadata’.  I can just hear the collective groans and yawns now.  Sorry, but metadata collection is one of those necessary evils that may not be fun in the doing, but having it available as a resource to understand your data and use it appropriately is invaluable.  And you just might find some interesting surprises along the way!

Metadata: What Is It & Why Do I Need It?

As you start your Root Cause Analysis (see last week’s primer), you first need to examine existing data definitions (or lack thereof).  Metadata is the foundation of good data management and forms the basis for Data Governance.   Pardon me for stating the obvious, but metadata is fundamental to investigating and resolving data issues and it is the first place to start when investigating data quality issues.

Metadata is “data about data”.  Plain and simple.  It includes descriptive information about electronic data used in common daily business practice.  Metadata includes items usually found in a data dictionary: field name, field length, retention rules, and security access, as well as additional descriptive information that may include data origin (source or system), creation/entry date, method of creation (key-entry or the result of a calculation), purpose of the data (its intended use), how frequently it gets updated or refreshed, and current location in a database (table, view, schema).  If a data element is the result of calculation logic or groupings (such as age categories), those business rules used to generate the resulting data values should be collected as part of the metadata.

A good example of metadata that you may use every day would be ‘document properties’ in a Word document.  This feature captures data on the original document creation date, most recent access and update times, document creator, count of characters, words and pages.  If the document should be private, this will be indicated in its properties.  You may also tag the document by indicating key words in order to make it easier to find by you or others.

A few of the benefits of Metadata Management include:

• Clarify rules for data entry
• Reduce ambiguity around appropriate use of data elements
• Eliminate problems associated with not having data definitions, business rules or transformation logic available
• Validate legitimate values at the data element level
• Provide evidence to regulators that security and confidentiality are protected
• Centralize the storage and accessibility of metadata for end-users
• Reduce the amount of effort required to research data results.

A Metadata Management Repository is a central location or system to collect and store metadata that may exist in disparate parts of the organization (data dictionaries, systems, spreadsheets, or people’s brains). The metadata repository will store detailed definitions centrally on a network where other users can find it.

There are three general sources of metadata that should be included in this repository:

Business Metadata – Business metadata attributes facilitate identification, understanding, and appropriate use of existing data elements.  These include clear business names and descriptions, relevant business rules, descriptions of the data sources, security and privacy rules, etc. 

Technical Metadata – Describes the technical attributes of data such as physical location (host server, database server, schema, etc.), data types, any transformations applied and domain of valid values, relationships to other data elements, precision, and lineage.  Technical metadata is used by business users and by IT staff to design efficient databases, queries, and applications, and to reduce duplication of data. 

Operational Metadata – Describes the attributes of routine operations on data and related statistics.  These include job schedules and descriptions, data movement and transformation processes, data read, update and performance statistics, volume statistics, backup and archival information.  Operational metadata is used by operations staff, and DBA’s to tune the system and ensure its continued efficient operations.  It is also used by business users to track such events as “last use” of a field, and “last load” of a data element.

Exciting stuff, huh?  Well, the whole point of metadata is to have the information about data available to a multitude of users when they need it, to keep it current, and to avoid confusion around usage.  So if you appreciate having a clean bathroom, and knowing where you keep your antiperspirant, you will also appreciate having good metadata!  The time for spring cleaning is well overdue.

---

Carol Newcomb is a Senior Consultant with Baseline Consulting. She specializes in developing BI and data governance programs to drive competitive advantage and fact-based decision making. Carol has consulted for a variety of health care organizations, including Rush Health Associates, Kaiser Permanente, OSF Healthcare, the Blue Cross Blue Shield Association and more. While working at the Joint Commission and Northwestern Memorial Hospital, she designed and conducted scientific research projects and contributed to statistical analyses.