In the Field with our Experts

Data Architect – Who is right for the job?

by Caryn Maresic, Senior Consultant

The Data Architect is the core of any BI team.  It is important to choose a person with the right skill set.  As I tried to put together a list of skills I looked to IT Toolbox and Database Answers for help, but my mind wandered a bit.  System Construction. Data Architect. Data Warehouse. Software Factory.  We like to portray what we do in terms of construction and/or manufacturing.  A recent client bemoaned her departments inability to move from “building custom cars” to “an assembly line”.  Comparing ourselves to these burly industries might make us feel strong, but it does it accurately represent what we aspire to be?

What is a Data Architect?  What should they know how to do?   I borrowed the following description from this article. Before you click, read on and see if you can guess what this is really describing.  I think it is a great description for a Data Architect:

A Data Architect is qualified by education, experience, and imagination to enhance the function and quality of systems. The purpose of this pursuit is to improve the quality of life, increase productivity, and protect the health, security, and welfare of the business.

The best Data Architects are capable of analyzing a client's needs, goals, safety and business requirements and integrating this information into a design that is both pleasing to the eye and functional. They will work with the client closely to develop preliminary design concepts that meet their aesthetic, functional, and economic needs while maintaining adherence to standards.

In essence, the best Data Architects are part detective, part artist, and part psychologist and they use these skill sets to create systems that fit a client's tastes and needs with their budget in mind.

Doesn’t that sound like a great job?  Sign me up!  What this is actually describing is an interior designer.  While I doubt that HGTV has any plans to showcase the next dashboard you build, we are indeed closer to Designing Women than Rosie the Riveter!  Stay tuned for future posts on the talents of a good Data Design Star.

photo by Annahape Gallery via Flickr (Creative Commons License)

---

Caryn has over 20 years experience in providing high-quality data solutions to clients in the areas of Business Intelligence, Data Warehousing and System Integration.  Caryn has expertise in across industries with an emphasis in Pharmaceutical, Manufacturing, and Insurance.  Prior to joining to Baseline, she ran her own consulting company.

SOA and Enterprise Applications

By Adam White, Senior Consultant

It’s frustrating to see the consistent mistakes enterprise and solution architects make when implementing and supporting Service Oriented Architecture (SOA).  The most common one is co-opting functionality that comes with a packaged product as an enterprise solution.  

One example is using CRM workflow as enterprise workflow.  Most products nowadays  come packaged with workflow capabilities.  This means that events can trigger a process and that processes can interface outside of the application to activate other events. The first inclination of many enterprise architects is to utilize that functionality to support the enterprise-class business needs.  The problem is that the functionality that comes with the product is designed to support the product, not the overall enterprise. 

However, there are workflow products that support the entire enterprise. The easiest way to tell is by applying two trusted rules of thumb.  The first is by mapping your functional requirements.  You will see that by using this yardstick, product level functionality is different than enterprise level functionality.  And the second simple rule is if you replace your packaged application (say, your CRM application) do you also have to replace or rebuild your workflow?  The answer should be no.  If it’s yes, then you have incorrectly repurposed the product’s workflow functionality. 

This also occurs in the integration world.  Just because an ETL solution provides integration and can call a web service does not mean it’s the product to use for your core applications. After all, these are systems that support business operations, the systems that you use to process your day to day customer transactions.  In the end, it all comes down to fleshing out your requirements, and using the right tool for  the right job.

---

Adam White is an expert in Customer Data Integration (CDI), Services Oriented Architecture (SOA), Systems Development Life Cycle Methodology, Project Management, IT Governance, and establishing IT Process and Organization. Since joining Baseline, Adam has been responsible for conducting enterprise CDI assessments, implementing CDI, implementing SOA architecture, and implementing development methodologies.

Does Your Data Warehouse Project Need a “House” Call?

By Bob Wall, Senior Consultant

  For the last several years I have been an avid fan of the television show “House”, an American medical drama.  Each week the episode begins with an unsuspecting patient falling prey to some unknown and mysterious disease. They typically fail to receive an initial correct diagnosis, rendering their case as rare and very complex.  Dr. House and his team of specialists then spend the remainder of the episode acting like the medical equivalents of Sherlock Holmes, trying to solve the root cause of the disease and discover a life saving cure. Throughout the episode, Dr. House is rude, obnoxious, condescending, and addicted to pain killers, and the team performs numerous ethically dubious procedures. For instance, one or more of the doctors often makes a visit to the patient’s home in search of clues that might suggest certain pathology. They enter the home with or without permission, sometimes breaking the law, to try to discover any underlying evidence.

Miraculously by the end of the hour show, the cure is almost always found; the patient lives, and House and his team are ready to solve the next big mystery the following week.

Wondering what this has to do with your data warehouse project? Well, imagine if you will, some slight changes in the show’s format and cast and maybe you will see the similarity.

In this episode a business manager mysteriously falls prey to a sick data warehouse environment and all indications are that it has something to do with bad data.  No one can figure out the underlying cause and in fact, the manager has received several misdiagnoses. Just as the manager is ready to give up, Dr. House and his team of diagnosticians step in to assist.  In this case House’s specialty is Data Management and the other doctors are specialists in Data Governance, Data Architecture, Data Modeling, Data Analysis/Profiling, and Data Quality.

The Data team first identifies the overall problem and then the symptoms:

Overall Problem

• Inability to understand source data and uncover the bad data disease, imperiling the data warehouse

Symptoms

• Data quality issues
• Multiple sources generating the same data
• Non-existent documentation of source systems
• Data models and file structures not understood
• Lack of knowledge about core data elements
• Mismatch between source system data and developers’ definition
• Analyzing multiple data sources for a high profile IT project

Next the team interviews all of the patient stakeholders (SMEs, Data Stewards, Business Analysts, IT, etc.) and makes a visit to the corporate home to gather  evidence that might help in the diagnosis (business plans, data models, system documentation, ETL specs, etc.).

Now the team concurs on several possible reasons for the problem and runs several diagnostic tests, the most prominent being:

• Data Source Analysis – the purpose is to attain an understanding of what data is available and where it resides within the patient’s myriad of systems and applications.
• Data Profiling – the purpose is to attain an understanding of an organizations data validating data types, domain values, etc.

Finally, the Data team is able to understand the patient’s disease and dispense a cure for the data warehouse problem by addressing the data quality problems. This cure includes offering short term data corrective actions, ongoing incorporation of data assessment—a data profiling tool and analysis process—as a permanent part of their source system infrastructures, and a long term common recommendation for the creation of a new role, Source Data Steward, for systems that are either highly complex or in demand by other business applications.

So, just like the television show, a “House” call for a sick Data Warehouse Project from the Data team can be seen as analogous to the House television episode. However, there are several noteworthy differences: The lead “Data Doctor” in this case is never rude, obnoxious or addicted to drugs! The team never breaks into a building to gather documents and evidence, but always asks permission first!  And, most importantly, there is no certainty that the cure will be found in only an hour episode!

---

Bob Wall is a senior consultant with Baseline Consulting. He is an information technology specialist with 30 years experience in all areas of data warehouse administration, data architecture, data resource management, training, and applications systems development, as well as in corporate management.