Solving the Metadata jigsaw puzzle
Recently Roger Nolan from Informatica wrote an interesting piece on the importance of metadata to the development process and highlighting how we can use great tools within Informatica like Metadata Manager and the Informatica Business Glossary. I’d like to extend this by considering the benefits of linking other kinds of metadata.
We’re seeing Informatica’s latest product developments helping to support a movement: a shift from IT to self-service Data Integration where the business positions itself as the driver of technology solutions.
So can we go even further with metadata to support this? Can we link the metadata sources and the tools that Roger discussed to the wealth of operational metadata from Informatica and the technology layers that surround our Data Integration workflows, sessions and mappings? This metadata would tell us more than just last night’s successes & failures but about trends in performance, resource usage, volumes processed and rejected data. Can we go further still and add the metadata about the development process itself? Can we add the ICC metadata, the ”Integration Factory”, into the mix and enable these new metadata sources to be linked to Metadata Manager’s understanding of sources, targets and mappings and their business context stored within Informatica’s Business Glossary. If we can achieve this then we’ll be rewarded with a 360-degree perspective on our Data Integration environment.
But where’s the business benefit? Let’s consider how this metadata could be used: take an everyday example of an overnight workflow running slowly at month-end. The Ops analyst is alerted. They recognize that it’s a DW process and this probably means that someone in the business is going to receive their reports late. But which reports are affected, which business users does this affect and has this overrun happened before making this problem part of a trend, perhaps caused by underlying infrastructure issues?
These questions can be answered by solving the Metadata jigsaw. This means linking operational metadata to those all-important metadata sources – Metadata Manager and the Business Glossary.
That’s why we are building a new version of HIPPO, for release at the end of 2011 to act as the linkage to enable a breakthrough in the use of metadata which extends its value beyond the design and development processes to include real time and historical operational information.
The IT department and the Business units which they serve will be linked by means of an operational metadata bridge between the two: the Business know which Informatica processes are critical to their daily reports and can monitor their activity trends via HIPPO. They can check if last night’s run was successful and what the trends for their daily report processes look like; will they overrun if data volumes continue to grow?
Likewise, when IT is faced with an overnight incident they can prioritise those failures with the most business impact and keep the relevant business consumers informed. They will do this by using the bridge between HIPPO’s Operational metadata and Informatica’s Metadata Manager and Business Glossary tools.
So if we have Informatica’s Business Glossary view linked to the Metadata Manager view of metadata about sources, targets and mappings, and with both now linked to Operational metadata by HIPPO, what’s missing? Well what remains is the ICC, the “Integration Factory”, metadata. By doing so we will be able to link a mapping to the developer and analysts who produced it (Factory) through to the reusable components, sources and targets that it uses (Metadata Manager) to the business units and processes that rely upon it (Business Glossary) and on to the operational metadata on its performance profile and infrastructure dependencies (HIPPO).
This is what we are prototyping at the moment and we plan to release these features in the new release of HIPPO at the end of this year. Contact me to find out more.