erwin Data Intelligence: A Data Partner’s Perspective

At Sparkle, we’re a holistic data partner helping organizations increase their data maturity in a strategic yet pragmatic way. One of the key ingredients to ensure data is really embedded in an organization, and one of the key enablers to increase the strategic impact of data, is the setup of a successful data governance program. While the essence of success in data governance is people and not technology, having the right tools at your fingertips is crucial. Technology is an enabler, and for data governance this is essentially having an excellent metadata management tool.

Next to data governance, data architecture is really embedded in our DNA. We will always start with having a good understanding of the customer’s needs, to ensure we draw up a conceptual architecture that will fulfill the customer’s current and anticipated future requirements. Technology wise, at Sparkle we’re a Microsoft first company. This means a couple things. We have a strategic partnership with Microsoft and invest in certifications and knowledge on Microsoft’s data offerings. At the same time, when Microsoft does not have the best or right technology available, we scan the market for complementary technology, to ensure we have a solid, integrated, future-proof architecture. Examples of complementary technologies include technology for data warehouse automation, change data capture and master data management.

For data governance, it’s more complex. Microsoft does have a solution with Microsoft Purview, which serves many organizations well. On the other hand, we also notice that for organizations with greater data governance needs, Purview does not tick all the boxes. That’s why we evaluated erwin Data Intelligence by Quest.

Decision drivers

For Sparkle to consider a new technology partnership, there are a couple of decision drivers evaluated. These are: completeness in terms of capabilities, flexibility, competitive advantage, user experience, vision, professionalism of the organization behind the software, pricing and most importantly, intuition. Can we fall in love with the product, and do we believe it’s more than a flirt? We will shortly go through and explain how we evaluated erwin Data Intelligence on each of these items.

Completeness in terms of capabilities

At Sparkle, we have had demonstrations from many different vendors of data governance solutions. And to say the least – we’re not an easy audience. We want to understand what works well, but at the same time have an inclined urge to understand the limitations. We will challenge the presenter whether the tool would be able to cope with needs we experienced in previous projects, needs we anticipate to come, and even needs that we dream up that would be answered by our ideal product. We can’t help it, that’s who we are. 😊And truth must be told, the demo of erwin Data Intelligence left us impressed.

There’s a multitude of capabilities that come to play in terms of evaluating metadata management tools, which would be more than we could cover here, but I will focus on two major areas: the business glossary and data lineage.

The Business Glossary

A business glossary needs to be easy to use and you need to be able to link and record all information considered relevant to capture for a business term. Also, it should not feel too technical, but really be geared to be used by business users. What I love in erwin Data Intelligence is that it not only allows you to enter a definition, description, data owner, data steward, data classification, and all the basics you expect – but that it also allows you to completely configure the fields you need to your unique data governance framework. If there are fields that are not relevant to your organization or other fields that you require that are a bit more specialized – you can tweak it all to your specifications. And that’s a constant throughout the software. The basic setup and out-of-the-box configuration is in line with the needs of most organizations, but you can completely transform it to make it your own.

Data Lineage

In terms of data lineage, it’s one of the most complete tools I’ve seen on the market, both in terms of the level of detail captured in the data lineage, as well as technologies that are covered. erwin Data Intelligence offers a library of nearly 120 erwin Standard and Smart Data Connectors supporting the ability to capture data-at-rest and data-in-motion metadata across diverse and complex data landscapes. Not only all relevant data-related on-premises and cloud-associated Microsoft technologies are covered, but really all major technologies including AWS, Databricks, Snowflake, SAP, Informatica, Tableau, Cloudera, Oracle, and many more.

It offers a complete column level tracking (from, for example, within a Power BI report) of all the transfers and transformations that have led to the contents of the report column. It actually reverse engineers the ETL/ELT technical code sets and transforms and visualizes it on a logical level. What’s more, it even allows you to forward engineer the code to a different technology. How cool is that? Imagine how useful this can be to support migrations.

Finally, automated data quality scoring from erwin Data Quality is completely integrated within erwin Data Intelligence data lineage. This means, in case of data quality issues, you can easily investigate where the data quality issue may be coming from. Does it exist at the initial data entry/data source? Or does the data enter the system correctly and then degrade along the way? In this case, erwin Data Intelligence allows you to visually pinpoint exactly where the data quality problem arises.

A Competitive Advantage

erwin Data Intelligence really excels in multiple areas. We’ve covered its flexibility and the exceptional data lineage but, for me, the cherry on top is the mind map view. In the mind map, you have all information linked to a business term in the blink of an eye. It brings together all business, technical and organizational information within one screen.

This information includes business processes, business rules, business policies, regulations and frameworks, KPI’s, dashboards, related business terms, systems, tables and columns.

You can start at a high level but also really drill down and investigate at a very granular level everything related to a business term.

User Experience

Although the user interface is good, there are tools with interfaces that feel even more modern, so for me that’s not the greatest differentiator. However, user experience is, of course, also about ease of use and continuously evolving capabilities to meet emerging needs. With the addition of erwin Data Marketplace this past year, I really like the direction that erwin Data Intelligence is taking to offer a data shopping experience fit for business users. This means data shoppers can “Amazon-like” select multiple data sets and compare them side to side, to help them decide which is right for them to use and then easily request and gain governed access to the needed data for their purpose.

Vision

In terms of vision, Quest is on top of the game and making the right investments in erwin Data Intelligence. Artificial Intelligence has already integrated to assist for example in detecting PII sensitive data, long before the hype. But the company goes beyond what you would expect from an average metadata management tool and is now for example evolving to help organizations in tackling the delivery of data products. With the addition of erwin Data Modeler, a leading data modeling tool for more than 30 years, erwin can deliver a Model to Marketplace approach to speeding data product delivery.

Professionalism of the Organization

It’s one thing to have a good product. It’s another thing to be professional in the way you interact with partner organizations and customers. As a partner, we have many different types of relationships with vendor organizations. At Sparkle, our values and company culture are integral to our success, and it has always been important for us to find similar values in the organizations we work with – both software vendors and customers. While we have at times been tempted to sacrifice the lack of culture and value match for the sake of interesting business opportunities, those relationships never last long. With erwin by Quest, the match was there from the start, and we appreciate the very proactive, responsive and collaborative partnership.

Intuition

Although we pretend to be rational human beings, in reality, a gut feeling is still crucial. We often make decisions from the gut and then rationalize why this makes sense to convince ourselves and others it’s the right decision to make. But the reverse doesn’t work. There can be many rational arguments to make a certain decision, but when your gut is telling you it’s not the right thing to do, it often turns out to be the case.

So, what is our gut telling us about erwin Data Intelligence? From the first demo we fell in love with the unique way they approach data governance, with a holistic view on business terms, with governance flexibility really baked into the system, with fast responses to all our questions and with the right values. A mindset of collaboration. And a vision aligned with what lives and works in real-world organizations today. Not from an ivory tower.

The Conclusion

At Sparkle, we really believe erwin Data Intelligence is a very strong player in terms of data governance and metadata management solutions, backed by an intelligent team that continues to evolve the product in the right direction and is making strategic improvements. But, of course, perfection is an illusion, and every product can be improved. Through continued partnership, we’ll continue to advise the erwin by Quest team where our customers can gain even more value.