Informatica aims to better track data lineage with AI-powered data catalog

December 10, 2019

How to win with prescriptive analytics

Analytics has evolved from the basics — visualizations, historicals and dashboards — to the more complex with recommendations and predictions of outcomes. Now it’s time to step it up and get prescriptive.

Informatica is updating its Intelligent Data Platform with tools to track data lineage, catalog information across various ecosystems and automation and artificial intelligence for improving customer records.

The Informatica updates come a week after the company expanded a partnership with Amazon Web Services to support the cloud giant’s databases and data lake and data warehouse services.

Informatica is a key enterprise layer in enterprise data management. Enterprise data has swelled to the point there it has outgrown the big data terminology. AWS CEO Andy Jassy in his keynote referenced huge data a few times, but that may not even capture what’s happening.

Among Informatica’s new features, its AI-powered data catalog, called Catalog of Catalogs is notable because it is trying to track data lineage across ecosystems. Catalog of Catalogs includes metadata scanners for business intelligence, data warehouses, big data and third party repositories. Here’s the gist in a slide:

The Informatica data catalog also includes profiling on Spark and the ability to use machine learning to find relationships between datasets and business entities.

Informatica’s bet is that AI can better scale data productivity and management. Amit Walia, president of products and marketing at Informatica, said:

AI and machine learning play a critical role in scaling the practices of data management. You cannot solve today’s data management challenges by simply throwing more engineers and developers at the problem. Traditional approaches to data management are inefficient. Projects are implemented in silos, with little end-to-end metadata visibility and limited automation. There is no learning, processing is expensive, governance and privacy steps are repeated over and over again. You need AI to automate and simplify tasks related to data management.

Other additions include:

A data marketplace within Informatica’s Axon Data Governance to access governed enterprise data. Automation makes datasets easier to find with transparency into the source, quality, sensitivity and access rights. Data categorization tools are also added.
Data quality updates to cover hybrid mapping of data, app integration, API management, Spark processing, data ingestion, cleansing, profiling and verification and integration.
Support for the latest spark engine across AWS and Microsoft Azure and support for data engineering use cases.
Customer data automation tools to enhance records, personalize and add reporting and analytics for customer segments.