Insurance Company - Implementation of the Data Catalog
Today, there are no cloud platforms without a Data Catalog as a service. Over time, numerous implementations of Data Catalogs on the market with different approaches and tools simplified the establishment of Data Governance policies and standards.
Our client is one of Nordic's largest non-life insurance companies. They provide insurance services and create secure conditions for companies to grow.
In our project for the insurance company, we were guided by the Business and IT needs and opportunities. The main goal was to invent a Data Catalog which would enable the organization of the Knowledge Catalog as part of a modern data platform with the help of taxonomy and ontology in the form of classification and Business Glossary.
The main challenges of the project:
- Design a data catalog in a way that it’s fully aligned with the central Data Governance repository
- Convince and educate the Business audience that the Data Catalog is an added value and adequately connects and recognizes such added value with innovative products
- Design Data Contract as a set of integrated product data used (critical data elements portfolios)
Step by step, with many MVPs, the project was developed to include and educate business quickly. For example, to enable “Fast failed” and modularly expand according to business needs that result in added value ASAP.
Examples of MVPs and functionalities:
- Integration Data Catalog with central Data Governance Repository in a way to be harmonized with the existing Data Governance methodology and defined policy, standards and processes
- Installation and configuration of Apache Atlas in several environments and integrating Apache Atlas with Apache NiFi, Apache Kafka, Hive, HDFS, Spark, Ranger
- Implementation of the standard Business Glossary for Insurance Industry according to ACORD standards and IBM Industry model
- Implementation and preparation of major data classifications for business and technical stakeholders
- Creating a service for synchronizing the central data management repository and the Apache Atlas using the REST API
- Defining Data Contract on the principle of publishing the subscribe concept between production and product consumers and defining Data Contract Graph between Data products.
- Construction of a unified authorization model that would include access rights (policy) and dynamic masking based on data classification within the data catalog at the lowest level of detail such as PII / GDPR, data storage, geolocation (data privacy)
We created a good baseline to make the Data Catalog expandable and sustainable, to be a “plug-in and play” for the new Data Mesh concept, and they eventually became a real and usable Knowledge Catalog. We have many ideas and planned MVPs in our mind to support Businesses to create fundamental values and innovative data products.
Technologies we used
Get in touch
Want to hear more about our services and projects? Feel free to contact usContact us