Have you ever wondered how accurate is the data you have access to in your organization? How often have you send the same document twice because of the wrong addresses in the system? Do the e-mails you send even get to the end-users or are you spending your money and time for nothing?
These are just some of the questions related directly to data quality. On the other hand, data quality management inside the organization depends on how the problem is recognized. Even if somebody sees the problem, they usually ignore it or don’t know how to solve it. A frequent problem is the fact that there isn’t a control point, in other words, someone who would control and manage data quality.
The result of not understanding the process of data quality is inaccurate data, which leads to misinterpretation of business results, and more important, it interferes with the insight of new added values or new potential problems. The problem with data is usually seen when it’s already too late to act, when the competition has already started to act.
What is the solution to data quality problem?
To get a concrete solution when it comes to data quality management, it is important to clarify to as many people possible to refer to data in the same way they would refer to other strategic assets or company’s resources. It is very important to define what is expected of data, using data quality metrics.
When establishing data quality control process, it is important to distance from implementation, functionality and problems inside the existing system – you should start from the very beginning. We should perceive data quality management as a multidisciplinary activity, that will constantly function, from the moment process has begun to its very end. We must act horizontally. Apart from that, data quality process must start from the question of what exactly do we want, what the key goals and KPIs, that are in sync with the company’s vision, are, so we could focus on important artefacts we need to propagandize to the lowest operative level. The next important step is understanding the business needs, defining and prioritizing key data sets and defining what the key disadvantages and what the possible benefits are. In parallel, it is necessary to define the data quality strategy, in which the management principles, that are in sync with important KPIs inside the organization, are launched. At the same time, the process should be led by the premise that data sets are an asset which has a clearly defined expectation and quality, and which is managed through a sustainable and continuous process of integration.
Nothing without strategy
Our DWH department has put together a consultant package, with the purpose of defining a way of programme development for data quality management, as a part of the ecosystem that would unite all key participants, processes and data into a sustainable and quality unit. The package defines key goals, describes the development plan and gives guidance recommendation on how to efficiently come to a sustainable process for data quality management.
The goal in client’s business environment is to create a clear data quality management strategy so it would be possible to:
- detect anomalies in data in time
- perceive different interpretation of the „same“ information in time
- define the satisfactory level of data quality using SLA for data quality which affects given information
- to create a good base for continuous improvement of data quality
DataTesThink application for data quality control and management
Regardless of the business field, every organization in the market needs quality data for competitiveness. We recently presented a unique data control and management application – DataTesThink. We didn’t find an adequate solution for our needs, so we created it ourselves! In contrast to other similar solutions in the market, DataTesThink application gives a series of innovations and functionality, which improves and accelerates the data quality management process.
Apart from the basic functionalities such as creating different text types, the application gives us extra individual solutions for report preparation (BI reports, control options), enables a quick reaction, i.e. printing of analytics in case of inaccurate data, gives an option for creating different text types such as automatic SQL generating, writing complex instructions and a check of recordings in real time.
One of the options that simplifies the testing is interactive creating of the test script, which allows the hierarchical performance of the test sets. Based on that, we can „pack“ tests into units for every change request or autonomous unit (functionality), and by doing that, clasp a bigger scope of testing, so we could cover the maximum impact on CR or the functionality we are testing.
We have tried to create an intuitive user interface which allows a simple entry and data collating.
A display of results in the DataTesThink application user interface.
DataTesThink application allows generating the display of results in a form of graphs, and by that gives information, in a visual way, about whether the data is becoming more accurate and quality in relation to previous stages.
A simple test entry, creating and managing test cases benefit the increase of data’s spreading rate and its accuracy.
Key advice
Data quality is hard to measure, so to establish a sustainable process of data quality, a lot of effort has to be put in, that is, to make the data „tangible“ to a number of participants in the process.
The key is to establish a sustainable process of data quality control, led strategically and run by methodology and framework, which integrates processes and organisation units within your environment the best, and which is conducted by concrete planned activities (iterations). The emphasis is on the assumption that the quality control process should be managed continuously, in small steps, and it can not be performed by one project with a fixed expiration date.
Finally, a message to all of those who handle data on a daily basis – you should be brave and start from a good foundation, which, with a help from (our) experts on data quality, and with our tools, you can establish in a short period of time.
