Taking care about data quality
Description:
Data quality is key to any software or data science related task. Also in software development research, findings are based on data. And if the data used for analysis does not meet specific standards, results might be biased or wrong. Therefore, it is of utmost importance to know methods and tools for assessing and improving the quality of data before using it.
Links:
https://www.frontiersin.org/articles/10.3389/fdata.2022.850611/full
Keywords:
Data quality, software quality, analytics quality
Motivation:
My main research area is on data quality, but I worked a software developer before and know how important it is to care about the quality of data used from databases.
Requirements/Prerequisities:
Domain experts to validate data quality
Level:
generic: high level abstract best practice, metalevel category (e.g. manage architectures)
Application domain:
Medicine/Healthcare, Education (Technology enhanced learning), Data science (analysis & visualisation), Industry (Production), Mobility, Energy, Software engineering
Main phase:
Generic: Requirements/Exploration, Generic: Design/Plan, Data Science: Preparation/Integration, Data Science: Modeling/Training/Evaluation, Development: Implementation/Code/Build, Development: Testing, Operations: Deployment/Release, Operations: Maintenance/Monitor
Related literature:
Ehrlinger, L., & Wöß, W. (2022). A survey of data quality measurement and monitoring tools. Frontiers in Big Data, 28.
https://www.frontiersin.org/articles/10.3389/fdata.2022.850611/full
Ehrlinger, L., & Wöß, W. (2017, October). Automated Data Quality Monitoring. In ICIQ: https://www.researchgate.net/profile/Wolfram-Woess/publication/323316592_Automated_Data_Quality_Monitoring/links/5a8d70020f7e9b27c5b4b266/Automated-Data-Quality-Monitoring.pdf
Sebastian-Coleman, L. (2012). Measuring data quality for ongoing improvement: a data quality assessment framework. Newnes.
In which projects do/did you use this practice?
In all of my current research projects (main project: COMET project Sebista)
Project manager / Researcher
>10 years of experiences
Software Competence Center Hagenberg
| 1. How do you rate the potential benefit for your projects? | 5 |
| 2. How often are you using that practice? | 5 |
| 3. What is the effort to introduce the practice in your project upfront? | 3 |
| 4. What is the effort to apply the best practice in your project daily basis? | 4 |
Questions 1, 3 and 4 (1 = Low, 5 = High)
Question 2 (1 = Never, 5 = Always)