What is dark data? It could be your organization’s greatest asset, if you know where to find it. We’ll tell you where.
There are insights about your business and EHSQ processes hiding in plain sight… if you know where to look.
Data that is collected — but never shared, mined, or leveraged to gain business insights — is called dark data. Every organization has dark data, which translates to missed opportunities for learning, insights, and performance improvement. Although it’s mostly discussed in the context of expanding the use of Internet of Things (IoT) devices in industrial contexts, there is another
Since storage has become relatively inexpensive (particularly cloud storage), the challenge presented by dark data is growing. While much of dark data resides in enterprise data warehouses, and databases that support Customer Relationship Management (CRM) and Enterprise Resource Planning (ERP) systems, there’s another source that is even more compelling: spreadsheets.
Spreadsheets, not IoT devices, are the number one source of dark data in most organizations today. Collecting data electronically is a good thing, though. It’s better to capture data in ways that can be emailed and archived, rather than on paper and clipboards that can be damaged, lost, or easily forgotten. But without the ability to search dark data for insights, the value of that data is greatly punctuated.
Extracting insights and meaning from dark data is one of the current research thrusts in machine learning. Until those methods are perfected, which may take several years, you can still take the first steps towards driving value from your dark data: help people make the transition from spreadsheets to centralized software systems or repositories. Any data management tool that makes it possible to access and search for data that was previously “dark” — stuck on someone’s laptop or other inaccessible places — will help you position your organization for these exciting advancements.
Freeman, G. (2018). Integrating Quality and Safety in Organizational Culture: A Cross-Industry Look. Intelex Insight Report. Available from https://www.intelex.com/resources/insight-report/integrating-quality-and-safety-organizational-culture
Freeman, G. (2018). How to Leverage QMS Software to Promote a Culture of Quality. Intelex Insight Report. Available from https://www.intelex.com/resources/insight-report/how-leverage-qms-software-promote-culture-quality
Hoyle, D. (2017). ISO 9000 Quality Systems Handbook – Updated for the ISO 9001: 2015 standard: Increasing the Quality of an Organization’s Outputs. Routledge.
About the Author: Nicole Radziwill is the Vice President, Global Practice Leader, Quality & Supply Chain at Intelex Technologies. Before Intelex, she was an Associate Professor of Data Science and Production Systems, Assistant Director (VP) End-to-End Operations at the National Radio Astronomy Observatory (NRAO), and manager and consultant for several other organizations since the late 1990’s bringing quality management to technologically-oriented operations. She is a Fellow of the American Society for Quality (ASQ) with a Ph.D. in Quality Systems from Indiana State University. Nicole serves as Editor of Software Quality Professional (SQP) journal and is a former Chair of the ASQ Software Division. She is an ASQ Certified Manager of Quality and Organizational Excellence (CMQ/OE) and Certified Six Sigma Black Belt (CSSBB).
This material provided by the Intelex Community and EHSQ Alliance is for informational purposes only. The material may include notification of regulatory activity, regulatory explanation and interpretation, policies and procedures, and best practices and guidelines that are intended to educate and inform you with regard to EHSQ topics of general interest. Opinions are those of the authors, and do not necessarily reflect the opinion of Intelex. The material is intended solely as guidance and you are responsible for any determination of whether the material meets your needs. Furthermore, you are responsible for complying with all relevant and applicable regulations. We are not responsible for any damage or loss, direct or indirect, arising out of or resulting from your selection or use of the materials.
Would you like to become a member of the EHSQ Community? Sign-up is free and easy.