Dark data is the new frontier for big data analytics.

Illustration of a man sitting at a computer with multiple screens analysing data.It’s been a long time since IT departments were nothing more than a few computer programmers relegated to a back room. Now, information technology is front and center at many businesses, and computer science jobs include a lot more positions than computer programming.

One of the fastest-growing areas in information technology is data analysis, with IBM predicting that demand for data scientists will increase 28% by 2020.* This expansion is being fueled by the increase in available data. And one of the most promising sources of this data is what’s known as dark data.

Dark Data Is Unused Data

The “dark” in “dark data” means unseen/kept in the dark. It applies to any data that’s being stored but isn’t being used and hasn’t been fully analyzed. Experts estimate that 52% of all data possessed by organizations is dark data, although the percentage could be much higher than that. The fact is, no one knows how much dark data is out there. Nor do they know how much of it could be valuable if properly analyzed.

Most Dark Data Is Unstructured

One of the primary reasons organizations don’t analyze their dark data is that most of it is unstructured. In other words, it exists in a format that isn’t geared for data analysis. A business’ word processing documents and e-mails are a good example of unstructured data. All those memos and conversations contain plenty of data, but that data isn’t formatted to be pulled apart and manipulated, and thus it’s difficult to analyze. Photographs, video, and audio also fall into the unstructured category, as does much of the data transmitted by devices in the Internet of Things.

Dark Data Exists on the Web

The unused data stored by organizations isn’t the only repository of dark data. Plenty of dark data resides on websites, too. While search engines index sites, they don’t analyze the information on those sites. Plus, not all sites appear on traditional web browsers. And yet, websites can possess a lot of useful data, such as reports from academics, governments, nonprofit organizations, and other groups. Additionally, websites—particularly social media sites and message boards—store all kinds of conversations and other interactions that are ripe for data analysis.

Dark Data Analysis Can Help Businesses Succeed

Dark data’s value is in its potential to provide businesses with new insights. For example, an analysis of a decade’s worth of e-mail exchanges with customers might uncover patterns that a business could then use to predict future market shifts or address problems before customers even begin to complain. Likewise, an analysis of social media conversations could reveal unmet consumer needs, which a business could address before its competition does. The more data available, the more easily data analytics can create models that increase a business’ overall knowledge and improve its ability to make smart decisions.

We Need More Data Analysis Leaders

Analyzing dark data takes two key elements: The creation of methods capable of sorting through vast amounts of unstructured data, and data analysis experts who know how to ask the right questions. Currently, most businesses not only lack the technological means to analyze their dark data, they lack the people whose understanding of data can make analysis worthwhile. After all, sorting data does little good if you don’t first know what you’re trying to find. The need for both technical solutions and data experts is one reason why, in a recent survey of chief information officers, CIOs identified the hiring of people with data analytics skills as their top recruiting priority.

How You Can Become a Data Analysis Leader With an MS in Information Technology

If you’re interested in leading the future of big data analytics, you should consider earning a master’s in information technology. This advanced IT degree is much more than a computer science degree. It’s your opportunity to gain an in-depth understanding of the modern challenges in information technology, which can prepare you for a leadership role as a data scientist, database administrator, systems analyst, systems administrator, or information technology executive.

If you’re worried that earning a master’s degree might require you to take too much time away from your current IT career, online education can be the solution you need. That’s because when you earn an MS in Information Technology degree online, you can complete the majority of your MS degree from home and on a schedule designed to allow you to continue working full time. It’s what makes online learning such a popular choice for working adults.

The analysis of dark data could revolutionize how organizations operate. When you earn your MS in Information Technology at an online university, you can put yourself at the forefront of that data revolution.

Walden University is an accredited institution offering an MS in Information Technology degree program online. Expand your career options and earn your degree in a convenient, flexible format that fits your busy life.


*L. Columbus, IBM Predicts Demand For Data Scientists Will Soar 28% By 2020, Forbes, on the internet at www.forbes.com/sites/louiscolumbus/2017/05/13/ibm-predicts-demand-for-data-scientists-will-soar-28-by-2020/#3241d8a97e3b.

† Veritas, Veritas Global Databerg Report Finds 85% of Stored Data Is Either Dark or Redundant, Obsolete, or Trivial (ROT), on the internet at www.veritas.com/news-releases/2016-03-15-veritas-global-databerg-report-finds-85-percent-of-stored-data.

‡ T. Kambies, et. al., Dark Analytics: Illuminating Opportunities Hidden Within Unstructured Data, Deloitte Insights, on the internet at https://dupress.deloitte.com/dup-us-en/focus/tech-trends/2017/dark-data-analyzing-unstructured-data.html.

Walden University is accredited by The Higher Learning Commission, www.hlcommission.org.

Man researching online education on laptop
Request Free Information
Request Free Information

Fill out this form to get free information on courses, admissions and financial aid from your personal advisor.


Please use our International Form if you live outside of the U.S.

Fill out this form to get free information on courses, admissions and financial aid from your personal advisor.


Please use our Domestic Form if you live in the U.S.
*Required Field
Academic Interests
Required
Required
Contact Information
Required
Required
Invalid email address
Please provide a valid phone number
Required
Please provide a valid zipcode
Required
Additional Information
Required
Submit

By submitting this form, I understand and agree that Walden University may contact me via email, text, telephone, and prerecorded message regarding furthering my education and that calls may be generated using automated technology. You may opt out at any time. Please view our privacy policy or contact us for more details.

Required