As internet usage continues to grow exponentially and become entrenched in daily life, the amount of data generated and stored, over-time, will undoubtedly follow a similar trajectory.
The term “big data” is used to describe data sets, that are so large and complex, that traditional methods of processing this data become inadequate.Similarly, as the ability to store data increases, so to does the amount of data available to store. As mentioned in class, Parkinson’s First Law states that data expands to fill the space available for storage. This remains true, as storage is getting cheaper and data compression is getting better.
“Big Data” is typically described by using Doug Laney’s 3 Vs:
- Volume: the sheer amount of data collected using various methods and from various mediums.
- Velocity: data is being collected constantly at overwhelming speeds and need to be organized in an efficient and timely manner.
- Variety: data comes in many different forms, and it can be structured or unstructured, related or unrelated, and qualitative or quantitative.
For businesses and organizations, analyzing all of this ‘big data’, compiled from transactions, and interactions with consumers, is becoming key in understanding and identifying trends, identifying strengths and weaknesses, and making decisions. Understanding and making sense of big data allows companies to see aspects of their business that haven’t seen before, and allows them respond to problems they didn’t know existed, or take advantage of strengths they didn’t know they had.