Information, information everywhere. There has been an exponential growth of information since the dawn of the internet era. From the early development of recording devices, data has been circulated and understood all across the globe. In the past floppy disks were the source of sharing data, soon Compact disks came into the picture with much larger capacity and then the advancement kept on happening. With storage devices becoming smaller but capacity becoming larger it’s been an era of technological revolution.
Floppy Disks had 1.44 MB data storage capacity.
Zip Drives had 100-250 MB capacity.
Compact Disks had 700 MB capacity.
And now, there are devices smaller than a human thumb which can store terabytes of data at once.
All these due to the innovation in technology. This has led a lot of information being sent, received and stored. A whopping 2.5 Quintillion bytes of data are generated every day (source: vcloudnews.com). These are not merely stored in drives and disks rather in a more intangible space called the ‘Cloud’.
Cloud storage is a new concept. The data stored in the cloud are physically in servers and on the internet throughout the globe. These data patches are sporadically scattered and recalled when needed.
For better understanding take the Internet for example, where a user sitting in one corner of the world can access the world’s information just after a few clicks. The data searched for is stored in places (servers) and are updating at a rapid rate. As we speak millions and millions of data packets are being sent, received and validated.
What does it all have to do with Big Data and what’s the physical significance anyway?
Well, you see this data is large, unaccountably large. If I had to explain in simple terms, the data in the cloud from all the devices we use to communicate and find information is just massive. But, a large amount of it is important to take a sound decision. At first glance, the data looks like random entries but if assigned to an algorithm, one can easily find patterns to use it to own benefit.
A decoder of Big Data networks can identify useful data to use for his/company’s advantage. The data if validated properly can range from Financial Advisory to Latest Trends, from Discoveries to Reason of Extinction, from Medical to Industrial strategies and much more. The whole conundrum arises on how to understand this raw data.
An IBM study visualized daily Wikipedia edits for a year, multiple terabytes of data were recorded to understand big data. This is what they got:
To give you a frame of reference, the colors signify the edited words and that’s just 1,06,776 data packets analyzed.
Solving the Big Data problem can do more good than harm. With the inception of Quantum computing, it can be achieved in the near future.
What are the aspects of Big Data?
The concept gathered a lot of hush in the early 2000s when industry analyst Doug stated the definition of big data as the three Vs:
Volume- There is an abundance of data ranging in various dimensions, like business transactions, social media, and data from sensor or machine-to-machine data. In the past, storing it was a problem – but new technologies have eased it up.
Velocity- Data streams in at a rapid pace and must be dealt with in a timely and precise manner. RFID tags, sensors, and smart gadgets are pushing the need to handle the torrents of data in real-time.
Variety- The formats of data are in abundance, from numeric data in traditional databases, structured, to unstructured docs, emails, audio, video and financial transactions.
Why is it important?
Here is a hypothetical scenario –
Let’s assume a person has a medical issue, say cancer. What does he do? He visits the doctors, goes under medical supervision and takes medicine. Pretty standard right.
Now, enters Mr. Big Data – with all chats and online discussions on Cancer and Cancer Treatment, still updating as a doctor just found a treatment and posted online, millions of testimonials analyzed to understand the physiological aspects of the patients.
Mr. Big Data summarizes all these and gives a feedback on 1 A4 size page with 12px font size.
What does the patient get?
He/she gets a real-time update on what to do next. He/she can decide on whether to continue with the current treatment/current medicine/current doctor/current hospital and how to cure himself/herself both mentally and physically, whom he/she can talk to for advice. Basically everything.
So you see, big data is not just random data if decoded and understood it can be life-changing.
So are there any companies adopting this Big Data?
Yes, many companies are doing this. They are consolidating all the data to give a better experience to their consumers, know the problem before it happens and be ahead of the curve.