Yes, it is just as the name suggests big data is a whole lot of data. What kind of data is it? Well, it could be data from an abundance of sources as well as from different platform-based observing systems, including satellites, and in-situ measurements.
What does big data do? The huge body of data can be used to analyze what no man till date has – it offers to delve into hidden and unprecedented opportunities. We can better understand, analyze and predict various patterns/phenomena in different spatial-temporal scales.
For example, when browsing at a shopping website you find that each time you visit the site; products of your interest are coming up. How do you think the computer presents this? Through a detailed analysis of what you have been searching for. The first step towards feeding human intelligence into a machine is through big data – large, unwieldy, multifaceted and layered data, that can be transmitted in the blink of an eye, through systems, sensors, mobile devices and more. And so ‘big data’ is being generated by devices all around us at all times.
Every digital process and social media exchange produce it. Big data is arriving from multiple sources with remarkable speed, volume, and variety. If we are to extract meaningful value from big data we need optimal processing power along with analytic capabilities and skill.
Using big data is a very challenging task. Think about how many individuals search the computer each day for various items. It is manually impossible to glean through this data and arrive at any cognitive results. Thus, data mining techniques are needed to facilitate the automatic extraction and analysis of specific information or patterns in this data.
Say, for example we need to know what the internet is being used for every morning, or after a disaster, or during political unrest in any given location in the world. An algorithm or a system is developed to find meaningful patterns that will lead us to what we are looking for. For example, this will tell us that promoting a top jewellery brand in the morning may not be a great idea. However, if it is a news update service, morning may just be the right time for promoting it.
Now, not everything is as simple. First, news updates may not appeal to everyone. Then again, human behaviour is subject to change. On a Sunday morning, users may be specifically looking to visit jewellery stores. Therefore, large, often noisy (erroneous, misleading), and heterogeneous datasets, are complex and difficult to interpret. For this reason design of efficient algorithms is essential.
The enormous growth of data in both commercial and scientific sectors are also been driven by advances in data generation and collection technologies. The current approach is to gather maximum data with the expectation of a value addition either for the purpose collected for, or its future use.