According to various sources, like this IBM post, 90 percent of the data in the world was created in the last two years. The main reasons are the rising popularity of social networks; our willingness to share more; and the type of information we create, consume and share. Let’s have a look at each of these in turn.
Rising popularity of social networks
Facebook, for example, grew from 197 million to 901 million active users within the space of three years (source: jeffbullas.com). Google+ figures tell a similar story; within a year, its user base grew from less than 50 million to 500 million (source: farotech.com). These are the two behemoths of social networking; however, the figures from other networks, such as LinkedIn and Twitter, suggest a similar story.
Willingness to share
This growth has made all of us much more comfortable using the networks, connecting with family and friends, and posting photos, videos, songs, etc.
In a sense, this is similar to e-commerce trends. In the early 2000s, only a small number of people were transacting over the Internet, and the security and privacy concerns were (justifiably) high. These days, few pay attention to how transactions occur. What is important is the result—a purchase made at an acceptable price with almost 100 percent guarantee of delivery.
Until recently creating and posting “rich media” for others to see and use required a certain amount of technical skill. At present, with a large percentage of populations (50.4 percent in the US, according to Nielsen) carrying smartphones in their pockets/handbags, it’s never been easier to create a three-minute video of little Johnny playing with his sister. The size of this video on iPhone 4 would be 240MB, while on iPhone 4S or 5 that figure would go to 540MB.
Even with compression—let’s say when uploading to Facebook—the data amounts are staggering compared to a spread sheet considered “state of the art” only a few years ago. (2005 seems prehistoric regarding “Internet chronology,” doesn’t it?)
This type of data explosion is certain to affect a growing number of professions. And information governance and security professionals must be among the first to adapt to this new reality. Organisations are already starting to recognise the importance of Big Data and how it can deliver competitive advantages. As a consequence, this realisation will further accelerate the drive to start using these technologies.
Thus, the question in the minds of many executives will become “How do we use it?” This much is certain—existing systems and infrastructure are not suited for Big Data purposes. Some of the more obvious options would be:
- Develop expertise in-house and introduce new infrastructure
- Outsource Big Data processing to companies specialising in this area
- A mixture of the above
- A new model, for example utilising Platform As A Service (PaaS), Software As A Service (SaaS) or something currently being dreamed/prototyped in a garage somewhere in the world
Any of these scenarios entail change. How this change is decided upon, implemented and then managed will be of concern to governance and security professionals. Why? They will need to provide advice, help other units within their organisation and monitor the new processes afterward.
This is a tide that is already irreversible. Our lives have already been enriched by new technologies—we are not going to stop sharing photos and videos. Organisations cannot afford to ignore Big Data because their external stakeholders have already embraced it.
I suspect many of us will soon ask for sensors to be attached to us so that we can monitor our blood pressure, or how well we exercise or sleep. Imagine teams of trained professionals monitoring our parents’ health, enabling them to live on their own for longer periods.
Big Data has made big promises. Now the general population is keen on seeing those promises delivered.
Chief Data Officer, Meta Business Systems
President, Board of ISACA-Brisbane
Note: For more information on Big Data, download ISACA’s free white paper here or visit the Big Data topic in ISACA’s Knowledge Center.