The advent of modern computer networks and the democratisation of information has exponentially multiplied the amount of digital data being generated. According to a Cisco estimate global IP Traffic will reach 2 zettabytes by 2019. (1 zettabyte = 1 billion terabytes).
The term Big Data has come into daily parlance owing to a shift in analysing structured relational databases toward working with massive quantities of unstructured databases generated through a variety of sources. There developed an urgent need to build tools that analyse and interpret such unstructured datasets.
Always connected portable devices have expanded the opportunities for inputs to the internet massively. Data, today, is being referred to as the lifeblood of digital organisations and the currency of the new digital economy, suggesting a level of significance and value distinctly higher than in previous times. While the price of computing continues to fall, data driven business models are becoming attractively affordable.
Research firm, Markets and Markets, tells us that Enterprise Data Management including software and services for migrating, warehousing, integrating and analysing all forms of data will approach USD 105 billion by 2020.
However, one of the major challenges to handling Big Data is a paucity of skills required in the workforce. Usage in industry is moving away from SQL (Sequential Query Language) to manage structured relational database sets to management and analysis of unstructured data. In order to handle Big Data, we need to anticipate the three phases of data analytics, viz., collection and storage; processing and organising and analysis and visualisation.
Since data analytics and management are relatively older areas in the tech world as compared to Artificial Intelligence, cloud systems and the Internet of Things, employers are somewhat aware of the skills that would be required. Nevertheless, the requirements of today and the future will depend more on the analytics part.
The most important question that we can then ask is: How can we intelligently use huge volumes of data for business transformation covering incremental revenue, trimming costs, enhanced and informed decision making and target marketing. Therefore, much of the skill gaps around data-handling now rest on real time analytics, predictive modeling, data security, distributed storage and data mining.
Data Interpretation and Visualisation encompasses job functions such as data analytics and business intelligence. In this domain, the competency requirement will emphasise upgrading the instruments of data analytics and warehousing, for example, Netezza, MicroStrategy, SAS and Tableau.
Data Processing and Management encompasses job functions like database administration, creation, data integration and data lifecycle management.
Workers in this profession must work on database software and platforms like Microsoft, Oracle, Cloudera and Hadoop.
Data Infrastructure is the platform upon which data management lies and job functions covering data storage, data centre management, business continuity and data security are emerging.
In order to gain entry into this interesting and much-in-demand field one would need to go through a comprehensive program designed and delivered by leading experts in the field. Different roles require different focus competencies and one must choose accordingly.