It is difficult to overstate the importance of data in today's economy. The tools we use and actions we take consume and generate a digital version of our world, all captured, waiting to be used. Data have become a real resource of interest across most industries and is rightly considered the gateway to competitive advantage and disruptive strategy.
Along with the rise of data has come two distinct efforts concerned with harnessing its potential. One is called Data Science and the other Big Data. These terms are often used interchangeably despite having fundamentally different roles to play in bringing the potential of data to the doorstep of an organization.
Although some would argue there is still confusion over the terms data science and big data, this has more to do with marketing interests than an honest look at what these terms have come to mean on real-world projects. Data science looks to create models that capture the underlying patterns of complex systems, and codify those models into working applications. Big data looks to collect and manage large amounts of varied data to serve large scale web applications and vast sensor networks.
Although both offer the potential to produce value from data, the fundamental difference between data science and big data can be summarized in one statement:
Despite this declaration being obvious, its truth is often overlooked in the rush to fit a company's arsenal with data-savvy technologies. Value is too often framed as something that increases solely by the collection of more data. This means investments in data-focused activities center around tools instead of approaches. The engineering cart gets put before the scientific horse, leaving an organization with a big set of tools, and a small amount of knowledge on how to convert data into something useful.
Bringing Ore to an Empty Workshop
Since the onset of the Iron Age, blacksmiths have used their skills and expertise to turn raw extracted material into a variety of valuable products. Using domain specific tools, the Blacksmith forges, draws, bends, punches and welds the raw material into objects of great utility. Through years of research, trial and error the blacksmith learned to use choice gases, specific temperatures, controlled atmospheres and varied ore sources to yield a tailored product bespoke to its unique application.
With the Industrial Revolution came the ability to convert raw material into valuable products more efficiently and at scale. But the focus on scaling wasn't the acquisition of more material. It was on building tools that scaled and mechanized the expertise in converting. With this mechanization came an even greater need to understand the craft since to effectively operate, maintain and innovate at scale one had to deeply understand the process of converting raw material into products that answered to the always-changing demands of the market.
In the world of data this expertise in converting is called data science. The reason it takes a science to convert a raw resource into something of value is because what is extracted from the 'ground' is never in a useful form. 'Data in the raw' is littered with useless noise, irrelevant information, and misleading patterns. To convert this into that precious thing we are after requires a study of its properties and the discovery of a working model that captures the behavior we are interested in. Being in possession of a model despite the noise means an organization now owns the beginnings of further discovery and innovation. Something unique to their business that has given them the knowledge of what to look for, and the codified descriptions of a world that can now be mechanized and scaled.
Conversion Should Scale Before Collection
No industry would invest in the extraction of a resource without the expertise in place to turn that resource into value. In any industry, that would be considered a bad venture. Loading the truck with ore only to have it arrive at an empty workshop adds little strategic benefit.
An unfortunate aspect of big data is that we look to the largest companies to see what solutions they have engineered to compete in their markets. But these companies hardly represent the challenges faced by most organizations. Their dominance often means they face very different competition and their engineering is done predominantly to serve large-scale applications. This engineering is critical for daily operations, and answering to the demands of high throughput and fault-tolerant architectures. But it says very little about the ability to discover and convert what is collected into valuable models that capture the driving forces behind how their markets operate. The ability to explain and predict an organization's dynamic environment is what it means to compete using data.
Understanding the distinction between data science and big data is critical to investing in a sound data strategy. For organizations looking to utilize their data as a competitive asset, the initial investment should be focused on converting data into value. The focus should be on the data science needed to build models that move data from raw to relevant. With time, big data approaches can work in concert with data science. The increased variety of data extracted can help make new discoveries or improve an existing model's ability to predict or classify.
Fill the workshop with the skills and expertise needed to convert data into something useful. The ore brought here will become the products that define a business.
An earlier version of this article was first published on KDNuggets.
Disclaimer: The statements and opinions expressed in this article are those of the author(s) and do not necessarily reflect the positions of Thoughtworks.
Ni vigumu kupita kiasi umuhimu wa data katika uchumi wa leo. Vifaa tunavyotumia na vitendo tunavyochukua hutumia na kutengeneza toleo la dijiti la ulimwengu wetu, yote yakikamatwa, yakisubiri kutumiwa. Takwimu zimekuwa rasilimali halisi ya maslahi katika sekta nyingi na kwa haki huchukuliwa kuwa lango la faida ya ushindani na mkakati wa kuvuruga. Pamoja na kuongezeka kwa data kumekuja juhudi mbili tofauti zinazohusiana na kutumia uwezo wake. Moja inaitwa Sayansi ya Takwimu na nyingine Takwimu Kubwa. Maneno haya mara nyingi hutumiwa kwa kubadilishana licha ya kuwa na majukumu tofauti ya kucheza katika kuleta uwezo wa data kwa mlango wa shirika. Ingawa wengine wangesema bado kuna kuchanganyikiwa juu ya maneno ya sayansi ya data na data kubwa, hii ina zaidi ya kufanya na maslahi ya uuzaji kuliko kuangalia kwa uaminifu nini maneno haya yamekuja kumaanisha kwenye miradi ya ulimwengu halisi. Sayansi ya data inaangalia kuunda mifano ambayo kukamata mifumo ya msingi ya mifumo tata, na codify mifano hiyo katika maombi ya kazi. Takwimu kubwa inaonekana kukusanya na kusimamia kiasi kikubwa cha data mbalimbali ili kutumikia maombi makubwa ya mtandao na mitandao kubwa ya sensorer. Ingawa zote mbili hutoa uwezo wa kuzalisha thamani kutoka kwa data, tofauti ya msingi kati ya sayansi ya data na data kubwa inaweza kufupishwa katika taarifa moja: Ingawa taarifa hii ni dhahiri, ukweli wake mara nyingi hupuuzwa katika haraka ya kukidhi silaha ya kampuni na teknolojia ya data. Thamani mara nyingi huelezwa kama kitu kinachoongezeka tu kwa kukusanya data zaidi. Hii ina maana uwekezaji katika shughuli data-kizingatia kituo karibu zana badala ya mbinu. "Kama ""ngoma ya uhandisi"" inawekwa mbele ya ""farasi wa kisayansi,"" inaacha shirika na seti kubwa ya zana na kiasi kidogo cha maarifa juu ya jinsi ya kubadilisha data kuwa kitu muhimu." Kuingiza madini katika karakana tupu Tangu mwanzo wa Enzi ya Chuma, wasanii wametumia ustadi wao na ujuzi wao kugeuza malighafi zilizochimbuliwa kuwa bidhaa mbalimbali zenye thamani. Kwa kutumia vifaa hususa, mfinyanzi hutengeneza, kuchora, kugeuza, kupiga ngumi na kulehemu vifaa hivyo kuwa vitu vyenye manufaa sana. Kupitia miaka ya utafiti, jaribio na makosa, fundi wa chuma alijifunza kutumia gesi za kuchagua, joto maalum, hali ya hewa iliyodhibitiwa na vyanzo tofauti vya madini ili kutoa bidhaa iliyoboreshwa kulingana na matumizi yake ya kipekee. Pamoja na Mapinduzi ya Viwanda alikuja uwezo wa kubadilisha malighafi katika bidhaa zenye thamani kwa ufanisi zaidi na kwa kiwango. Lakini lengo la kuongeza ukubwa halikuwa upatikanaji wa vifaa zaidi. Ilikuwa juu ya kujenga zana ambazo zilipanda na kuunganisha utaalam katika kubadilisha. Pamoja na uendeshaji huu wa mashine, ilikuja haja kubwa zaidi ya kuelewa ufundi, kwani ili kuendesha, kudumisha na kubuni kwa ufanisi kwa kiwango, mtu alikuwa na kuelewa kina mchakato wa kubadilisha malighafi kuwa bidhaa ambazo zilijibu mahitaji ya soko yanayobadilika daima. Katika ulimwengu wa data utaalam huu katika kubadilisha inaitwa sayansi ya data. Sababu inachukua sayansi kugeuza rasilimali ghafi katika kitu cha thamani ni kwa sababu kile kinachotokana na 'ardhi' kamwe hakikuwa katika fomu muhimu. 'Data katika mbichi' ni littered na kelele zisizo na maana, habari zisizo na maana, na mifumo ya kupotosha. Ili kugeuza hii kuwa kitu cha thamani tunayofuata inahitaji utafiti wa mali zake na ugunduzi wa mfano wa kazi ambao unashika tabia tunayoipenda. Kuwa na mfano licha ya kelele inamaanisha shirika sasa linamiliki mwanzo wa ugunduzi zaidi na uvumbuzi. "Kitu cha kipekee kwa biashara yao ambacho kimewatoa maarifa ya nini cha kutafuta, na maelezo yaliyoorodheshwa ya ulimwengu ambao sasa unaweza kuwa na mashine na kupandishwa.""" """Hakuna sekta ambayo ingewekeza katika uchimbaji wa rasilimali bila utaalam wa kugeuza rasilimali hiyo kuwa thamani." Katika sekta yoyote, hilo lingehesabiwa kuwa jambo baya. Kupakia lori hilo madini ili lifikie kwenye karakana tupu hakuna faida kubwa ya kimkakati. Jambo la kusikitisha la data kubwa ni kwamba tunatazama kampuni kubwa zaidi kuona ni suluhisho gani wamebuni ili kushindana katika masoko yao. Lakini makampuni haya hayawakilishi changamoto zinazokabiliwa na mashirika mengi. Utawala wao mara nyingi humaanisha wanakabiliwa na ushindani tofauti sana na uhandisi wao hufanywa hasa kuhudumia maombi ya kiwango kikubwa. Uhandisi huu ni muhimu kwa ajili ya shughuli za kila siku, na kujibu mahitaji ya throughput ya juu na makosa-uvumilivu usanifu. Lakini inasema kidogo sana juu ya uwezo wa kugundua na kubadilisha kile kilichounganishwa kuwa mifano yenye thamani ambayo inashika nguvu za kuendesha nyuma ya jinsi masoko yao yanavyofanya kazi. Uwezo wa kuelezea na kutabiri mazingira ya nguvu ya shirika ni nini inamaanisha kushindana kwa kutumia data. Kuelewa tofauti kati ya sayansi ya data na data kubwa ni muhimu kwa kuwekeza katika mkakati wa data ya sauti. Kwa mashirika yanayoangalia kutumia data zao kama mali ya ushindani, uwekezaji wa awali unapaswa kuzingatia kubadilisha data kuwa thamani. Lengo linapaswa kuwa juu ya sayansi ya data inahitajika kujenga mifano ambayo hoja data kutoka ghafi kwa muhimu. Baada ya muda, mbinu kubwa za data zinaweza kufanya kazi kwa kushirikiana na sayansi ya data. Tofauti ya data iliyotokana inaweza kusaidia kufanya uvumbuzi mpya au kuboresha uwezo wa mfano uliopo wa kutabiri au kuainisha. Kujaza semina na ujuzi na utaalam inahitajika kugeuza data katika kitu muhimu. Madini yanayoletwa hapa yatakuwa bidhaa zinazotambulisha biashara. Toleo la awali la makala hii lilichapishwa kwanza kwenye KDNuggets. Taarifa na maoni yaliyoonyeshwa katika makala hii ni ya mwandishi (s) na si lazima kutafakari msimamo wa Thoughtworks.