Big Data innovations continue to drive business intelligence and integrate into everyday life. Whether you are an experienced data scientist or an aspiring one, whether you are in big business or a one-man shop, whether you are worried about your weight or what your government is doing – Big Data is a part of everyone’s future.
Big Data made a Big Difference in the biggest story of 2016 – the US Presidential election. Although President Trump had pooh-poohed the impact of Big Data during his initial campaign, he rallied a last minute expert team just months before the polls that just may have made the difference.
Using sophisticated analytics and digital targeting, President Trump’s technology strategy collected characteristics from online and offline sources to find potential voters. With over 4,000 finely tuned messages, a specific one was placed after assessing the potential voter’s Facebook, Pandora and snapchat activity. Virtual grassroots at its finest.
Bringing Big Data to the people. So, what is Big Data and what concepts do you need to know right now?
What is the big deal about Big Data?
Big Data is the collective term for the accumulation, processing and utilization of lots and lots (and lots) of data. Big Data is huge quantities of data – Volume. Big Data is an array of types of data, from an equally diverse set of sources – Variety. Big Data is collection and interpretation at ever-faster rates – Velocity. These are the “3 Vs” often referred to in discussion of Big Data.
Although humans have been collecting information about what they do and create since the beginning of recorded history arguably somewhere in the Roman Empire, Big Data is the relatively recent capability to capture and process such significantly larger and more robust data sets. Although computers began the data accumulation in the 1950s-70s, the phenomena of Big Data evolved as recently as 2001 when the term was coined by analyst Doug Laney. What makes the BIG in Big Data is the exponential increase in the 3 Vs discussed. Here’s a couple of examples.
The Big Picture
When the Sloan Digital Sky Survey began in 2000, its telescope in New Mexico collected more information in the first few weeks than had been amassed in the entire history of astronomy. By 2010, there was over 140 Terabytes of information. That amount of information can now be collected every 5 days.
When scientists first decoded the human genome in 2003, it took them a decade of intensive work to sequence the three billion base pairs. Now a single facility could sequence that much DNA in a day. The cost of that processing went from $40 million to $5000.
What data you can store and process on your phone today in 24 hours has probably more capability than all computer processing up through the 1970s. In 2005, a cell phone – without even a camera – had more processing power than NASA’s mission control during the Apollo flights that put men on the moon.
To understand why you need to know about Big Data, let’s start with The Fab Five.
#1 Not a Fad
In the past decade plus years, the 3 Vs of Big Data – Volume, Velocity and Variety has gotten a lot of attention from techies, industry and the public. There’s even been a fourth (Veracity) and Fifth V (Value) to further explore its opportunities and challenges. Like any popular uprising, the hype or substance of Big Data (depending on how you look at it) reached a certain level of attention before the naysayers began to cast the first predictions of if being a passing fad.
To some, Big Data melts into a crucible of technology slugs and ingots that are pedestrian and passing. But it’s not. The volume, velocity and variety of data available today, versus last year or ten years ago are not about to peak. Following the Second Law of Thermodynamics, its disorder only increases.
Big Data is still in the flat slope climbing the learning curve of what Big Data is and isn’t or what it can and cannot do. Utilizing its capability has considerable challenges, ranging from how it is initial collected to how to get to its mined “gold” – prediction. The philosophic trellis supporting Big Data is complexity and chaotic systems. It’s tricky stuff that the best experts are still beginning to explore.
It’s all emerging technology with all the nubile stumbling of a toddler. As its potential is only unfolding, the impact of Big Data is less like a popular novel and more like the Gutenberg bible. The bell can’t be unrung; it is here to stay.
Business uses it. Government uses it. Non-government organizations and non-state actors – both beneficent and malevolent (terrorist) – use it. And you use it too.
#2 You’re Wearing It
Wearables continue to infiltrate everyday life. Right now, the obvious example is your mobile phone. Somewhere in 2014, the number of cell phone subscriptions rose to equal the world population. (Land lines in the US never made that ratio, peaking way back in 2000.)
Cell phones provide you with more and more capability that is also your identity. It’s not just contacts and email connectivity. It’s not just communication. It has your banking information. It has your pics and music and social media, all brimming over with the 3 Vs of data. It entertains you and provides you with convenience. Some argue it is also security. It tells you where you are as well, and as it captures everywhere you have been.
Wearables have become increasingly popular with connecting into more robust medical applications – blood content, vital signs, respiration. Shoes have been designed to give directions to the blind. Socks can charge batteries with walking. These may seem like cool or awkward technologies but their implementation will break barriers in ways that aren’t obvious to the casual technology observer.
Wearables aren’t just for humans either. Wildlife is tracked for numbers and habits. Domestic animals also wear their own version of biometric sensors. The data analysis is used to optimize breeding and feeding practices. Even a honey bee can be fitted out for tracking movement for scientific experiment. These are data points that have been available in small portions before, but as the cost has gone downward, the capacity of data to be analyzed has gone up. Before it was a few discrete points; now it is a flow with more robust and significant and actionable outcomes.
Wearables are moving into more platforms and becoming more ubiquitous. They can be literally woven into fabric and painted or embedded into the skin. The Big Data doesn’t stop capturing your life though with wearables. It keeps going.
#3 It’s All Around
Wearables are just a subset of the propagation of sensors embedded in every aspect of life. Sensors will continue to combine with increased ability to interact and utilize that information. This – the Internet of Things (IoT) – started as a cool idea, but you can bet it already has effect in your life. You are always “on”.
Mobile phones and wearables are examples already provided, but there are others you already know. A suite of home monitoring products on the market provide remote control and observation to check on your electricity usage, environmental status, fire protection, doors locked. You can add monitoring to your car as well, and new models are incorporating more and more sensors that analyze its operation, alerting the driver to hazardous operating conditions and providing maintenance observations.
The Internet of Things (IoT) monitors crop growth. It’s used to drive building space utilization and builds maintenance plans for that building. Big Data and the IoT predict the weather and provide direction for recovery efforts when weather goes awry. The IoT is tolls tags in your car that don’t impede traffic and intelligent labels in your clothing that provide wardrobe inventory analysis and suggestions.
As the Internet itself is the eruption of software – bits and bytes that have become the blood of life, the Internet of Things (IoT) is essentially the physical hardware that we touch and manipulate connecting to the data flow. The embedded technologies weaving together your daily life are becoming more robust, providing an increase in productivity, an increase in relevance, and increase in well-being.
Consumers and society want this capability and they are willing to sacrifice at least some privacy and security for the perceived benefits. See Who’s Betting On the IoT.
#4 It’s Your Business
Big business has been the early adopter of Big Data and it touches all aspects of business – product/service development, manufacturing, operations, distribution, marketing, sales. More importantly, Big Data affects the most important function of business – the bottom line. Big business has had the deep pockets to explore the emerging technology, recognizing the not only the potential return on investment but also the danger of competitive advantage. As Big Data expands, the cost of entry is decreasing as the availability of resources extends to smaller businesses and individuals.
At last year’s (2016) Paris Air Show for example, Bombardier showcased its C Series jetliner that carries Pratt & Whitney’s Geared Turbo Fan (GTF) engine, which is fitted with 5,000 sensors that generate up to 10 GB of data per second. A single twin-engine aircraft with an average 12-hr. flight-time can produce up to 844 TB of data. In comparison, at the end of 2014, it was estimated that Facebook accumulated around 600 TB of data per day; but with an orderbook of more than 7,000 GTF engines, Pratt could potentially download zeta bytes of data once all their engines are in the field. It seems therefore, that the data generated by the aerospace industry alone could soon surpass the magnitude of the consumer Internet.
http://aviationweek.com/connected-aerospace/internet-aircraft-things-industry-set-be-transformed
We live in a world of increasing choices. The Mad Men marketing schema are iconic caricatures of what capability has begun and will continue to evolve. Your computer already learns from your search history what products and services you are at just thinking about purchasing. That’s a linear example. You search; the sites you visit take the information from your activity to pitch you products and services you are more likely to want. In a way, it’s annoying. In a way, it is convenient.
Big Data will make the message more compelling and more satisfying as it is derived from multivariate activity that accumulates from the 3 Vs. It’s going to start passing products and services you didn’t’ know you need (or want.)
“A lot of times, people don’t know what they want until you show it to them.” – Steve Jobs
Big Data marketing will know your transaction history, your lifestyle patterns and deviations, and fashion a very, very personal sales message to you (whether you like it or not).
#5 Your Tax Dollars at Work
Governments are getting into Big Data, not so much by leaps and bounds, but more by specific experiments. The United States uses Big Data in several agencies. Fraud, default and illegal activities can be detected or even predicted by observing the huge volumes of data available from agencies that use a huge volume of transactional data, like the Social Security Administration, the Federal Housing Authority and the Securities Exchange Commission. In the interest of public health, the Food and Drug Administration and Department of Health and Human Services utilize Big Data for better decision-making on the impact of individual lifestyle choices.
The Department of Homeland Security is another obvious player, utilizing the 3Vs of data available from not just federal sources, but state and local law enforcement entities. In the aftermath of the Boston Marathon bombing, over 480,000 images were ingested for investigation. Cross pollination of NASA and the US Forest System Big Data resources coordinated to better predict weather patterns affecting ground and space events.
The next wave of Big Data in government goes even further. It’s a bit more “out there,” and it is a little scary. China citizens have stopped using wallets and instead use their phones for all transactions. At first it was simple and convenient for buying groceries or renting a bike, but it has evolved into personal credit and social monitoring. Big Data or Big Brother, only the Chinese government algorithms know.
Greater Good
The 2018 Guide to Big Data has the 5 things to know about Big Data; it’s not just big business, although that group will continue to invest for both ROI and competitive advantage. Big Data also isn’t just about lifestyle choices. Wearables and the Internet of Things are building a Big Data trellis that grows the fruit of your life. Businesses that utilize Big Data will nurture that fruit, providing the tools and subsistence to grow the optimal grape.
Big Data is also about a bigger picture too. Ill intent will continue to undermine the soil and bind the vines. The bad guys aren’t going away; they will continue to find new ways to steal, or worse.
Big Data can do really great things. It is used for disaster search and rescue as well as damage assessment. It’s used for wildlife assessment. It brings together the people throughout the world who want to help.
