Come and get it!!

My pre-launch book sale campaign is underway!! Here’s a taste of what’s up for sale!

What’s Your Problem

Solving world hunger or fending off COVID-19 are a different problem space than what’s for dinner. It’s pretty easy to follow that distinction; however, it’s important now because we do have much more information and many more tools for tackling the former. Twenty years ago, solving world hunger seemed to be a matter of getting the best minds together with a sufficient checkbook. The matrix of issues and identities though is much more complicated than just ideas and the finances to back them. Although it does take funds to make things happen, the funds don’t solve the problem of itself.

Better understanding the dimensions of problems determines solution space as well as tools and methods for addressing its aspects and effects – because it may or may not be solvable.

“We are doomed to choose” – Isaiah Berlin

The world and its problems are becoming more complex. 

The world and its problems are becoming much more intricately entwined.

We all need to make better data decisions because subsequently the world we live in is subject to greater reactions and effects.

We gotta figure this out. We can’t just keep pushing more and more words and numbers into documents without providing a way to comprehend them better.

I may vomit. This part of writing a book is the hardest – actually trying to get people to buy it. For those who’ve been following my book posts on LinkedIn and Facebook, this is the pre-sale campaign I’ve been hinting at for the past few months. Now that I’m in the revision phases of my manuscript and have New Degree Press as my publisher, I now get to worry about funding the actual publication process!

If you’d like to help me on my publishing journey, please pre-order a signed copy by clicking here!


#privacy  #personalization

Wouldn’t it be nice if what you were just thinking of something you needed to get . . . and it came right to you. I imagine that’s the futuristic goal of #amazon. Haven’t they made our lives sssoooo much more convenient? On the one hand, it’s so easy to “shop around” at home. It’s so nice when a solution isn’t that hard to find. Right? But it’s a fine line to cross into where you’re not driving the train for what you need (or think you need.) Who is looking into your #personalinformation?

#theguardian reported last week on Amazon in “The Data Game: what Amazon knows about you and how to stop it.”. Some things you may know already, but others maybe not. Their source document, though, came from a #wired magazine article “Amazon’s Dark Secret: It Has Failed to Protect Your Data” (link in second comment) printed in November 2021 – that packs a punch:

“According to internal documents reviewed by Reveal from the Center for Investigative Reporting and WIRED, Amazon’s vast empire of customer data—its metastasizing record of what you search for, what you buy, what shows you watch, what pills you take, what you say to Alexa, and who’s at your front door—had become so sprawling, fragmented, and promiscuously shared within the company that the security division couldn’t even map all of it, much less adequately defend its borders.

In the name of speedy customer service, unbridled growth, and rapid-fire “invention on behalf of customers”—in the name of delighting you—Amazon had given broad swathes of its global workforce extraordinary latitude to tap into customer data at will. It was, as former Amazon chief information security officer Gary Gagnon calls it, a “free-for-all” of internal access to customer information. And as information security leaders warned, that free-for-all left the company wide open to “internal threat actors” while simultaneously making it inordinately difficult to track where all of Amazon’s data was flowing.”

Both articles are well worth your time to understand the power and vulnerability of #personalinformation. This is my take on #bigdata pricing and what the future of hyper-niche looks like. It’s in my upcoming book #thefallacyoflayingflat.

The future – the Big Data pricing – is that the price will shift according to your ability as much as willingness to pay.  That means the (online) webinar product that I didn’t want to pay $497 would have been further discounted to the $247 that would have made me uncomfortable but ready to use my credit card.

How is this possible?  For one, it is a product (online webinar) with almost zero production cost.  Unlike the widget which requires materials, labor, manufacturing, storage, and distribution, the digital warehouse doesn’t need a water supply or a janitor.  The one stored copy (and backup) replicates instantly.  

The next component is YOU. Big Data knows YOU along the same lines I mentioned that I use to make a purchase.  Big Data figures out if I have enough cash or credit for the sale.  Big Data knows if I spend money on these types of products already.  Big Data knows whether this fits my spending lifestyle or if it is a reach.  Big Data tells me I can or cannot deduct it as a business expense.   Even more so, Big Data knows whether I need deductions at this point, against how many deductions I have already accrued in my fiscal year.

The more uncomfortable aspect? Big Data Pricing will charge me a different amount than the next sale to a different customer.  An entity with a bigger bankroll may get charged more, or they may be offered a morphed package of sales for services I cannot afford:  X downloads, unlimited downloads, additional webinars or custom services.  In the lesser bankrolls, perhaps we will get a deeper discount or an extended opportunity to buy or a few more promptings in our email inbox.

“So pricing in the virtual world has not gone into our personal pocket books yet (that we know.)  The online market does use digital information such as browsing history and location to triangulate your willingness to pay a certain price.  This is still within the Small Data genre of capability, utilizing mean and median sources. Big Data Pricing though – and I believe it will – knows YOUR personal bottom line.  This is not a random variable calculated through the local and not so location population supply and demand.  Big Data Pricing knows exactly what price to set for you from all your transaction history in stores and online, your taxes, your job, your household status, and much more.

Is that scary?  Perhaps.  But it is already very close to possible.  

The world out there is waiting to sell you the next Best Thing and Big Data or not, marketing will continue to morph to find the magic price you are willing to pay.  Big Data Pricing though will be oh-so intimately familiar with you and your money.  In the end though, Big Data Pricing can only posture the question:  will you buy?  

The answer is still up to you.

Great News!

I have been “greenlit” by #newdegreepress. They’ve reviewed my work so far on my book “The Fallacy of Laying Flat”. I’ve got the words, the format and the concept in control and on track.  My first draft is due late February for publishing by the fall of #2022. Wish me luck! This is tough stuff!! I couldn’t have done it without the wisdom and support of #bookcreators and #georgetownuniversity#publishing #authorlife #bigdata #decisionmaking #decisionintelligence #data4good #data

The physical and the mental . . . and the Information in Between

Chapter X of The Fallacy of Laying Flat

Before we delve into the art and science of how Big Data derives solutions, let’s look at the problem space itself. We err with small data decisions in a Big Data world.  As Big Data capability evolves, it should be moving out of the flat into structure. The following classifications evoke the thinking needed to stretch data sets into approaches that provide more productive and realistic decision capability. These ways and means are not being applied today as they could and not progressing as they should be. 

Problems have a spectrum of simplicity and entanglement. Figuring out world peace isn’t the same as what’s for dinner. Big Data hasn’t really changed the essence of some of those problems or the schema. Finding a plumber or combating global pandemic is still a slate of options and contingencies, just with more information.  

Big Data though provides so much information that it needs to be interpreted differently than the ways we have previously used small data.  The data sets are not just exponentially larger; they are continuous and “always on”. They cost less to capture and process; they’re not the expensive discrete points of little data. Big Data is variety too – images and videos and texture beyond the characters and digits to which humanity has only been availed previously. The ability to capture and manipulate information is extraordinary. Machines are learning from us and their intelligence artificial – by our hands from our minds.

Big Data doesn’t fit into excel spreadsheets.  How we interact and interpret data is both divergent and nascent. We will talk more about what those differences are but for now we’ll break down the problem space into categories of how they occupy mental space. Basically, how we understand a “problem” needs to grow from what standard concepts and practices are now.

The constraints of the physical still cannot be overcome without physical manifestation, but In a Big Data world, that interaction is so much more augmented with information. You still have to jump to catch a moving train but the ability to catch it is much easier, more predictable, repeatable … rapid… with Big Data. 

So let’s take a look at this Big Data rendition of examining problems – in four dimensions. The concept is to realize the “space” for decisions in an ever-increasing information world. As we use data more and more naturally, the problems adjust from single and two-dimensional thinking to more robust insight and action. For now perhaps the characters are overwhelming. We ride upon the churning waters of the ocean. The depths below contain mystery, and danger and possibility. We stay afloat by what we can see and feel and touch and by any and all means to survive or gain advantage. Soaring overhead is yet more potential – the ability to fly above it all. 

The potential of Big Data is to observe and eventually coalesce the other domains that are tangent and perhaps critical to survival. It starts with one . . .

ONE – A Point in Time

The ONE dimension problem is a single moment.  What to have for breakfast?  Take or reject a job offer or a new relationship or a moral judgment.  The classic scenario is “this or that”  – the famous road diverging in a yellow wood.  Robert Frost penned an iconic work of literary art in “The Road Not Taken” for choosing one path or another to symbolize the decisions we make in life.  Standing at a fork, you can look down either of two roads and ponder what is or isn’t there.  Most likely the traveler cannot return so the decision may be lamented afterwards, perhaps years or even a lifetime.  Robert Frost knew FOMO well ahead of the Information Age.

A specific pivot point in time, it seems singularly lacking any width or depth of character or choice. Although we usually see the point in time as an either/or solution set, it has potential in an infinite set of choices in any direction.  Given the opportunity to stop and derive solutions in place, most likely you can only see a couple of options. 

I’d argue though that standing at that point, there are far more than two paths.  The traveler could stand still or sit down and wait until something or someone came to assist or order or demonstrate.  The traveler could decide not to take a road but instead trounce through the underbrush.  Perhaps he or she could even turn around and go the way they came.

A point in time has a solution in multiple – or perhaps infinite –  directions.  We often shave the choices down to two or three, which is poignant for advancing through the dimensions of problems. In business decision etiquette presenting more than three courses of action usually shows less than thorough research of the issue and a lack of ability to present concise thinking. Perhaps the broader scope of options is too much.  Perhaps many of the choices are not even considered consciously or subconsciously because of undesirable effects.  Embedded prejudices and experiences alter the frame of acceptable choices. We carve down the infinite set into tangible options from learned experience, which is often good but not always.  We also eliminate what we don’t see.  

Keep these limitations in mind as we step further into the TWO dimension problem.

TWO – The Plan

November 9th 1965, New York City and areas with over 30 million people and 80,000 square miles miles of civilization underwent an electricity black out of epic consequence.  The city was without power for an unprecedented 13 hours.  Heavy investigation into the circumstances traced to a single point of failure – a safety relay that tripped as programmed.  The relay opened the line disrupting power because of the heavy demand signal.  Unfortunately the load was a “normal” albeit extensive load on the line due only to heavy usage from deep cold, and not the catastrophic power surge it was designed to prevent. A waterfall of effects followed as the excessive load now shot to other re-directed lines, likewise shutting them down. 

This interconnected network poignantly was created in order to prevent blackouts instead of beget the proliferation of overload trips.  Also of note were the exceptions – numerous “islands” were able to escape the blackout via having the right off/on switches. Staten Island and parts of Brooklyn evaded the effects when people with quick thinking disconnected them from the grid before the programming shut them down.  

It’s complicated

The TWO dimension problem is complicated but not complex. Complicated is likened to a tangled set of corded earbuds that you pull out of the backpack.  Always wadded into an artistic nest, it takes a minute or so to unravel, but then the cords are “laying flat.” The power loss wasn’t lack of energy but the distribution and safety systems in place.  The solution was derived from unraveling the knot of the issue to an ah-ha moment. The plan works but it’s not perfect.

The TWO dimension problem is characterized and best recognized by its product – a PLAN.  This is the lion’s share of today’s active problem solving. Two dimension thinking works best/most with complicated but not complex problems. If you can press down the frayed and curled edges of the PLAN, the map is “laying flat” and everyone can read the landscape and follow the directions.  The map nails down points of interest and position and context and perhaps some texture. “The map” in your organization is a powerpoint brief or spreadsheet or POAM (Plan of Action and Milestones) or a policy document or any of a plethora of business planning products.

One of many challenges with The Plan is its limitations for capturing the situation. Most often it appears dumbed down to make it actionable, but accordingly, it is ignorantly sourced.

Flattening the curve for COVID-19 was quintessential decision making within the plane. Positive cases and death counts tempered against healthcare resources of bedspace and ventilators were used to forecast the ability to handle ill patients. The data points told us something. The goal became keeping the numbers from exploding. The desired effect was keeping hospital resources from being overwhelmed. The public centered on those numbers, weighing success and failure from the daily counts.

The numbers were scalar. They didn’t address the defiance within gradients of symptoms or effect of measures put in place. It didn’t include testing practices or policy or most poignantly, data sources for the efficacy of all of the above. The data most assuredly had considerable variation and instability (better known as “noise” in data talk); however, the numbers became ground truth – firm footing for taking next steps. The reality is those numbers can never explain the effects. The numbers were mistaken proof of themselves instead of the reality of complexity which was too difficult to maneuver or provide guidance. (See “Will I die of COVID-19?”)

It’s classic causality error. Does US spending on science, space and technology increase suicide occurances?

So causality often uses numbers for quantifying things that are a bit fuzzy. But when is it causality or its misunderstood cousin – correlation? This graph represents a strong correlation between US spending on science and technology and an increase in suicide by strangulation. A strong correlation does not mean that financing more STEM leads to more suicides. That’s the difference between correlation and causality, a fine line we are able to appreciate given an obvious scenario.

Looks like a dead give away. We can save lives and dollars by cutting resources to the favored STEM programs. This is an OBVIOUS example of representation of data that shouldn’t plan anything but a casual remark of “interesting.” When data sets are not so obvious and life and death are on the line – as with COVID-19 – it is hopefully easy to see how important understanding data interpretation is to global pandemic response. Like a smooth talking salesman, the “truth” can be manipulated.

Lots of buts

This TWO dimension of decision space is where the law of averages and linear thinking nestle in and take over.  Laying flat nurtures a confidence in comprehending the problem and the expected solution – that may or may not be reality. It inspires conviction in the linear logic of IF>THEN. Causality is the greatest desire and yet the most elusive truth. If bee stings or certain foods or situations cause an allergic reaction, then avoiding it prevents the pain. if you stick your finger in an electric socket, it’s going to shock you (or kill you depending on the socket.) Got it.

Then you slide toward more generalized IF/THens. These have subtle rules embedded in their sequence. If you wear your seatbelt, you have a better chance of surviving an accident, but seatbelts don’t ensure survival. If you complete a college education, will you make more money? Most likely, but not absolutely and the increase in college debts makes the uphill climb more monumental. And that logic doesn’t apply to Bill Gates and Steve Jobs (among a sometimes surprising list of stellar dropouts). Most laws and policy follow this gradient of IF/THEN. Whatever is generally good for one or for most is applied to support the greater good. 

Drunk drivers come in all ages; however, raising the drinking age has had a lasting impact in reducing alcohol related car fatalities. Turning 21 years old doesn’t have a miraculous ah-ha moment of responsibility but eighteen year olds collectively and decidedly do not.

Then there’s plenty of insanely simple and wrong examples of causality. If a woman floats, she must be a witch; otherwise, she drowns. If everywhere you look is flat, then the world must be flat and sailing too far into the sunset risks falling off. We may be more “enlightened” now but we apply the same logic with today’s issues. COVID-19 included.

Normally we would

The law of averages is another deeply ingrained mindset within TWO dimension. We tend to lace over design with the bulge of the bell curve, regardless of its application to the situation. The previous examples are just a couple of ways that outliers foul the comfort of “average.” 

The natural setting though is often nonlinearity. Logarithmic, power, and exponential relationships are the prevalence of many phenomena. These are “tipping point” behaviours and seemingly sudden or unexpected permutations – exactly what bites when you are thinking linearly. 

It’s very comfortable in the map but the real world will never become the representation we create. Wishful linear thinking and projected norms are not reality. Breaking this plane of thinking takes training, tools and systems that support the non-linear, chaotic world that is constantly surprising us with its bends and tricks. 

Getting to the next dimension is critical to utilizing the data capabilities at hand as well as those evolving. Let’s go there now.

THREE – Out of the plane and into the fire

The THREE dimension solution set pulls out of the plane and tackles complexity.  Wherein complicated can be untangled, complexity problems do not lay flat. The variables continuously morph, pressing in or fading out of contention for individual or shared interest. With complexity, the set of earphones would change length or separate and reattach (or not) or switch color or stretch/compress; other people would be trying to fix them for their benefit and your detriment (or not.) Complexity isn’t just a problem with too many moving parts; complexity players and events enter and exit the problem space with disparate probability sets of their own.  The plan can never be laid on a table and “flattened,” such as traffic and nature and happiness. Or global pandemic.

The THREE dimension problem can be bounded, but the space is vastly larger and boundary control is a variable of itself.  Suspending the system for evaluation does little to preclude the participation of influencing factors. Like slicing an integral to examine an instance, freezing traffic for a moment or an hour or any time increment does only little to dilute its effect or understand the issues. Just as a road construction project creates a suite of ill effects to be mitigated, bringing economies to near all-stop has consequences too. We plan for such a stoppage with road construction but it doesn’t prevent the need for repairs in the first place, nor the resulting pile ups that will happen when it is underway. 

The leverage in understanding traffic is within the flow – a vector quantity of magnitude and direction. Traffic doesn’t cease to exist because more information defines and tracks it. You don’t solve traffic so much as assuage its effects. Big Data learns to avoid or go around traffic if possible. That flow itself inserts influence on the problem set. 

Boundary lines

To give a simple visualization of a complex problem, consider the systems engineering example of air conditioning a space. A box is created to make a comfortable space for some creature to inhabit (and possibly WFH). It has walls and ceiling and floor to contain the desired temperature, but it also needs a door – for food and water and mental relief and other assorted desirable bodily needs. How does air conditioning it work? How much energy is needed to sustain the effort? What state is necessary to survive or to thrive? Like the 3 little pigs, will it be made of straw or sticks or bricks? Who’s going to pay for it?

Will the walls be thick or thin? Wherein, no insulation permeates heat, insulating a room to even extremes still subjects the contents to weather because it exists on earth that has variable weather. Think about it.  The more the problem is mitigated only increases its dependence, in this case –  on the weather.  Trying to “control” the weather only makes the problem more subject to it. The internal condition is influenced by the external conditions. 

This case also visualizes the boundary passage aspect.  Physically entering and leaving is necessary for utilizing the space; otherwise, why have the space?  Air flow must pass the boundary as well for the heat exchange necessary to maintain the desirable temperature and utilization of the space. And there will be inefficiencies, waste, and slippage.

Note too that this example is “solvable” – with a given tolerance.  I can set the control to 74 degrees.  The “answer” is an easily established and visualized integer representing a desirable outcome.  There are several feedback loops but making a decision, such as putting the thermostat at 74, establishes a policy that will not be instantaneous. The effect depends upon the system working as anticipated within a certain time period. That may or may not happen depending upon the physical operation capabilities of the system and the less measurable patience of the participants.  Is there sufficient funding and engineering to design, build and then maintain that solution?

Although a solution set of options attends the problem, the solution sets are multi-dimensioned. Each has time, cost, risk and opportunity in addition to width and depth of variables.  The matrix likely has gray strata of pros and cons, at least some of which are subjective.  If the desired state is comfortable living conditions, does “74 degrees Fahrenheit” maximize that accomplishment?  Other variables are pressing.  How great a variance is tolerable, detrimental, disastrous?

What It Takes

Solving a problem or preventing a disaster in three dimensions has several important components. First, there’s never a stasis. Entities as well as the influencers and structures are fluid, and not necessarily predictable or linear or normalized. The solution space has volume and does not remain within a plane; it is unlikely to become flat. If it does, the moment is measured but not enduring. 

Studying three dimension problems is about utilizing vector quantities to appreciate the flows. The solution set is subjective measurement with associated tolerances, costs, and risks. “Acceptable” is both subjective and variable, although it can be measured and visualized. 

Finally, the THREE dimension problem has to be comfortable with the gray and oh-so fuzzy in order to attain desirable effects. “74” is a number attached to a goal, but it’s not the end state.  “74-ish” likely describes its acquiescence to variables observed or mitigated. 

The next dimension pops out of the box. 

FOUR – A Stitch in TIme

The morning of September 8th, 1900, was a mild, partly cloudy day, with beach dwellers lingering to enjoy the surf of Galveston, Texas. The peaceful day was disrupted by a local weather forecaster on horseback riding the streets sounding the alarm of impending disaster. Whether they heeded his warning or not, the day was the first of several days unloading all the hurricane forces feared. Most fateful was a tidal surge of over 15 feet, easily covering the entire island’s paltry 8 feet above sea level. Buildings simply floated off their foundations and crashed like bumper cars into other buildings. The death toll of 6000-10000 lives still remains the worst fatal US natural disaster.

Like perhaps all disasters, there were signs and signals that were either ignored or denied or incorrectly interpreted or promulgated. Data points indicated potential hurricane capability, but sparity of sources and lack of communication left them adrift to be tossed in the consuming waters. The local forecasters actually broke policy by announcing the impending hurricane disaster, which dictated that warnings could only be broadcast with national center blessing. The reparation is decades of research and reform and refinement developing some of the most sophisticated forecasting and modeling on the planet – hurricane tracking. 

Predicting hurricanes is not like finding out today’s chance of precipitation. It requires a suite of forecasting tools. First, the atmosphere movement is captured via supercomputer dynamical modeling. Then hIstorical models consume all the behaviors of past hurricanes to project active storm potential.  Add to that trajectory models that focus solely on predicting the eye over land and ground. A slate of surge, wave and wind models each work to predict the variety of storm forces. Statistical-dynamic models encompass the influence of those two types.  Finally ensemble forecasts incorporate a suite of models.

Multiple approaches utilize different but sometimes overlapping data sets to tackle pieces to the puzzle. Only the most powerful tools can even attempt to consummate the results. For now too, trying to put all the factors together for a comprehensive picture is likely to dilute accuracy from the parts. We’re still not there yet.

Where the rigor meets the road

The result of using multiple models with competing and converging resources and reasons is the FOUR dimension problem space – building the plane as it flies – working all the issues of three dimensions in the reality of the march of time while handling injections and nonlinearity. The solution set for the four dimension problem is another integration up. Whereas the THREE dimension problem captures complexity, this dimension captures the lags and leaps of acceleration and unknowns. 

Four dimensions is where the issue is most closely mapped to the territory. The models have fidelity and tangency and vibration. The result is never a “map” and surely, the body and edges never lay flat. The potency is vigilance, iteration, and constant tweaking of all the resources available. The boundary flickers and moves.

Modeling is rigor. Creating the model, especially in four dimension, exacts a deliberation in understanding the facets and facilities surrounding a problem as well as potential solutions. Only through this work can the intricacies and complexity be fathomed. The model itself becomes a talking point – an opportunity to share and critique. A model engages players and encourages interaction. The model extracts data from the Big Data world. It incorporates the “corporate knowledge” of its keepers, and it manipulates the interpretations, which are multiple and varied for most likely and most deadly. Taskings evolve from the model and return to add to its color and texture.

Get it

This is the essence of Big Data problem solving – humans leveraging knowledge through technology. It extends the Industrial Age to the Information Age and keeps going. This bridge from possible to actual is delicate and yet formidable. 

It’s delicate because building the best modeled problem does not necessarily offer the most ideal solutions.  Models are representations of the world – or the portion thereof that we want to control or correct or predict – but they are not reality. “Confusing the map for the territory” is a siren to which all humans are drawn. It is primal behavior. The model will never be reality though. Models are tools for working a problem. Recreating a problem set to exacting proportions results in . . . another world to manage.  Plus, being humans integral to the system itself, artificial intelligence development has proven we add our own delusions through bias and conscious and subconscious interpretation.

That doesn’t mean it’s useless; it’s the algebra that needs to be done. And by algebra, I mean it’s the math not enough people can do. The FOUR dimension problem is the real world and it’s really, really messy. It’s full of noise, non-linearity, and sensitivity to initial conditions. The four dimension problem is where testing reality lies. We probably can’t solve problems in the four dimension with today’s science, technology and resources but this dimension is the evolution or perhaps quantum leap toward that capability.  All problem solving isn’t about recreating the situation but manipulating it for desirable effects.  

“You can’t navigate well in an interconnected, feedback-dominated world unless you take your eyes off short-term events and look for long-term behavior and structure; unless you are aware of false boundaries and bounded rationality; unless you take into account limiting factors, nonlinearities and delays.  You are likely to mistreat, misdesign, or misread systems if you don’t respect their properties of resilience, self-organization, and hierarchy.”

It could be argued all problems exist in the FOUR dimension because time affects us all whether we recognize its influence or not.  Not all problems though need the special effects of FOUR dimension.  Some problems may recur or morph slightly over time; the risk and return of the solution though does not vary sufficiently to warrant FOUR dimension analysis.

This dimension also ventures to see the edge of chaos. Greater complexity systems study actually reveals that strategic intuition often pushes the lever the wrong direction from the desired outcome. Studying FOUR dimension problems and models is about trial and error and observation.

∑ – In the End

The answer is there is no answer. Not ONE like we want it to be. But we need to keep evolving with the Big Data world.

These four dimensions are hardly the best definition of problem solving.  Many, many more academics can slice and dice and better explain the logistics and psychology of how and why attack an issue but this book will return to the factor that we are making small data decisions in a Big Data world. 

Breaking down problem space into dimensions demonstrates at what level we are currently stuck as well as the potential of stepping up a dimension. The tools and systems we use now are based upon training and education with small data interpretation. Realizing the next dimension has greater insight and better prediction capability is a template for designing new tools and systems that do solve the more complicated and complex problems that can be tackled with Big Data. 

It’s not just a cool factor of utilizing emerging technology. Big Data is necessary for solving the problems of a Big Data world. We had to stop the world to stop a pandemic. We can count positive cases and recoveries and deaths, but that doesn’t portend the effects of economic impact. These numbers shape the iceberg as seen from the surface. The impact of hunger, substance abuse, unemployment, depression, and less apparent “excess” death rates are under the water. 

Using those numbers is valuable for comprising algorithms and assessments for short and long term reverberations. Those numbers in combination with other Big Data sources can build scenarios for rebuilding and resurgence. What if this pandemic or the next one was much worse? 

The Navigator’s Balls

So let’s consider the ancient mariners. Four thousand years before Christ, seafaring souls took to crossing waters while hugging the shoreline. A couple thousand years later, they used the stars and developed tools to create maps. (Yah, I totally believe it would take hundreds of years of staring at the stars to figure out how to navigate by them.)  The two-dimension problem – here to there – was to facilitate commerce. They risked because the reward was financial. But predicting the weather over the horizon – that was the gods’ will.  

Going back a couple hundred years to the sailing days of Columbus, they didn’t know the world was round, but they could use the very crude data of falling barometric pressure to appreciate weather not yet seen. They didn’t understand exactly why but falling pressure meant prepare for the worst. Not an easy signal to find but once discovered, they knew the consequences. We need to figure out Big Data barometry.

Big data and its solutions are not going to look like what we have been doing or the type of results we have gotten. Today we have very sophisticated means to predict the weather – supercomputers and Big Data – but although the local forecast is usually close, it’s rarely 100%. As the forecast stretches further out into the future, it becomes less and less accurate.  That’s the world of chaos and a whole additional elephant in the room that comes later. 

So, there’s a lot of room for opportunity to grow, which is a nice way of saying we fall short of a lot of ways to solve problems. When 95% of the world we have yet to understand, solutions are raison d’etre.  Life exists in understanding our world and marveling at the accomplishments of our creator – whomever you choose.  Einstein quipped that if given an hour to solve a problem, he would spend 55 minutes studying the problem before 5 minutes of conjuring solutions.

The call to action with this book though is the data world is amassing information much more ominously than we are adapting to using its power. When we still use small data tools in a Big Data world, it’s bringing a knife to a gun fight. 

Next we will explore what it takes to break the plane. These new decision spaces demand things we are not used to needing or accepting while creating a solution. Then we will dive into the triad of tools, training and systems that are needed to implement Big Data solving Big Problems

Life before Facebook Newsfeed

Almost 15 years ago, finding out what your friends were up to meant going to individual FB pages to check on them. Click. Read. Click. Read. In 2006, one of Zuckerberg’s famous notebook sketches came to life – news feed. Hence the scroll was born and we’ve been scrolling ever since. And Mark Zuckerberg has been answering time and again over its effect.

Now the norm, news feed’s arrival was wildly unpopular. Actually “This SUCKS” was the collective comment; complaints of creepy and stalker-like were the reaction. Ten percent of FB users joined groups (ironically on Facebook) demanding its removal. FB itself simmered internally with controversy over keeping it versus trashing it.  The solution came as a compromise: opt-in privacy settings that allow control over who sees the news feed. Makes sense . . . because that’s what we’ve lived for years now. Scroll. Scroll. Scroll.

This isn’t a recall lesson for FB devops; this is for us to remember. 1) We consumers don’t like change for the most part. Unless there’s a huge visible upside, we fight the upsets to our expectations of how life should go. 2) Great ideas – even game changers – need tweaking. Don’t be afraid or too proud to see the faults in your work. Take the edits; listen to the critics. There’s going to be unintended consequences to any magical kingdom. 

Mark Zuckerberg may always have the ego to conquer the world with his creation. But he’s also admitted to failure. He has also adapted to the responsibility. His famous “Move fast and break things” became people’s lives – sometimes literally. And I doubt he’s seen his last appearance before Congress.

How well can we predict the future of #coronavirus?

Did COVID-19 start in a lab or naturally progress from animals to humans?

While there’s no shortage of speculation on #COVID-19 – especially how it got started – the answer(s) have yet to become clear. It’s doubtful we will ever learn all the specifics that seem simple enough to determine and necessary to prevent the next iteration. COVID-19 and the next pandemic are subject to the butterfly effect and likely will remain a cloudy opinion. 

The butterfly effect is casually referred in pop culture but usually incorrectly. Edward Lorenz developed the concept while working meticulous calculations for predictions in the weather.  He observed that the smallest deviations in a system can lead to dramatically different results.  Hence a butterfly flapping its wings in Brazil causes tornadoes in Texas. This sensitivity to original conditions is why we can’t well predict the weather beyond 2 weeks.

His work led to the founding of chaotic study – which is not randomness. Chaotic systems are ordered but they vary – within limits. The solution to predicting the next pandemic is like hurricane watching. We can learn the pertinent path with variance but only know its exact movements as it unfolds.

“You’re touching too much”

#coronavirus #techhacks

Touching is tangency. We didn’t realize how much we touched stuff until health and life demanded you pay attention. Realizing how much you touch stuff other people touched is really creepy.

There can be an app for that – touching too much. Your phone’s sensitivity can pick up how much you touch things – door handles, drinking & eating, your wallet, private/public transportation/buildings. It knows your meeting sked.  It can remind you to wash hands or wipe down surfaces. At given intervals too, it politely says, “You need to wipe me down too; I’m getting kinda funky.”

There’s also already an app for one of the dominant touch activities – paying for stuff. ApplePay and GooglePay (did you know you can add money as an attachment with gmail??) already provide touchless options for transactions from your phone. Even before the outbreak, I cringed whenever I pulled out a credit card for payment. Either the cashier is touching it (who touched a hundred before) or I’m touching a kiosk innumerable people before me have.  Contactless is exactly that – no touching! Although quite prolific in Europe, many kiosks in the US have the symbol but it isn’t active. Having universal contactless transactions would hack a major TOUCH vulnerability.

Up next: Your health record: you can take it with you

what is google pay googlepay5
what is google pay googlepay3

Artificial Intelligence Rule #7: Close Enough


Rule #7 of artificialintelligence: close enough.

Number 7 is pretty far down the list, but “close enough” is an equally important concept for AI. Have you ever queried Google or Bing and gotten a single entry? Aka “the answer” to your question? No. I know I’ve gotten a single page of items in return (I ask some weird questions) but it always provides a menu of options.

The page ranking algorithms of Google are legendary and as closely guarded as military secrets. They aren’t carved in stone. Indeed the algorithms are manipulated in order to adjust for specific hacks as well as smoothing trends.

But artificial intelligence doesn’t provide solutions like an algebraic math problem. It’s stoic in its reply, showing no emotion and yet posing a voluminous suite of possibilities to be considered by the inquirer. Indifferent to the vicissitudes of fortune, the ai sweeps the oceans of the internet to provide you what is . . . close enough.

Humans calculate those algorithms and only you decide what is “the answer.”

#machinelearning #aibots #algorithms #aibot #deeplearning #ml


Rule #4 of Artificial Intelligence: no context.  

#AIbots have been trained to do many mundane, repetitive jobs. Training involves utilizing data sets with often millions of data points. Without enthusiasm or angst, the AI ingests these volumes of data and returns outcomes by direction of its creator.  Given x-ray images of healthy and diseased lungs, we have to tell the AI which is good and bad. AIs are “rewarded” for correctly identifying the difference.

But an AI doesn’t know what a lung is, how it works, that it actually exists inside a person or what a person is. 

#Google was the first major developer in #recognition when it looped together 16,000 computer processors with one billion connections in order for it to watch #YouTube and find . . . cats.  This was a tremendous breakthrough for 2012, but isn’t this what a 5 year old understands without seeing millions of videos?

And a toddler knows colors and textures and if the cat is missing an eye or leg or tail. He or she knows cats are pets and most are found in or around homes. Cats walk on four legs; they don’t swim. Not everyone likes cats but they are loveable. 

This is context and the richness of a five year old’s perception outweighs a million data points.



Pet peeve – use of “literally.”

If you don’t understand literally vice figuratively, #artificialintelligence can set you straight. Rule#2 of #AI is everything is literal. AI does exactly as you tell it.  That can be annoying from a child or spouse or customer service chatbot. AI doesn’t have the contextual preferences of humans – which emotes angst and joy in the uncovering. Given a problem, AI is going to take the tasking literally. For example:

I hooked a neural network up to my Roomba. I wanted it to learn to navigate without bumping into things, so I set up a reward scheme to encourage speed and discourage hitting the bumper sensors. It learnt to drive backwards, because there are no bumpers on the back. – @smingleigh

This is an interesting concept because the “bugs” that you deal with your computer, your phone, your network, your business are likely a synergy of literal translation. Code knows 0 or 1, and coders get that.  The rest of us are swimming in “why the hell is this broken” when the answer is a literal question return.

#saywhatyoumean #meanwhatyousay