Data Lakes are an integral part of Big Data.

Many great inspirations take years to develop in concepts and more time yet again to become  physical manifestation.  The Global Maritime Sensing concept has been met with positive if not raving reviews by US and international navy partners.  The implementation has still been challenged  as in even the best of financial times, there are but limited resources.  These are definitely austere times and thus more creativity is required.

Regardless of budgetary limitations, GMS has always focused on the extraction of value and from the meaning world around us as individuals and not as institutions.  As time continues to take us by leaps in bounds through emerging technologic miracles, the promise, the capability, of Big Data is now the foundation for GMS implementation.

Big Data is most definitely a paradigm shift, and one in which those that don’t get it will be left behind with failures.  Those that do see its potential are still grappling with its power, much as we all do with the digital world.  Although it would be good to give a couple posts relating the general fit of GMS and Big Data, I’m  instead going to jump into my upcoming attendance at the Big Data Tech Conference in Boston, 30 MAR – 2 APR.  As I go to the tutorials and classes on broad Big Data implementation and specific Big Data tools, I will use the GMS lens as subject for the infinite opportunities of Big Data and the action plan for successful deployment of GMS.

Solutions to complex problems such as Maritime Domain Awareness are best approached with a combination of creativity and quantification.  Harvard Business Review’s article Learning to Live with Complexity (SEP 2011) suggests that in attaining better risk mitigation of complex systems in the business venue, four elements make sense.  These include storytelling and triangulation – which is a powerful combination of what they refer to as combining “soft” and “hard” approaches.

Storytelling and counter-factuals are visualizing potential shock events and sharing anecdotes of “near-misses” respectively.  This aspect of mitigation is relevant to capturing the risk that is “unlikely but potentially catastrophic.”  This dream weaving and arm chair quarterbacking is creative and interpretive.

Triangulation then reigns in the infinite array of possibilities that can be woven.  Triangulation asserts varying methodologies and perspectives through technical or algorithmic lenses.

This risk mitigation concept marries well with game theory and its application to military strategy.  The numerous elements at play in a given situation are mapped out and given weighted probabilities.  Decision making is then aided with rigorous calculations to provide substance to consequence engagement.  

To that extent another methodology derived in 2007 that could be utilized is computer generated fiction.  Computer software is fed scenario elements and probabilities.  The fiction novel then crafts original stories based on computerized representations of emotions and tensions between characters.

The premise then in looking for solutions to Maritime Domain Awareness is creating the elements that derive the “characters” and “emotions” of the maritime domain and then quantitatively weighing them.  This  which is within the cognitive ability of the individual players and then applying these types of algorithms to triangulate the unlimited potential into strategic plans or comprehensive courses of action.

To further address the challenge of identifying the “dark” vessels of Maritime Domain Awareness then lies with citizen science and crowd sourcing to feed random and specific data points to these lines of possibility.  The data points then twist the algorithms to fit real time events.

Because MDA is not an isolated issues but one shared with the world’s maritime forces, using the ISS as the forum for storytelling and counter-factual collection is an opportunity integral to solving the “dark vessel” gaps.

Prettymaps is an excellent example of open source information geographically displayed.  This page explains GIS well.  http://prettymaps.stamen.com/201008/about/

Complicated or Complex?
Maritime Domain Awareness is a complex system, not merely a complicated one. Although the electrical grid for the New York City metropolis area is vast and intricate, it’s workings are none the less only complicated because it consists of a basic network on/off switches. The linear relationship makes it easy to diagnose past incidents and anticipate potential problems. The massive blackout of 1977 could be traced to a series of lightning strikes that triggered unintended switches in the power grid algorithms with catastrophic consequences.

Complexity is different because the elements are not independent and the relationships are nonlinear. Accordingly, MDA, as a complex system, cannot be predicted through probabilistic methods such as extrapolating averages and interpretting median deviations. A complex system also overwhelms the cognitive ability of a decision maker to see the emerging patterns or significant elements. In the case of MDA, that cognitive ability currently encompasses the collection capabilities of several institutions and all nations and navies.

Signal in the Noise
Like most intelligence collection, complexity research is also about finding the anomalies and outliers that have significance – of finding signal within the noise. Data mining of complex systems such as MDA involves three buckets of information: lagging, current, and leading. Lagging is information such as ships’ movements; current is ships’ positions. For MDA this data has tremendous gaps. Traditional AIS tracking fails to capture “dark vessels,” those that are defiantly misleading and those that are too small for AIS. Collection, fusion and analysis of information on these dark vessels are the higher priorities of Maritime Domain Awareness.

Finally, the golden egg of the system is leading information, which is best described by former US Coast Guard commandant ADM Allen’s comment that what you need for piracy is a pirate and a pirate-able ship. Harvard Business Review’s suggestion for risk mitigation of complex systems includes storytelling and counterfactuals in conjunction with triangulation of those creative outcomes.

As in any counterterrorism textbook, capturing the near misses is as important as the responses to real events. Arm chair quarterback analysis of past events and hypothesizing even grand scenarios is much more productive now than ever before with the aid of information systems such as Global Information Systems (GIS).

Global Information Systems
Utilizing Global Information Systems is the key to answering the MDA complex system risk mitigation, especially dark vessels. Using the open source data bases and combining deep intelligence is the first step in the process. Next is sharing the practices, near misses, and possible scenarios that currently reside within silos within the various United States’ government and non-government organizations. Finally, the results are triangulated by assessment from various angles that provide unique perspective that collectively minimize the risk terrain. A more rich and comprehensive solution set would be possible in expanding the cognitive efforts of the world’s navies at such forums as the International Seapower Symposium.

Why GIS?
Utilizing techniques such as creative storytelling in conjunction with quantitative chop has not been used extensively until recently. Global Information Systems are an open source, intuitive capability that captures real time data and overlays it geographically. As such it is perfect for such sophisticated analysis of complex systems of significant emphasis in MDA.

Why the International Seapower Symposium?
The International Seapower Symposium is a biennial event that collects the world’s navies to socialize and share the collective issues of the sea. By far the most prevalent issue is Maritime Domain Awareness. The capabilities of GIS were just budding two years ago for the last meeting, and now the digital power curve for this technology is rising exponentially. This critical technology needs to be presented and socialized at this forum. Because of the scale of its simplicity to sophistication as well as its open source basis, anyone can use it. That also means the “dark vessel” owners. The ISS is the time and place to ensure gaps in MDA are illuminated and leveraged.  The ISS is the opportunity to socialize current practices and draft scenarios for testing.  [polldaddy poll=5462876]  Now is the time.

A geographic information system (GIS) is a system used to describe and characterize the earth and other geographies for the purpose of visualizing and analyzing geographically referenced information.

Many have characterized GIS as one of the most powerful of all information technologies because it focuses on integrating knowledge from multiple sources (for example, as layers within a map) and creates a crosscutting environment for collaboration. In addition, GIS is attractive to most people who encounter it because it is both intuitive and cognitive. It combines a powerful visualization environment—using maps to communicate and visualize—with a strong analytic and modeling framework that is rooted in the science of geography.

This combination has resulted in a technology that is science based, trusted, and easily communicated using maps and other geographic views.
(http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#/Introduction/00v200000002000000/)

The critical question is whether photos already on the web and for future concept of operations are “readable” for the geotagging information. The answer is arguably yes and no. The use of geotagging for social networking sites suggests a plethora of data available for the sole or enhanced purpose of advertising or sharing location and yet there is opportunity for misdirection. Wikipedia provides an interesting footnote.

Dangers of geotagging
Following a scientific study[11] and several demonstrative websites,[12][13] a discussion on the privacy implications of geotagging has raised public attention.[14][15][16] In particular, the automatic embedding of geotags in pictures taken with smartphones is often ignored by cell-phone users. As a result, people are often not aware that the photos they publish on the Internet have been geotagged. Many celebrities reportedly gave away their home location without knowing it. According to the study, a significant number of for-sale advertisements on Craigslist, that were otherwise anonymized, contained geotags, thereby revealing the location of high-valued goods—sometimes in combination with clear hints to the absence of the offerer at certain times. Publishing photos and other media tagged with exact geolocation on the Internet allows random people to track an individual’s location and correlate it with other information. Therefore, criminals could find out when homes are empty because their inhabitants posted geotagged and timestamped information both about their home address and their vacation residence. These dangers can be avoided by removing geotags with a metadata removal tool for photos before publishing them on the Internet. (http://en.wikipedia.org/wiki/Geotagging)

In regard to the unintended consequences geotagging has listed in Wikipedi for Craigslist and celebrities, it is easy to recognize ill intentions are already using the geotag capability. The intent of GMS is to leverage this vulnerability against such efforts in the Maritime Domain. Hopefully, US forces are preventing the exploitation of this gap against the US and its allies.