Thursday, November 3, 2011

The Future of Contextual Mobile Search - Predicting what we want before we even know what we want.

My life is defined by seats - airline, restaurant, concert, theater, train, car, conference - but the next minute of my existence is defined by my oversized Japanese tatami bed.  The alarm, seconds away from screaming the arrival of a new day.  My brain already races as it preprocesses the details of the day ahead.  Absolute quiet is disturbed only by the muted soft sounds of gears slowly turning as the tightly wound coil of my old mechanical alarm clock unwinds.  I stare at the minute hand - with its ceaseless radial movement - it momentary reaches apogee - 6:00 a.m.  Like Thor and his anvil - there’s nothing that replicates two metal surfaces violently colliding - not processed or synthesized.  There’s something so old school in the sound of real metal striking metal.  Yet even a throw away phone can digitally reproduce the sound of my analog alarm in 16 bit clarity.  Am I holding onto a delusional past, like audiophiles with their vinyl records? As I ponder my nostalgic thoughts, the cacophony of musical disorder from the alarm announces my day of more seats ahead: toilet, kitchen stool, taxi, airline, limo and lastly, the Aeron of a stealth startup.

Normally I would have moved at a slower pace after numerous martinis and champagne toasts the night before.  But excitement and anticipation masks any hangover I might be experiencing.  Being a so-called tech documentary screenwriter, I meet famous people. Though, sadly my Flickr account is pathetically devoid of any celebrity actually seen on the silver screen.  Instead, my online album is more like a Stanford and MIT alumni gathering.

As the vibrations that just proclaimed the start of my day slowly dissipate, I stare at the box.  One slowly opens the lid as if it were an Egyptian Pharaoh’s burial chamber to reveal the true treasure inside - the world’s first mobile phone that natively supports the so-called semantic web.  Like a connoisseur of fine wine, I subject my nose to the electronic aroma of the hermetically sealed device. Its polyethylene protective wrap, like the cork of a vintage Chateau Latour; once opened, the aromatic senses are overwhelming, sending chills down one’s back.  

This morning, neither the electronic aroma nor the actual device is the cause of my adrenaline.  Like most seismic shifts in one’s life or disjointed nonlinear jumps in the evolutionary process, one doesn’t typically see it immediately.  Most so-called revolutionary jumps only turn out to be an over-hyped ‘TransMeta’ or a ‘Ginger’.

The rush of dopamine and my innate instinct overwhelm me - by nightfall I’ll witness that rare moment in time that changes everything.  Like that hot summer evening in 1879 when a certain carbon-based filament results in the first sustainable burn for an incandescent lamp or that chilly winter morning of 1947 when the first amplified signal from a solid state device was detected.

When Markus Demetrius called the other week demanding that I bring the device to D.C for an interesting demonstration, I knew by his understated boast and the fact that publicly no one even knew I was getting this device was all the more reason for today’s adrenaline. 

In my university days of neuro and computer sciences, Markus’s hyperkinetic energy and his IQ made other PhD candidates feel as if they were in a remedial class.  After he spent the last decade in the NSA, conversations with him were infrequent and dry.  When Markus informed me that he has been surreptitiously working together with Wolfgang Reinhold the last few years, I knew not to take his invite casually.

The deal was, I would provide them with the first semantic phone and in return would witness what they referred to as ‘Singularity’.  My first thought was of Kurzweil and his nano bots running amuck in my brain or some bio-interface.  Either way, I’ll soon be leaving the comfort of my Tribeca loft to deal with the magnetic wand of airport security.

After an uneventful flight guarding my carry-on like it’s the Hope Diamond, I exit the terminal at Dulles into a waiting Yakuza-like car; windows tinted the same shade as asphalt.  I tell the driver to head to McLean via 495.  I don’t know where I’m heading; my final destination is to arrive by text. 

As we pass signs welcoming us to McLean, my mobile vibrates, alerting me of my next steps.  After bidding adieu to my driver, I get into a parked car nearby.  With the keys that were Fedex’ed to me early this morning, I fire up the ignition and rev the cylinders within the engine’s anodized aluminum block.  The explosive combustion of oxygen and vaporized fuel injected gasoline roar like a backyard mower. Oh well - what was I expecting - it’s a ‘Green Car’.  Another text soon arrives.

It’s late morning as I drive through the heavily wooded residential areas of McLean.  Large bricked Georgian colonial and federalist houses remind me of my childhood in Norman Rockwell-like communities outside Boston.  As I approach the end of the cul-de-sac, one home stands out.  Darkened Frank Lloyd Wright lines clashed with shiny Gehry curves.  As I pull into the secluded driveway, a pair of eyes from behind a heavily draped window silently announces guests are not welcomed.

After a quick retinal scan at the side entrance, the sound of pneumatic locks unbolting is heard as the door swings open and my old roommate Markus of years past greets me.

Markus, with a day’s old beard and skin looking like it’s not seen sun in months,  is not about to win an award for healthy living.  His clothing hangs on his tall wiry frame as if he is missing critical body mass.

After finishing my coffee, Markus informs me Wolfgang is ready to meet.  Another retinal and finger scan.  I feel the reverberation of reinforced bolts unlocking and the sudden rush of ionized basement air as the heavy steel door opens.  The hum of an overworked ventilation system is at first irritating, but as we descend the steel steps, the noise diminishes and the temperature noticeably decreases.  It’s quickly obvious why.  With enough oversized flat screen monitors and computer equipment, it could pass for mission control at AT&T network central.

Most programmers’ inner sanctuaries look like an archeological treasure trove with layers of their daily life stacked upon each another.  Here it was different, almost having a clinical feel to it.  One could sense something is intricately being crafted.
His shoulder length tangled blond hair and his tall lanky frame is my first details I notice of Wolfgang.   Eyeing my carry-on bag; it’s obvious where his interests lie.

We walk over to a steel table. A few white papers on NSA security protocols and a laptop are the only things on it.  Like a diamond or drug buyer about to inspect the goods, Wolfgang and Markus hover around the table in anticipation.

As I pull the device out of its protective box and lay it on the table, Wolfgang’s first words are ‘Is it Charged’ and can he put his SIM into it?  I answer affirmative to both queries.  With surgeon like precision, an exacto knife is drawn across the protective polyethylene and a new SIM inserted.  From the laptop on the desk, he informs me he’s already Bluetooth bonded and unlocked the phone.

With heuristic voice recognition long being native to mobile phones, the need for a physical keypad seems dated as a rotary dial.  The device is a thing of simplicity and beauty. The rectangular slab of semi opaque glass like material is roughly the size of an iPhone, but a quarter of the thickness.  There are no external buttons, plugs or power inputs.  With wireless syncing and near field battery charging, it’s like holding the neo Paleolithic monolith from Kubrick’s Space Odyssey. But looks are secondary for this device; it’s what’s running the inside where the magic starts. 

While Wolfgang ferociously attacks the keys of his laptop, he starts to explain what I’m about to witness.  In his slight German accent, “The old Web 2.0 revolution that started the toy industry of mash-ups and so-called distributed web services, was actually building the foundation and protocols of what we now know as the semantic web.  No longer do you need human intervention for trivial queries like, ‘Show me nearby sunny holiday destinations with water sports, Saint Tropez-like nightlife and Adour Alain Ducasse-like French cuisine’.  Remember how long it would take a travel agent to process and find a destination that match a similar query?  Now with the semantic web, these queries are easy.”

Wolfgang goes on to explain, “Search as we know ends today.  Like the transition from ape to human with genetic mutation of a neuropsin-based protein that led to the evolution of a brain capable of understanding language and mathematics.  If I need to give a logline of what Singularity is, I would say it’s where search understands you both socially and contextual. But it’s so much more…”

Wolfgang continues, “First a little background. You know you’ve reached a stagnant period in your life when you’re even sweating boredom.  That was me when I was analyzing the vulnerabilities of data.  Like an artist with a wet brush wanting to create.  I felt stifled touching up other people’s work.  Like Rembrandt forced to paint by numbers.  So I went to work for Fifth Third Processing Solutions, one of the largest credit card processing companies with 30 billion transactions processed annually.  I headed an internal project that data-mined credit transactions in real-time.  At an aggregate level, I knew details on every purchase.  From basic statistics like Boston drinks 5% more Coke than Pepsi, but Atlanta drinks 15% more Pepsi than Coke or blue shirts are more popular in the south than the west.  We never kept track of who bought that blue shirt; just that 10 percent of shirts sold in January were blue and 20% of all shirts sold in June were blue.  Really boring stats, but so monetarily important in the world of business intelligence.

We amassed intricate details on trends in fashion, clothing sizes, food tastes, color trends, etc.  Practically every competitive statistic you could imagine. We could tell you the average height, weight or overall health of any city.  For example, people in Detroit on average were 3 inches taller and 30 pounds heavier than those living in Seattle. We could slice it based on year of birth, gender, salary etc.  All without having to do any door-to-door surveys, saving researchers, economists and drug companies millions of dollars in their own surveys or data sampling.  In the case of foods scare or if certain foods were found to be beneficial, real-time changes in sales could be tracked. 

Then 9/11 occurred and the Patriots Act required us to open our credit card transactions processing to the NSA.  They wanted to perform their own deep and persistent data-mining, searching for unusual activities in purchases of fertilizer, ball bearings, nitrates or whatever they might conclude was a potential threat.  This changed the whole notion of data-mining credit card purchases.  What was once done at an aggregate or collective level, the NSA was now going to do it on an individual basis - taking multi-dimensional data-mining to a level unheard of before.  Why do you think grocery and convenience stores give out those so-called ‘Saver cards’? They know the value of persistent data.”

Wolfgang continues, “This is how Markus and I met.  He was at the NSA defining access and security protocols and writing white papers justifying why they needed hundreds of million of dollars for purchase orders consisting of enough heavy metal to support the MIPS needed to handle data-mining 100 million daily purchases.  This now meant every transaction would be associated with an individual.  If you bought a book six months ago on Anarchy on Amazon or at your local Barnes & Noble and four weeks ago you bought 100 pounds of sodium nitrates, the NSA would know immediately.   Credit card companies loved this idea since it was a leap forward in fraud detection. If grandma buys tickets to a hip hop concert and has no grandchildren, red flags are automatically raised.  Credit card companies even advertise the fact that they are doing deep data-mining on all your purchases -- all in the name of protecting your valuable credit history.  Just try to buy something unusual and see how quickly your credit card company calls you to validate that purchase.

Wolfgang, calmly continues, “The next stage was predictive probability forecasting.  By deep data-mining, we could predict if husbands were about to cheat on their wives; sudden changes in clothing, cologne, expensive jewelry purchases not around any holiday or spouse’s birthday, drinking habits, changes in restaurants frequented, size 2 lingerie purchases when the wife is a size eight, frequent hotel stays in his home city.  Even if he pays cash upon checking out, if he used a credit card to reserve the room, it will show up in our records.   With deeper cross referencing data-mining we can even determine the identity of his mistress.  If she buys a drink with her credit card while waiting for him at a hotel, restaurant or Starbucks and we see a constant repeat pattern over time that a certain woman overlaps in location and time, we can be almost certain that they know each other.”

Wolfgang, now somewhat impatient, “But now this is so yesterday.  Every credit card company does this.  Deep data-mining and selling aggregate models and predictive trend analysis is a huge business. They’re just careful only to sell data at an aggregate level.  The market for this is insatiable - imagine in almost real-time, knowing changes in the trends of clothing colors, changes towards cheaper or more expensive designers, food trends, changes in sizes, trends towards shopping closer at home because of fuel costs.  We can even predict product recalls before the manufactures.  A disproportionate number of front disk brakes or fuel pumps purchased for last year’s GM Rover truck.  Is a clear sign there’s either a defect or an issue with the design.  

Over the next two years, the amount of supercomputer power within the NSA increased to two-third of all governmental MIPS and it was directly related to the data-mining activities occurring over at the NSA.  Too bad global warming and environmental modeling took a backseat when it came to who was getting funding for the number crunching metal.

Because of the ever evolving nature of what the government defines as a threat, the decision was made to track and mine everything.  That glossy nail polish in the future could be mixed with some new composite which in turn masks plastic explosives from detection.  Basically the government could never predict what might be considered a threat so we treated everything you buy as a potential threat.

Then one late night over a few too many martinis, Markus and I started to discuss opening backdoors that would allow us to define our own queries. Once we decided to move forward, we enabled the self masking backdoor processes that we had written into the NSA data-mining and collaborative filtering engines to allow for external queries. The queries would not be tracked or logged.

With all search engines including this mobile phone natively supporting natural query processing for semantic web, it’s absurdly easy for us to support verbal queries like ‘Show Washington D.C. residents who buy trendy labels, drive fuel efficient cars, subscribe to moderate political magazines and take tropical holidays’.  All data now has meaning other than just size, cost, weight, color. It now has amorphous properties and associations. You don’t have to define what you mean by ‘Trendy’ or ‘Fuel Efficient’.  With semantic definitions and tagging, Singularity learns. It’s also an evolving property, what’s ‘trendy’ or ‘fuel efficient’ this week will be different ten months from now.  

By data-mining semantic properties and credit card purchases; it’s now possible for queries like. “Show middle-aged slender Georgetown women who enjoy Petrus or Margaux type Bordeaux, Siam style Thai and are active in winter holidays.  With all ATM and all credit cards purchases now tracked and match to individuals and social security numbers. These queries were trivially easy to generate a list of specific individuals who match our search criteria.  The only problem was we knew that Michelle Halenarova on M street in Georgetown apartment 2G was a size 3 and 5 feet 8 inches tall, single, but we had no idea what she actually looked like.

Then in the name of cross department coordination, including the CIA, NSA, Home Land Security and an updated Foreign Intelligence Surveillance Act, the division heads asked if we could enhance their queries by overlaying the output with social networks.  Now, if a so called ‘terrorist’ were to lack common sense and have a Facebook, Myspace, LinkedIn or Friendster account, we could extract photos from their profiles and online albums to visually identify the person in question.  This was the key piece that would tie these highly rich data models with the real world, to put it so bluntly - to add a face behind the data.

Through a simple sodium nitrate purchase in a farm store in Oklahoma, we knew before that individual even returned home, the history of his purchases, his visual appearance and through his social networks all possible associates.  We could then cross reference other purchases within the last few months, such as truck rentals, storage bins, aluminum piping and match it with anyone in his network.  This was powerful shit. 


No comments: