New interface could offer massive boost to nascent hands free technology

A group of natural language parsing and artificial intelligence veterans led by former Siri, Inc. CEO Dag Kittlaus highlighted the work of their stealth startup Viv Labs in a new piece in Wired.  In the piece, industry experts rave that the technology could trigger a bidding war between the smartphone industry's superpowers and could become "the future of intelligent agents and a multibillion-dollar industry".
We've uncovered a number of details about Viv Labs' history, its extensive poaching of Apple, Inc. (AAPL) employees, and even its patents.  But first let's review the facts we know, including the
I. The Global Brain
Wired writes:
Viv founders say you’ll access its artificial intelligence as a utility, the way you draw on electricity. Simply by speaking, you will connect to what they are calling “a global brain.” And that brain can help power a million different apps and devices.
That's nearly accurate, but based on our research there's one subtle nuance that was lost.  According to data we found on Trademarkia, Viv Labs' slogan for its product is "The Global Brain".  In their eyes the assistant service they're developing is one-of-a-kind -- or the first of its kind at least.  They envision a system that is no mere knowledge graph.

Viv the Global Brain

The product is named "Viv" after the Latin verb for "life".  Under the three "pillars" adopted by Viv Labs, the Viv digital assistant service will learn something new every day from people worldwide and will have algorithms to make new knowledge connections not directly taught to it.  

It will be able to understand ambiguous, abstract commands with complex conditionals -- for example you can ask it to buy the cheapest available item that meets some criteria and it will hunt it down and initiate a purchase for you.  It will even know the social context of what to do in in certain cases such as knowing to call a taxi if you tell it "I'm drunk."

In his vision laid out to Wired, Viv Labs, Inc. CEO Dag Kittlaus indicates that the sale of Siri, Inc. to Apple, Inc. (AAPL) was a learning experience.  This time around he isn't interested in selling his Siri successor, he's determined to license it (or so he says).  He states:

Siri is chapter one of a much longer, bigger story.

Intelligence becomes a utility.  Boy, wouldn’t it be nice if you could talk to everything, and it knew you, and it knew everything about you, and it could do everything?

Let me just cut through all the usual founder bullshit.  What we’re really after is ubiquity. We want this to be everywhere, and we’re going to consider all paths along those lines.

Viv co-founders
From left to right: Viv Labs cofounders Adam Cheyer, Dag Kittlaus, and Chris Brigham.
[Image Source: Ariel Zambelich/Wired]

Siri, Inc.'s top engineer, Adam Cheyer, and its cloud services architect, Chris Brigham, at first were deeply enamored with late Apple CEO Steven P. Jobs' vision of deep integration for Siri.  But after his death and the leadership stumbles of iOS, the passion was wearing thinner by the day as Apple shut out many of Siri's partners, preferring instead to focus on linking it to its own platform apps and services.
Mr. Brigham remarks:

The only way to make this ubiquitous conversational assistant is to open it up to third parties to allow everyone to plug into it.

In Sept. 2012 Mr. Kittlaus invited the pair to consider his vision of a Siri successor.  He helped them realize what they knew all along -- under Apple Siri's potential had been handcuffed, its original open nature slowly closed shut.  So Mr. Cheyer followed Mr. Kittlaus and left Apple to chase new life with his stealth startup.  He recalls:

I’m extremely proud of Siri and the impact it’s had on the world, but in many ways it could have been more.  Now I want to do something bigger than mobile, bigger than consumer, bigger than desktop or enterprise. I want to do something that could fundamentally change the way software is built.

One of the most interesting details of the Wired report is its accounting of Viv's funding.  According to Wired it's primarily backed by a single venture capitalist -- Solina Chau.  Ms. Chau is the business partner and spouse of China's richest man, Li Ka-shing.  Mr. Ka-Shing and his venture capital fund, Horizons Ventures, led a Series B funding round in Nov. 2009 that helped Siri, Inc. gain the momentum it need to launch.

Solina Chau
Viv Labs scored $10M USD in venture capital funding from Solina Chau. [Image Source: Getty Images]

This time around, Siri's founders reached out to Ms. Chau, who without hesitation offered them $10M USD.
II. Adam Cheyer Had Siri Almost Working in 1993
The quest for a digital secretary/helper -- a nearly human digital assistant -- predated the modern personal computer, as both the literal and figurative "stuff of science fiction".  The acronym "personal data assistant" (PDA), the first widely used miniature mobile device, itself echoed that desire.
To create a personable digital assistant some of the top minds of academia had to first labor to develop voice recognition, natural language parsing, knowledge graphing, and artificial intelligence technology.  In other words, they had to reverse engineer the human mind.
The story of Viv, Siri, and its pack of digital assistant rivals mostly began in California in the 1990s, at the Stanford Research Institute (SRI), the nonprofit research lab of Stanford University.  The project proved a fertile breeding ground for AI ideas.  Among the bright-eyed young college researchers to work on the effort was Adam Cheyer.
Adam Cheyer, in part, received inspiration from the Knowledge Navigator -- a research and development project at Apple in the late 1980s under CEO John Sculley.

Mr. Cheyer developed a program [PDF] -- initially in C and Prolog -- using SRI's DECIPHER voice recognition and GEMINI natural language parsing software.  He called it the "Open Agent Architecture" [PDF] -- OAA, for short.  

Adam Cheyer OAA

The idea behind the OAA was identical to Siri and Viv -- to create an assistant that leveraged the internet, natural language parsing, and voice recognition to help its user.  From 1995-1997 the project was funded under the Defense Advanced Research Projects Agency's (DARPA) (the research funding wing of the U.S. Department of Defense (DOD)) CommandTalk program.  Then from 1998-1999 it was funded under DARPA's Communicator program.

OAA -- concept
OAA is depicted in here in diagrams from Adam Cheyer.

Later ported to Java and other languages, some might be surprised that this Siri precursor was initially demonstrated not on the Apple Newton PDA, but on a largely forgotten rival Windows 3.1 device that narrowly beat the famous Apple portable to market.
Designed by Dauphin Technology Inc. and International Business Machines Corp. (IBMthe DTR-1 was a black-and-white Windows 3.1 handheld PC/early tablet from San Francisco, Calif.-based Dauphin Technology Inc. and International Business Machines Corp. (IBM).  The Dauphin DTR-1 was in some ways much more advanced than the Apple Computer's 1993 release -- the Original Newton MessagePad H1000.  It carried more than six times as much memory, a higher resolution screen, and an internal hard drive.
The President of Dauphin, Alan Young, had bragged to InfoWorld at the device's Jan. 1993 launch:
We are trying to beat [Apple CEO] John Sculley to the punch.
The good news for Mr. Young is he certainly did that.  The bad news was that Dauphin Technology would collapse into bankruptcy [PDF] in Jan. 1995 ending its nascent tablet/laptop effort.  But the tablet proved a crucial early test bed for the OAA.
Dauphin computer
The Dauphin DTR-1 in pen tablet mode [Image Source:]

In an interview with Pioneers, an entrepreneurial magazine, Mr. Cheyer discusses that early implementation and how close it was to Siri.  In the interview, he recalled:

Many of the ideas that I made into companies – they were not instant ideas – I had been thinking about or working on for a long time.

Most people do not know this, but my first prototype of Siri was in 1993. I had a mobile tablet, a lot like an iPad and it had almost all the capabilities that came out by Apple almost 20 years later.

You could use speech recognition, there were contacts, calendars, email, and maps. Many of the things you’d say – "schedule a meeting tomorrow at 6pm' – all worked in 1993, which was before I ever saw a web browser.

The work in the 1990s would prove crucial for what was to come.
III. Military-funded CALO Project Hones Assistant Tech
The final road to commercialization began in 2005, when the SRI was elected to lead a massive DARPA artificial intelligence (AI) project to produce a personal assistant.  Dubbed the Personalized Assistant that Learns (PAL) project within DARPA, it was termed the Cognitive Agent that Learns and Organizes (CALO) by Mr. Cheyer's group at SRI.
Backed by an unprecedented $200M USD in funding, the project was the world's largest AI project at the time.  It would draw 300 scientists from multiple universities and private sector firms.  This diverse group of many undergraduates, Master's degree students, Ph.D. candidates, postdocs, and faculty produced a massive volume of work -- over 500 papers over the next 4 years.

Siri looked to productize the SRI's CALO after its 2007 spinoff and 2008 rebranding. [Image Source: SRI]

Siri was the first of several spinoffs to come from the project.
After being impressed by the launch of the original iPhone in 2007, a Norwegian entrepreneur -- Dag Kittlaus -- approached CALO about licensing its technology (e.g. U.S. Patent No. 6,859,931) to make a mobile assistant app.  Mr. Kittlaus was then an executive at Motorola, but having seen the iPhone and read about the CALO project in the literature, he recognized a tremendous opportunity.

Dag Kittlaus
Motorola veteran Dag Kittlaus jumped at the opportunity to transform the CALO assets into an iPhone personal assistant. [Image Source:]

In many ways Mr. Kittlaus had a similar style to Apple's CEO Steve Jobs.  In the Wired interview he sounds downright "Jobsian" when he states:

I felt that computers were invented for me.  I live my life by what I call verbally stated goals. I crystallize a feeling, a need, into words. I think about the words, and I tell everyone I meet, ‘This is what I’m doing.’ I say it, and then I believe it. By telling people, you’re committed to it, and they help you. And it works. 

He had worked as a telecommunications, web, and mobile manager in Norway for 10 years.  Sometime around 2006 Motorola convinced him to join their firm and move to the U.S. as part of its xProducts subsidiary, located near Chicago, Ill.  
At Motorola he founded the Motorola Interactivity Group.  His most notable work at Motorola was the Screen3 newsfeed push technology.  An intriguing side note: according to a lengthy interview published by Cooler By the Lake, while at Motorola Mr. Kittlaus worked briefly on its upcoming Android project -- so before his Apple tie-ups, the Siri cofounder was briefly steering Motorola's Android phone efforts.
But the temptation of CALO and the iPhone proved too great.
IV. Siri Goes Live, Quickly Gets Scooped Up
Mr. Kittlaus convinced SRI to license the tech needed, and he convinced Adam Cheyer to join his new firm -- Siri, Inc. -- as its VP of Engineering.  He also successfully wooed a number of other SRI and CALO project team members into joining.  His new executive legion also included Tom Gruber (a Ph.D. computer scientist who was of the earliest developers of CALO's underpinning technology -- the Semantic Web), and Norman Winarsky (SRI's spinoff chief guru who holds a Ph.D. in mathematics, but is even better at business).
Siri worked with voice recognition/dictation firm Nuance Communications Inc. (NUAN) to give the new app responsive voice recognition.  Nuance was known for its Dragon Dictation PC software and its work on voice recognition in Ford Motor Comp.'s (FSYNC and MyFord Touch voice recognition software.  The Siri app launched in Feb. 2010 in the iTunes App Store.

Siri before Apple
Siri launched in Feb. 2010 and was purchased by Apple in just two months. [Image Source: PCMag]

Siri was capable of responding to a number of canned voice commands with some flexibility to recognize variation.  The app was ahead of its time -- understandable given its high-caliber research pedigree.  It was also one of the first apps to draw upon a cloud backend for offloaded computation.

It would respond to conversational inquiries with tongue-in-cheek replies based on words parsed out of the question.  But challenged with more complex inquiries, or even some simple ones not yet supported, and Siri would draw a blank unable to do simple task like finding a book in iTunes.

iPhone 4S Siri

At its launch in Feb. 2010, Siri supported over 40 third-party apps.  Just three weeks after the launch, Steve Jobs himself called Mr. Kittlaus and offered to purchase his intriguing project.  The deal was closed in April 2010 and was rumored to be worth around $200M USD.  The Siri sale made Mr. Kittlaus and his fellow co-founders multimillionaires.
Mr. Kittlaus and others were hired on at Apple, but from the start many of the team had problems with Apple.  Whatever these problems were under the leadership of Steve Jobs in his final days, when Steve Jobs resigned in Aug. 2011 due to his deteriorating health, the situation got worse.  The Wired report states that the post-Jobs Apple management was "less respectful of [the Siri team's] vision than Jobs was."
One key bone of contention was the idea of third party support.  Apple was not a fan of this pillar of Siri, Inc.'s business plan, as it preferred to keep tight integration to only its core apps.  Many of Siri's early investors were disappointed to see this outcome.  Gary Morgenthaler bemoaned in retrospect in the Wired piece:
Siri in 2014 is less capable than it was in 2010.
By the time Siri re-released in Oct. 2011 with the launch of Siri alongside the launch of the iPhone 4S, third party support had been slashed from over forty apps to less than ten.
Some Siri, Inc. team members immediately left Apple in the aftermath of their disagreements with Apple's management.  The most notable departures were the Siri unit's former CEO Dag Kittlaus (who resigned on "amicable" terms a day after the iPhone 4S launch, the same day Steve Jobs died) and Siri, Inc. CFO Byron Rovegno who left after the acquisition was announced in 2010.
Others like Adam Cheyer and Chris Brigham stayed a bit longer.  But they too grew frustrated with Apple's modifications to their original goals for the app, modifications they largely blamed on new CEO Timothy Cook.
V. A Crowded Field
Dag Kittlaus' bid at a Siri successor is made more challenging by the arrival of fresh blood to the digital assistant scene.
The field of voice recognition and natural language parsing in mobile devices would soon grow more crowded.  Google Inc.'s (GOOG) Google Now has emerged as one Siri competitor, launching in July 2012 with Android 4.1 "Jellybean".

Google Now
Google has added a similar in-house developed assistant called "Google Now". [Image Source: DroidLife]

Microsoft Corp. (MSFT) arrived the latest to the game, with its voice assistant Cortana, which recently debuted with the launch of Windows Phone 8.1.  

From Apple's Siri to Microsoft's Cortana (pictured), natural language processing and voice-assistant AI has been rapidly improving.

Despite the late arrival, Cortana is perhaps the "smartest" of the voice assistants.  It is not only able to parse natural language commands, but also to handle complex commands that require inter-app connectivity (e.g. checking the schedule and then performing a command conditionally).  Google Now and Siri can't do that -- yet -- although similar capabilities are pending in Android 5.0 'L' and the Siri update integrated into iOS 8.
Facebook Inc. (FB) has an AI, language parsing, and deep-learning group of its own, fueled by key academic expert Professor Yann LeCun, who the internet giant poached from New York University (NYU).  Facebook is rumored to be preparing deeper voice recognition into its mobile apps and messaging clients., Inc. (AMZN) is also incorporating voice recognition and voice commands into its Fire OS, both on the mobile front and for its new set-top box product, Fire TV.

In short, over time smartphone natural language parsing software has grown much smarter, but also much more crowded.
VI. From Five Six Labs to Viv
For all the negativity over how Apple changed the direction of Siri, it should be noted that there was some good as well.  Probably Apple's most praise-worthy contribution was showing the potential of the technology.  The raw post-SRI version of Siri launched in Feb. 2010 had a rather unsightly user interface, which may have scared some away from its terrific promise.
By the time of the relaunch, the UI was greatly refined and simplified.  And Apple allowed Siri to be integrated more closely to the core apps than a third-party app could have been.  In that regard Apple arguably convinced some non-technophile skeptics that the market was ready for natural language processing in mobile devices and mobile personal assistants.
Mr. Kittlaus' path to a Siri successor was "draped in secrecy", according to Wired.
But for those carefully watching corporate filings and Mr. Kittlaus' speeches, the stealth startup shouldn't have come as too large a surprise.  The Norwegian entrepreneur dropped a strong hint at the Technori conference in Chicago, Ill. in March 2012, stating:

When I look at artificial intelligence in just what’s gone on in the last 3 years, in that space, it is astounding.  So there’s a lot coming.  Be prepared for the fact that life is going to change a lot in the next few decades. Quite incredible.
There’s an incredible amount of things that are coming with this and Siri’s really just the beginning, even for Siri. What we’ve seen today is just really scratching the surface of what’s coming.
I’m brainstorming [my next startup]. I love technology and high tech obviously as you can tell, so I’m thinking about what comes next in the same way that I did — search was kind of the thing at the time.

I think that this notion of a virtual personal assistant with artificial intelligence is going to be a huge thing. I think our kids are going to look back at us and go, “you didn’t have an assistant?  Oh I’ve got one, but it sends emails to my friends and tells me jokes.” I truly believed that everyone will have an assistant.  There’s so many different areas that that’s going to be relevant in terms of verticals and industries that that’s going to disrupt.  I think that’s going to be way bigger than search ever was.  I’m thinking a lot about that in terms of where I spend my time next, but it’ll be something along those lines.

The Wired piece mentions that Viv Labs' office is decorated in wall art bearing two numbers -- "6 5".  The significance of that number in Viv Labs' history isn't fully explained, but we've uncovered some of these murkey histories.

Viv Lab decor
Before it was Viv Labs, it was Six-Five Labs.  [Image Source: Ariel Zambelich/Wired]

A careful search of Delaware corporate records also shows that in Jan. 2012 -- roughly two months after leaving Apple, Mr. Kittlaus quietly incorporated a new startup -- Six Five Labs, Inc. -- in Delaware.  It's unclear where the name comes from, but the term six-five has become popular, more recently, in reference to a promising hand to hit in blackjack.  Before that it was known for the Six-Five Special, an influential early rock-and-roll TV segment on the BBC, which was named after its time slot.

One possible reason for the choice is that it was ideal for providing the startup with anonymity during its early days.  After all there were a number of other similarly branded businesses: The latter company -- Fivesix Media -- maintains offices in Vancouver.  One of its employees -- web design Jason Villasoto -- owns the URL, according to WHOIS records.  A landing page with as simple graphic of an alarm clock reads "Six Five, Inc."  A Facebook announcement from 2013, reveals this page was indeed tied to Mr. Kittlaus's effort, perhaps created by Mr. Villasoto on commission.  (Note the suite number, which is the same one that Viv Labs is cited as being in.)

In July 2012 Six Five Labs was rebranded under its eventual name -- Viv Labs, Inc. -- according to updates to its Delaware registration.  However, it would continue to use the Six Five Labs name for some time to come.
Around the time of the rebranding, Mr. Kittlaus took a major step towards the launch of Viv Labs -- assembling his core team.  He approached Mr. Cheyer and Mr. Brigham, arguably the two brightest engineering talents on the Siri project.  Inviting them to his home in Chicago, they brainstormed about future personal assistants.  The experience helped the pair see what Mr. Kittlaus saw far sooner -- Apple was holding the potential of their first personal assistant back.
On a most basic level Apple's approach of a more closed personal assistant was antithetical to the vision all three men shared -- a vision of a "do engine".  Mr. Kittlaus believes "do engines" will eventually replace search engines.  He states:
[Back in the early days of Siri] I was doing this demo next to Google — it was all about doing versus searching.  I didn’t want to be just a little bit of a smarter search engine.  I wanted it to be a “do engine.” I wanted it to do something for me and I can just go in…. In all the demos we did, we just typed in next to a google box, and we showed them how bad Google was and how great we were.
But as much as Mr. Kittlaus and his colleagues want to supplant traditional search, they at least pay homage to its tried and true monetization strategies of partner commissions and advertising opportunities -- strategies they plan to use to monetize a more open Siri successor.
In Sept. 2012 Adam Cheyer and Chris Brigham officially left the Siri team at Apple, in preparation to join the stealth Viv Labs startup.  But much work remained.
VII. Building a Siri Successor
In December 2012, Mr. Kittlaus's San Francisco, Calif. law firm, Morgan, Lewis, and Bockius LLP officially incorporated Five Six Labs, Inc. as a Viv Labs, Inc. subsidiary in the state of California.  The next month the stealth startup would be joined by Siri, Inc.'s former CFO, Rovegno Byron, who serves as part-time CFO and consultant to Viv Labs.
In Jan. 2013 Viv Labs/Five Six Labs moved into suite 560, a 4,164 sq. ft. office in San Jose.  Located inside the sleek 17-story glass Ten Almaden tower, the office has views of San Pedro Square in San Jose's downtown district.  Square footage in the high rise goes for around $0.25 USD/sq. ft./month so in its 20 months so far at the site, Viv Labs likely paid a modest $20,820 USD, or so in rent.

10 Almaden building
10 Almaden, San Jose, Calif. [Image Source: Facebook]

Over the course of 2013 Viv Labs worked laboriously laid the foundation of their upcoming personal assistant, Viv.  They were assisted in the endeavor by SRI, who they license technology from and remain closely aligned with.
As noted by Wired, one key external acquisition was the addition in May 2014 of David Gondek as the principle learning engineer at Viv Labs.  Mr. Gondek had previous been a top researcher on IBM's Watson project from 2005 to 2012.
But equally notable were Viv Labs 2013 additions, which included former Siri, Inc. engineer Daniel Lord (poached from Apple) and Joshua Levy, a veteran of both SRI and Siri, Inc. who followed Mr. Cheyer to Viv Labs in Feb. 2013.  Viv Labs also added Marcello Bastéa-Forte as a design engineer earlier this year.  Mr. Bastéa-Forte was a rising young Bay Area web-coding talent and part of the CelloSoft design collective.
Currently Viv Labs has ten employees, of which nine are engineers, according to Wired.  (Dag Kittlaus is not an engineer by trade.)  Wired revealed four of these individuals.  We revealed four more.  Of those eight, six appear to have defected straight from Apple.
Above all else, it's intriguing how actively Viv Labs has poached Apple while maintaining such a low profile.
VIII. Launched at Last -- What's Next?
Earlier this year preparations intensified for a public unveil, as did secrecy.  A keynote speaker at the May 2014 Technori Pitch event at the Chase Auditorium in Chicago, Ill., Dag Kittlaus was far more tight-lipped this time around when asked what was next for him.  He responded:
What’s next for you, and how can we help?  I’ll get back in the game at some point soon.  When I’m ready to talk a little more about that, I will.
In a May 2014 interview, Mr. Cheyer was similarly silent on the pending announcement.
Secrecy was important, as others appeared intent to spill the secret -- or hint at it at least.  In Nov. 2013, by Matt Moog who namechecked the prestigious Built in Chicago alumni in a Chicago Tribune interview.  He stated:
Dag Kittlaus moved his family to San Francisco, set down at the research lab at Stanford University and founded Siri and sold it to Apple. It was a direct result of Stanford’s research and intellectual property. He’s now back here doing something new. 
Domain records indicate that just the month before Joshua Levy had reserved the Anguila domain  And in June Viv Labs filed for four trademarks, which included the phrase "The Global Brain", it's 'V' letter icon, it's full product name + slogan ("Viv: The Global Brain"), and the basic Viv icon.  And in July, it finally filed for its trademark on a Viv Labs logo.
Viv Labs 'V'

Viv Logo

This week, as the Wired piece went live, so too did the webpage.  It's still only a teaser.  But it's drawn a lot of interest.  Perhaps most intriguing is the infographic that Viv Labs released to Wired detailing how "The Global Brain" "do engine" parses speech and turns it into actions.

Viv Labs has already demoed the technology to Google, whom it is court as a licensing target.  Reportedly current and former Google Now staffers are impressed at the engine's current level of sophistication.  Microsoft is another likely licensing target.

Given all the buzz it will be hard to resist the temptation of another acquisition, but there's reason to believe that Viv Labs will stay independent and true to its principles of openness and a licensed business model.  After all, Siri was a learning experience for the Viv Labs cofounders.  This time around they want to make sure no one gets in the way of their ambitious vision, an effort that's been two decades in the making, since Adam Cheyer first started his OAA campaign at SRI.

Apple is surely a little irked after paying so much for Siri only to have it pillaged and cannibalized by poaching from the firm's former CEO.  But that's the risk you run when you try to constrain and cage the vision of such tireless, ambitious visionaries.  Sooner or later they're going to break free.

Sources:, BetterWHOIS, Linked In, Wired

"The whole principle [of censorship] is wrong. It's like demanding that grown men live on skim milk because the baby can't have steak." -- Robert Heinlein

Latest Blog Posts
Apple in the News
Saimin Nidarson - Apr 4, 2017, 9:03 AM

Copyright 2017 DailyTech LLC. - RSS Feed | Advertise | About Us | Ethics | FAQ | Terms, Conditions & Privacy Information | Kristopher Kubicki