One Line of Code Created a World of Woe for HTC, Carrier IQ, et al.
December 9, 2011 4:44 PM
comment(s) - last by
Code sloppily, pay the price; or as a source puts it "It really appears like Carrier IQ got screwed by HTC."
Cautionary lesson to programmers: code sloppily and it may cost your business dearly.
One line of code, presumably authored by HTC Corp. (
) appears to have been the spark that ignited
a wave of press and public hysteria
regarding smartphone privacy invasions and a world of woes for HTC, a small telemetry company called Carrier IQ, and Carrier IQ's numerous industry partners.
Ed./Update V2- Based on a bit more research, I added theory on the potential of a reference design from Carrier IQ, a case that would send some of the blame back to Carrier IQ (though HTC appears to be the primary negligent party in this one-line botch job.
I. The Panic
Customers are already queasy about data mining, regardless of whether it is for the purpose of improving products or for financial gain. While there's always the assurance of anonymity, customers fear -- and perhaps justifiably so -- that their private data could be exposed or maliciously exploited by data miners.
On the surface it appeared that those worst fears were confirmed over the course of the past couple months, when it was found that telemetry firm Carrier IQ appeared to be printing a host of private data to the debug stream in Android, accessible by other apps. Media jumped to the conclusion that ALL Carrier IQ apps must be doing the same, and the hunt was on for Carrier IQ on various smartphones.
The search yielded results. Carrier IQ was found on Android, on the iPhone, and more. Meanwhile media and members of the public jumped to more and more wild conclusions. As one reader wrote in my previous post on Carrier IQ:
They are reading every key press, that is an epic fail. It is beyond a significant security/privacy issue...
The only problem is that nobody bothered to dig into the actual code and see if that claim was right. Or almost nobody.
I started decompiling the pieces of Carrier IQ last week, and while my analysis of the three programs...
... off a newly rooted EVO 4G device I have access to is ongoing, the early results indicate that another member of the security community who also went to the trouble of
actually bothering to look at the code
-- Dan Rosenberg -- was correct in stating that there's no evidence that Carrier IQ is exposing customers' private information to malicious third parties.
II. The Breach
But what about the debug stream information? That's a
huge security flaw
[PDF] in its own right, correct?
Of course it is, but it's important not overblow the security risk here. Here's the compromise scenario:
IF you install an app with debug permission android.permission.READ_LOGS in its AndroidManifest.xml ...
AND IF you run that app...
AND IF it's malicious...
AND IF it or some other installed app from the same malicious party, which is also installed and running have network permissions ( android.permission.INTERNET)....
Then yes, absolutely your private data (specifically GPS position, SMS text contents, URLs, including uncensored HTTPS strings, and numbers dialed) could be captured, packaged, and sent out to a malicious third party. Again, let's consider the worse case scenario if this happens:
IF you use an https encrypted website (there's no keylogging outside the dialer app so apps are safe...)...
AND IF that site doesn't properly censor your username and/or password in its string after the base domain in the https URL...
AND IF all of the above (debug app, internet permission, malicious) is true, you could lose your username and/or password to a malicious party.
Again it's easy to overstate the risk, but any data loss is unacceptable, so let's for now consider this an atrocious security breach, even if it only affects a handful of customers out of millions of subscribers.
III. The Blame
So who's to blame?
Dan Rosenberg argued that "com.htc.android.iqagent" was the sole work of HTC, and that it was the app to blame for egregious data prints, but that Carrier IQ's core apps (iqd and iqfd) didn't log any of this information, and didn't even pass along all the things the HTC app was printed (i.e. it did pass along URLs, but didn't pass along text message contents, etc.).
Anyhow the important point Mr. Rosenberg raised is that the exposure of this data to malicious third parties via the debug stream was the (accidental) work of HTC -- NOT Carier IQ.
And while I was unable to confirm definitively that Carrier IQ was indeed solely authored by HTC, the nature of data flow, the code itself, and even the title seemingly suggests that HTC is the primary author of this code. And those egregious prints?
Ed. - I added this thought upon further examination of anothe CIQ code...
Of course this code looks an awful lot like a reference design from Carrier IQ, given its similarities to code seen elsewhere on Samsung Electronics Comp., Ltd.'s (
) devices. If this is the case the blame becomes a bit more complicated -- Carrier IQ may have had the debug statement turned on in the reference design it may have been given by Carrier IQ, but it clearly modified the code and was responsible for the finished product. Why didn't it have the common sense to turn off the debug flag?
They all come down to
one fateful line
DBG = true
(found near the top of com.htc.android.iqagent (class AgentService)
Which rears its ugly head in this conditional:
public void handleMessage(Message paramMessage)
AgentService.this.result = Controller.IQInit();
Intent localIntent = (Intent)paramMessage.obj;
Log.v(AgentService.TAG, "Action[" + AgentService.submitTime + "]:" + localIntent.getAction());
For the non-Android developers out there, here's what the code cumulatively does:
The HTC program (
Ed. - perhaps based on a Carrier IQ reference code
) gets a message from another HTC program or a program it modified to gain access to additional information (such as the default webkit browser).
NOTE: Obviously it can only make these modification to core system apps it distributes on its smartphones, not apps downloaded from the Android Market.
It starts the main Carrier IQ app (in /system/ on HTC devices).
It sends an Intent for the main Carrier IQ to read and log, echoing the data it received.
It checks to see if the debug flag is set.
If this flag is set, it writes a log to the system's debugging cache file with the full details of the Intent (which include various juicy details like SMS contents, etc.).
The problem? The debug flag is set in the production build.
IV. The Aftermath
If HTC had not left this flag set, perhaps somebody someday might have found out about Carrier IQ. But the security flaw in the debug stream would never have existed. And it would have been unlikely that novice developer Trevor Eckhart would have found it while poking around in the debug log -- because nothing would have been printed to the debug log. And the public witch hunt against Carrier IQ never would have ensued.
Of course this was probably just a careless, but innocent accident on the part of some HTC developer. But it was a costly one, one that sent ripples of fear, uncertainty, and doubt through a public already fearful of data mining.
That line of code would lead to a world of hurt.
A source close to another devicemaker seconded Mr. Rosenberg's opinion that the module in question appeared to be HTC code and not Carrier IQ code, when I showed them it. After showing them the fateful line, they remarked, "It really appears like Carrier IQ got screwed by HTC."
"Screwed" indeed. Who knows how much business Carrier IQ has lost because of this mess.
Perhaps that explains the
of Carrier IQ spokesperson Andrew Coward (a special thank you to
's Senior Editor John Cox who pointed me to this quote during a discussion),"We're as surprised as anybody to see all that information flowing. It raises a lot of questions for the industry -- and not [only] for Carrier IQ."
V. The Blame, Redux
The bottom line here is that it increasingly appears that there's a strong likelihood that Carrier IQ was not the one directly responsible for the inadvertent exposure of private data. The direct responsibility falls on HTC and likely HTC alone (Dan Rosenberg does not mention finding a similar flaw in Samsung implementation of the data passer).
But in terms of a more broad umbrella of responsibility, there's two perspectives you can take here:
The "Tank Argument:
I made this argument to a colleague, writing about the data breach. The premise here is that Carrier IQ is somewhat to blame for the products it puts out jointly with its partners, even if the partner made the component that did the damage.
I give the analogy of a tank software maker who passes the responsibility of firing the main gun to a third-party who then botches the job and delivers a code that unbeknowst to testers on an extremely rare but predicable occasion erratically fires due to a known bug. Servicepeople die and the software is scrutinized. The bug is found and the families of the deceased sue the tank software company AND its partner, even though the partner wrote the flawed subprogram. The main program maker should have kept track of what was going on its joint finished product, the lawyers might argue.
The Small Developer, Big Developer Argument:
Even with all its recent success Carrier IQ has no where near the resources of HTC. So you could make the counterargument that HTC's faulty software should be blamed on HTC alone as Carrier IQ as a (respectively) smaller third party who services scores of different device makers and carriers doesn't have the time to individually monitor its partners' associated codes for flaws (and perhaps doesn't even GET uncensored access to these partner codes).
This is certainly a fair argument as well.
Ed. - If my reference code hypothesis is confirmed, this offers some additional stock in the philosophical argument that Carrier IQ should be partly to blame, even if HTC was negligent and lacked common sense in modifying the reference design.
Whichever argument you buy, the Carrier IQ mess does paint an interesting picture:
HTC accidentally leaves its debug on in a single line of code, exposing Carrier IQs metrics (and more) to the world (and malicious parties).
A non-developer media fails to properly research the issue and assumes its endemic to all Carrier IQ devices (or at least all Android devices) and goes wild, accusing Carrier IQ of installing rootkits and violating user privacy.
A non-developer public fails to understand this is an HTC issue and panics, when in many cases they are at no risk.
Carrier IQ potentially loses millions in business.
Now the exposed information does offer some decent philisophical debates (albeit at the cost of Carrier IQ,
.'s financial well-being and creation of perhaps unwarranted paranoia among non-HTC device owners):
Should Carrier act as a platform manager and collect anonymized data further safeguarding privacy?
Should they act as a dumb pipe at the cost of perhaps providing inferior service that might have been improved by telemetry?
The debate is analogous to the question on the merits monitoring/usage data collection in an IT setting, given that your devicemaker and/or carrier are in effect your smartphones' administers and you're just a user, unless you hack your phone and root it. (Hence why Carrier IQ is not a rootkit, even if it keylogs -- it was installed by the administer of the device -- that's the fundamental difference between an administrative monitoring software and a rootkit, by definition.)
The question in IT terms would be something like:
Should an IT department monitor users, at the cost of privacy, in order to counter extreme abusive misbehavior and/or improve service quality?
Should they act as a dumb pipe at the cost of perhaps providing inferior service and potential abuse?
The world of commercial IT computing has virtually unanimously voted in favor of the former approach -- active administration.
But hey, perhaps the smartphone, despite being an analogous situation (customers wanting to pass of the icky administrative responsibilities on to a service provider who lowers the customers' permissions to user level) -- should be handled differently.
It's a good philisophical debate to be had.
But members of the public and media, please, let us be rational and look at the actual code before we pass judgement and condemn Carrier IQ, Samsung, or anyone else.
In my past piece, I relied primarily on the debugging cache a built in log file readable by Android apps or PC-side developer tools like ADB. I personally used "
", a free app by Nolan Lawson, for convenience. When verifying that it was com.android.htc.iqagent that was doing the talking, I used "
" by Tak Kuji to grab and match the pIDs in the log.
In this followup I:
Root my test device.
Mounted in ADB and transferred the appropriate .apk (iq-related) off the phone.
Opened a cmd prompt
(Windows button > type "cmd" in the bar)
Turned the .apk files into .zip files via a rename.
Extracted the contents to a folder.
(free: from a Google Code project) on the classes.dex compressed .jar file:
(free: from a French developer) to view the resulting classes-dex2jar.jar file.
Poked around in the code!
You can of course try this for yourself and test it out on your Android smartphone of choice. But beware of the following:
Rooting devices voids their warranty (though a reinstall of the factory rom could obfuscate that fact, should you need a repair... hint, hint).
Decompiling apps you did not make is a gray area of the law. Typically it's viewed as legal for security researchers. But be aware that it is a sort of hazy region if your ultra-worried about liability.
Happy reverse engineering!
This article is over a month old, voting and posting comments is disabled
12/11/2011 6:14:00 PM
I have to agree with Flash on some parts.
As corporate IT you own and control all parts of your network so watching everything and being able to control it is by design and law.
Now to that end what control do wireless carriers have over devices running on their networks? Or maybe the question is how much control should they have?
The reason I ask it the following. Let say someone loads some custom non standard OS on a device, it could be a laptop with wireless data card, it could be a phone or tablet.
It then goes out and creates all sorts of havoc on the network, it could be anything form redirecting DNS queries to overloading cell sights with data. It could even be actively spying on other devices.
What control or right does the carrier have to find out what is going on and shut down, block monitor the device causing the problem as it is now affecting other customers?
The reason I am using the case above is because while you don't work for a carrier, you do sign some sort of agreement with the carrier about running a device on their network which you do not own.
Also one point of disagreement. Trevor while an enthusiast does not show a long as sorted carrier as a programmer or security expert.
for his resume.
He does show experience in Windows, management using Cisco UI and desktop and laptop support, but nothing regarding custom script implementation across linux builds, large scale security infrastructure planning, nothing.
Now before you send the lynx mob my way I am saying this because:
1. He has a right to experiment, and look into things.
2. He has a right to formulate a theory and present it to peers to support or contradict.
3. He has a right to say what ever he wants to whom ever he wants about what ever he wants*** But there is a catch.
He posted a fire starting storm of a video to millions of non technical people, and it seems now most of it can be shot down, or at least explained.
While I agree there is an HTC implementation issue, I don't know if anything was legally done wrong, so the AFF better stick around his camp for a while.
12/12/2011 12:03:13 AM
In answer to your question on "how much control should they have"
Same as what my corporate ISP would have if my server got infected and started flooding the network with spam. They wouldn't require me to install ISP spyware on my server. They would very quickly just cut off my network access. I don't see why wireless phones should be any different. If my phone is misbehaving they just block my id (IMSI/MSISDN) or my phone id (IMEI/MEID) from registering on the network until I have fixed the problem. If some customers want the carrier to administer and spy on their phone, that is fine. This, however, should be opt-in ONLY.
I repeat again that the US cellular experience where the phone is tightly bound to the carrier is largely unique. In **most** countries it is very easy to buy the phone separate from the cellular service. If you don't like your service you just pop in a SIM from another carrier and you are good to go. Overseas carriers just don't have the opportunity to install their spyware on your phone...yet I have not heard about these foreign networks keeling over due to hoards of misbehaving phones.
Whilst techies have known for sometime that carriers are up to these spyware games (e.g. look up flash2011 on slashdot). Trevor Eckhart did a great service by making his youtube video. It started to make the media and public in general aware of what is going on. This truly needs a proper debate, not only about Carrier IQ (which by the way even Eric Schmidt called Carrier IQ a "keylogger" - you would think he would be more careful with his words if he was unsure), but on a whole range of issues related to privacy on mobile devices.
I do find it rather outrageous your attempts to "shoot the messenger" (Trevor Eckhart). First you (and Jason Mick) suggest we should disregard him because he doesn't have the qualifications, even though we are not talking about some obscure hole in the TLS protocol or in AES encryption. We are talking about data logging on a phone. His many years of IT experience **more** than qualifies him to make statements about Carrier IQ (even though he uses "Windows" and started at "Staples" - gee you would think we should ignore **anyone** in IT who never completed a university degree - **cough** Bill Gates **cough ** Steve Jobs **cough** Mark Zuckerberg **cough**).
Then you bring up his recent move to Telogis. Apparently they also write software to handle device metrics. What are you trying to insinuate here? He may now know **too much** about the topic? I may have missed it but, unlike Carrier IQ, I didn't see where Telogic install spyware on consumer devices. Perhaps you are trying to insinuate corporate sabotage? I guess if you throw enough mud, even though none of it is true, some of it will stick. Classic "shoot the messenger".
Your final, most revolting argument, is to suggest that what he did was wrong (another "shoot the messenger" tactic). This is not a case of finding a zero-day vulnerability in Windows and spreading the news all over the web without giving Microsoft the opportunity to fix it. Instead this is a case of shedding some sunlight on a hidden backdoor installed by carriers to vacuum up private data (if you look up my slashdot submission it appears that Verizon is already selling this data, though it comes from a different program). I don't think he claimed to provide a complete analysis (that will come from discovery for the class action lawsuits - I am looking forward to this. Hopefully they don't settle out of court to try to "hush it up"). He just reported what he saw. To say that the tech community needs to "self censor" to protect corporations is just not worthy of a citizen of a country that supposedly prides itself on the First Amendment.
"I f***ing cannot play Halo 2 multiplayer. I cannot do it." -- Bungie Technical Lead Chris Butcher
Research: Carrier IQ Underscores Deeper Hole in Android OS's Security
December 5, 2011, 9:41 AM
Apple Posts Record Fiscal Q3 Profit of $7.7B, Sells 35.2M iPhones
July 22, 2014, 4:57 PM
Quick Note: Mobile Use Surpasses PC in China for First Time
July 22, 2014, 11:18 AM
WSJ: Up to 80 Million iPhone 6 Smartphones to be Produced by Year's End
July 22, 2014, 10:57 AM
Quick Note: NVIDIA’s SHIELD Tablet Gets Official
July 22, 2014, 10:00 AM
Motorola Moto G Successor Reportedly Uncovered, Moto X Discounted by up to $75
July 21, 2014, 1:11 PM
JJ Abrams Unveils X-Wing Starfighter for New "Star Wars" Movie
July 21, 2014, 12:24 PM
Most Popular Articles
Microsoft Kills Entertainment Unit, May Shelve Flagship Lumia "McLaren"
July 18, 2014, 7:40 PM
Boeing 777 Malaysian Airlines Flight 17 Crashes in Ukraine
July 17, 2014, 1:00 PM
FBI Report Suggests That Self-Driving Cars Could Be Used as Rolling Bombs
July 16, 2014, 11:02 AM
Tesla Confirms “Model III” EV with 200+ Mile Range, Blames Ford for Missed "SEX"
July 15, 2014, 9:12 PM
Google Signs Controversial Deal With Pharmaceutical Giant Novartis
July 16, 2014, 7:32 PM
Latest Blog Posts
Space Terrorism is a Looming Threat For the United States
Apr 23, 2014, 7:47 PM
Facebook Aims to Provide Internet to "Every Person in the World" with Drones, Satellites
Apr 1, 2014, 10:20 AM
Retail Mobile Sites Experience Outages in Light of Simplexity's Bankruptcy
Mar 14, 2014, 8:48 AM
Tesla vs. BMW: Who Has the Safer EV?
Feb 1, 2014, 2:56 PM
Justice Leaks Details of Next HTC One Two Flagship Phone
Dec 5, 2013, 4:04 PM
More Blog Posts
Copyright 2014 DailyTech LLC. -
Terms, Conditions & Privacy Information