Print 31 comment(s) - last by flash2011.. on Dec 13 at 12:54 AM

Code sloppily, pay the price; or as a source puts it "It really appears like Carrier IQ got screwed by HTC."

Cautionary lesson to programmers: code sloppily and it may cost your business dearly.  

One line of code, presumably authored by HTC Corp. (TPE:2498) appears to have been the spark that ignited a wave of press and public hysteria regarding smartphone privacy invasions and a world of woes for HTC, a small telemetry company called Carrier IQ, and Carrier IQ's numerous industry partners.

(Ed./Update V2-  Based on a bit more research, I added theory on the potential of a reference design from Carrier IQ, a case that would send some of the blame back to Carrier IQ (though HTC appears to be the primary negligent party in this one-line botch job.)

I. The Panic

Customers are already queasy about data mining, regardless of whether it is for the purpose of improving products or for financial gain.  While there's always the assurance of anonymity, customers fear -- and perhaps justifiably so -- that their private data could be exposed or maliciously exploited by data miners.

On the surface it appeared that those worst fears were confirmed over the course of the past couple months, when it was found that telemetry firm Carrier IQ appeared to be printing a host of private data to the debug stream in Android, accessible by other apps.  Media jumped to the conclusion that ALL Carrier IQ apps must be doing the same, and the hunt was on for Carrier IQ on various smartphones.  

The search yielded results.  Carrier IQ was found on Android, on the iPhone, and more.  Meanwhile media and members of the public jumped to more and more wild conclusions.  As one reader wrote in my previous post on Carrier IQ:
They are reading every key press, that is an epic fail. It is beyond a significant security/privacy issue...

The only problem is that nobody bothered to dig into the actual code and see if that claim was right.  Or almost nobody.

I started decompiling the pieces of Carrier IQ last week, and while my analysis of the three programs...


... off a newly rooted EVO 4G device I have access to is ongoing, the early results indicate that another member of the security community who also went to the trouble of actually bothering to look at the code -- Dan Rosenberg -- was correct in stating that there's no evidence that Carrier IQ is exposing customers' private information to malicious third parties.

II. The Breach

But what about the debug stream information?  That's a huge security flaw [PDF] in its own right, correct?

Of course it is, but it's important not overblow the security risk here.  Here's the compromise scenario:
  1. IF you install an app with debug permission android.permission.READ_LOGS in its AndroidManifest.xml ...
  2. AND IF you run that app...
  3. AND IF it's malicious...
  4. AND IF it or some other installed app from the same malicious party, which is also installed and running have network permissions ( android.permission.INTERNET)....
Then yes, absolutely your private data (specifically GPS position, SMS text contents, URLs, including uncensored HTTPS strings, and numbers dialed) could be captured, packaged, and sent out to a malicious third party.  Again, let's consider the worse case scenario if this happens:
  1. IF you use an https encrypted website (there's no keylogging outside the dialer app so apps are safe...)...
  2. AND IF that site doesn't properly censor your username and/or password in its string after the base domain in the https URL...
  3. AND IF all of the above (debug app, internet permission, malicious) is true, you could lose your username and/or password to a malicious party.
Again it's easy to overstate the risk, but any data loss is unacceptable, so let's for now consider this an atrocious security breach, even if it only affects a handful of customers out of millions of subscribers.

III. The Blame

So who's to blame?

Dan Rosenberg argued that "" was the sole work of HTC, and that it was the app to blame for egregious data prints, but that Carrier IQ's core apps (iqd and iqfd) didn't log any of this information, and didn't even pass along all the things the HTC app was printed (i.e. it did pass along URLs, but didn't pass along text message contents, etc.).

Anyhow the important point Mr. Rosenberg raised is that the exposure of this data to malicious third parties via the debug stream was the (accidental) work of HTC -- NOT Carier IQ.  

And while I was unable to confirm definitively that Carrier IQ was indeed solely authored by HTC, the nature of data flow, the code itself, and even the title seemingly suggests that HTC is the primary author of this code.  And those egregious prints?  

Ed. - I added this thought upon further examination of anothe CIQ code...
Of course this code looks an awful lot like a reference design from Carrier IQ, given its similarities to code seen elsewhere on Samsung Electronics Comp., Ltd.'s (KS:005930) devices.  If this is the case the blame becomes a bit more complicated --  Carrier IQ may have had the debug statement turned on in the reference design it may have been given by Carrier IQ, but it clearly modified the code and was responsible for the finished product.  Why didn't it have the common sense to turn off the debug flag?

They all come down to one fateful line:

protected boolean DBG = true;

(found near the top of (class AgentService)

Which rears its ugly head in this conditional:

public void handleMessage(Message paramMessage)
      AgentService.this.result = Controller.IQInit();
      Intent localIntent = (Intent)paramMessage.obj;
      if (AgentService.this.DBG)
        Log.v(AgentService.TAG, "Action[" + AgentService.submitTime + "]:" + localIntent.getAction());

For the non-Android developers out there, here's what the code cumulatively does:  
  1. The HTC program (Ed. - perhaps based on a Carrier IQ reference code) gets a message from another HTC program or a program it modified to gain access to additional information (such as the default webkit browser).  
    NOTE: Obviously it can only make these modification to core system apps it distributes on its smartphones, not apps downloaded from the Android Market. 
  2. It starts the main Carrier IQ app (in /system/ on HTC devices).
  3. It sends an Intent for the main Carrier IQ to read and log, echoing the data it received.
  4. It checks to see if the debug flag is set.
  5. If this flag is set, it writes a log to the system's debugging cache file with the full details of the Intent (which include various juicy details like SMS contents, etc.).
The problem?  The debug flag is set in the production build.

IV. The Aftermath

If HTC had not left this flag set, perhaps somebody someday might have found out about Carrier IQ.  But the security flaw in the debug stream would never have existed.  And it would have been unlikely that novice developer Trevor Eckhart would have found it while poking around in the debug log -- because nothing would have been printed to the debug log.  And the public witch hunt against Carrier IQ never would have ensued.

Of course this was probably just a careless, but innocent accident on the part of some HTC developer.  But it was a costly one, one that sent ripples of fear, uncertainty, and doubt through a public already fearful of data mining.

That line of code would lead to a world of hurt.

A source close to another devicemaker seconded Mr. Rosenberg's opinion that the module in question appeared to be HTC code and not Carrier IQ code,  when I showed them it.  After showing them the fateful line, they remarked, "It really appears like Carrier IQ got screwed by HTC."

"Screwed" indeed.  Who knows how much business Carrier IQ has lost because of this mess.

Perhaps that explains the dumbfounded statement of Carrier IQ spokesperson Andrew Coward (a special thank you to Network World's Senior Editor John Cox who pointed me to this quote during a discussion),"We're as surprised as anybody to see all that information flowing.  It raises a lot of questions for the industry -- and not [only] for Carrier IQ."

V. The Blame, Redux

The bottom line here is that it increasingly appears that there's a strong likelihood that Carrier IQ was not the one directly responsible for the inadvertent exposure of private data.  The direct responsibility falls on HTC and likely HTC alone (Dan Rosenberg does not mention finding a similar flaw in Samsung implementation of the data passer).

But in terms of a more broad umbrella of responsibility, there's two perspectives you can take here:
  1. The "Tank Argument:
    I made this argument to a colleague, writing about the data breach.  The premise here is that Carrier IQ is somewhat to blame for the products it puts out jointly with its partners, even if the partner made the component that did the damage.  

    I give the analogy of a tank software maker who passes the responsibility of firing the main gun to a third-party who then botches the job and delivers a code that unbeknowst to testers on an extremely rare but predicable occasion erratically fires due to a known bug.  Servicepeople die and the software is scrutinized.  The bug is found and the families of the deceased sue the tank software company AND its partner, even though the partner wrote the flawed subprogram.  The main program maker should have kept track of what was going on its joint finished product, the lawyers might argue.
  2. The Small Developer, Big Developer Argument:
    Even with all its recent success Carrier IQ has no where near the resources of HTC.  So you could make the counterargument that HTC's faulty software should be blamed on HTC alone as Carrier IQ as a (respectively) smaller third party who services scores of different device makers and carriers doesn't have the time to individually monitor its partners' associated codes for flaws (and perhaps doesn't even GET uncensored access to these partner codes).  

    This is certainly a fair argument as well.
(Ed. - If my reference code hypothesis is confirmed, this offers some additional stock in the philosophical argument that Carrier IQ should be partly to blame, even if HTC was negligent and lacked common sense in modifying the reference design.)

Whichever argument you buy, the Carrier IQ mess does paint an interesting picture:
  1. HTC accidentally leaves its debug on in a single line of code, exposing Carrier IQs metrics (and more) to the world (and malicious parties).
  2. A non-developer media fails to properly research the issue and assumes its endemic to all Carrier IQ devices (or at least all Android devices) and goes wild, accusing Carrier IQ of installing rootkits and violating user privacy.
  3. A non-developer public fails to understand this is an HTC issue and panics, when in many cases they are at no risk.
  4. Carrier IQ potentially loses millions in business.
Now the exposed information does offer some decent philisophical debates (albeit at the cost of Carrier IQ, et al.'s financial well-being and creation of perhaps unwarranted paranoia among non-HTC device owners):

Should Carrier act as a platform manager and collect anonymized data further safeguarding privacy?


Should they act as a dumb pipe at the cost of perhaps providing inferior service that might have been improved by telemetry?

The debate is analogous to the question on the merits monitoring/usage data collection in an IT setting, given that your devicemaker and/or carrier are in effect your smartphones' administers and you're just a user, unless you hack your phone and root it. (Hence why Carrier IQ is not a rootkit, even if it keylogs -- it was installed by the administer of the device -- that's the fundamental difference between an administrative monitoring software and a rootkit, by definition.)

The question in IT terms would be something like:

Should an IT department monitor users, at the cost of privacy, in order to counter extreme abusive misbehavior and/or improve service quality?


Should they act as a dumb pipe at the cost of perhaps providing inferior service and potential abuse?

The world of commercial IT computing has virtually unanimously voted in favor of the former approach -- active administration.

But hey, perhaps the smartphone, despite being an analogous situation (customers wanting to pass of the icky administrative responsibilities on to a service provider who lowers the customers' permissions to user level) -- should be handled differently.

It's a good philisophical debate to be had.

But members of the public and media, please, let us be rational and look at the actual code before we pass judgement and condemn Carrier IQ, Samsung, or anyone else.


In my past piece, I relied primarily on the debugging cache a built in log file readable by Android apps or PC-side developer tools like ADB. I personally used "Catlog", a free app by Nolan Lawson, for convenience.  When verifying that it was that was doing the talking, I used "Process List" by Tak Kuji to grab and match the pIDs in the log.

In this followup I:
  1. Root my test device.
  2. Mounted in ADB and transferred the appropriate .apk (iq-related) off the phone.
  3. Opened a cmd prompt
  4. (Windows button > type "cmd" in the bar)
  5. Turned the .apk files into .zip files via a rename.
  6. Extracted the contents to a folder. 
  7. Ran dex-translator- (free: from a Google Code project) on the classes.dex compressed .jar file:
    QR code --- dex translator
  8. Ran jd-gui.exe (free: from a French developer) to view the resulting classes-dex2jar.jar file.
  9. Poked around in the code!

You can of course try this for yourself and test it out on your Android smartphone of choice.  But beware of the following:
  1. Rooting devices voids their warranty (though a reinstall of the factory rom could obfuscate that fact, should you need a repair... hint, hint).
  2. Decompiling apps you did not make is a gray area of the law.  Typically it's viewed as legal for security researchers.  But be aware that it is a sort of hazy region if your ultra-worried about liability.
Happy reverse engineering!

Comments     Threshold

This article is over a month old, voting and posting comments is disabled

What can we do now?
By ghost.image on 12/11/2011 5:55:14 PM , Rating: 2
Good Article..
Sometimes the follow on discussions are just as interesting.
From a legal stand point I think we can probably deduce or at minimum assume the following ideas and follow on questions.
1. The carriers probably have something somewhere in their end user agreements regarding data collection and monitoring. What should be changed in their regard? I think most folks are just comfortable with just not having it buried in the end user agreement.
2. Even though you buy the phone and yes for those enthusiasts out there running custom ROMs should be your choice, but what if any control can the carrier, and for that matter the manufacturer hold over their device as dictated by law?
3. I think there has been some confusion on who Carrier IQ's customers are. They are not you, the end user of the phone. They are the carriers. It is like a car analogy, pieces and parts of the car might come from all over, lots of different sub contractors, but the name on the car says Ford. In this instance the people who need to opt in or out is the carrier not you, assuming the end user agreement you signed covers data collection.
4. I would have to say more than likely specifically with HTC and Carrier IQ, HTC would probably hold the majority of responsibility. It is their device and like you said all the other phones as far as we know don't seem to have the debug mode turned on. And while most programs have a debug mode option, the fact it was turned on means it probably was a mistake by an HTC employee.
5. Someone had mentioned that Trevor had ten years of experience. While he is an Android enthusiast to be sure, experienced no. He was working at Staples until 2005. After that he is working at a single company as a systems administrator which is Windows centric. He does not show any programming work history or security work history with regard to specific wireless data protocols. NOTE: this is based off of his online resume, which may hold inaccuracies.
So for reference or further work on Trevor's initial theory I would suggest reading Dan Rosenberg's analysis. Tim Armstrong from Kaspersky Labs. Also there was a blog release from Trend Micro with regard to Carrier IQ. While the Kaspersky, and Trend statements are not as detailed as this, thank you by the way for the details, they are official statements from top quadrant anti virus firms.

This raises a question I would like to pose to everyone to think about:
Android OS is open source. As such it is imperative, and critical that developers pick apart phones, and the OS to look for flaws, and bugs. It is what drives the development of the OS and makes it better faster stronger…
So here is my question and this goes beyond the current issue, and I think is a valid inquiry to any developer or enthusiast.
If, and I mean if Carrier IQ is cleared of any charges or wrong doing, which I think there is a good chance, is Trevor in any way responsible?
I am not picking on Trevor, I really want to pose the question. Based on the article Carrier IQ more than likely didn't do anything wrong in the eyes of the law or isn't the super evil company the media first ran with.
The reason I ask is the following.
1. We can deduce that the information released on Youtube while correct in the fact that yes it is an Android debug log set to verbose, it doesn't show what the software is doing on the back end, so lets say that bad things were stated and what it is doing, or heavily inferred and incorrect for the most part.
2. This was put on Youtube, not an Android developer forum. while I think if you have a theory, great! Post it to the forum, ask people to check it out, respond back with comments. I think this would have more than likely not have been the fire storm we have seen had it not been put on youtube.
Developers would have checked into it, found as you did the flaw, tested other devices, and found it is HTC specific. And to be honest I think the issue could have been resolved quickly and easily.
Instead it was posted to millions on a non technical forum causing all sorts of heart ache I am sure for a small startup.
Write or wrong there more than likely has been thousands spent shoring up security, making sure everything is backed up etc. Anyone who has ever read a class action lawsuit electronic preservation statement knows what a pain in the butt it can be. This all equals dollars spent by the company.
3. While free speech is a right here in the US, the the unfounded statements are proven to be false, even if only partly false, can the person making them be held responsible in some way?

Why am I asking this?
Answer: I am asking this question because it raises the question of a code of conduct regarding development and bug tracking within the open source community. Yes we have a right to say what we want, unless we are threatening heads of state, but what is the best way to handle these things.
While the C&D issued by the software maker might not have been the best move, it was warranted. The AFF carries a lot of weight so the back down as I am assuming to take a different approach.
As developers, and enthusiasts, hackers just sounds bad, how should we conduct ourselves in the future? If things turn out such that Trevor gets dragged through the mud over this, wont this discredit the community as a whole?
I do admit there was a lot of bad journalism going on. People ran with the story which became headlines overnight, and really no one until later decided to actually look at the code like yourself, and Dan.
What can be done as a development community as a whole to ensure that issues are resolved, supported by fact, and delta with in such a way as so shore up the Android community as a whole. We don't want to wind up looking like a bunch of amateurs who cried wolf. That helps no one in the future.

“So far we have not seen a single Android device that does not infringe on our patents." -- Microsoft General Counsel Brad Smith

Copyright 2016 DailyTech LLC. - RSS Feed | Advertise | About Us | Ethics | FAQ | Terms, Conditions & Privacy Information | Kristopher Kubicki