backtop


Print 10 comment(s) - last by ven1ger.. on Feb 19 at 5:49 PM

Typically accurate algorithm may need tweaking to deal with new trends

Launched in 2008Google Flu Trends is one of Google Inc.'s (GOOG) most intriguing internet services.  Leveraging Google's mastery of data mining and industry-leading search engine position, the system injects data on health-related searches into a complex model which spits out an estimation of the number of people in each region that are infected with the flu virus.  

I. Flu Trends' Bad Forecast

The system performs extraordinarily well, typically matching up closely with data from the Centers for Disease Control (CDC).  The system typically is several days ahead of CDC predictions in 29 countries and today also tracks a second disease -- dengue.

But this flu season was a rough one for the Google service.  Google badly over predicted the Christmas peak of flu season, estimating that over 10 percent of the U.S. population had flu, versus the CDC's data, which showed only around 6 percent had it.

Google Flu Trends miss
Google Flu overpredicted, while Flu Near You underpredicted. [Image Source: Nature]

So what happened?

Experts believe part of the problem was public fear influencing search behavior in an unusual way.  The flu season got off to its earliest seasonal start in nearly a decade, starting in November and peaking in December.  Of the three major flu virus strains, the most virulent strain -- H3N2n -- dominated, as in the severe 2003 outbreak.

There were multiple deaths, particularly among the elderly from the outbreak.  As a result, the mass media latched onto the story and focus a great deal of coverage on the flu -- more so than usual.  Experts say that may have led people to increase searches for flu terms, even if they didn't have flu.

It was also a bad season for norovirus outbreaks, which may have led to people searching for flu terms.  Norovirus is a separate pathogen that manifests itself via intestinal symptoms.  However, many in the public mistake norovirus infections for flu.

II. Self-Tracking Efforts Also Miss

Professor John Brownstein, a Harvard Medical School epidemiologist, told Nature Magazine that the miss by Google shows the difficulty in adapting algorithms to new variables.  He comments, "You need to be constantly adapting these models, they don’t work in a vacuum.  You need to recalibrate them every year."

Professor Brownstein helps maintain "Flu Near You" a Boston Children's Hospital/HealthMap project which tracks flu in the U.S. via volunteer self-reporting.  The 2011 program tracks a sample set of 70,000 people and has 46,000 participants (who report on their own health and health of family members).

Flu Virus
Tracking the flu-virus in real time is a daunting challenge. [Image Source: Navco]

To Google's credit, Flu Near You missed nearly as badly, underestimating the peak infection rate.  It peaked around a little over 4 percent.  It's unclear why that underestimation happened.

Digital tracking of influenza-like illness (ILI) -- symptoms such as fever and sinus issues -- first began in 1985 in France with the creation of the Sentinelles network.  Today France's GrippeNet.fr continues that tradition.  Similar to Flu Near You, GrippeNet uses self-reporting from volunteers and has 5,500 participants.

The CDC's tracking comes directly from a network of 2,700 health institutions, which cover 30 million patients.

III. What About Twitter Tracking?

The misses of Flu Near You and Google Flu Trends open the door to new analysis projects.  A pair of new efforts -- MappyHealth and Sickweather -- look to track Twitter posts to determine flu rates.  Johns Hopkins University Professor Michael Paul argues that Twitter monitoring may offer less noise than search term monitoring, while offering big sample sets than self-monitoring.

He comments, "I suspect that passive monitoring of social media will always yield more data than systems that rely on people to actively respond to surveys, like Flu Near You."

Twitter
Some are turning to Twitter for flu tracking.

But some are skeptical of Twitter-mining.  Lyn Finelli, head of the CDC’s Influenza Surveillance and Outbreak Response Team, argues that Twitter crowdsourcing is less effect that search-term efforts, because Twitter's userbase tends to be younger, less reliable users.  She comments, "The Twitter analyses have much less promise."

The 2012 miss was the worst since 2009 for Google, which led to some major algorithmic adjustments.  In 2009 Google under predicted the H1N1 swine flu outbreak.  That mistake was the subject of a PLOS One paper entitled "Assessing Google Flu Trends Performance in the United States during the 2009 Influenza Virus A (H1N1) Pandemic".

Source: Nature Magazine



Comments     Threshold


This article is over a month old, voting and posting comments is disabled

Why should Google be wrong, and not the others?
By UpSpin on 2/18/2013 1:16:44 PM , Rating: 5
I also had flu this year, but no medical institution knows it, except Google because I did a few searchs on it. I was one week at home, ill. Just because you're ill, doesn't mean you need medicine or visit the doctor. There's no easy 'medicine' against flu, you have to give your body time and rest, and keep an eye on your temperature and the other signals your body gives you and the flu is gone in about one, up to two, weeks.
But many people also tend to, if they start to cough, to take common painkillers which also lower fever, like Ibuprofen, and don't really get that they have flu. They're just dizzy several weeks and think they 'just' have a persistent cold, yet had, or still have, flu.
And then there are people who run to the doctor because of every cough.

So maybe Google's data are the most accurate ones, at least in my case, Google would know that I had flu, a medical institution not.




By ChronoReverse on 2/18/2013 2:17:15 PM , Rating: 2
That's an interesting thought. I've been recovering from a bit of the flu this weekend but I didn't see a doctor for it since I wasn't running a serious fever or anything that could be dangerous. This wouldn't be recorded by the medical community but I'd wager that my comments in social media would have given it away to the aggregators.


By Mitch101 on 2/18/2013 4:33:44 PM , Rating: 2
Same here I knew I was sick but didn't want to be in the doctors office waiting room with other sickos and non-sickos and lose the co-pay to what I already knew. Why subject myself and others to it and possibly get other sicknesses from a weaken immune system.


By chripuck on 2/19/2013 12:52:37 PM , Rating: 1
Not to disagree with your point, but if you get the flu no amount of OTC pain killers will keep you ambulatory. If you can take two Advil and get on with your day then you don't have the flu.


Circular algorithm...
By tayb on 2/18/2013 12:56:28 PM , Rating: 3
The algorithm predicts the flu using search data, which is published by the media, which causes people to perform flu related searches, which is picked up by the algorithm, etc, etc.

Ironic.




By blowfish on 2/18/2013 9:31:13 PM , Rating: 1
A cold, untreated, will last about a week. Treated, it will last about seven days. Why do people in the US put so much faith in the medical profession - when US healthcare has simply perfected the means of sucking as much money as possible from the young, the old or the chronically ill? Instead of wasting money on quack treatments, the US should ensure that mothers to be have a decent diet - since the first 1000 days are key to everything that follows - your resistance to disease and your ultimate longevity.




By ven1ger on 2/19/2013 5:49:27 PM , Rating: 2
Don't believe colds are life-threatening. The flu can be life-threatening for many, especially the elderly and the very young.


just because...
By talikarni on 2/18/2013 8:55:13 PM , Rating: 2
So? I suspect 10%+ would be true since over 60% of flu sufferers do not go to a doctor since they are otherwise healthy and not hypochondriacs. I've never gone to see a doctor or ever reported when I had the flu or cold. All 4 of my immediately family, and all 3 inlaws including wifes grandfather had the flu in the 3rd or 4th week of Jan. Only the grandfather went to the doctor since he is in his upper 80s.




I don't know about you
By johnsmith9875 on 2/19/2013 12:54:24 AM , Rating: 2
But in my state it was at epidemic levels. Some grade schools shut down because 25% of the students had it.




silliness
By chromal on 2/18/2013 12:13:27 PM , Rating: 1
I always thought that, while interesting, zeitgeist-like data is at best 'soft' data, saying more about what people are thinking about than any actual cold hard reality. Further distorting any meaning of the data would be flu-related searches inspired by the new media flu coverage, probably driven (ironically) in part by the same Google predictions.

It's all sillyness. Who cares what people *think* or *believe*, what is actual and real is what matters. I realize that the flu is tricky to track. Because some folks are asymptomatic or don't seek medical attention, it's a tip-of-the-iceberg sort of situation.




"It's okay. The scenarios aren't that clear. But it's good looking. [Steve Jobs] does good design, and [the iPad] is absolutely a good example of that." -- Bill Gates on the Apple iPad

Related Articles
Google Predicts the Flu
November 12, 2008, 10:10 AM













botimage
Copyright 2014 DailyTech LLC. - RSS Feed | Advertise | About Us | Ethics | FAQ | Terms, Conditions & Privacy Information | Kristopher Kubicki