Lying with statistics

In the wake of Jimi Hes­elden’s trag­ic Seg­way acci­dent Mon­day, media out­lets have seized upon a small study done at George Wash­ing­ton Uni­ver­si­ty Hos­pi­tal and breath­less­ly issued sen­sa­tion­al head­lines. MSNBC says, «Seg­way Scoot­ers Can Lead To Seri­ous Injuries, Experts Warn» ABC News asks, «Are Seg­ways Safe?» Even Web­MD claims, «Seri­ous Injuries From Rid­ing Seg­ways Increas­ing» with­out any indi­ca­tion of whether the increase is in pro­por­tion to the use of Seg­ways. 383,000 peo­ple died in the US in 1900. A cen­tu­ry lat­er, the US had 2.4 mil­lion deaths. You could say that death is «on the rise» except that there were about twen­ty times more peo­ple in the US in 2000 than there were in 1900. The mor­tal­i­ty rate per capi­ta has actu­al­ly been cut in half.

The George Wash­ing­ton Uni­ver­si­ty Hos­pi­tal study com­pared the injuries of peo­ple who arrived at the emer­gency room after a Seg­way acci­dent with the injuries of peo­ple who arrived at the ER after being hit by a car. Just using that as a yard­stick sounds like an attempt at cre­at­ing bias. They found that pro­por­tion­al­ly more peo­ple who came in to the ER after a Seg­way acci­dent end­ed up being admit­ted to the hos­pi­tal than were admit­ted after being hit by a car.

It makes it sound like rid­ing a Seg­way is there­fore much more dan­ger­ous than get­ting hit by a car. Which just does­n’t pass the smell test.

Actu­al­ly it real­ly means that falling off a Seg­way is more dan­ger­ous than get­ting hit by a car. That does­n’t pass the smell test either. Actu­al­ly, none of the 41 patients in the study died. I don’t know how many of the nine pedes­tri­ans killed by auto­mo­bile in DC in 2008 end­ed up at GWUH’s ER, but if even one did fail­ing to men­tion it in a study com­par­ing car acci­dents with Seg­way acci­dents seems misleading.

Con­sid­er the dif­fer­ence in what a rea­son­able per­son would do after get­ting hit by a car or after falling off a big elec­tric scoot­er. Even if knocked down with noth­ing but scrapes and bruis­es it’s like­ly some­one hit by an auto­mo­bile will make a trip to the emer­gency room. A sim­ple fall from a Seg­way won’t trig­ger the same auto­mat­ic response. Typ­i­cal­ly after a short fall that does­n’t involve an auto­mo­bile a per­son will dust them­selves off, check them­selves to make sure that every­thing is OK, then get back up and go about their business.

So of all the Seg­way-relat­ed injuries that hap­pen, only a small number—the very worst ones—will end up in the emer­gency room. Of all the pedes­tri­ans struck by auto­mo­biles, many will end up in the ER even if they aren’t injured.

A bet­ter com­par­i­son would be to com­pare Seg­way acci­dents with skate­board or per­haps bicy­cle acci­dents. Or bet­ter yet, com­pare Seg­way acci­dents sta­tis­ti­cal­ly with the aggre­gate of all acci­den­tal trau­ma cas­es in the emer­gency room.

Trou­ble is, even-hand­ed facts and analy­sis might not sup­port the evil-tech­nol­o­gy-turned-on-its-cre­ator nar­ra­tive that makes such good head­lines. Infor­ma­tion does­n’t cap­ture the atten­tion the way that scare­mon­ger­ing does.

It would be use­ful to learn how safe the Seg­way is, and those who are col­lect­ing data about peo­ple injured while using Seg­ways should be laud­ed. How­ev­er, col­lect­ing and pre­sent­ing this infor­ma­tion should always be a process of uncov­er­ing and inter­pret­ing the truth, not a dis­tor­tion of truth to fit an agen­da or sell news­pa­pers. This sort of thing makes us less safe. Our deci­sions about what to do are based on what we learn to be safe or unsafe. At best we can’t trust the infor­ma­tion we are hand­ed, and at worst our deci­sions will be based on false information.

We should expect bet­ter than this from our media out­lets. More impor­tant­ly, we should expect bet­ter than this from the sci­en­tists and researchers upon whose data and analy­ses we are counting. 

And for the record, it’s not iron­ic that the own­er of the com­pa­ny that man­u­fac­tures the Seg­way died while rid­ing a Seg­way. It would be iron­ic if he died in a skate­board­ing acci­dent, or if he fell over the embank­ment just after step­ping off the Seg­way. Dying while rid­ing his own prod­uct is just tragic.