Tesla vs. NYT: Do the Data Really Tell All?

18 Feb 2013

I’ve enjoyed so far the back and forth between Tesla Motors and New York Times reporter John Broder. The short version is

• Broder tested one of Tesla’s new Model S all-electric sedans on a drive from Washington, D.C. to Groton, CT. Part of the reason for this specific trip was to make use of Tesla’s new supercharger stations along the route (one in Delaware and one in Connecticut).
• Broder’s trip appeared to have some bumps, including running out of electricity at one point and requiring a tow.
• After the review was published in the New York Times, Elon Musk, the CEO/Founder of Tesla, was apparently livid. He published a detailed response on the Tesla blog explaining that what Broder wrote in his review was not true and that “he simply did not accurately capture what happened and worked very hard to force our car to stop running”.
• Broder has since responded to Musk’s response with further explanation.

Of course, the most interesting aspect of Musk’s response on the Tesla blog was that he published the data collected by the car during Broder’s test drive. When revelations of this data came about, I thought it was a bit creepy, but Musk makes clear in his post that they require data collection for all reviewers because of a previous bad experience. So, the fact that data were being collected on speed, cabin temperature, battery charge %, and rated range remaining, was presumably known to all, especially Broder. Given that you know Big Brother Musk is watching, it seems odd to deliberately lie in a widely read publication like the Times.

Having read the original article, Musk’s response, and Broder’s rebuttal, one things is clear to me–there’s more than one way to see the data. The challenge here is that Broder had the car, but not the data, so had to rely on his personal recollection and notes. Musk has the data, but wasn’t there, and so has to rely on peering at graphs to interpret what happened on the trip.

One graph in particular was fascinating. Musk shows a periodic-looking segment of the speed graph and concludes

Instead of plugging in the car, he drove in circles for over half a mile in a tiny, 100-space parking lot. When the Model S valiantly refused to die, he eventually plugged it in.

Broder claims

I drove around the Milford service plaza in the dark looking for the Supercharger, which is not prominently marked. I was not trying to drain the battery. (It was already on reserve power.) As soon as I found the Supercharger, I plugged the car in.

Okay, so who’s right? Isn’t the data supposed to settle this?

In a few other cases in this story, the data support both people. In particular, it seems that there was some serious miscommunication between Broder and Tesla’s staff. I’m sure they also have recordings of those telephone calls too but they were not reproduced in Musk’s response.

The bottom line here, in my opinion, is that sometimes the data don’t tell all, especially “big data”. In the end, data are one thing, interpretation is another. Tesla had reams of black-box data from the car and yet some of the data still appear to be open to interpretation. My guess is that the data Tesla collects is not collected specifically to root out liars, and so is maybe not optimized for this purpose. Which leads to another key point about big data–they are often used “off-label”, i.e. not for the purpose they were originally designed.

I read this story with interest because I actually think Tesla is a fascinating company that makes cool products (that sadly, I could never afford). This episode will surely not be the end of Tesla or of the New York Times, but it illustrates to me that simply “having the data” doesn’t necessarily give you what you want.