Doing Statistical Research

25 Jun 2013

There’s a wonderful article over at the STATtr@k web site by Terry Speed on How to Do Statistical Research. There is a lot of good advice there, but the column is most notable because it’s pretty much the exact opposite of the advice that I got when I first started out.

To quote the article:

The ideal research problem in statistics is “do-able,” interesting, and one for which there is not much competition. My strategy for getting there can be summed up as follows:

• Consulting: Do a very large amount
• Collaborating: Do quite a bit
• Research: Do some

For the most part, I was told to flip the research and consulting bits. That is, you want to spend most of your time doing “research” and very little of your time doing “consulting”. Why? Because ostensibly, the consulting work doesn’t involve new problems, only solving old problems with existing techniques. The research work by definition involves addressing new problems.

But,

A strategy I discourage is “develop theory/model/method, seek application.” Developing theory, a model, or a method suggests you have done some context-free research; already a bad start. The existence of proof (Is there a problem?) hasn’t been given. If you then seek an application, you don’t ask, “What is a reasonable way to answer this question, given this data, in this context?” Instead, you ask, “Can I answer the question with this data; in this context; with my theory, model, or method?” Who then considers whether a different (perhaps simpler) answer would have been better?

The truth is, most problems can be solved with an existing method. They may not be 100% solvable with existing tools, but usually 90% is good enough and it’s not worth developing a new statistical method to cover the remaining 10%. What you really want to be doing is working on the problem that is 0% solvable with existing methods. Then there’s a pretty big payback if you develop a new method to address it and it’s more likely that your approach will be adopted by others simply because there’s no alternative. But in order to find these 0% problems, you have to see a lot of problems, and that’s where the consulting and collaboration comes in. Exposure to lots of problems lets you see the universe of possibilities and gives you a sense of where scientists really need help and where they’re more or less doing okay.

Even if you agree with Terry’s advice, implementing it may not be so straightforward. It may be easier/harder to do consulting and collaboration depending on where you work. Also, finding good collaborators can be tricky and may involve some trial and error.

But it’s useful to keep this advice in mind, especially when looking for a job. The places you want to be on the lookout for are places that give you the most exposure to interesting scientific problems, the 0% problems. These places will give you the best opportunities for collaboration and for having a real impact on science.