Good reporting of survey results tells how many respondents answered individual questions

Puzzling over the data from a survey where law departments rated the performance of their law firms, it occurred to me to make a point of methodology (See my post of July 21, 2008: survey methodology and 48 references cited.). The article, from Inside Counsel, July 2008 at 47, explains that 548 in-house counsel responded to the survey. What it does not say he is how many of those respondents answered specific questions.

I have no reason to doubt that the question that shows that in-house counsel have given fewer A grades each year since 2006 elicited responses from significantly fewer of those 548 respondents, but it is possible. For that reason, the best survey methodology for reporting data is to indicate the so-called “N” – number of answers to the question – for each question.