An open question has been whether the time of participation in a benchmark metrics survey correlates to statistically different samples. Put differently, do law departments that submit data early in the collection period differ materially on key benchmarks from departments that submit their data months later, in the final stages of data collection?
To research this methodological issue, I took the law departments from U.S. and Canadian companies that signed up for the General Counsel Metrics benchmark survey this year for the first time. My reasoning was that participants in the 2010 or 2011 survey who returned had responded to different factors than newcomers. There are 191 law departments in the newcomer set, which I divided into thirds by the date they completed the online survey.
The earliest third of the participants – very roughly the Spring participants – are about a half lawyer per billion smaller than the late participants – think of those as the Fall participants. The Spring participants also had much lower total legal spending as a percentage of revenue than the Falls (0.35% compared to 0.55%). In terms of size, the median number of lawyers, three, was dead even for both groups.