How can matter management systems average an ROI of 36% of outside counsel spend?

“Average reported savings from using matter management systems were 36.8% of outside legal spending.” Incredible, and not to be believed.

The claim comes from the 2008 ACC/Serengeti Managing Outside Counsel Survey, which obtained survey responses from hundreds of ACC-member law departments. I have twice before challenged similar claims drawn from this survey, and won’t repeat my criticisms here (See my post of Aug. 5, 2005: average savings on matter management systems from survey of more than 250 law departments in 2002 was 16%; and April 13, 2007: from same survey in 2006, “average reported savings from using matter management systems was 11% of outside legal spending.”). Little of this do I believe can be substantiated.

I will add three points, the first being that the purported average savings last year doubled the savings of the two previous reports. Doubled! Rob Thomas, who runs the survey for Serengeti and is thoughtful about these operational points, advised me that the average over the past several years has been closer to 10 percent, so last year’s anomalous bulge may be some artifact of the process. Perhaps medians would be more realistic than averages.

Second, the sentence, from ACC Docket, Vol. 28, Sept. 2009 at 12, concludes “electronic billing systems average savings of 12.8% of outside legal spending.” If a law department has both are we to understand that the savings from matter management and from e-billing are additive? No, suggests Thomas and I agree.

My third and most basic observation complains that it is impossible for a general counsel to isolate the effects of software from all the other efforts presumably made to rein in outside counsel spend. At the most charitable, the estimates of savings collected in the survey are wishful thinking by whoever answered the survey; at worst, they are misleading and might become a much-touted urban legend.