Law Department Management Blog
Some surveys and the interpretation of their results deliberately lean a particular way because the surveying organization has a bias and seeks certain results. Admittedly, no survey can be completely neutral, but that should be the goal. Examples on this blog have been plentiful of possible survey bias. Here is a sampling where, in order, bias might have seeped in due to financial gain, ideological goals, or manipulated methodology.
Law firms, consultants and vendors apply some spin so they can sell their services (See my posts of April 7, 2006 on surveys conducted by law firms; and June 18, 2007 on a consultant’s headlines about law departments “firing” law firms; April 3, 2005 regarding research by Blackberry on PDA’s; Feb. 26, 2005 on compensation data from executive search firms; May 4, 2007 on international arbitration costs and PwC data; and May 27, 2007 on some data from the American Arbitration Association.).
Proponents of causes want to find data that strengthens their cause, which might tilt their survey efforts (See my post of May 27, 2007 about an ADR survey; Oct. 18, 2005 on latent desires for telecommuting; Oct. 18, 2006 on diversity efforts; July 18, 2006 on women researchers finding advantages with women managers; and June 7, 2006 regarding the US Chamber of Commerce and its ranking of states’ judicial systems.).
Some surveyors trip up with sloppy methodology, which influences their findings (See my post of Nov. 15, 2005 on questions by Bottom Line Technologies tilted toward favorable ratings of law firms; and April 9, 2005 on very low participation rates in a survey by Serengeti.).
I am not saying that the organizations that produced these survey results deliberately manipulated their methods or data, but I am suggesting that self interest may have warped the objectivity of their efforts.