Selection bias in surveys of law departments

A methodological spoke in the wheel of many surveys is selection bias. If survey data comes mostly from what surfaces on its own, because someone chooses to respond, the data may not be representative of the universe of data. It may suffer from what statisticians refer to as “selection bias” (See my posts of Dec. 1, 2006; Aug. 27, 2005; and Dec. 3, 2006: general discussions.).

Several posts have raised a red flag about the possibility that data commented on was skewed by this bias (See my posts of May 14, 2005: an example regarding knowledge management; Aug. 30, 2006: those who chose to reply to an internet survey; Oct. 16, 2006: diversity statistics; May 27, 2007: former users of arbitration service; and July 2, 2007: discounts granted by law firms.). Since no surveyor of law departments can force their members to respond, all data in this industry suffers from some degree of selection bias.

Selection bias is not the same phenomenon as either adverse selection or survivor bias (See my posts of Feb. 9, 2008: adverse selection; and March 20, 2007: expert witness testimony.).