The Institute for the Advancement of the American Legal System (IAALS), part of the University of Denver, recently published a survey report. www.du.edu/legalinstitute/form-chieflegal-success.html The academics who conducted the survey of legal departments excel in their background explanation of how they accumulated and organized the data in the report. All surveyors should emulate such methodological clarity and completeness.
Source of Population Sent the Survey: The IAALS obtained the mailing list of the Association of Corporate Counsel (ACC) and removed duplicates or multiple names for the same company. Once the list was ready for survey administration, it contained 5,257 individual email addresses. All surveys should clearly describe the group to whom they sent surveys.
Effort. The IAALS, teaming with survey specialists, sent the group four e-mails during November 2009 through January 2010. All four e-mails contained instructions for requesting a hard-copy version of the survey. All surveyors should explain what they did to obtain their data.
Quality Assurance. The specialists eliminated respondents who did not provide an answer to any substantive questions and studied descriptive statistics to detect and eliminate clear errors (such as answers outside the permissible ranges). After the data verification process, a total of 485 valid responses to the survey were left. Of that total, 112 respondents (almost one-quarter) indicated that their company had “less than 5 state or federal civil cases in the last five years so these respondents did not answer any of the substantive questions concerning the legal system and process.” That left 367 respondents (over three-quarters), which is a clear statement of the survey pool.
Demographics of the Survey Group. 20% of the respondents (.2*367) had revenue of less than $100MM; another 38% had revenue between $100MM and $1 billion. In terms of employees, 11 percent a hundred or less! 19 percent a 100 to 500 and 13% 500 to 1,000. Thus, almost half of the respondents (.43 of 367) had 1,000 or fewer employees. The report says that “The vast majority (92%) of respondent companies are headquartered in the United States. However, respondent companies are also located in the following regions: Canada (1%); Caribbean (2%); Western Europe (3%); Central Europe (<1%); South and Central Asia (1%); East Asia (1%); Australia and the Pacific (1%).” Demographics of a survey study are crucial.