Comments on the methodology of survey results on morale boosters vs busters

For its biennial survey of career satisfaction, covered by InsideCounsel, May 2008 at 61, InsideCounsel obtained responses from 1,200 in-house counsel. One question asked them to choose from a list of 11 the MOST rewarding part of working in-house. A second question asked them to choose from another list of 11 the LEAST rewarding aspect of their jobs. This post comments only on the methodology of the survey questions. Later posts will turn to the results.

It is an excellent sample size (See my posts of April 22, 2007: power tests and sample size; Dec. 9, 2005: margin of error and sample size; and Oct. 31, 2007: benchmark data and a formula.). For that reason, if data were collected it would be possible to compare responses to years out of law school or years in present position. A more sophisticated analysis might account for multiple responses from the same law department (See my post of Sept. 5, 2007: probability-weighted samples.).

Multiple choice questions often need an “other” category, which this survey did not offer, or possibly did not report on (See my posts of April 12, 2006 and July 14, 2005: importance of “Other” as a list choice.).

It would have been even better to have asked respondents to rank their top 3 or 4 morale boosters and busters, especially if as here one or two choices dominate the selections The top two accounted for 48 percent and 53 percent of the selections on their respective lists. Rankings allow much more analysis, yet don’t require much more effort on the part of respondents (See my posts of Oct. 17, 2005: ratings compared to rankings; June 10, 2007: a better way for surveys to rank; March 10, 2005: setting priorities; April 8, 2007: ranking compensation by practice area; and April 4, 2008: rankings and percentages.).

Finally, analysis of the results would be cleaner if the lists had used the same wording for similar aspects. For example, “perks” as a morale booster corresponds pretty closely to “benefits package” as a morale buster. Even so, I think of benefits as primarily monetary, whereas perks might include non-monetary benefits such as parking spots or size of office. Likewise, the booster choice “management opportunities” (8th on the list at 3.1%) compares roughly to the buster choice “managing people” (6th on the list at 5.2%).