Poor survey methodology, tied to the effectiveness of web sites for law firm marketing

A survey in February 2006 by the Australian companies Mahlab Recruitment and Harris Cost Lawyers gave law department respondents 14 choices to answer the question: “During 2004, what were the five most common ways that you located new law firms?”

The five methods that respondents most frequently selected were closely clustered: (1) “referral from in-house counsel at my company,” (2) “panel firm [which had the most number 1 responses],” (3) “referral from law firms,” (4) “referral from other employees at my company,” and (5) “referral from in-house counsel at other companies.” Word of mouth dominates.

What interests me is that the sixth most frequently selected was “search of law firm websites,” even though it dropped to half the frequency of the top five. That drop-off gives a clue regarding the usefulness of this source of information, but we still can’t evaluate except quite loosely the popularity of this method of finding new firms.

Methodologically, when you ask people to “pick the top five,” there is no way to judge the relative importance an individual gives to the five. It is better to have people pick the five and allocate 100 points among them. That way, it is possible to see that the first choice got 75 points and the fifth got 5 points; the two are barely in the same contemplation of the respondent. If the first gets a 1 and the fifth gets a 5, you cannot tell that degree of difference. Had the survey of the Australian law departments done this, we might indeed find that websites help firms get found, but that the marketing method is very rare, much more common, or something in between.

One Response to Poor survey methodology, tied to the effectiveness of web sites for law firm marketing

We welcome comments

Your email address will not be published. Required fields are marked *