Picking apart an effort to collect metrics on knowledge management while still acknowledging the worth of the effort

“Does your law department have a formal knowledge management system or practice?”
In answer to this poll on the Association of Corporate Counsel website as of the afternoon of July 25th, 7 people had answered “Yes” and 32 had answered “No.” htt Managers of law departments need metrics. That is my mantra and this poll does provide metrics. Having acknowledged the potential value of this poll, at the same time it seems to be riddled with problems.

What does the adjective “formal” add? I suppose it means that actions are described somewhere in some detail, people know about them, and the whole system has some official, departmental imprimatur.

Even if a law department has a “formal policy” to preserve knowledge, how effectively is the policy followed? Many people own dusty exercise equipment.

Does any component of knowledge management (KM) count and count equally? One department may specify in its outside counsel guidelines that law firms who represent the department must provide electronic copies of all documents produced. Would that alone suffice for a “Yes”?

Are enthusiasts of KM more likely to notice the poll and take the time to answer it? My hunch is yet, and survey methodologists have found that to be true.

Might a law department have multiple respondents? No individual from a legal department would know that another has responded. For that matter, a zealot (or a forgetful person) could take the poll more than once.

Finally, were all these methodological bumps smoothed, the ultimate question remains untested: To what degree does the implementation of a formal KM policy make a difference to the effectiveness of the department?