I thought about an encore to my recent thoughts on key benchmarks for legal department managers, those that are known and some that are not (See my post of July 9, 2009: ten most fundamental benchmarks; and July 10, 2009: ten benchmarks general counsel may wish they could obtain.). I decided to herd together the many benchmarks that have not been collected previously. Thus I excluded posts on practice group benchmarks. The demarcation lines are not watertight, however, so I grouped the assemblage under my categories.

Cost: (See my post of March 5, 2009: percentage of legal spend paid governments for patent costs; May 24, 2005: legal spending as a percentage of profit margin; May 23, 2007: profit per lawyer; April 24, 2009: other denominators for benchmarks; Feb. 25, 2009: occupancy expenses per lawyer; March 28, 2006: inside spend per lawyer; July 21, 2008: inside spend per lawyer over time; Aug. 4, 2008: decline in ratio of internal spend as department size grows; Sept. 7, 2008: total legal costs expressed as cents per share; May 26, 2007: market capitalization as benchmark denominator; July 2, 2007: metrics that use market capitalization; Feb. 6 2009: payments made to inventors; Aug. 14, 2005: spending of $4,000 per lawyer on technology; May 31, 2005: legal spending per resident; Feb. 4, 2008: cost for corporate secretaries per entity maintained; Jan. 20, 2009: legal resolution costs – settlements and judgments; and Jan. 19, 2008: unknown metrics about non-publicly traded companies.).

Outside Counsel: (See my post of April 10, 2006: total law firms retained; Dec. 21, 2008: outside counsel spend as a percentage of revenue; May 21, 2008: percentage of in-house attorneys who manage outside counsel; May 4, 2009: outside counsel spend per lawyer, about $600,000; and July 16, 2005: law firms paid more than $100,000 per billion of revenue.).

Productivity: (See my post of May 22, 2009: total lawyer hours worked per billion of revenue; Feb. 12, 2008: percentage of work done inside; July 4, 2009: European trend upward of work done inside; Dec. 11, 2006: the proportion of work that they send out; March 19, 2006: hours inside vs. outside in Canada; June 10, 2007: odd aspects of numbers of matters sent outside; July 11, 2008: autarky and doing work inside; Jan. 25, 2006: number of major lawsuits pending; and May 28, 2005: cycle time for cases in Federal District Courts.). Structure: (See my post of April 18, 2009: lawyers as percentage of total legal staff and more with one to one; March 26, 2006: EMC ratio of one-to-one; Nov. 28, 2007: trend toward more lawyers per staff; March 2, 2009: specialists headquarters and commercial lawyers in international regions; Sept. 27, 2005: median of 3.0 paralegals in US departments; Feb. 7, 2008: department size and likelihood of having an administrator; Feb. 16, 2009: FBI has 30 lawyers per billion of budget; March 11, 2009: US Postal Service; Aug. 27, 2005: one IT support person for every 24 people in the legal department; Dec. 23, 2005: less than one IT support for every 35 people; Dec. 23, 2005: prosecutors’ offices; and June 15, 2008: percentage of lawyers outside the home country in relation to percentage of revenue generated internationally.).

Talent: (See my post of Feb. 25, 2009: attrition rates for inside lawyers; July 27, 2008: offers extended and accepted; and Nov. 28, 2007: years of in-house lawyer experience.).

At one point this Spring I set off on an ambitious series of posts about what I call “descriptive metrics.” I persuaded myself that I had hit upon a higher-level way to quantify and depict legal department performance and characteristics. Eight posts eventually saw the light of day, but I think the series will now go dark (See my post of Feb. 19, 2009: supervisory responsibility; Feb. 26, 2009: start of a series on descriptive metrics; March 8, 2009: in-house lawyers; March 9, 2009: fee concentration with firm size and effective rate; March 11, 2009: management initiatives; March 26, 2009: degree of client reliance on the legal department; May 19, 2009: workload; and May 28, 2009: h-index of law firm use.).

Any time a legal department counts something that metric becomes a benchmark candidate if enough departments shared their respective counts. Hence, “benchmarks” are “descriptive metrics.” To describe a law department through metrics, singly or in combinations such as indices, is no new kind of beast, so I will retire the concept and end the series.

Having presented the ten most fundamental benchmarks for legal department managers, all of them available from various sources, I mention here ten benchmarks that many general counsel may wish were more available (See my post of July 9, 2009: ten crucial metrics.).

1. Effective hourly rates of law firms that account for 75 percent or more of fees paid
2. Law firms paid more than $5,000 in a year per billion dollars of revenue
3. Settlements, fines and judgments paid as a percentage of revenue
4. Lawyers, who practice some law but are not part of the legal department, even by decentralized reporting – as a percentage of legal department lawyers
5. Paralegals by practice area
6. International legal spend as a percentage of total legal spend
7. Percentage of lawyers other than in largest location
8. Average years of lawyers with the law department as that number relates to lawyers per billion
9. Outside counsel managed per lawyer in the department who manages outside counsel
10. Chargeable hours actually worked

For US law departments that have more than five lawyers, here are the fundamental ten metrics:

1. They spend approximately 0.5% of their corporation’s revenue each year on their inside plus their external spend.

2. That benchmark for “total legal spending” does not include settlements, judgments and fines, which vary widely but are typically considerably less.

3. A legal department’s internal expenditures are typically about 40 percent of total legal spending.

4. On the order of 75 percent of the internal departmental budget is salaries, bonuses, and benefits overhead.

5. The next largest category of internal spend is (or should be) for facilities, at 5 percent or so.

6. About 60% of the total legal spend typically goes to law firms and other external service providers (approximately 90% to firms).

7. Of the external spend roughly a half is litigation-related.

8. Commonly, US legal departments have between three and seven lawyers per billion dollars of revenue, depending on the industry.

9. The fully-loaded cost of an internal lawyer ranges from $160 to $250 per hour.

10. To have one lawyer for every non-lawyer in a department is a normal ratio.

I wrote about the article by Bill Turner of Womble Carlyle on Monte Carlo simulations, and emailed him with some questions (See my post of June 26, 2009: Monte Carlo and sensitivity analysis.). One was about law departments that have used the firm’s Monte Carlo capabilities. While unwilling to disclose the name of a specific law department that has used the software, “we have used this tool to develop value-driven alternative engagements for some clients.”

Turner also wrote clearly about multivariate regression. “In multivariate regression, you're often testing to see how much of a variable, X, contributed to the forecast Y, and whether X is significant in the model, overall. The Monte Carlo tools perform sensitivity analysis by creating rank correlation coefficients between the assumptions and the model forecasts while the simulation is running. These coefficients indicate the strength with which the assumptions and the forecasts change together. The coefficients are squared and normalized to 100%.

One of the key differences between using these techniques vs. multivariate regression is that multivariate regression analysis is usually run against actual data (where variables are then tested for significance, multicolliniarity, heteroskedasticity, auto correlation, etc.) whereas Monte Carlo analysis creates the data (and then runs the sensitivity) based on defined parameters. In principle, the concepts are similar, though the sensitivity analysis demonstrated in the paper involves how much the model assumptions contributed to the variation in the forecast rather than how much the assumptions contributed to the forecast itself. The goal is to measure risk.” (See my post of Dec. 31, 2008: regression statistics with 6 references.).

Thank you very much, Bill!

A clear explanation of the statistical model, known as the Monte Carlo simulation appears in Womble Carlyle’s FocusExtra, 2Q2009 at 1, by Bill Turner. The newsletter explains the technique in the context of estimating the cost of a lawsuit through trial. Essentially, for four stages of a litigation, the law firm and the law department -- or just the law department -- need to estimate the cost for each stage, as well as a low-cost estimate and a high-cost estimate. Software can then run multiple iterations where it uses data from that table to prepare a bell shaped curve of likely outcomes (See my post of May 15, 2005: Monte Carlo simulations as computational models.).

Because the outcome curve meets the requirements of a normal, Gaussian distribution, the law department can calculate confidence intervals for any given total cost. That means you can say, for example, with 80 percent confidence that the total cost will be $1.2 million.

A law department can also use the data and software for a sensitivity analysis. A sensitivity analysis tells which factors within the model create how much of the variation, the factors here being the four stages of litigation. This analysis is similar to what a multiple regression analysis can calculate.

The piece suggests that a law department should do this calculation on its own and then compare a proposed fee by the law firm to the likelihood of that fee being the total cost according to the Monte Carlo simulation.

Queuing theory and its models often assume that the rates of arrival of work and delivery of service can be described by a Poisson distribution (See my post of Jan. 20, 2006: one of many kinds of distributions of numbers; and Aug. 16, 2006: predicts likelihood of event during a given time period.).

Let me describe what a Poisson distribution looks like graphically. Visualize a column chart that shows on the horizontal, bottom axis the number of client requests for legal services that arrive in a legal department each week. The number of requests per week increase as you move to the right.

The vertical axis shows the relative frequency of each of those weeks, expressed as percentages increasing from quite low to perhaps 20 percent. Thus, the tallest column in the middle might be 15 requests a week, which happens during 18 percent of all weeks; the lowest column on the left corresponds to one request during a week, which happens five percent of the time; the lowest column on the right represents 25 requests for legal assistance during a week, which happens one percent of the time. The overall shape of the columns is somewhat like a bell curve, but with a longer tail to the right.

Equivalently, according to William J. Stevenson, Operations Management (McGraw-Hill, 2005, 8th Ed.) at 782, the time between arrival of a request and completion of the service time can be described by a negative exponential distribution. Imagine a line on a chart sloping from high on the left (a high relative frequency percent of numbers or service requests during a week) down to the right where there are unusual demand levels at low percentages of occurrences.

In Stevenson’s example, many times service is provided quickly, but some services take a long time. “That is, if service time is exponential, then the service rate is Poisson. Similarly, if the customer arrival rate is Poisson, then the interarrival time (i.e., the time between arrivals) is exponential.”

How well we understand and can model turnaround time in a law department makes a significant difference. These two statistical tools provide some insight.

The challenge when a legal group commissions a custom benchmarking project is not creating the questions, finding comparable law departments, confirming the accuracy of the metrics, or analyzing the results. Rather, it is the weeks and weeks needed to persuade other companies to submit data and the passage of time while you await their decisions. The delays of benchmarking are like the delays of selling your house: you can’t push buyers to make an offer and you can’t strong-arm general counsel to agree to take part. So you wait and wait.

Eventually you have to decide that the data is sufficient and to close down the drawn-out effort to get one more company into the data set.

I have done well more than a score of benchmark projects for legal departments and am continually frustrated by how hard it is to pull the teeth of metrics. I can be confident on deliverables, effort, cost (even to the point of guaranteeing a fixed cost) and value, but I can’t control the waiting game.

A physicist, Jorge Hirsch, devised a formula to determine the quality of scientific papers published by a scientist. “The h-index is the number n of a researcher’s papers that have been cited by other papers at least n times. High numbers = important science = important scientist.” According to Wired, June 2009 at 94, “similar statistical approaches have become standard practice in Internet search algorithms and on social networking sites.

Legal departments could translate this idea into the number of matters for which a law firm was hired. Let me explain. Law firm One has been retained during the past three years 100 times. Law firm Two has been retained for 50 matters, while law firm Three has handled 25 matters. Down the list, law firm Twelve has been retained 12 times. Thus the number 12 (“n”) law firm has been retained by your department at least 12 (“n”) times. Your “h[law firm]-index” is 12. Every legal department can figure out its own h[law firm]-index and, if that information is shared, compare their measure of concentration of outsourced work. The lower the number, the more concentrated the work.

The index is a descriptive metric for concentration of work given to your firms. An employment firm may top the list if it handles many smallish matters. Or a patent boutique may if it handles many patents. But eventually, down the list a ways, some firm will hit the h[law firm]-index point.

From the standpoint of a typical general counsel, the benchmark surveys you can buy leave several scars and irritations.

1. Sometimes subsidiaries submit responses, which skews the results.
2. Many questions on the survey may not matter to you.
3. Generic questions applicable to all don’t fit the nuances of your industry
4. They take forever to complete, irritating staff and administrators.
5. The surveyors badger you to take part and then to check or complete answers.
6. The surveyors shoehorn participants into industries, sometimes very broad ones.
7. They cost an arm and a leg for what you get.
8. The time between submission and receipt of the report is months and months.
9. Once a year they come out, even though you want data sooner than that.
10. The reports generated may only be tables.
11. The reports don’t suggest what you might do if your metrics are out of kilter.
12. The survey may be bundled with data you don’t care about, such as compensation or performance measurements.

A solution? A targeted benchmark effort where you select the companies and the data. Modestly, I urge you to visit my website and consider a fixed-price offering.