Information equals how much you learn from unexpected events

Claude Shannon, the father of information theory, calculated “that the information content of any event was proportional to the logarithm of its inverse probability of occurrence.” On the off chance that quote leaves you with any doubt, the Economist, April 24, 2010 at 82, says that it means “an unexpected, infrequent event contains much more information than a more regular happening.” If a class action has a one in one thousand chance of being certified, the logarithm of that chance’s inverse (1,000 is the inverse of 1/1,000) is 3. A one in five hundred case has a log of 2.69 so the rarer event has 11 percent more informational value, as that term was defined by Shannon.

We notice deviations from patterns because the oddity contains more “information.” We can learn more from the unusual than from a continued series of normals. Three examples suggest the range of this axiomatic proposition.

Power-law formulas tell us much because they push us to consider unlikely outcomes and enable us to frame their likelihood (See my post of July 25, 2005: bell curve compared to power-law functions; Nov.13, 2005: power laws and the ratios of litigation costs; May. 27, 2007: Zipf’s law; Feb. 24, 2009: explanation of power-law distributions; Feb. 24, 2009: power laws in law department management; and Sept. 27, 2009: my article on the esoterica of power laws.).

Exception reporting brings to the attention of managers the unusual event, the informative event, the event that stands out and might require special attention. When we pay attention to something that breaks the mold, that spikes, we learn more than we do from the undulations of the commonplace (See my post of May 17, 2006: part of third-wave technology for matter management systems.).

Benchmarks and outliers. While means and medians are the common currency of benchmark studies, the extremes of any metric tell us the boundaries. The large law department that has very few administrative assistants, for example, may be more “informative” than all those with the usual support configuration because it pushes us to wonder how they accomplish the work.

All these are modest manifestations of information theory as it applies to legal departments (See my post of May 23, 2008 #4: information theory; March 6, 2009 #1: diminishment of information with each transmission; May 23, 2008: every relay doubles the noise and cuts the message in half; and July 17, 2009: applications of information theory.).

We welcome comments

Your email address will not be published. Required fields are marked *