Those legal departments whose attorneys grade law firms most commonly do the exercise every six months or once a year. As an alternative choice, departments can follow a policy of evaluations when major matters end.
My objection to semi-annual or annual reviews comes down to mélange. Everything that happens during the period goes into the stew pot. Good, bad and indifferent swirl together in a dimly remembered mix. Whereas, at the end of a matter, the evaluation focuses on one team, doing one kind of legal service.
Unfortunately, that neatness comes at a cost – long time delays, sometimes. You can’t very well evaluate a litigation firm after four years of work on a case (See my post of Nov. 16, 2005: evaluations of law firms with 9 references.). You may not even know that a matter has concluded. For those reasons, the best solution is to evaluate matter by matter when enough has been spent on a firm to create useful insights, or on a schedule, whichever comes first.