The commentovirus pandemic

An open letter to an imaginary evaluation manager

Stephen Lister

Dear Evaluation Manager

Thank you for asking us about the commenting process for the draft report we have just submitted. We don’t have much time if a revised report is to be circulated before the holiday season. However, we see this as an opportunity to pilot a remedy for the commentovirus pandemic that is debilitating the evaluation world.

The commentovirus infects evaluations at the stage when evaluators submit a draft report for review. The common symptoms of this virus are twofold: in evaluation managers it manifests as commentosis – the production of a morbidly excessive number of comments (albeit most of them very superficial) which triggers commentitis (an inflammation to which evaluators succumb as result of an excess viral load of comments).

The commentovirus generates fatigue and debilitation  – with both physical and mental symptoms – resulting from the workload and tedium of dealing with an inordinate number of comments. The mental debilitation is aggravated by sufferers’ knowledge that they will experience at least three waves of the virus, and sometimes many more. The virus disproportionately affects seniors (team leaders in particular) because they are the ones least able to distance themselves from its ravages.

Although the effects on individuals are unfortunate, the virus’s effects at the systemic level – on the quality of evaluations – are even more serious, and are aggravated by inappropriate treatment. Evaluation managers succumb to commentosis with the best intentions, mistakenly regarding the number of their comments as an indicator of thoroughness and quality when, most often, the reverse is true. Insightful comments result when a reviewer reads a whole report and reflects before making any comments. The contrary habit – starting to comment as you read the first page and not stopping until there is graffiti on every section – may feel thorough,  but  it prevents the reviewer from distinguishing the wood from the trees, and the number of comments is in inverse proportion to their value added.

A changing e-climate has made it much easier for comments to multiply, with evaluators routinely receiving literally hundreds of comments. Unfortunately, the number of comments is rarely correlated in any obvious way with the quality of the report submitted. Disabling Word’s track-changes and comment bubble features would stop the virus in its tracks; these features in Word have completely changed the terms of trade, enabling evaluation managers almost effortlessly to generate huge amounts of work for evaluators.

Strangely, evaluation managers are well aware that “more means worse” – at least when it comes to the rules they impose on evaluators. In particular, it is standard practice to restrict evaluators to ten recommendations, so that they will focus on what is important. We believe that is an excellent rule: it forces evaluation teams to think properly about – and discuss amongst themselves –what is most important.

This points to a simple way for all of us avoid our holiday season being blighted (or even abolished) by this virus, and – more importantly – to raise the quality of the evaluation as we do so (no trade-off here!).

Here is the proposed therapy: (a) do not submit any comments in track-change form; (b) meet with your reference group and agree on not more than ten practical ways in which you think we can improve our draft report during the short period you have allowed us between receiving your comments and handing in a revised draft.  This will help both you and us to focus on what’s important without getting bogged down in detail.

We predict that this will be a healthier and less stressful route to a report of the highest possible quality.  Let us at least evaluate this approach. There is huge demand for a therapy  that can alleviate the effects of an ubiquitous virus for which, unfortunately, vaccination seems an unlikely prospect.

Thank you and  stay well

SEL

Download this article.