This in the end is the worst problem with the proposition that what we need to do is write more letters, make more metrics, indulge in a fetishistic mimicry of “science” in the evaluation of our peers. Not only would such a change dramatically amplify the uncompensated chore of writing assessments and evaluations (rules are like rabbits: require ten letters and I guarantee you in a decade, the same logic will grow the requirement to twelve, fifteen, twenty, twenty-five), it goes in entirely the wrong direction. Outside letters are already a form of outsourcing: they relieve faculty of the burden of having to become literate enough in the work of their colleagues that they can judge for themselves its value and integrity, and therefore simultaneously relieve us all of the obligation to communicate and disseminate our work in a manner that anticipates and welcomes those collegial readings.
If Chance wants to know whether to trust those adjectives, to believe in the professionalism of his profession, to know whether he’s the only person who invests time and effort in such letters or merely one in a legion, the first step is not to build a graven idol out of statistics, to believe that a bigger N is a magic path to truth and trust. The first step is a more humane gesture: to learn to read for oneself, as much as one can, the work of any colleague you’ve been asked to evaluate, and to build a culture that expects academic work to provide signposts for such readings. That’s work too, but it’s a more gratifying kind that doesn’t require putting our human sensibilities and intellectual abilities in a blind trust, yoked to a process that must be made as remote as possible from the ways we actually work and know one another.