How to measure the reputation of bureaucracies

The behavior of administrative agencies largely depends on the bureaucratic reputation that units and departments enjoy with different audiences. However, the current scholarship has not yet produced a coherent measure of bureaucratic reputation that is applicable to multiple agencies across different countries and over time. In his latest paper, published in the American Journal of Political Science, Bocconi Post-Doc and DONDENA Researcher Luca Bellodi addressed this gap by devising a new strategy to compute scores of bureaucratic credibility using state-of-the-art techniques in the field of Natural Language Processing.

The paper employs a technique called word embedding, which computes the probability that two or more groups of words occur in the same linguistical context. In this case, the authors are interested in computing the probability that the name of any agency is brought up close to words symbolizing high or low reputation and efficiency. This exercise was carried on using the names of 465 bureaucratic agencies in the U.S. and in the U.K. mentioned, over 40 years, in the parliamentary speeches made by members of Congress and of the British Parliament.

The results of this strategy then passed additional validation tests, proving to be robust against other text corpora and alternative measures. In particular, the author was able to identify boosts and drops in terms of credibility that happen in the presence of a crisis as it occurred to the British Home Office in the wake of the Windrush Scandal. The paper also finds that the U.S. Environmental Protection Agency has suffered a steady decline in the last 30 years whereas the British Department of Health improved its status in the same period.

The findings suggest that members of the party in government tend to praise bureaucratic agencies more than the opposition. At the same time, this difference decreases when the mentioned agencies are more independent.

In fact, the index of reputation can be seen to vary according to the political affiliation of the speaker. Polarization can be observed in terms of how a member of parliament talks about a given agency conditionally on her party affiliation and it appears to be consistently higher in the U.S. than in the U.K. Parties in power tend to speak better of governmental agencies overall, especially if the agency is under tighter governmental control. This indicates that political affiliation is an important component in the formation of beliefs regarding the work of the bureaucracy. A proof of this is that when agencies are more independent, they correlate with significantly less polarization.

Designing a new measure of reputation applicable to different political and institutional environments can help to answer several research questions and to foster our understanding of bureaucracies in a comparative perspective. Agencies with different degrees of credibility may exhibit divergent patterns of external communication and outreach. Likewise, a positive reputation can allow them to generate greater support, recruit more qualified personnel and protect themselves better against political attacks. Clearly, a trade off exists between nuance and coverage. Measures that are applicable to various environments are bound to lose on the minute peculiarities of every context. However, by focusing on the references that members of parliament make on bureaucratic agencies in their speeches, the authors can leverage on an informed and diverse audience which should proxy the opinions of key stakeholders.