Smoke and mirrors: how referencing hides shortcomings and may lead to a mistrust in evidence use

It is interesting to note how policy documents, be they English white papers, Norwegian green papers or messages to Parliament, or French national inspectorate reviews often tend to hide opinion behind a veil of "evidence" in the form of references. These references are in turn often poorly worded, such as a 2016 report by the French general inspectorate of national education who simply wrote: "based on data from the ministry" or "source: web-page of MENESR" without any further direction as to where on the site said information could be found (if indeed it still could be). At other times, we may observe (should we be lucky enough to know, and be able to identify, the reference) that content is taken out of context and repackaged to justify opinion.

This arguably has a dual purpose of granting control over information and the information narrative, and legitimizing points of views through obscuring what is really going on. It enables an allure of knowledge and justification rooted in evidence of what ought to be done because it is what works. In effect, it creates a form of smoke and mirrors that not only obscures opinion as fact, but also discredit other opinions as not fact based, or not based on the "right" facts. It also becomes a form of bombardment of information, where it is hard for non-experts and experts alike to discern what is good use of evidence and what is not.

The dangers of such approaches to the use of evidence and references in policy documents and policy discourse are the de-legitimization of all truth. In other words, once evidence becomes whatever fits ones opinion at a given time, we enter a post-truth society. Where considered debate over what ought to be done, rooted in philosophy and supported by considered use of empirical observations and their theorizations, is replaced by shouting of evidence that dictate what should be done only to be faced by other sources of reported evidence that dictates a different approach. At that point no one is in reality standing up against nonsense, as the entire debate has entered ridicule and may be considered nonsense itself. Instead, evidence should be used with vigour and rigour, and referencing to evidence should be done correctly, with the important components (including caveats!) of the research presented in the text, as opposed to hidden behind a veil.

To conclude, policy and political discourse may benefit from being less focused on invoking evidence, and more focused on reducing the use of smoke and mirrors. To focus on the correct use of evidence (see my blog post on what evidence can and cannot do), when evidence cannot be of use without philosophy denoting purpose and moral choices, and to focus on the risks and limits of past observations that are invoked as gospel. That way, once evidence is used and used well, it is not attacked as "fake" nor as something to be discredited simply because it does not fit ones opinion, but rather it may become a foundation for future political debate as to what society is desired and how we may achieve such a society. Doing that would arguably enhance the democratic legitimacy of any policy, while avoiding any further erosion of scientific or expert legitimacy.