Confirmational Bias

I’ve got a rant. This essay is sure to raise the blood pressure for some people. If you are not in the mood for an attack on horse professionals marketing science to sell their product, please skip this post and go to YouTube to watch videos of kittens.


It’s hard to ignore that some trainers are leaning on science to explain and justify their training principles and methods. However, none of the trainers I am familiar with have a background in science and research. None that I know read the peer-reviewed literature. None of them appear to have the depth of knowledge to gauge the merit of the science they are relaying. They depend on a third party to explain it to them in a way they can slot into their training and teaching.


When a person doesn’t know much about the science they espouse they fail to ask the hard questions. And that’s always obvious when somebody else does. A scientist asks the hard question relating to proof, but a horse trainer asks only how they can fit the idea into their business.


In science, there is a term for this. It’s called “confirmational bias”. It means having a bias towards something because it confirms what you already believe or want to believe.


Confirmational bias is a grave error in science because it leads to mistakes and false trails. Science is the practice of testing a theory. But confirmational bias causes researchers to design studies that prove their theory, not test it.


So it is with several horse trainers and behaviourists. They grab onto a theory because it confirms what they already believe or want to believe. If they have a high profile in the horse world, that theory then becomes an accepted fact among their followers. None of them ask for incontrovertible proof or even ask critical questions. If the theory appears anywhere with the words “studies” or “data” or “science” appearing, the horse guru and their followers accept it as fact.


In a recent example, I read a post by a trainer that described a way they could detect different emotions in their horses through energy transmission. The trainer touted it as a way of reading a horse’s emotions. They claimed there were scientific studies to confirm their experience. I wrote to the trainer to ask for the reference to the scientific studies they were using to support the claim. So far, no references have been forthcoming. But with a little detective work, I was able to find an abstract (basically a one-paragraph summary) of a study presented at an obscure symposium in 2017 that supported the trainer’s post. The study was done with only one horse, with no details on how it was performed, and appears to have never been published in a quality peer-reviewed science journal. The work appears to have disappeared.


I could cite many claims by many trainers, but the story is always the same.


But I want to say that the problem is not that trainers get excited by a scientific explanation of their methods and principles. I think that is great. But in most cases, it is not the science that is important to them. The science is only of interest if it supports their claims. Science is only useful if it confirms their belief and what they are marketing to horse people. They have no time for science that contradicts their ideas.


I have often been the target of criticism and even anger when I have questioned the merit of the science a trainer cites to support their work. I am fortunate enough to have sufficient knowledge of scientific methods and physiology and endocrinology to know the critical questions to ask and evaluate the validity of most scientific studies. These questions must be asked before embracing the claims of any scientific study. But trainers who use science as a crutch to justify their work, hate it. They suffer from the Dunning-Kruger effect where they know so little about science and scientific method that they don’t know how much they don’t know.


They will use science to explain and offer proof of their work. Science is there to confirm to their followers they are speaking the truth. But they have no interest in whether or not the science they quote is correct. In fact, many attempt to shut down any debate about the merits of the science they cite. I know this from firsthand experience with several trainers.


I don’t believe you can have it both ways. It is dishonest to embrace the science that you like, but reject the science that doesn’t confirm your beliefs irrespective of how well the studies were performed. Good science is good science and poor science is poor science. If a trainer is going to be honest with themselves, their students, and their horses they have to accept what the good science is telling them and reject what the poor science is saying irrespective of what they want the science to say. They have to be ready to reject their belief when the evidence is not there to support it. Until there is irrefutable scientific proof their ideas remain a theory, not a fact.


I need to be clear that I am not rejecting behavioural science or biomechanical science or endocrine science one iota. But it is the nature of science that it is always evolving. It must evolve. Our understanding of gravity has come a long way since Isaac Newton published Principia Mathematica in 1687 because the nature of science demands an evolution of ideas.


The only way that science evolves is for every study and every conclusion to be scrutinised, questioned, and re-tested. A scientist accepts that it is the job of other scientists to find the holes in their work rather than slap them on the back for a good job. It is the only way science moves forward. If a trainer quotes a study or data or something with the words “scientifically proved” in it, ask them for verification of the claims, ask them how the trial was performed, ask them how many animals and what species were used, ask them what other supportive studies have been published and where, etc. If your questions don’t get satisfactory answers, treat it with acute scepticism. I’m not saying you should automatically reject any claim of scientific proof coming out the mouth of your favourite guru. However, there should be a burden of proof for an idea to transition from a theory to a fact.


But horse trainers don’t get that. Horse trainers don’t open themselves to critical analysis either of their ideas or their work. It is my experience many horse trainers (but not all) openly practice censorship when it comes to their work. It seems some horse trainers suffer from a terminal case of confirmational bias.

Both students (human and horse) are drilling me for answers to their questions - as it should be.

Both students (human and horse) are drilling me for answers to their questions - as it should be.