[intro music]

Host – Dan Keller

Hello, and welcome to Episode Ninety-eight of Multiple Sclerosis Discovery, the podcast of the MS Discovery Forum. I'm Dan Keller.

Today's interview again features Dr. David Baker, Professor of Neuroimmunology at Queen Mary University of London in the U.K. We spoke at the ECTRIMS conference last fall. In part one of our interview he raised the issue of why there has been very poor translation from animal models to clinical trials. Today, Dr. Baker, also known as the "Mouse Doctor" for his work with animal models, lays out why this situation exists and what to do about it.

Interviewee – David Baker

I think there's many reasons why, and I think we all have our failings. And one can point the finger at the animal models, which a lot of the clinicians do, saying it's the animal model's fault, which is possible. But I think also we have to look at humans and how humans use their animal models. And then how humans translate the data from the animal models into the clinic, because I think there's many failings along the line, and I think that's one of the reasons for the failing between the two.

I think one of the failings is, in terms of the animal models, that when we do our animal models for these, we're looking for mechanisms not treatments. And so about 70% of studies give drug before disease is ever induced, which never happens in a human. You know, you go after you've had one or two or more attacks before you're given drugs. We also use the drugs in a way that are never used in a human, so people will do what they call a prophylactic drug where they'll give it before the disease manifests itself. Or a therapeutic dose, which is probably when the animals are showing their symptoms. But in reality, a human would be getting steroids at that time point. They would never get a DMT. So you're not comparing, you know, apples with apples. You're comparing apples with pears, and I think that's one of the problems.

And I think, you know, if you try and block an immune response from being generated, that's quite easy compared to stopping an immune response once it's been generated, because immunity's about giving life-long protection against infections. And so I think it's a different type of beast to target. So I think this is where the animal models could do it, because EAE is one of the few where you have this relapsing-remitting disease course. But it's very, very rare that people actually start to treat in between attacks to block further relapses. I think that's one of the problems.

The other big problem is the dose; the dose relationship between animals and humans. There's a tendency we just keep giving more and more and more and more, and eventually the drugs will work. But you've got this problem that animals are very liable to be stressed, and we call it the building site effect, so construction site effect. And if you have lots of loud noises, it scares animals. They get very stressed, and your EAE just disappears. And likewise, if you just give lots and lots of drug, that probably tastes nasty. They get stressed out as well. And I think many of the so-called wonder cures – cures of the week – are because we're just giving too much, which doesn't have a relationship to what the human dose is going to be.

And then, likewise, I think we've got too much of a publication bias for the need to generate positive data. And I think what we then have to do is we have to look at the quality of the data. And I think there has been a lot of failure to replicate data. I think some of that is because some studies lack quality control, and the way I look at that – and I could be wrong; obviously it's an opinion – but if you look at the way that EAE is scored (it's normally a scoring system 1 to 3 or 1 to 4) and then you have your drug, which may be, you know, takes your control down from 3 down to a 1. But then, every now again, you look at the studies where it goes either way, and your controls are at 1 and it goes up to 3, and I ask the question how do you get a score of 1? Because if you had four animals, they're all scoring 1. Or is it three animals score 0 and one score 4, and that will give you a score of 1. And I think if people were made to actually put the data about how many animals got disease, we'd be able to interpret those line graphs. Because I feel that, in many cases, some of those graphs lack quality control.

If you have a robust quality control system, your control group should be giving you roughly the same type of scores every time. But in individual papers you can see, in some groups you have a score of 1 in the control group. The next experiment it's a score of 3. To my mind I think if you look at that, then those are probably the experiments are much more likely not to replicate. So I think you have to be, obviously, skeptical, but I really would like people to actually probably give us the information about how many animals got disease – what is their mean score – in addition to those line graphs. Because without that, they're impossible to interpret.

So that's, you know, kind of one problem of the animals. And then for the humans, you have the same problems. So they over-interpret the animal data. The people doing the clinical trials are very, very rarely the people who came up with the idea. So if there's a weird side effect that you may know about, you know, that's not translated to the person who's actually doing the study, because they don't talk to the basic scientists. Then they probably underpower the studies. They don't necessarily pick the right outcome measurements. So I think there's many failings in both sides of the equation, and it's not always the animal model. But I think unless we kind of up our game, I think it's going to be very difficult for the people who are working on animal models, because you know, there are treatments that come along for, you know, the immune part of multiple sclerosis.

And if you're thinking about the ethical use of animals, it's much harder to make the ethical argument that you should be using disease models which are very severe for the animals to try and work out fundamental parts of biology. And, therefore, I think we'll find that you know the funding agencies start to say, well, why are we funding this work? So I think we need to have good quality work, because if we don't have good quality work, it allows that clinical view that animal work doesn't really deliver the treatments. And I think they can deliver the treatments, but we just have to use our animal studies wisely to ask questions rather than, you know, blindly saying this will work in multiple sclerosis because it works in EAE. That doesn't make sense to me.

Interviewer – Dan Keller

Do you have any succinct tips for people who are either reviewing papers on animal studies or people who are reading those papers once they're published or even the general public reading a news story?

Dr. Baker

Well my first tip would be probably – and this is okay as an opinion – but, you know, EAE data is nonparametric. It goes 1, 2, 3, 4; it's not a continuous scale, so first tip is don't use, you know, the t-test of parametric data on nonparametric data. And that does make a difference. There is a Nature paper published this year that was analyzed with a t-test. If you analyze it with a Mann-Whitney test, which you should have done, the data becomes nonsignificant. So rather than the take home message is, you know, this is a new wonder drug for multiple sclerosis, their answer should have been you have to go back and reproduce your EAE experiment because it didn't work. So I think that would be the first tip. And then the second tip, I would really like people to say, tell us how many animals get disease and on what level and when, so we can interpret the line graph.

MSDF

This is something that you routinely see in oncology done right. They talk about percent of responders, and among responders, what was the shrinkage of the tumor? They don't average it out among all the people who dilute it out by not responding.

Dr. Baker

Well I think one of the problems as well is we've also got this publication bias. We've got you know this urge to see positive data, and I think that skews the whole system.

MSDF

Has anything changed since you came out with a response to the animal checklist?

Dr. Baker

I think, sadly, no, but we're actually doing the checklist again, so we will be able to see if things have changed. I don't think it has. I think the message hasn't gotten through. But I think – this is, again, another one of those nails in the animal model coffin that, if we don't up our game, we'll be seen to be doing an inferior quality work and eventually we'll get discarded. So I know that some of the grant councils are, as you know, saying this is a condition of your grant. But I think you know it's been slow to change, and I think one of the reasons is actually people who are leaders of the field actually are some of the people who are some of the worst offenders. So we're leading by bad example rather than good example.

MSDF

We don't want to leave the listener with the impression that you're against animal models. I mean, you're known as "Dr. Mouse," so you know I guess you just want to see them done well.

Dr. Baker

Yes, I'm passionate. I mean, I really you know believe animal models have a real positive impact to do. And I've been really lucky in the recent years is that, you know, some of those animal models – and work we've done from animal models – is going through into humans and you know is starting to make the ...