Ask the Experts: Alzheimer’s fabrication controversy

By PaulNews


BioTechniques News
Beatrice Bowlby

A couple months ago we learned that an Alzheimer’s research paper from 2006, detailing the toxicity of an amyloid fragment, may have contained fabricated images (read more about it here).   

We asked Becky Carlyle (University of Oxford, UK), Mark Dallas (University of Reading, UK) and John Hardy (University College London, UK) to provide insight into what this allegation means for Alzheimer’s disease (AD) research, drug development, scientific publication and the public.

Questions

Where did you stand on the amyloid-beta (Aβ) hypothesis of Alzheimer’s before this allegation was publicized?

Becky Carlyle (BC): As a relatively late entrant to the AD research field, really immersing myself in it when I joined Massachusetts General Hospital (MA, USA) in 2017, I’ve tried to take a nuanced view of the importance of the Aβ hypothesis. There was strong initial evidence for this hypothesis from cell biology and genetics, and it makes common sense that having large deposits of insoluble protein hanging around in your cortex is probably not great for cortical function.

However, the presence of a very significant number of resilient individuals in the wider population (people with a high load of amyloid plaques but little to no cognitive impairment) certainly suggests that amyloid is not the only factor contributing to cognitive decline in the aging population. The reality is likely to be a subtle mix of genetic susceptibility, lifestyle risk factors and environmental exposures, accumulated across a long lifespan.

Mark Dallas (MD): Prior to these revelations, our research has looked at the toxicity of the Aβ peptide in driving neurodegeneration evident in AD. I do not believe it is the sole contributing factor to the disease, but it has a role to play. Here, more research is key to understanding its role and what impact targeting it can have on those living with AD. Our research has moved to examine common pathways across brain diseases and those that occur prior to amyloid accumulation. These might prove more fruitful in our efforts to combat AD and understand the changes that take place in our brains prior to diagnosis.

John Hardy (JH): When I started working on AD, there were very few people in the field, and there were a number of different ideas. I was a neurochemist, and I was looking to see exactly which nerve cells were lost for transmitter replacement therapies. Inspired by Gusella’s 1983 paper, which investigated where and how Huntington’s disease starts, I realized we should try and understand how AD starts.

When I came to London to work at St. Mary’s (UK), with the clinician Martin Rossor, we started to collect families with AD. In one of those families, we found the amyloid gene mutations. I wrote two very simple reviews. One with David Allsop, reporting that amyloid was how the disease started, and one with Gerald Higgins, which detailed the amyloid cascade hypothesis.

The amyloid theory is based on the observation of the pathology in the brain, but it’s mainly based on the genetics. Lesné’s paper suggests a particular form, which is 12 molecules of amyloid stuck together, is the toxic species that leads to disease. I think the hypothesis has a lot of truth to it, and I think that the evidence says that amyloid is where the disease starts.

How did these revelations change your perception of the hypothesis?

BC: While not wishing to minimize the severity of the likely fabrication of data in this particular case, which is appalling, I believe the reporting of these revelations has also been extremely irresponsible. In all honesty, as someone who joined the field 5 years ago, I had not heard of the Aβ*56 peptide. Since these revelations, I’ve read statements from a number of people who were in the field around the release of this data who had tried and failed to replicate this work and then moved on. These failures to replicate were not published, which is a large problem that all researchers face; it’s difficult to publish negative results and isn’t hugely beneficial to your career progression. I don’t believe, therefore, that the existence of this particular peptide really swayed the amyloid hypothesis one way or another and certainly this potential data fabrication does not change the strong original evidence for the amyloid hypothesis.

MD: These allegations do not change my perceptions of the hypothesis, merely that an author had misled others through their deceit.

JH: I have to say when that paper came out, I was not convinced by it. I don’t think I’ve ever cited it. I don’t think it’s particularly important. I don’t think it says anything about the amyloid hypothesis. It didn’t really fit with what my view of the disease was, and it’s strictly irrelevant to my research.

What does this allegation mean for amyloid-based Alzheimer’s research conducted between 2006 and now?

BC: In reference to my answer to the previous question, I believe the field would have looked mostly the same. The vast majority of funding awarded that relied on the amyloid hypothesis was not awarded as a result of this work, and the hypothesis that removal or reduction of amyloid from the brain may help disease progression was worthy of pursuit. The failure of Aduhelm and secretase inhibitors – the enzymes that produce Aβ peptides and many other peptides, too – to contribute meaningfully to protection against cognitive decline was of far greater impact to this field than the fabrication of these papers.

MD: I don’t think these allegations have a massive impact on the amyloid field. It certainly has not tainted other research efforts which have led to clinical trials to modulate Aβ production. The studies in question focused on the Aβ*56 species, suggesting it was pivotal in the cognitive decline observed in animal models. Other researchers have not been able to replicate these finding or detect the specific form of Aβ in question.

JH: The paper containing potentially fabricated images has been cited 3,000 times. So, although it hasn’t influenced my work at all, it’s influenced other people. I’m sure that there have been multiple grant applications and multiple labs trying to replicate it. I’m sure it had influence. And if it was fraudulent, then that influence has been toxic; people have wasted time and money chasing it down.

What is the impact on funding and drug development moving forward?

BC: My hope is that these revelations, and the failure of Aduhelm, may cause funding agencies and pharmaceutical companies to lessen their mono-focus on amyloid as the sole cause of AD. In my experience, this is already happening and is strongly reflected in the Requests for Applications currently being issued by the National Institutes of Health in the USA and a noticeable shift in the pharma landscape towards immunomodulation and endosomal/lysosomal biology.

I had some really interesting discussions at a recent conference with staff from Alzheimer’s Research UK, where they mentioned that with other complex disorders associated with aging that we understand much better, such as cardiovascular disease, renal disease and cancer, there is no single causative pathway, no single diagnostic test and no single miracle treatment. Yet for some reason, in brain disorders (I found the same in molecular psychiatry), we have remained relatively mono-focused.

Is the cause amyloid, or it is tau? Why not both, with a healthy dose of inflammatory dysregulation and metabolic changes alongside? I believe this is where the field must progress, and there’s tons of fantastic academic and commercial work already heading in this direction.

MD: This will have a minimal impact going forward. Drug development programs are well established and providing valuable information to the research community. Even those that do not make clinical trials provide pieces of the jigsaw that we must complete to unravel the full complexities of AD.

JH: This allegation is bad for the field. One of the things that worries me is that heads of pharmaceutical companies might be reluctant to invest in AD and amyloid treatments if they read about this potential fabrication. The same sort of thing happens on grant review bodies. You get to a grant review body, let’s say there’s 30 people on the grant review committee. Of that 30, maybe 25 of them will not be AD researchers. Your Alzheimer’s grant will be competing with all the other grants in neuroscience. The AD reviewers might think this is a good grant, but then the other reviewers, who really don’t know the field completely, won’t want to give more money to researchers in AD because of this potential fabrication. It’s harm by reputation rather than harm by science. So that summarizes what I think really. I’m sure it stopped people investing in amyloid. I’m sure it’s done some harm.

Are there any drugs on the market that you think need re-evaluating after these findings?

BC: Honestly, no. I’ve mentioned Aduhelm a couple of times, and I still think there’s the potential that an anti-amyloid treatment given earlier in the disease may be beneficial. The problem with this particular drug is that it’s difficult to administer and requires expensive, regular follow-up imaging to detect some really bad side effects. But to sound like a broken record, the recent data fabrication was likely of no consequence when it came to Aduhelm’s pathway to market.

MD: Since January 2022, only 29% of potential Alzheimer’s drugs are looking at targeting Aβ. Additionally, the data supporting anti-amyloid drugs progressing to clinical trials is independent of the research that has been questioned. The one drug which was under scrutiny was Simufilam, which reportedly works through modulation of filamin A; efforts to stop clinical trials of this drug have so far failed.

JH: No. However, the Aducanumab controversy also plays into amyloid-targeted drug development. In summary, Biogen tested an antibody drug called Aducanumab, which clearly removes amyloid from the brain. The drug went into clinical trials and halfway through, the reviewing committee stopped the trials because the drug wasn’t working. Biogen looked very carefully at the data and found that for very mildly affected participants, they did show some improvement. So, although the trial overall failed, for those who were very mildly affected, it was beneficial. That’s called a post hoc analysis. By that, I mean, it isn’t what they went to test. Post-hoc analyses are very dangerous, because you can test any number of things. A statistician would not like it. Sure enough, when they went back to the FDA, the statisticians for the FDA weren’t sold on the post-hoc results and advised the FDA against Aducanumab’s approval. However, the FDA went against their statistical advisors and approved it.

I think that Biogen were scientifically correct. I think that there is a signal in very mildly affected individuals but if I’d have been on the FDA panel, I’d have voted against approval too because post-hoc analyses are dangerous and because identifying those individuals who might get beneficial results is extremely difficult and would not happen in normal clinical practice. So a lot of people might get the drug, and for the majority of them it would do no good even under this optimistic scenario.

Now there are two other drugs, which are coming to their phase III results now. I’ve seen their phase II results, and I am optimistic that these drugs are indeed marginally beneficial. I’m optimistic that we are going to see drugs which are based upon amyloid, which are going to have marginal clinical benefit next year. If either of these drugs do work, then it will bring money back into the field because it’ll say the disease is tractable.

What are the lessons taken from this situation and how can it be avoided in future?

BC: A lot of lessons have already been learned between 2006 and now; screening software to look for image manipulation, the provision in the supplement of non-cropped raw western blot images and improvements in the level of antibody validation required have all been adopted by the larger journals. The biggest thing I’ve been thinking a lot about recently is having paid peer reviewers, employed by the journals. We have a huge employment problem in science, with a tiny number of professor positions available and a huge swath of incredibly skilled post-docs with nowhere to go.

These post-docs are the ones looking at raw data every day, who actually run the experiments that require a critical eye in review and who know which common errors and issues to look out for. Academics are increasingly pressed for time and being asked to review for free on top of the huge workloads we already have is an enormous pressure and will lead to important flaws in papers being missed.

In the last year I have reviewed a couple of papers that have taken me well over two days to review properly, with large errors in their analysis, and when I’ve received the other reviewer comments post-submission, they’ve been a couple of sentences long. The system is broken. Professional post-doctoral level reviewers would be a really great addition to the peer review process.

There’s the argument that once you leave the lab, you’re too detached from the experiments to critically review them, but let’s be honest, this applies to professors reviewing papers too. How many of them have actually been at the bench running the type of experiments they’re now reviewing? And there’s always the option to bring in an academic reviewer with a particular skill set if this is not covered by your professional staff.

MD: I think there will need to be a greater effort at screening images submitted to journals for publication. It can be added to the workflow to act as a triage for incoming manuscripts prior to sending out for peer review.

JH: This is difficult because fraud is the issue, and fraud is very difficult to prove. Universities are not set up to deal with fraud. Of course, you have to remember that people are innocent until proven guilty. Even if you see an image used twice, people could say that it was a mistake and it’s not important. You can’t prove that they’re guilty of anything nefarious because you can’t reach that standard. Universities are not really well set up to deal with that. I mean, if you found out that one of your staff members was producing fraudulent papers, you’d like to fire them. However, they’ve got legal rights, which include employment rights. So, it’s just very difficult to deal with.

What is the biggest repercussion of this allegation?

BC: Unfortunately, I think the irresponsible reporting of this allegation is going to be of much larger significance than the actual fabrication of the data. In the weeks following the reports in Science and the wider press, linking the fabrication as a direct driver of the amyloid hypothesis, I spent a lot of time on various places on the internet reading about people with relatives who’d passed away from AD who were extremely upset that their relatives had “received the wrong treatment.”

There was a widely shared online article from a pretty dubious looking outlet called “Wall Street Pro” that very clearly made this connection, which to be honest, was not a huge leap from the way it was reported in the mainstream press. As researchers, we rely on the trust and the extraordinary generosity and kindness of the general public to fund our research and to donate their time and their bodies to take part in our studies, and stories reported like this are hugely damaging to that trust.

MD: It is a stain on the neuroscience community and publishers, but this was one author that has set out to deceive the research community and progress their own agenda.

JH: It brings the field into disrepute. He brings the amyloid hypothesis, by implication, into disrepute. Certainly, it’s not good for the public image of science in general and AD research, specifically. One of the things that has really changed is image analysis. So, the allegation against Lesné is that he’s doctored images, and that type of doctoring of images was really impossible, for example, for reviewers to see.

Today, computerized image analysis can suddenly expose frauds that could not be exposed years ago because this technology can go through all of a certain author’s papers and check if they have ever used the same image twice. That is a development, which presumably Lesné never anticipated, that could suddenly scrutinize past papers in an entirely new way. That’s been a very important thing and has exposed other frauds in this type of image analysis.

What do you see as the positive outcomes from this situation?

BC: I hope a more nuanced view of this complex disease will be possible and fundable, and I believe this was already happening regardless of this situation.

MD: It is a real positive that research is securitized and that there are open forums to do this (e.g., PubPeer). These allegations arose through studies about an AD drug, but the tenacity of the researcher led to the wider investigation calling into question the role of the Aβ*56 form and this author’s work.

JH: It encourages informal discussion. This is in a way why scientific meetings are useful, not so you hear platform presentations but because people say I can’t reproduce this paper, and you don’t waste your time doing the same thing. So that’s part of how it’s dealt with.

PubPeer is a website that allows for the informal discussion of papers. I think it’s extremely useful. I think it’s a good thing. There’s also a website called ForBetterScience, which has some useful things on it and exposures of fraud. So, there are informal ways of doing things.

What areas of Alzheimer’s research do you think are particularly promising?

BC: It sounds like a cliché at this point, but I think big data can be really helpful here. I think ‘omics studies of carefully selected tissues and biofluids from individuals with AD and those who are resilient to the effects of the pathology will be very useful. Large population studies like Generation Scotland, UK Biobank and the ROSMAP projects have already revealed novel risk factors and phenomena that we’d never have seen otherwise and will continue to do so as these populations age. But we need to make sure we join the dots: the epidemiologists need to talk to the ‘omics specialists, who have to be in touch with the cell biologists and clinicians, if these findings are to be translated to the clinic.

MD: I have long been an advocate of the non-neuronal cells and think they will offer real potential to combat a range of brain diseases. The question remains when and how we should be targeting these cells to offer a real therapeutic difference to those living with AD.           

JH: Polypharmacy is an approach taken in nearly all complicated diseases. So, I think that eventually, for AD, we’ll get to polypharmacy involving an anti-amyloid drug, an anti-tau drug and a microglial drug, perhaps. That’s where we’re going to end up. All of the genetics to do with late-onset risk have really been pointing at microglial biology, which almost every group is now trying to modulate.

How do you think reviewers and publishers will be affected by this allegation?

BC: It’s clear that the peer review process in its current form doesn’t work. Academics don’t have the time to do this job properly and software isn’t capable of detecting every possible means of data falsification. In a previous question I mentioned paid peer reviewers; I think this would go a huge distance towards improving the process. Echoing the thoughts of many before me, it must become easier to publish studies that fail to replicate the findings of others. The constant search for novelty in articles is likely biasing acceptances in the wrong direction and has been for years.

MD: I think reviewers should be more confident in calling out concerns in submitted images; these might be genuine errors in the handling of images, but the authors should be made to account for any discrepancies. Most publishers have put in place sophisticated detection mechanisms, but these are always open to manipulation for those wishing to deceive. In addition, publishers need to work collectively to quash the rise in image manipulation. It is pleasing to read about new initiatives being set up, and I am hopeful that this will lead to a real-time reduction in these types of ‘fakery’ stories, which damage the public’s opinion of scientific research.

JH: The issue is with proving fraudulent research. The trouble is that if you get on a stage and claim that a researcher is a scientific fraud, you could be subject to libel. So, people are very careful to make that type of accusation.

Will this change the way we teach about imaging, analysis and critical thinking in science?

BC: I think there’s already a shift towards an improvement. Most graduate-level students can now code and many labs are supplying analytical code with paper submissions to ensure that others can reproduce their analysis. Of course, many of the analytical packages used can be black boxes, which may also be subject to misuse or incorrect usage, so it’s important to include strong teaching of basic statistical principles for all students.

Teaching the importance of sharing “raw” data, such as intact blots, and making sure that peer reviewers look at them is going to be important. We’re going to be discussing these papers in our next journal club to identify red flags and features that may help lead a reviewer to uncover these practices. And ultimately, teaching people to stay sceptical as a basic tenet of scientific thinking is going to continue to be very important.

MD: Most scientific publications reflect true observation, and these have given so much to society, but we should not believe all that we read or see. It is important that the next generation of scientists can critique both the science and the presentation of data in an open forum that leads to action, such as the retraction of published papers or editors voicing their concerns about an article.

JH: We should always look at papers critically. This is the purpose of journal clubs. And just because a paper is in a “big” journal does not mean we should believe it.

Meet the Experts:

Becky Carlyle

Alzheimer’s Research UK Senior Research Fellow at Oxford University

Dr Becky Carlyle is an Alzheimer’s Research UK Senior Research Fellow at Oxford University where she researches how molecular changes in the brain lead to neurodegenerative disease and their biomarkers. She uses induced pluripotent stem cell derived models as well as post-mortem human brain tissue.

 

Mark Dallas

Associate Professor in Cellular Neuroscience, School of Pharmacy at the University of Reading

Dr Mark Dallas is an Associate Professor in Cellular Neuroscience at the University of Reading. His research investigates the regulation of ion channels and transporters, with special interest in glial cells within the central nervous system. This has opened an exciting area of research pointing to their use as therapeutic targets to tackle brain disease.

 

John Hardy

Chair of Molecular Biology of Neurological Disease at University College London

Sir John Hardy is a geneticist and molecular biologist working at the Reta Lila Weston Institute of Neurological Studies at University College London. His research focuses on the genetic basis of neurological diseases, such as Alzheimer’s, Parkinson’s and motor neuron disease.

The post Ask the Experts: Alzheimer’s fabrication controversy appeared first on BioTechniques.

Full BioTechniques Article here

Powered by WPeMatico