Not a week passes without reproducibility in science or the lack of it hitting the headlines. Although much of the criticism is directed at the biomedical sciences or psychology, many of the same problems also pervade the chemical sciences.
A survey of over 1500 scientists conducted by Nature last year revealed that 70% of researchers think that science faces a reproducibility crisis. Over half, however, still have faith in published literature in their field with chemists being amongst the most confident despite reporting the most difficulty replicating other researchers or their own work. Although this observation seems contradictory, an explanation might be that chemists are more often looking to repeat experiments exactly, says Rick Danheiser, a synthetic organic chemist at the Massachusetts Institute of Technology.
Chemical journal articles suffer from the inability of people to name compounds accurately
Anita Bandrowski, University of California, San Diego
Danheiser is the editor-in-chief of the unconventional journal Organic Syntheses that has verified the experiments of all the papers it has published since it launched in 1921. The journal does this by having the research replicated by independent chemists before publishing them a practice that is almost unheard of in chemistry or any other research field (the exception being a few brief instances in history). All experiments are checked for reproducibility in the lab of one of the journals board of editors, often by graduate students and postdoctoral researchers working under the supervision of the Organic Syntheses editor. Danheiser, who has written about reproducibility, tells Chemistry World that the journal usually aims to check the work in submitted papers within six months.
But Danheiser says that rolling out this replication process to other fields is challenging as biology often has too many variables and physics and some areas of chemistry entails the use of specialised or costly equipment that may not be readily available in other labs. Between 2010 and 2016, the journal rejected 7.5% of submissions due to irreproducibility of yield or selectivity, Danheiser notes. Most chemists would consider that to be frightening, he adds, as papers in conventional journals are therefore less likely to be reproducible.
Chemistry is to some extent in a sweet spot when it comes to reproducing experiments as the apparatus used is usually not too exotic, notes Derek Lowe, a medicinal chemist who pens the In the pipeline blog. But he notes that, for instance, in natural product chemistry almost none of the multi-step syntheses used to produce the compound are a feasible way of getting to the product. As a result, most are never completely reproduced in the first place, explains Lowe. In fact, he says, products are often easy enough to obtain from their original source in most cases. What people do build on, Lowe says, is the unique techniques researchers use when creating compounds.
Medicinal chemistry and drug discovery literature are some of the more reproducible fields in chemistry, adds Lowe, as theyre more extensively conducted by industry scientists, who have fewer incentives to publish fake or substandard results. Lowe points out that industry researchers contribute more to patent literature, and patents lacking reproducibility and rigour could result in legal issues.
Chemistrys reproducibility problem should be a fairly easy thing to fix
Phil Baran, Scripps Research Institute
Furthermore, Lowe believes irreproducibility is a bigger problem at high profile journals publishing cutting edge research, as well as low-end journals. Most solid papers are likely to be in what Lowe calls middle journals those that are respectable, but not glamorous.
This month, the Chemical Probes Portal saw its 275th compound added. This portal is part of the campaign to make science more reproducible and holds a database of many of the small molecules used in drug discovery and to investigate biological processes. The problems being addressed by the portal are twofold, says Amy Donner, its director in Boston. The smaller problem, she says, is that of making sure appropriate reagents are used in experiments. Whereas the larger issue that results in a tremendous amount of irreproducibility that we see in the literature is misunderstanding how to use these highly potent chemical probes to tweak biological processes in living organisms.
Donner says the number of chemical probes out there is in the tens of thousands, and that in itself is a problem. Biologists who are most likely to use probes in experiments dont necessarily have the expertise to carry out relevant chemical validation tests to determine which molecules would work best in their experiments, says Donner. This is where the chemical probes portal comes into its own offering advice on which probes are suitable for specific circumstances and which are best avoided. Researchers can search their target protein and find a list of molecules that interact with it, Donner explains.
Theres currently no checklist outlining a compounds information rather, says Donner, scientists rely on aggregating information about probes as they go. Sharing that information on the portal should save others from replicating that effort as well as informing investigators how they should be designing experiments taking into account a probes strengths and weaknesses.
But the portal is still in its infancy and researchers face a huge task logging tens of thousands more molecules, Donner says. At present, the priority is highlighting the best and worst probes. In the short term, thats how we can have the biggest impact, she adds.
Research Resource Identifier (RRID) is another initiative with the goal of improving research reproducibility by helping to standardise how reagents, software, antibodies and model organisms are cited in the literature. Although over 200 journals have already started encouraging authors to use RRIDs to cite laboratory resources, chemistry journals are, on the whole, not on board yet, says Anita Bandrowski, a neuroscientist at the University of California, San Diego, who is coordinating the RRID project.
According to Bandrowski, chemical journal articles also suffer from the inability of people to name compounds accurately. As its difficult to teach everyone to use appropriate nomenclature, says Bandrowski, attention has mostly focused on pushing researchers to at least use numerical identifiers like the PubChem ID. Our findings show that authors are highly accurate when they provide this type of information, she says.
Another phenomenon that has taken industrial chemistry by storm is electronic laboratory notebooks (ELNs), which store experimental data and procedures, allowing researchers to access data years after it was collected. One feature of ELNs is that they allow researchers to search molecular structures of reactants and products, making it easier to keep track of reactions. Inside a research group or a company, ELNs may be set up to allow scientists to search their colleagues notebooks, which is useful to avoid duplicating work and assess differences in repeat experiments. But ELNs have seen slower uptake in academia probably due to their high costs.
There are many ELNs out there with different ways of collecting and storing data, says Richard Whitby, an organic chemist at the University of Southampton, UK, who leads the Dial-a-molecule network, which aims to speed up the process of synthesising new molecules. One of the networks other goals is to promote a standard for ELN data to enable exchange and data mining. ELNs which are designed for chemistry pay you back for using them, Whitby adds. He notes that they also have some useful features such as stoichiometry calculations and experimental safety data, which partly makes up for the fact that its actually more difficult to use an ELN than a paper notebook.
Phil Baran, a synthetic organic chemist at the Scripps Research Institute in La Jolla, California, says that blogging at the Open Flask has helped his group tackle issues like reproducibility and transparency around experiments. Weve gotten more use out of this than conventional peer review, Baran notes. He says chemistrys reproducibility problem compared with other fields should be a fairly easy thing to fix. The solution, he says, is to put the onus on authors to make their supporting information clear and to encourage them to collaborate with other labs to get their work checked or field-tested. Journals should also make rules specific to different disciplines, and text only supporting information should no longer be allowed, particularly for methodology papers where the primary outcome is for others to reproduce a protocol, Baran says.
Read more from the original source:
Taking on chemistry's reproducibility problem - Chemistry World (subscription)
- VeruTEK Reports Very High Increases In Heavy Oil Production Using ProBoost Enhanced Oil Recovery Chemistry - January 15th, 2013 [January 15th, 2013]
- Broncos looking to find proper balance of talent and chemistry - January 22nd, 2013 [January 22nd, 2013]
- Chemistry Movie Trailer - Video - January 22nd, 2013 [January 22nd, 2013]
- Research and Markets: Austria In Vitro Diagnostics Market Outlook to 2018 - Clinical Chemistry Genetic Testing ... - February 1st, 2013 [February 1st, 2013]
- Ayrshire news: Student lands top chemistry award - February 1st, 2013 [February 1st, 2013]
- JoVE expands scientific video publication into chemistry - February 5th, 2013 [February 5th, 2013]
- Stability of transition metal alkyls - Video - January 8th, 2014 [January 8th, 2014]
- Chemistry Surface Chemistry part 3 (Adsorption Examples) CBSE class 12 XII - Video - January 15th, 2014 [January 15th, 2014]
- 3 BASIC CONCEPT OF CHEMISTRY - Video - January 16th, 2014 [January 16th, 2014]
- 5 Minute Lesson: CHEMISTRY #3: Electron Configuration - Video - January 17th, 2014 [January 17th, 2014]
- MUSHROOM CLOUD - MALAYALAM - Fun with chemistry! - Video - January 19th, 2014 [January 19th, 2014]
- "Arrow's" Stephen Amell Talks Oliver and Felicity's Chemistry - Video - January 23rd, 2014 [January 23rd, 2014]
- Eq 6.1: Le Chatelier's Principle - Introduction - Video - January 28th, 2014 [January 28th, 2014]
- Conservation of Mass 020 - Video - February 2nd, 2014 [February 2nd, 2014]
- 2423 PE01ASA 024C - Balancing Combustion Reactions 013 - Video - February 2nd, 2014 [February 2nd, 2014]
- Isobutane vs. Butane : Chemistry Lessons - Video - February 2nd, 2014 [February 2nd, 2014]
- How Being Present Can Improve Your Child's Brain Chemistry - Video - February 6th, 2014 [February 6th, 2014]
- Science interesting experiment Chemistry experiment 00004 - Video - February 9th, 2014 [February 9th, 2014]
- Music Box Inspires Portable $5 Chemistry Lab - April 10th, 2014 [April 10th, 2014]
- A Clever New Chemistry Kit Your Kid Will Actually Want to Use - April 10th, 2014 [April 10th, 2014]
- Stanford Scientist Creates $5 Chemistry Set Inspired By Music Box - April 10th, 2014 [April 10th, 2014]
- Chemistry - Wikipedia, the free encyclopedia - April 10th, 2014 [April 10th, 2014]
- Chemistry.com | An Online Dating Site for Singles - April 10th, 2014 [April 10th, 2014]
- thermo - Video - April 10th, 2014 [April 10th, 2014]
- Sexy Chemistry of Siddharth and Shraddha in Villain! - Video - April 10th, 2014 [April 10th, 2014]
- lecture 29-part 5- Analytical Chemistry and Chromatography for Graduate Students - Prof Peter Carr - Video - April 10th, 2014 [April 10th, 2014]
- lecture 29-part 4- Analytical Chemistry and Chromatography for Graduate Students - Prof Peter Carr - Video - April 10th, 2014 [April 10th, 2014]
- c i - Video - April 10th, 2014 [April 10th, 2014]
- Jee-Main (AIEEE) 2014 - Chemistry- Q22 - Video - April 10th, 2014 [April 10th, 2014]
- Moles calculations examples - IGCSE Chemistry - Video - April 10th, 2014 [April 10th, 2014]
- Form 2 chemistry calculation with MOLES - IGCSE Chemistry - Video - April 10th, 2014 [April 10th, 2014]
- Physical Chemistry Lab: Parallel First Order Reactions - Video - April 10th, 2014 [April 10th, 2014]
- UNedited Chemistry Project By The Dream Team - Video - April 10th, 2014 [April 10th, 2014]
- we - Video - April 10th, 2014 [April 10th, 2014]
- Teacher of Chemistry - April 15th, 2014 [April 15th, 2014]
- Head of Chemistry - April 15th, 2014 [April 15th, 2014]
- About.com Chemistry - Chemistry Projects, Homework Help ... - April 15th, 2014 [April 15th, 2014]
- PARKnSHOP 'Chemistry Class' Mrs Wong's Academy - Video - April 15th, 2014 [April 15th, 2014]
- AP chemistry kinetics lab green food coloring/bleach - Video - April 15th, 2014 [April 15th, 2014]
- The Chemistry of Flubber - Video - April 15th, 2014 [April 15th, 2014]
- Chemistry Cooking Video - Video - April 15th, 2014 [April 15th, 2014]
- Chemistry Intro Animation Videos HD 2014 - Video - April 15th, 2014 [April 15th, 2014]
- Chemistry Today: An Out of Class Chemistry Project - Video - April 15th, 2014 [April 15th, 2014]
- Chemistry - Naming Ionic Compounds - Video - April 15th, 2014 [April 15th, 2014]
- Chemistry - Stoichiometry - Video - April 15th, 2014 [April 15th, 2014]
- Chemistry - Hybridization - Video - April 15th, 2014 [April 15th, 2014]
- Arjun Kapoors speaks about his chemistry with Alia Bhatt in 2 states - Video - April 15th, 2014 [April 15th, 2014]
- Chemistry Official Promo - Video - April 15th, 2014 [April 15th, 2014]
- Leah4Sci's Awesome Free Online Organic Chemistry I and II Workshops! - Video - April 15th, 2014 [April 15th, 2014]
- org - Video - April 15th, 2014 [April 15th, 2014]
- What is the Difference between Process Free and Chemistry Free Plates? - Video - April 15th, 2014 [April 15th, 2014]
- Stanford bioengineer creates $5 chemistry set - Video - April 15th, 2014 [April 15th, 2014]
- Reduction of Ketones and Aldehdyes Made Easy! - Organic Chemistry - Video - April 15th, 2014 [April 15th, 2014]
- Catch Salman and Daisy's sizzling chemistry in the superhit Jai Ho! - Video - April 15th, 2014 [April 15th, 2014]
- Chemistry Lab Technician - April 15th, 2014 [April 15th, 2014]
- Science teacher Imogen Herrick wins ExCel Award at Academy @ Shawnee - April 15th, 2014 [April 15th, 2014]
- equlibrium - Video - April 15th, 2014 [April 15th, 2014]
- Veazie police officer lauded for saving UMaine students life - April 16th, 2014 [April 16th, 2014]
- IGCSE Chemistry - Video - April 16th, 2014 [April 16th, 2014]
- Chemistry - Chemical Kinetics (11 of 30) Finding the Order - Using Initial Rates - Video - April 16th, 2014 [April 16th, 2014]
- CHEMISTRY TUTORIAL:Colligative Properties and Van't Hoff - Video - April 16th, 2014 [April 16th, 2014]
- Quantum Chemistry - Schrodinger Eq / Particle in a Box Review - Video - April 16th, 2014 [April 16th, 2014]
- Science Nation - Solar Fuels: A grand challenge of 21st century chemistry - Video - April 17th, 2014 [April 17th, 2014]
- Making Sense: A' Level Chemistry Made Easy! (Extended) - Video - April 17th, 2014 [April 17th, 2014]
- Making Sense: A' Level Chemistry Made Easy - Video - April 17th, 2014 [April 17th, 2014]
- Artificial Chemistry - Video - April 17th, 2014 [April 17th, 2014]
- "Materials of the Future"- Chemistry of Materials Video Contest - Video - April 17th, 2014 [April 17th, 2014]
- "We can do something about [global warming]." Nobel Laureate Mario Molina (1995 Chemistry) - Video - April 17th, 2014 [April 17th, 2014]
- Chemistry - Chemical Kinetics (1 of 30) Definition - Video - April 17th, 2014 [April 17th, 2014]
- Chemistry: A Molecular Approach at eCampus.com - Video - April 18th, 2014 [April 18th, 2014]
- Chemistry - Reaction Prediction and Precipitate Solubility - Video - April 18th, 2014 [April 18th, 2014]
- Brady Hoke: Michigan has shown great chemistry during offseason - April 19th, 2014 [April 19th, 2014]
- Brady Hoke says Michigan football team has shown great chemistry during offseason - April 19th, 2014 [April 19th, 2014]
- IIT Delhi Fire Incident Blast At Chemistry Lab - Video - April 19th, 2014 [April 19th, 2014]
- [Chemistry] [Chemistry II] - 14 - Reaction Rates Concept Video Part I Intro - Video - April 19th, 2014 [April 19th, 2014]
- Chemistry - Polarity - Video - April 19th, 2014 [April 19th, 2014]
- Basic Organic Chemistry Nomenclature of Alkanes and Bonding of Non-Metals - Video - April 19th, 2014 [April 19th, 2014]
- Introducing Green Chemistry into the High School - Video - April 21st, 2014 [April 21st, 2014]
- Chemistry - Ionic and Covalent Bonds - Video - April 21st, 2014 [April 21st, 2014]
- Tucson, AZ | Continuous News and Weather - April 22nd, 2014 [April 22nd, 2014]