Independent review

Independent review is the practice of having an expert, but independent evaluation of a set of results or artifacts produced by an author or organization.^[1] The practice appears in a large number of fields, including science, engineering, public policy, finance, medical practice, etc. In science, the term is often used synonymously with peer review.^[2] The term is sometimes used to contrast it with an "in-house review" performed by someone inside an organization.^[3]

Academic publishing

When scientific papers are considered for publication in scientific journals, there are generally submitted to peer review first. Often this is done in confidence (see anonymous peer review).

Sometimes researchers will bypass the pre-publication review process (see "science by press conference") or will refuse to share their data and methods with other scientists. In general, the scientific community frowns on this, as it makes it difficult or even impossible for other scientists to verify the data and interpretations of the research (see data sharing).

Medical practice

An article in Physician's News Digest says that there's a need for independent review, because hospital peer review process is broken.^[4]

Conflicts of interest

It is considered common knowledge that financial interests affect research results, and many people believe that funding can influence a scientist's findings. In the well known tobacco industry case, manufacturers funded studies seeming to prove that tobacco smoking was harmless. Scientists whose funding did not come from tobacco companies eventually proved a link between smoking and cancer. The British Medical Journal reported that, "Systematic bias favours products which are made by the company funding the research."^[5]

Science

Independent review of scientific results is the cornerstone of scientific accountability (see also peer review). By giving other scientists access to one's own data and methods (see data sharing), and giving them an opportunity to gauge the reproducibility of one's results, one ensures that errors due to incompetence, unconscious bias, or other causes can be found by others. It's the scientific version of proofreading.

Fringe science

Another possible reason for bypassing traditional peer review is when reporting results which are radically at odds with mainstream scientific views. In several historical cases, discoveries announced in this way have eventually reached the mainstream (see Semmelweiss and his theory of an "invisible substance" infecting women after childbirth; see also Continental drift).

Government agencies in the USA

The Office of Research Integrity employs independent reviewers to guard against scientific misconduct.^[6]

The state of California uses independent review to evaluate conservation proposals and to ensure that the general public can understand scientific publications written in mind-numbingly foggy technical prose.

... California law, which requires independent scientific review at several stages of the conservation planning process, “so consultants working for counties or developers are not able to get away with using flawed scientific methodologies.” The published reports of such independent reviews can help the public navigate an otherwise impenetrable, seemingly arcane debate.^[7]

Quotes

The best means for assessing risks and benefits is through independent review of the proposed research by individuals who have no direct vested interest in its outcome.^[8]
A central tenet in the protection of research participants is the independent review of research protocols to assess their scientific merit and ethical acceptability.
All protocols involving human participants should undergo an independent and rigorous scientific review to assess scientific quality, the importance of the research to increase knowledge, and the appropriateness of the study methodology to answer a precisely articulated scientific and, in some cases, clinical question. For example, the design of clinical trials should be based on sound statistical principles and methodologies, including sample size, use of controls, randomization, population stratification, stopping rules, and the feasibility of relating endpoints to objectives.^[8]

But knowing a study is industry funded isn't enough. More important is having independent reviewers evaluate the validity of the study design, methods, protocols, and determine whether the conclusions are consistent with the data. This is how rigorous peer review is supposed to work. But peer review can miss fatal flaws in studies—even when authors provide disclosure statements. That's why many researchers working at the intersection of science and policy want to see transparent review processes and independent scientific review panels oversee research results destined for use in policymaking (or drug approval) decisions.^[7]