Progress in rehabilitation depends, like progress in medicine in general, largely on theoretical considerations, empirical results and reports on the widest variety possible of applications of ‘classical’ and new treatments in practice. The results of these findings get lost, however, if they are not communicated to fellow-scientists and fellow-practitioners. This is commonly done by publication of the findings (whether theoretical or practical) in the form of journal articles and as books or book chapters, as well as by presentation at conferences or personal contacts at scientific meetings.

However practical and efficient this may have been in the earlier days of science, this is no longer the most effective way of scientific information transfer: there are too many scientists to meet all relevant specialists personally, and there are far too many books and journals to be able to read all relevant literature, even if it is on a very specific topic. For the time being, this problem is tackled (but fundamental changes in approach will be necessary within a few years – or maximally a few decades) by the publication of review articles [1], commonly published in review journals (e.g., the ‘International Journal of Medical Reviews’ and the ‘European Review for Medical and Pharmacological Sciences’). The authors of such review articles should be highly praised, because writing such an article is painstaking and time-consuming.

The scientific value of a review article depends, obviously, on its reliability. Such a reliability is however, in spite of the fact that the manuscripts for such articles tend to be scrupulously reviewed by several specialists, not self-evident, due to several reasons: (1) the author(s) of one or more of the reviewed articles may unintentionally have included incorrect data/findings in his/her/their article(s), (2) the author(s) of one or more of the reviewed articles may willingly have included incorrect information (scientific fraud), (3) the author(s) of the review article may have misinterpreted the results of one or more of the reviewed articles and/or unintentionally have drawn incorrect conclusions, (4) the author(s) of the review article may willingly have presented incorrect conclusions (scientific fraud), (5) the author(s) of the review article may have overlooked articles that might influence their conclusions, (6) the author(s) of the review article may be unaware that only part of the relevant studies carried out has been published in the open literature.

Unintentional errors

Scientists are humans, so they can make mistakes. The above errors (1) and (3) can consequently not be avoided for 100%; it may, however, be expected from authors that they do their best to avoid errors. This implies that it may be expected from the authors of review articles that they analyze whether the data in the primary articles that they review are reliable. Unfortunately, not all authors of medical review articles adhere to this viewpoint: Valencia-Chulián et al. [2] even state in a reply to a comment on mistakes in a review article that they wrote; “It is not our duty, however, to question the integrity of primary data”. Considering the possible consequences of such an attitude for public health, such ‘unintentional errors’ come close to scientific misconduct. On the other hand, it should be realized that it is physically impossible for authors of review articles to collect and analyze all primary articles in detail; this is rather a task for organizations such as the Cochrane Library (see: https//www.cochranelibrary.com/cdsr/about-cdsr).

Fortunately, good scientific journals have capable reviewers (and, in principle also editors) who should detect such errors and should require corrections in a revised manuscript. In this context, a major problem is that few capable reviewers, particularly in medical disciplines, can take sufficient time for thorough reviews. A consequence is that reviews of medical manuscripts tend to be much shorter (and much less in-depth: [3]) than reviews of manuscripts in particularly the natural sciences

Scientific fraud

Scientific fraud (errors 2 and 4) is one of the reasons why part of our society has lost trust in science [4,5]. Unfortunately, scientific fraud is not uncommon, mainly because of the ‘publish or perish’ philosophy adhered to by many universities [6], but it is relatively frequent in medical publications [7-11]. The ‘publish or perish’ philosophy has led to an incredible growth in the submission of low-quality manuscripts. In order to protect the reputation of Chinese science, the Chinese government even stopped a few years ago to pay scientists for publications in ‘foreign’ journals. Particularly the COVID-19 pandemic has raised a tsunami of low-quality manuscripts, most of which were rejected by high-reputation journals.

Numerous examples that show how unreliable medical publications (and even medical authors!) are, have been given by Persson and Persson [12], who refer to studies published regarding COVID-19 aspects. They also quote Vervoort et al. [13] who mention that “a total of 367 articles related to COVID-19 [were] published in journals on Beall's List [14] between January and May 2020 alone…”. These authors [13] mention three major concerns: “1. Loss of potentially valuable biomedical and epidemiological information. 2. Spread of misinformation with potentially harmful or negligent consequences (e.g., some articles claimed efficacy of homeopathy). 3. Money earned off of unknowing authors in times where many scientists and clinicians have taken pay cuts.”

It is noteworthy in this context that a manuscript dealing with COVID-19 was accepted and published as "peer-reviewed" in the Asian Journal of Medicine and Health, but that it was a hoax After the hoax had been revealed, the article was retracted (see htpp://journalajmah.com/index.php/AJMAH/article/view/30232), but the example shows how badly needed a critical view of authors of review articles is.

Although less frequent in articles about rehabilitation, it is also present in this discipline, and probably increasing [15]. Consequently, it must be presumed that many medical review articles are based on at least partly fraudulent material.

Overlooked literature

Overlooking relevant publications by the author(s) of a review article (error 5) is probably a serious problem, but it is hardly possible to determine to which extent this plays a role. Although databases offer commonly the possibility to trace at least most of the relevant literature, a problem remains that authors may fail to choose the best keywords; another problem is that neither all relevant articles, nor all relevant journals are covered by all relevant databases; this refers particularly to articles in not commonly accessible journals and to new journals that did not yet become known.

In addition, it has become clear in the course of time that particularly pharmaceutical industries have not always handled morally correctly [16]: they are known to have prevented publications with negative outcomes of tests with new medicines while allowing publication of positive results of similar tests; the most well-known case [e.g., 17] concerned thalidomide (α-N-[phtalimido]glutarimide; also known under names such as softenon and contergan) in the middle of the past century.

It is, obviously, possible that several of the above aspects play a role in a single review article. An interesting example has been published recently; it concerns a review article that will be dealt with in this section.

Background

The review article dealt with here [18] is a systematic review of 16 studies and case reports about dry needling for the management of spasticity, pain, and range of movement in adults after stroke. Most questions raised by this review article concern a 2018 review article [19]. A co-author of this article appeared to be also a co-author of two other studies [20-21] dealt with in the review study [18].The three studies [19-21] in which this author was involved dealt with essentially the same topic: the effects of dry needling on post-stroke spasticity (either or not with some additional aspects). Moreover, the studies [19-21] had several more authors in common. It was not clear, however, whether the groups of patients showed overlap, which is, obviously, of great importance for the statistics of the findings.

Errors, inconsistencies and ambiguities

Another problem concerns one of the studies [20] in the review study [18]. It shows enigmatic data: a group of 20 subjects is mentioned, but tables show results for only 19 subjects. A question about this discrepancy could not be answered by the corresponding author, even after specific questions about this missing subject.

A statistical analysis of the effects of a specific type of medical treatment has, obviously, little or no value if the data are not (1) reliable (= correct, complete and unambiguous) and (2) mutually independent. Moreover, the data should be up-to-date [22]. If these requirements are not met, the conclusions might suggest incorrectly more negative than positive effects (or the reverse). More details about the flaws in the review article under debate [18] can be found in a recently published comment [23].

The questions raised by the review article [18] on the basis of questionable data in some of the reviewed articles [19-21] affect its reliability. Because review articles are nowadays fundamental for the progress of science, and certainly also for the progress of medicine and rehabilitation [24], it is worthwhile to find ways to reduce the number of uncertainties in them.

One of the commonly sufficient measures is the process of peer reviewing [25]. This certainly has helped for a long time to reduce the number of error sources 1 and 3 mentioned above. The present-day philosophy that employees should help increase profits for the employer (whether a hospital, a university or a commercial company) has, however, the negative result that hardly any time is left for thorough reviewing, so that specialist often have to decline invitations for reviewing, or review a manuscript in unduly much haste; this may well result in overlooking weak points. It seems that this problem could only be solved if large publishing companies employ professional reviewers that are experts in the fields of the topics of the journals published.

Such an approach might also help to detect scientific fraud (error sources 2 and 4) more quickly. It appears that some individual researchers/authors tend to commit scientific fraud relatively frequently, so that it might be helpful if publishing companies inform each other about the detection of fraudulent authors; a problem might be that in several countries ‘black lists’ and exchange of this type of information are illegal on the basis of the right on privacy.

The problem that authors writing a review article overlook relevant literature should be solved by the journals or their publishers. It is, unfortunately, fairly common that articles are published in a journal outside the interest field of the journal. This may be due to inexperienced authors who are insufficiently aware of the most suited journals, but also to differences in the height of author fees. Moreover, rejection of a manuscript often leads to submission to a lower-rank journal that more easily accepts manuscripts, whatever the topic may be. Particularly editors and reviewers should in this respect take their role as gatekeepers of sound science more seriously Finally, the problem of biased information in review articles because not all primary information has been made public, regards particularly articles about the effectiveness (and possible side effects) of newly tested medicines. This certainly has played a major role in the past (as shown by the thalominid affair), but it is commonly assumed that this problem can be avoided, for instance by asking authors of primary articles on the effects of drugs to provide information that the producer of the medicine states that no results of comparable tests are withheld.

Thorough analysis of review articles may reveal significant flaws. These may be due to a number of errors that are not always easy to detect by reviewers. A main problem in this context is that specialist scientists get increasingly less time for review activities; moreover, the work of reviewers is often not or insufficiently acknowledged [26]. It should therefore be considered by the large publishing companies to employ specialist scientists as professional reviewers. Moreover, both editors and reviewers should be more aware than in the past that the exponentially increasing flow of scientific manuscripts does not help science, if not of high quality and reliable. This holds particularly for review articles.

We thank the two anonymous reviewers of an earlier version for their suggestions to improve the manuscript.