(C) PLOS One [1]. This unaltered content originally appeared in journals.plosone.org.
Licensed under Creative Commons Attribution (CC BY) license.
url:https://journals.plos.org/plosone/s/licenses-and-copyright

------------



Accuracy of novel antigen rapid diagnostics for SARS-CoV-2: A living systematic review and meta-analysis

['Lukas E. Brümmer', 'Division Of Tropical Medicine', 'Center For Infectious Diseases', 'Heidelberg University Hospital', 'Heidelberg', 'Stephan Katzenschlager', 'Department Of Anesthesiology', 'Mary Gaeddert', 'Christian Erdmann', 'Fh Muenster University Of Applied Sciences']

Date: 2021-08

In this study we found that Ag-RDTs detect the vast majority of SARS-CoV-2-infected persons within the first week of symptom onset and those with high viral load. Thus, they can have high utility for diagnostic purposes in the early phase of disease, making them a valuable tool to fight the spread of SARS-CoV-2. Standardization in conduct and reporting of clinical accuracy studies would improve comparability and use of data.

We registered the review on PROSPERO (registration number: CRD42020225140). We systematically searched multiple databases (PubMed, Web of Science Core Collection, medRvix, bioRvix, and FIND) for publications evaluating the accuracy of Ag-RDTs for SARS-CoV-2 up until 30 April 2021. Descriptive analyses of all studies were performed, and when more than 4 studies were available, a random-effects meta-analysis was used to estimate pooled sensitivity and specificity in comparison to reverse transcription polymerase chain reaction (RT-PCR) testing. We assessed heterogeneity by subgroup analyses, and rated study quality and risk of bias using the QUADAS-2 assessment tool. From a total of 14,254 articles, we included 133 analytical and clinical studies resulting in 214 clinical accuracy datasets with 112,323 samples. Across all meta-analyzed samples, the pooled Ag-RDT sensitivity and specificity were 71.2% (95% CI 68.2% to 74.0%) and 98.9% (95% CI 98.6% to 99.1%), respectively. Sensitivity increased to 76.3% (95% CI 73.1% to 79.2%) if analysis was restricted to studies that followed the Ag-RDT manufacturers’ instructions. LumiraDx showed the highest sensitivity, with 88.2% (95% CI 59.0% to 97.5%). Of instrument-free Ag-RDTs, Standard Q nasal performed best, with 80.2% sensitivity (95% CI 70.3% to 87.4%). Across all Ag-RDTs, sensitivity was markedly better on samples with lower RT-PCR cycle threshold (Ct) values, i.e., <20 (96.5%, 95% CI 92.6% to 98.4%) and <25 (95.8%, 95% CI 92.3% to 97.8%), in comparison to those with Ct ≥ 25 (50.7%, 95% CI 35.6% to 65.8%) and ≥30 (20.9%, 95% CI 12.5% to 32.8%). Testing in the first week from symptom onset resulted in substantially higher sensitivity (83.8%, 95% CI 76.3% to 89.2%) compared to testing after 1 week (61.5%, 95% CI 52.2% to 70.0%). The best Ag-RDT sensitivity was found with anterior nasal sampling (75.5%, 95% CI 70.4% to 79.9%), in comparison to other sample types (e.g., nasopharyngeal, 71.6%, 95% CI 68.1% to 74.9%), although CIs were overlapping. Concerns of bias were raised across all datasets, and financial support from the manufacturer was reported in 24.1% of datasets. Our analysis was limited by the included studies’ heterogeneity in design and reporting.

SARS-CoV-2 antigen rapid diagnostic tests (Ag-RDTs) are increasingly being integrated in testing strategies around the world. Studies of the Ag-RDTs have shown variable performance. In this systematic review and meta-analysis, we assessed the clinical accuracy (sensitivity and specificity) of commercially available Ag-RDTs.

Competing interests: I have read the journal’s policy and the authors of this manuscript have the following competing interests: CMD is a member of the Editorial Board of PLOS Medicine.

Funding: The study was supported by the Ministry of Science, Research and Arts of the State of Baden-Wuerttemberg, Germany (no grant number; https://mwk.baden-wuerttemberg.de/de/startseite/ ) and internal funds from the Heidelberg University Hospital (no grant number; https://www.heidelberg-university-hospital.com/de/ ) to CMD. Further, this project was funded by United Kingdom (UK) aid from the British people (grant number: 300341-102; Foreign, Commonwealth & Development Office (FCMO), former UK Department of International Development (DFID); www.gov.uk/fcdo ), and supported by a grant from the World Health Organization (WHO; no grant number; https://www.who.int ) and a grant from Unitaid (grant number: 2019-32-FIND MDR; https://unitaid.org ) to Foundation of New Diagnostics (FIND; JAS, SC, SO, AM). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright: © 2021 Brümmer et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

With our systematic review and meta-analysis, we aim to close this gap in the literature and link to a website ( https://www.diagnosticsglobalhealth.org ) that is regularly updated.

With the increased availability of Ag-RDTs, an increasing number of independent validations have been published. Such evaluations differ widely in their quality, methods, and results, making it difficult to assess the true performance of the respective tests [ 7 ]. To inform decision makers on the best choice of individual tests, an aggregated, widely available, and frequently updated assessment of the quality, performance, and independence of the data is urgently needed. While other systematic reviews have been published, they include data only up until November 2020 [ 8 – 11 ], exclude preprints [ 12 ], or were industry sponsored [ 13 ]. In addition, only 1 assessed the quality of studies in detail, with data up until November 2020 [ 7 , 11 ].

As the COVID-19 pandemic continues around the globe, antigen rapid diagnostic tests (Ag-RDTs) for SARS-CoV-2 are seen as an important diagnostic tool to fight the virus’s spread [ 1 , 2 ]. The number of Ag-RDTs on the market is increasing constantly [ 3 ]. Initial data from independent evaluations suggest that the performance of SARS-CoV-2 Ag-RDTs may be lower than what is reported by the manufacturers. In addition, Ag-RDT accuracy seems to vary substantially between tests [ 4 – 6 ].

Two types of sensitivity analyses were planned: estimation of sensitivity and specificity excluding case–control studies, and estimation of sensitivity and specificity excluding non-peer-reviewed studies. We compared the results of each sensitivity analysis against the overall results to assess the potential bias introduced by considering case–control studies and non-peer-reviewed studies.

We aimed to do meta-regression to examine the impact of covariates including symptom duration and Ct value range. We also performed the Deeks test for funnel-plot asymmetry as recommended to investigate publication bias for diagnostic test accuracy meta-analyses [ 18 ] (using the “midas” command in Stata, version 15); a p-value < 0.10 for the slope coefficient indicates significant asymmetry.

For categorization by sample type, we assessed (1) nasopharyngeal (NP) alone or combined with other (e.g., oropharyngeal [OP]), (2) OP alone, (3) anterior nasal (AN) or mid-turbinate (MT), (4) a combination of bronchoalveolar lavage and throat wash (BAL/TW), or (5) saliva. Analyses were preformed using R 4.0.3 (R Foundation for Statistical Computing, Vienna, Austria).

In an effort to use as much of the heterogeneous data as possible, the cutoffs for the Ct value groups were relaxed by 2–3 points within each range. The <20 group included values reported up to ≤20, the <25 group included values reported as ≤24 or <25 or 20–25, and the <30 group included values from ≤29 to ≤33 and 25–30. The ≥25 group included values reported as ≥25 or 25–30, and the ≥30 group included values from ≥30 to ≥35. For the same reason, when categorizing by age, the age group <18 years (children) included samples from persons whose age was reported as <16 or <18 years, whereas the age group ≥18 years (adults) included samples from persons whose age was reported as ≥16 years or ≥18.

We extracted raw data from the studies and recalculated performance estimates where possible based on the extracted data. The raw data can be found in S2 Table . We prepared forest plots for the sensitivity and specificity of each test and visually evaluated the heterogeneity between studies. If 4 or more datasets were available with at least 20 positive RT-PCR samples per dataset for a predefined analysis, a meta-analysis was performed. We report point estimates of sensitivity and specificity for SARS-CoV-2 detection compared to the reference standard along with 95% confidence intervals (CIs) using a bivariate model (implemented with the “reitsma” command from the R package “mada,” version 0.5.10). When there were fewer than 4 studies for an index test, only a descriptive analysis was performed, and accuracy ranges are reported. In subgroup analyses where papers presented data only on sensitivity, a univariate random-effects inverse variance meta-analysis was performed (using the “metagen” command from the R package “meta,” version 4.11–0). We predefined subgroups for meta-analysis based on the following characteristics: Ct value range, sampling and testing procedure in accordance with manufacturer’s instructions as detailed in the instructions for use (IFU) (henceforth called IFU-conforming) versus not IFU-conforming, age (<18 versus ≥18 years), sample type, presence or absence of symptoms, symptom duration (<7 days versus ≥7 days), viral load, and type of RT-PCR used.

We examined whether a study received financial support from a test manufacturer (including the free provision of Ag-RDTs), whether any study author was affiliated with a test manufacturer, and whether a respective conflict of interest was declared. Studies were judged not to be independent from the test manufacturer if at least 1 of these aspects was present; otherwise, they were considered to be independent.

The quality of the clinical accuracy studies was assessed by applying the QUADAS-2 tool [ 17 ]. The tool evaluates 4 domains: patient selection, index test, reference standard, and flow and timing. For each domain, the risk of bias is analyzed using different signaling questions. Beyond the risk of bias, the tool also evaluates the applicability of each included study to the research question for every domain. The QUADAS-2 tool was adjusted to the needs of this review and can be found in S3 Text .

We differentiated between clinical accuracy studies (performed on clinical samples) and analytical accuracy studies (performed on spiked samples with a known quantity of virus). Analytical accuracy studies can differ widely in methodology, impeding an aggregation of their results. Thus, while we extracted the data for both kinds of studies, we only considered data from clinical accuracy studies as eligible for the meta-analysis. Separately, we summarized the results of analytical studies and compared them with the results of the meta-analysis for individual tests.

At first, 4 authors (SK, CE, SS, and MB) extracted 5 randomly selected papers in parallel to align data extraction methods. Afterwards, data extraction and the assessment of methodological quality and independence from test manufacturers (see below) was performed by 1 author per paper (SK, CE, SS, or MB) and controlled by a second (LEB, SK, SS, or MB). Any differences were resolved by discussion or by consulting a third author (CMD).

A full list of the parameters extracted is included in S1 Table , and the data extraction file is available at https://zenodo.org/record/4924035#.YOlzWS223RZ . Studies that assessed multiple Ag-RDTs or presented results based on differing parameters (e.g., various sample types) were considered as individual datasets.

Two reviewers (LEB and CE, LEB and SS, or LEB and MB) reviewed the titles and abstracts of all publications identified by the search algorithm independently, followed by a full-text review for those eligible, to select the articles for inclusion in the systematic review. Any disputes were solved by discussion or by a third reviewer (CMD).

Viral culture detects viable virus that is relevant for transmission but is available in research settings only. Since RT-PCR tests are more widely available and SARS-CoV-2 RNA (as reflected by RT-PCR cycle threshold [Ct] value) highly correlates with SARS-CoV-2 antigen quantities, we considered it an acceptable reference standard for the purposes of this systematic review [ 16 ]. It is of note that there is currently no international standard for the classification of viral load available.

Ag-RDTs for SARS-CoV-2 aim to detect infection by recognizing viral proteins. Most Ag-RDTs use specific labeled antibodies attached to a nitrocellulose matrix strip, to capture the virus antigen. Successful binding of the antibodies to the antigen either is detected visually (through the appearance of a line on the matrix strip [lateral flow assay]) or requires a specific reader for fluorescence detection. Microfluidic enzyme-linked immunosorbent assays have also been developed. Ag-RDTs typically provide results within 10 to 30 minutes [ 6 ].

We excluded studies in which patients were tested for the purpose of monitoring or ending quarantine. Also, publications with a population size smaller than 10 were excluded. Although the size threshold of 10 is arbitrary, such small studies are more likely to give unreliable estimates of sensitivity and specificity.

We included studies evaluating the accuracy of commercially available Ag-RDTs to establish a diagnosis of SARS-CoV-2 infection, against reverse transcription polymerase chain reaction (RT-PCR) or cell culture as reference standard. We included all study populations irrespective of age, presence of symptoms, or study location. We considered cohort studies, nested cohort studies, case–control or cross-sectional studies, and randomized studies. We included both peer-reviewed publications and preprints.

We performed a search of the databases PubMed, Web of Science, medRxiv, and bioRxiv using search terms that were developed with an experienced medical librarian (M. Grilli) using combinations of subject headings (when applicable) and text-words for the concepts of the search question. The main search terms were “Severe Acute Respiratory Syndrome Corona-virus 2,” “COVID-19,” “Betacoronavirus,” “Coronavirus,” and “Point of Care Testing.” The full list of search terms is available in S2 Text . We also searched the Foundation for Innovative New Diagnostics (FIND) website ( https://www.finddx.org/sarscov2-eval-antigen/ ) for relevant studies manually. We performed the search up until 30 April 2021. No language restrictions were applied.

We developed a study protocol following standard guidelines for systematic reviews [ 14 , 15 ], which is available in S1 Text . We also completed the PRISMA checklist ( S1 PRISMA Checklist ). Furthermore, we registered the review on PROSPERO (registration number: CRD42020225140).

Results

Methodological quality of studies The findings on study quality using the QUADAS-2 tool are presented in Figs 2 and 3. In 190 (88.8%) datasets a relevant patient population was assessed. However, for only 44 (20.6%) of the datasets was patient selection considered representative of the setting and population chosen (i.e., they avoided inappropriate exclusions and a case–control design, and enrollment occurred consecutively or randomly). PPT PowerPoint slide

PNG larger image

TIFF original image Download: Fig 2. Methodological quality of the clinical accuracy studies: Risk of bias. Proportion of studies with low, intermediate, high, or unclear risk of bias (percent). https://doi.org/10.1371/journal.pmed.1003735.g002 PPT PowerPoint slide

PNG larger image

TIFF original image Download: Fig 3. Methodological quality of the clinical accuracy studies: Applicability. Proportion of studies with low, intermediate, high, or unclear concerns regarding applicability (percent). https://doi.org/10.1371/journal.pmed.1003735.g003 The conduct and interpretation of the index tests was considered to have low risk for introduction of bias in 113 (52.8%) datasets (through, e.g., appropriate blinding of persons interpreting the visual readout). However, for 99 (46.3%) datasets, sufficient information to clearly judge the risk of bias was not provided. In only 89 (41.6%) datasets were the Ag-RDTs performed according to IFU, while 100 (46.7%) were not IFU-conforming, potentially impacting the diagnostic accuracy (for 25 [11.7%] datasets the IFU status was unclear). In 81 (37.9%) datasets, the reference standard was performed before the Ag-RDT, or the operator conducting the reference standard was blinded to the Ag-RDT results, resulting in a low risk of bias. In almost all other datasets (132/61.7%), this risk could not be assessed due to missing data. The applicability of the reference test was judged to be of low concern for all datasets, as cell culture and RT-PCR are expected to adequately define the target condition. In 209 (97.7%) datasets, the sample for the index test and reference test were obtained at the same time, while this was unclear in 5 (2.3%) datasets. All samples included in a dataset were subjected to the same type of RT-PCR in 145 (67.8%) datasets, while different types of RT-PCR were used within the same dataset in 50 (23.4%) datasets. For 19 (8.9%) datasets, it was unclear. Furthermore, for 11 (5.1%) datasets, there was a concern that not all selected patients were included in the analysis. Finally, 32 (24.1%) of the studies received financial support from the Ag-RDT manufacturer, and in another 9 (6.8%) studies, employment of the authors by the manufacturer of the Ag-RDT studied was indicated. Overall, a competing interest was found in 33 (24.8%) of the studies.

Detection of SARS-CoV-2 infection Out of 214 clinical datasets (from 124 studies), 20 were excluded from the meta-analysis because they included fewer than 20 RT-PCR positive samples. A further 21 datasets were missing either sensitivity or specificity and were only considered for univariate analyses. Across the remaining 173 datasets, including any test and type of sample, the pooled sensitivity and specificity were 71.2% (95% CI 68.2% to 74.0%) and 98.9% (95% CI 98.6% to 99.1%), respectively. If testing was performed in conformity with IFU, sensitivity increased to 76.3% (95% CI 73.1% to 79.2%), while non-IFU-conforming testing had a sensitivity of 65.9% (95% CI 60.6% to 70.8%). Pooled specificity was similar in both groups (99.1% [95% CI 98.8–99.4%] and 98.3% [95% CI 97.7% to 98.8%], respectively).

Comparison with analytical studies The 9 analytical studies were divided into 63 datasets, evaluating 23 different Ag-RDTs. Only 7 studies reported a sample size, for which 833 (90.6%) samples originated from NP swabs, while for 86 (9.4%) the sample type was unclear. One of the 2 studies not reporting sample size used saliva samples [198], while the other used the sample type specified in the respective Ag-RDT’s IFU [173]. Overall, the reported analytical sensitivity (limit of detection [LOD]) in the studies resembled the results of the meta-analysis presented above. Rapigen (LOD, in log10 copies per swab: 10.2) and Coris (LOD 7.46) were found to perform worse than Panbio (LOD 6.6 to 6.1) and Standard Q (LOD 6.8 to 6.0), whereas Clinitest (LOD 6.0) and BinaxNOW by Abbott (LOD 4.6 to 4.9) performed better [191,256,282]. Similar results were found in another study, where Standard Q showed the lowest LOD (detecting virus up to what is an equivalent Ct value of 26.3 to 28.7), compared to that of Rapigen and Coris (detecting virus up to what is an equivalent Ct value of only 18.4 for both) [208,274,275]. However, another study found Panbio, Standard Q, Coris, and BinaxNOW to have a similar LOD values of 5.0 × 103 plaque forming units (PFU)/mL, but the ESPLINE SARS-CoV-2 by Fujirebio (Japan), the COVID-19 Rapid Antigen Test by Mologic (UK), and the Sure Status COVID-19 Antigen Card Test by Premier Medical Corporation (India) performed markedly better (LOD 2.5 × 102 to 5.0 × 102 PFU/mL) [173]. An overview of all LOD values reported in the studies can be found in S3 Table.

[END]

[1] Url: https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1003735

(C) Plos One. "Accelerating the publication of peer-reviewed science."
Licensed under Creative Commons Attribution (CC BY 4.0)
URL: https://creativecommons.org/licenses/by/4.0/


via Magical.Fish Gopher News Feeds:
gopher://magical.fish/1/feeds/news/plosone/

You are viewing proxied material from magical.fish. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.