Usefulness of a Quantitative Olfactory Test for the Detection of COVID-19

Marcos A Lessa^1,2,3*, Stella M Cotta-Pereira¹, Frederico R Ferreira¹, Therezinha M Castiñeiras⁴, Rafael M Galliez⁴, Débora S. Faffe⁴, Isabela de C Leitão⁴, Diana Mariani⁵, Erica R Nascimento⁶, Flávia S Lessa⁷, Isabella B Succi⁷, and Carlos A Pedreira⁸

¹Laboratory of Clinical and Experimental Pathophysiology, Oswaldo Cruz Institute - Fiocruz, Rio de Janeiro, Brazil

²Department of General Surgery, Anesthesiology Division, Faculty of Medical Sciences, State University Rio de Janeiro, Rio de Janeiro, Brazil

³Extension and Research Teaching Center, Rector Hesio Cordeiro University Hospital, Rio de Janeiro State University, Cabo Frio, Brazil

⁴Department of Infectious and Parasitic Diseases, Faculty of Medicine, Federal University of Rio de Janeiro, Rio de Janeiro, Brazil

⁵Department of Genetics, Institute of Biology, Federal University of Rio de Janeiro, Rio de Janeiro, Brazil

⁶Laboratory of Molecular Virology, Institute of Biology, Federal University of Rio de Janeiro, Rio de Janeiro, Brazil

⁷Occupational Health Center, General People Coordination, Oswaldo Cruz Foundation, Rio de Janeiro, Brazil

⁸Systems and Computer Engineering Program, Alberto Luiz Coimbra Institute of Postgraduate Studies and Engineering Research, Federal University of Rio de Janeiro, Rio de Janeiro, Brazil

^*Corresponding author: Marcos A Lessa, Department of Surgery, Discipline of Anesthesiology, Faculty of Medical Sciences, State University Rio de Janeiro, Rio de Janeiro, Brazil.

Received: 20 December 2023; Accepted: 01 January 2024; Published: 01 August 2024

Introduction

Before the advent of the SARS-CoV-2 vaccine and large-scale immunization of the population worldwide, coronavirus disease 2019 (COVID-19) caused severe injury and death. It quickly became a pandemic, as the transmission of the virus often occurred before the onset of symptoms. To further complicate matters, the symptoms associated with COVID-19 were variable, and many could be mild and hard to identify objectively or even not present at all (asymptomatic). Thus, it was a challenge for science and medicine to stop rapid viral transmission and identify suspected cases of this wide range of symptoms. Among the clinical manifestations of COVID-19, olfactory dysfunction (OD), which consists of anosmia or hyposmia, was frequently reported by many patients, and studies based mainly on patient surveys have indicated that OD is a prevalent and early symptom of SARS-CoV-2 infection [1-4]. In contrast to nasal congestion typically found in viral upper respiratory tract infections, COVID-19-induced OD is associated with the presence of the virus in cells adjacent to olfactory neurons [5] and, by a mechanism that, until recently, has not been fully elucidated, causes alterations in the odor perception function of olfactory neurons [5]. The rapid onset of anosmia is highly suggestive of SARS-CoV-2 viral infection, and in many cases of COVID-19, OD and ageusia are the only presenting symptoms [3,4,6]. However, OD, especially hyposmia, may be unnoticed unless it is formally and objectively tested with a measurable olfactory test. There is a wide range for the prevalence of COVID-19-associated OD in the literature, from 15% to 85%, and most of the data are from subjective evaluations of patients' self-reports of loss of smell [1,7-9]. The use of an olfactory test to diagnose COVID-19 infection has been reported in different studies during the pandemic [7,9-13]. However, nearly all those studies only tested the OD in PCR-positive patients. Thus, the test's specificity and degree of association with the disease (full odds ratio) are unclear. Similarly, how quantitative olfactory tests compare to temperature testing, a quantifiable symptom commonly used for COVID-19 screening, is unclear.

Population testing is a crucial strategy for efficiently identifying and isolating suspected cases, mainly because of the high infectivity of COVID-19. The principal tools for diagnosing COVID-19 are reverse transcriptase-polymerase chain reaction (RT-PCR) and antigen tests. Although PCR is the gold standard with excellent sensitivity and specificity, it is costly. It requires sample collection by others, special handling, instrumentation, and analysis—often at distant locations—which can cause significant delays. These issues pose challenges for very wide-scale population testing and are largely confined to testing a subpopulation of suspected infected people. There is a global need for an alternative, affordable, reliable, albeit imperfect, test to be used on a large scale to identify infected individuals and block transmission more easily. Here, we compared a scratch-and-sniff-style smell identification test (SIT) that has five odorant windows on a single card as a potentially quick, inexpensive, and easy prescreen test for COVID-19 in symptomatic patients, with RT-PCR as the reference standard to investigate the feasibility of using a quantifiable olfactory test to help identify COVID-19 infection. Additionally, we examined the effectiveness of infrared forehead temperature screens in the same patient cohort.

Methods

The study was conducted at the Center for COVID-19 Diagnosis of the Federal University of Rio de Janeiro (UFRJ). A total of 165 individuals who presented with mild cold-like symptoms at our center were enrolled from June to August 2020. Nasopharyngeal swabs were obtained from each participant, and a diagnosis of COVID-19 was made via RT-PCR using the CDC protocol, with primers and probes for N1 and N2 targets. Clinical and demographic data were self-reported by the patients. The study was approved by the local ethics committee of Clementino Fraga Filho University Hospital (CAAE: 30161620.0.0000.5257). Written informed consent was obtained from all participants. The inclusion criterion was symptomatic outpatients older than 18 years who were scheduled to have their nasopharyngeal swabs collected for PCR at the UFRJ testing facility. Patients who had rhinorrhea or nasal congestion were excluded, and eligible volunteers whose forehead temperature was checked with an infrared noncontact thermometer (temperature gun) were excluded. After that, the volunteers received two u-Smell-it™ (Connecticut, USA) olfactory SIT cards (5 scents each) and were asked to scratch and sniff each of the ten areas containing the microencapsulated fragrances and mark the best choice of 5 options (4 scent choices and ‘no scent’) on a response card. In our protocol, three versions of the test were used. Cards #1414 and #1515 have the same scents; however, they are presented in different orders, and card #1313 has different scents from #1414 and #1515—supplemental Figures 1 and 2 present examples of card tests and response cards, respectively. For the first protocol, we used one card, #1313, in combination with another card, #1414 or #1515, to have 10 different smells presented to the patients. For the second protocol (reproducibility analysis), we used #1414 and #1515 cards to have 2 smells repeated on both cards. Supplemental Table 1 shows the scent options and the right response for all cards used in this study.

Figure 1: Sensitivity, specificity, and ROC curve for u-Smell-it™ using two cards (10-scent).

Figure 2: Sensitivity, specificity and ROC curve for the use of u-Smell-it™ as a single card (5-scent).

Table 1: Demographic data.

For tests with two olfactory cards (10 scents), the optimal cutoff to most accurately distinguish COVID-19-positive and COVID-19-negative patients, as determined by RT-PCR, was achieved using £5 correct responses (‘hits’) scored as ‘positive’ for loss of smell (LOS); ³ 6 hits were considered negative for LOS. Figure 1 shows the sensitivity and specificity for different cutoffs and the ROC curve for the test using two cards. To evaluate whether the use of a single olfactory card (5 scents) presented similar results to the use of two olfactory cards (10 scents), we used a bootstrap statistical procedure (see below) to simulate the use of a single card and determine the statistical results. Figure 2 shows the sensitivity and specificity for different cutoffs after the bootstrap statistical procedure and the ROC curve. With the goal of testing for a positive association between the u-Smell-itÔ olfactory test and RT-PCR-based COVID-19 diagnosis, two binary variables were considered, one for the tested scheme and another for the gold/reference standard diagnosis. We constructed contingency tables and calculated the sensitivity, specificity, and positive and negative predictive values. We also applied the chi-square Pearson test [14,15] to demonstrate that the proposed method does not produce an independent outcome compared to the gold standard diagnosis. Bootstrap [16] is a statistical procedure that creates many simulated samples by resampling the dataset. The parameters are estimated by averaging the results of all runs. Here, we randomly sorted 5 out of 10 odorants for each patient and calculated the parameters after passing through the complete dataset. This procedure was repeated 10,000 times for the whole dataset, generating 10,000 values for each parameter. New random draws were performed for each patient in all the runs. In the end, the average of these 10,000 runs was used to estimate the parameters and confidence intervals, as calculated through the 2.5th and 97.5th percentiles. For reproducibility, a two-branch experiment was run with five odorants in each for the same group of individuals to evaluate differences between branch responses. The Wilcoxon signed-rank test was used for post hoc analyses. All the data were statistically analyzed using MATLAB software (R2019b) and confirmed with MedCalc® software.

Results

One hundred sixty-five patients were eligible for the olfactory test and nasopharyngeal swab collection for PCR. Five patients were excluded because of inconclusive PCR results (n=2) or missing data (n=3). A total of 160 patients completed the full protocol. The PCR positivity rate for COVID-19 was 27.5% (n=44). The demographic data are shown in Table 1. The forehead temperature check did not reveal any fever (0/165), which was defined as a temperature of 37.8°C or above. The frequency distribution of the temperatures is shown in Figure 3. The contingency table and the statistical analysis of the olfactory test performance using two u-Smell-itÔ cards (10 scents) compared with the RT-PCR results for SARS-CoV-2 are shown in Tables 2 and 3, respectively. Olfactory testing showed a specificity of 94.8% (95% CI, 89.1–98.1), sensitivity of 47.7% (95% CI, 32.7–63.3), positive predictive value of 77.8% (95% CI, 59.6–88.8), negative predictive value of 82.7% (85% CI, 78.7–86.7), accuracy of 82% (95% CI, 75–87), and odds ratio of 16.7. The contingency table and the statistical analysis of self-reports of LOS performance compared with RT-PCR are shown in Tables 4 and 5, respectively. Although the sensitivity and PPV of self-reports of the LOS test were greater than those of the u-Smell-itÔ quantitative olfactory test, the specificity, NPV, and accuracy of self-reports were lower. Interestingly, using an RT-PCR test as a reference standard, the odds ratio for olfactory dysfunction, as determined by the u-Smell-itÔ olfactory quantitative test, was 3.6-fold greater than the odds ratio of self-reported LOS (4.6).

Figure 3: Frequency distribution of the forehead temperature of patients according to RT-PCR results.

Table 2: u-Smell-itÔ olfactory test (two cards) vs RT-PCR.

Table 3: Statistical findings of the u-Smell-it^TM olfactory test (two cards) vs RT-PCR

Table 4: Self-reports of loss of smell vs RT-PCR.

Table 5: Statistical findings when comparing self-reports of LOS to PCR.

To evaluate the reproducibility of the quantitative olfactory test, another group of 66 patients was tested with two u-Smell-itÔ cards containing the same five scents but arranged in a different order. Forty-one out of 66 tested individuals produced the same number of correct hits in both branches; 23 had a divergence of just one scent, and two patients mismatched two or more scents. Wilcoxon signed rank analysis produced a p value= 0.58, indicating that there is no evidence that the ‘smells’ were perceived as diverse in the two branches of tests. Subjects did not report any side effects or complaints about test participation. Most patients took approximately 90 sec or less to complete the test with two olfactory cards (10 scents).

Discussion

The present study showed that a large fraction of SARS-CoV-2 PCR-positive patients had olfactory dysfunction, and this symptom, when detected by an olfactory test, was highly specific (95%). Our results demonstrated that the odds ratio of the u-Smell-itÔ test for detecting COVID-19 infection was 3.6-fold greater than that of self-reports, suggesting that a quantitative test outperforms self-surveys. These findings were observed in the aggregate and after using a cutoff score of three on a 6-point scale (LOS = score 0, 1, 2 out of 5 maximum). The results indicate that a simple 5- or 10-window olfactory smell identification test can specifically differentiate, with only approximately 5% false positives, people infected by SARS-CoV-2 with 82% accuracy. Interestingly, the statistical performance metrics of the five-window test were similar to those of the 10-window test, which is consistent with the relatively intense loss of smell in our study population. Dramatically, in contrast to olfactory tests, temperature checking failed to detect any cases of COVID-19 infection. This study underscores this weakness and likely the limited usefulness of using temperature tests and infrared camera monitoring to identify persons infected with COVID-19. Our results showed that none of the 44 patients who tested positive for SARS-CoV-2 had a fever (typically a temperature greater than 38°C). Notably, fever is a nonspecific symptom of viral infection, and because of that, the usefulness of temperature screening for identifying suspected cases has been called into question [17–20]. Nevertheless, body temperature checks are routinely applied as the primary screening test to identify individuals with fever at the entrance to many public places, such as schools, airports, hospitals, etc.

Notably, the literature shows that only approximately 20% of people who have COVID-19 have a fever, which generally occurs very early in the course of the infection and has a short duration (under three days) [21]. The body temperature of our SARS-CoV-2-positive patients was not significantly different from that of SARS-CoV-2-negative patients (Table 1 and Figure 1), and one possible explanation is that the outpatients observed in our study population may have arrived after transient fever. It also shows the poor performance of temperature-based screens and further questions their usefulness as a screening test. Our study tested two different symptoms of COVID-19 infection (OD and temperature) and provided strong evidence that an olfactory test would significantly outperform a temperature test. Another key finding of the study was that an olfactory test significantly outperformed self-reported LOS. Specifically, for patients who were positive for COVID-19 according to RT-PCR, the odds ratio for self-reports was only 4.6, while for patients who were positive according to a real olfactory test, it was 16.7. This result suggested that patients with OD who were diagnosed using olfactory tests were more strongly associated (3.6×) with a positive test for COVID-19 than with self-reported LOS. This finding is consistent with other reports in which an objective test outperformed nonobjective and nonquantitative surveys [22]. Patients who did not have a LOS data included those with both anosmia and hyposmia.

Our results showed that the olfactory test specificity for detecting OD in patients infected with COVID-19 was very high (~95%), which was consistent with other reports [23], including the CDC [24], indicating that "a new loss of taste or smell" was the single best indicator symptom of COVID-19, as indicated by the odds ratio. Although OD is acknowledged by many research teams and listed on the WHO website as a finding consistent with SARS-CoV-2 infection, little attention has been given to whether OD is a better indicator of COVID-19 than other symptoms. Considering this symptom's high specificity, the OD should be much better suited for rapid screening than the ubiquitous temperature test. The journal STAT News proposed that, for screening suspicious cases of COVID-19 infection, a smell test would be a better option than temperature tests [25]. The sensitivity of the test with u-Smell-itÔ (ranging from 48% in the 10-scent test to 55% in the 5-scent test) was lower than expected, considering other reports that showed OD detection via smell testing with a sensitivity of 76% (51–91%)²³. Given that no patients had fever in our population and that the average duration of loss of smell was reported to be approximately 7 to 8 days [11], one possible interpretation is that some patients may have arrived after recovering from LOS. Consistent with this possibility, no patients had an elevated temperature above baseline. Second, for self-reported LOS, the sensitivity was greater; however, the specificity was very poor. Considering that the average duration of LOS is only approximately one week (note that a subset of patients (~10%) can have a longer-term loss) and that RT-PCR positivity may persist for three weeks or more, it is conceivable that, in our study, some patients were tested for OD after recovery; this is consistent with the finding indicating that they had a recent LOS on the questionnaire but showing standard performance in an olfactory test. However, properly addressing this issue would require longitudinal testing or other mechanisms, such as serology, to accurately identify disease staging.

The study's limitations include that we recruited only outpatients who came to the clinic for a diagnostic test at variable points of the disease, which, as mentioned above, we have not ascertained, and we did not investigate asymptomatic patients. Moreover, our cohort did not include children, pregnant individuals, or elderly individuals, so our results may not be directly generalizable to these groups of patients. It was anticipated that a short 5- or 10-odorant smell identification test for patients with minor hyposmia may be missed compared to a 40-odorant UP-SIT test [11,26].

In conclusion, our study demonstrated that a positive SARS-CoV-2 RT-PCR result was strongly associated with olfactory dysfunction, which was 3.6-fold greater in patients who were tested with a short olfactory test than in those who were detected via self-reports. The results of the present study suggest that quick olfactory tests may be useful for detecting COVID-19 infection in symptomatic patients.

Funding

CNPq: Brazilian National Research Council

FAPERJ: Rio de Janeiro Research Support Foundation

References

1. Zahra SA, Iddawela S, Pillai K, et al. Can symptoms of anosmia and dysgeusia be diagnostic for COVID-19? Brain Behav 10 (2020): e01839.
2. Sayin I, Yasar KK, Yazici Taste and Smell Impairment in COVID-19: An AAO-HNS Anosmia Reporting Tool-Based Comparative Study. Otolaryngol-Head Neck Surg Off J Am Acad Otolaryngol-Head Neck Surg 163 (2020): 473-479.
3. Mohindra R, Sainath KG, Kanta P, et al. Anosmia and ageusia as presenting complaints of coronavirus disease (COVID-19) J Fam Med Prim Care 9 (2020): 4406-4408.
4. Lechner M, Chandrasekharan D, Jumani K, et al. Anosmia as a presenting symptom of SARS-CoV-2 infection in healthcare workers - A systematic review of the literature, case series, and recommendations for clinical assessment and management. Rhinology 58 (2020): 394-399.
5. Brann DH, Tsukahara T, Weinreb C, et al. Non-neuronal expression of SARS-CoV-2 entry genes in the olfactory system suggests mechanisms underlying COVID-19-associated anosmia. Sci Adv 6 (2020).
6. Zhang Q, Shan KS, Abdollahi S, et al. Anosmia and Ageusia as the Only Indicators of Coronavirus Disease 2019 (COVID-19). Cureus 12 (2020): e7918.
7. Moein ST, Hashemian SM, Mansourafshar B, et al. Smell dysfunction: a biomarker for COVID-19. Int Forum Allergy Rhinol 10 (2020): 944-950.
8. Lechien JR, Chiesa-Estomba CM, De Siati DR, et al. Olfactory and gustatory dysfunctions as a clinical presentation of mild-to-moderate forms of the coronavirus disease (COVID-19): a multicenter European study. Eur Arch Oto- Rhino-Laryngol Off J Eur Fed Oto-Rhino-Laryngol Soc EUFOS Affil Ger Soc Oto- Rhino-Laryngol - Head Neck Surg 277 (2020): 2251-2261.
9. Lechien JR, Chiesa-Estomba CM, Place S, et al. Clinical and epidemiological characteristics of 1420 European patients with mild-to-moderate coronavirus disease 2019. J Intern Med 288 (2020): 335-344.
10. Lee JM, Lee SJ. Olfactory and Gustatory Dysfunction in a COVID-19 Patient with Ankylosing Spondylitis Treated with Etanercept: Case Report. J Korean Med Sci 35 (2020): e201.
11. Lee Y, Min P, Lee S, et al. Prevalence and Duration of Acute Loss of Smell or Taste in COVID-19 Patients. J Korean Med Sci 35 (2020): e174.
12. Yan CH, Faraji F, Prajapati DP, et al. Self-reported olfactory loss associates with outpatient clinical course in COVID-19. Int Forum Allergy Rhinol 10 (2020): 821-831.
13. Yan CH, Faraji F, Prajapati DP, et al. Association of chemosensory dysfunction and COVID-19 in patients presenting with influenza- like symptoms. Int Forum Allergy Rhinol 10 (2020): 806-813.
14. Fisher On the interpretation of χ2fromcontingency tables and the calculation of P. J R Stat Soc 85 (1922): 87-94.
15. Pearson KX. On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. Lond Edinb Dublin Philos Mag J Sci 50 (1900): 157-175.
16. Bradley Efron. 5. The Bootstrap. In: The Jackknife, the Bootstrap and Other Resampling Plans. CBMS-NSF Regional Conference Series in Applied Mathematics. Society for Industrial and Applied Mathematics (1982): 27–36.
17. Buoite Stella A, Manganotti P, Furlanis G, et al. Return to school in the COVID-19 era: considerations for temperature J Med Eng Technol 44 (2020): 468-471.
18. Mitra B, Luckhoff C, Mitchell RD, et al. Temperature screening has negligible value for control of COVID-19. Emerg Med Australas EMA 32 (2020): 867-869.
19. Chen H-Y, Chen A, Chen Investigation of the Impact of Infrared Sensors on Core Body Temperature Monitoring by Comparing Measurement Sites. Sensors 20 (2020).
20. Quilty BJ, Clifford S, Flasche S, et al. Effectiveness of airport screening at detecting travellers infected with novel coronavirus (2019-nCoV). Euro Surveill Bull Eur Sur Mal Transm Eur Commun Dis Bull 25 (2020).
21. O’Reilly GM, Mitchell RD, Rajiv P, et al. Epidemiology and clinical features of emergency department patients with suspected COVID-19: Initial results from the COVID-19 Emergency Department Quality Improvement Project (COVED-1). Emerg Med Australas EMA 32 (2020): 638-645.
22. Agyeman AA, Chin KL, Landersdorfer CB, et al. Smell and Taste Dysfunction in Patients With COVID-19: A Systematic Review and Meta-analysis. Mayo Clin Proc 95 (2020): 1621-1631.
23. Pang KW, Chee J, Subramaniam S, et al. Frequency and Clinical Utility of Olfactory Dysfunction in COVID-19: a Systematic Review and Meta-analysis. Curr Allergy Asthma Rep 20 (2020): 76.
24. Payne DC, Smith-Jeffcoat SE, Nowak G, et al. SARS-CoV-2 Infections and Serologic Responses from a Sample of U.S. Navy Service Members-USS Theodore Roosevelt, April 2020. Morb Mortal Wkly Rep 69 (2020): 714-721.
25. Fever checks can’t catch all Covid-19 cases. Smell tests might help. STAT-Health (2020).
26. Moein ST, Hashemian SM, Tabarsi P, et al. Prevalence and reversibility of smell dysfunction measured psychophysically in a cohort of COVID-19 patients. Int Forum Allergy Rhinol 10 (2020): 1127-1135.

Supplementary Files

Supplemental Figure 1: Example of a scratch-and-sniff (u-Smell-it™) olfactory test.

Supplemental Figure 2: Example of a response card to the olfactory test.

Supplemental Table 1: Scent options and the right response for all cards used in this study.