Advertisement
Research Article| Volume 10, 100461, 2023

Detection and diagnosis of posterior circulation calcification: An interobserver agreement study

Open AccessPublished:December 08, 2022DOI:https://doi.org/10.1016/j.ejro.2022.100461
Detection and diagnosis of posterior circulation calcification: An interobserver agreement study
Next ArticleImaging biomarkers to stratify lymph node metastases in abdominal CT – Is radiomics superior to dual-energy material decomposition?

      Abstract

      Background

      Posterior cerebral ischemic stroke and its underlying causes can be easily misdiagnosed in routine practice. Therefore, more than a third of positive cases can be easily missed during routine CT image reporting unless expert neuroimaging radiologists carefully report it.

      Objective

      To assess the inter-rater agreement level between senior residents and general radiologists and a specialized expert neuroradiologist when detecting and diagnosing posterior cerebral circulation calcification.

      Methods

      This was a cross-sectional observational study. A total of fifteen senior radiology residents (SRRs) and fifteen general radiologists (GRs) at four different hospitals in Riyadh, Saudi Arabia, were included in the study. A total of four CT-scanned brain cases with the presence of posterior circulation calcification (PCC) with different degrees of severity and one brain case with negative PCC were selected. These cases were predefined by expert neuroradiologists at our center. The cases were uploaded into the picture archiving and communication systems (PACS) at four different centers as outsider cases. These cases were then randomly assigned to the participating SRRs and GRs for reporting. All radiologists were blinded to the findings of the cases. Inter-observer agreement was assessed using the weighted kappa coefficient of agreement (k) between the two groups.

      Results

      The cerebral calcification misdiagnosis rate for the SRRs and GRs was > 93% for most of the positive cases. There was 1) poor inter-observer agreement between the SRRs and GRs for the detection of severe posterior cerebral calcification(PCC) in a negative stroke case (agreement for misdiagnosis, k = 0.93; correct diagnosis, k = 0.00), 2) poor inter-observer agreement for mild PCC in a negative stroke case (agreement for misdiagnosis, k = 0.93; correct diagnosis, k = 0.00), 3) moderate PCC in a positive posterior stroke case (agreement for misdiagnosis, k = 0.92; correct diagnosis, k = 0.00), and 4) poor interobserver agreement for severe PCC in a positive posterior cerebral stroke case (agreement for misdiagnosis, k = 0.846; correct diagnosis, k = 0.00). There was excellent agreement between the SRRs and GRs when reporting negative cases of PCC and stroke.

      Conclusion

      Our study concluded that most of the SRRs and GRs missed the diagnosis of posterior cerebral calcification in the presented cases.

      Abbreviations:

      PCC (Posterior circulation calcification), SRR (Senior Radiology Resident), GR (General Radiologist), NECT (Non-enhanced computed tomographic), CT (Computed tomography), PACS (Picture archiving communicating system)

      Keywords

      1. Introduction

      Stroke is considered one of the main reasons for death and disability globally [
      • Venkat A.
      • et al.
      Factors associated with stroke misdiagnosis in the emergency department: a retrospective case-control study.
      Neuroepidemiology. 2018; 51 (123–27)
      ]. A stroke can occur in the anterior or posterior part of the brain. The diagnosis of the anterior stroke diagnosis is well established among radiologists and mostly seen via cross-section imaging techniques. The definition of a posterior cerebral ischemic stroke is an infarction within vascular regions that receive blood supply from the vertebrobasilar arterial system [
      • Nouh A.
      • Remke J.
      • Ruland S.
      Ischemic posterior circulation stroke: a review of anatomy, clinical presentations, diagnosis, and current management.
      Front. Neurol. 2014; 5: 1-16
      ]. Most posterior cerebral ischemic strokes are caused by arterial atherosclerosis and calcification. Unfortunately, posterior cerebral ischemic stroke and its underlying causes can be easily misdiagnosed. Therefore, more than a third of positive cases can be easily missed during routine CT image reporting unless expert neuroimaging radiologists carefully report it.
      Furthermore, according to Banerjee, Stone, and Werring (2018), posterior brain cerebral circulation strokes are missed three times more than anterior circulation strokes [
      • Banerjee G.
      • Stone S.P.
      • Werring D.J.
      Posterior circulation ischaemic stroke.
      BMJ. 2018; 361: 1-7
      ]. Arterial calcification is considered a significant risk factor for ischemic stroke [
      • Gökçal E.
      • et al.
      The association of vertebrobasilar calcification with etiological subtypes, stroke recurrence and outcome in acute brainstem ischemic stroke.
      Neurol. Neurochir. Pol. 2018; 52 (188–93)
      ]. Most stroke patients develop ischemic symptoms, including sensory loss, visual field defects, and vertigo [
      • Banerjee G.
      • Stone S.P.
      • Werring D.J.
      Posterior circulation ischaemic stroke.
      BMJ. 2018; 361: 1-7
      ]. Vertebrobasilar artery calcification is commonly attributed to hypertension, age, smoking, and diabetes [
      • Van Der Toorn J.E.
      • Engelkes S.R.
      • Kamran Ikram M.
      • Arfan Ikram M.
      • Vernooij M.W.
      • Kavousi M.
      • Bos D.
      Vertebrobasilar artery calcification: prevalence and risk factors in the general population.
      Atherosclerosis. 2019; : 46-52https://doi.org/10.1016/J.Atherosclerosis.2019.05.001
      ].
      Non-enhanced computed tomographic (NECT) scanning is a superior technology that can detect vascular calcification [
      • Venkat A.
      • et al.
      Factors associated with stroke misdiagnosis in the emergency department: a retrospective case-control study.
      Neuroepidemiology. 2018; 51 (123–27)
      ]. Regardless of its drawbacks, NECT is widely available and a fast diagnostic modality, which enforces its common application for the detection of ischemic and hemorrhagic stroke [
      • Jensen-Kondering U.
      Hyperdense artery sign on computed tomography in acute ischemic stroke.
      World J. Radiol. 2010; 2: 354
      ]. However, interpreters can miss posterior cerebral circulation calcification (PCC) for several reasons, such as a lack of knowledge and experience, as well as interpreting too many cases during long and variable duties. From observing numerous brain CT scan reports for patients examined between 2015 and 2021 in our center, we observed that most positive PCC cases were not noted or mentioned in corresponding reports. Therefore, it was essential to address this issue in a scientific manner and examine the medical interpreters’ abilities while blinded to the reporting of the positive and negative PCC cases.
      It was assumed that PCC detection, diagnosis, and interpretation by cross-sectional imaging could be missed by both SRRs and GRs. Therefore, the purpose of this study was to determine how frequently PCC is misdiagnosed by SRRs and GRs because they are the first interpreters of CT brain scans in emergency departments. Moreover, we also assessed the PCC detection rates of SRRs, and GRs compared to an expert neuroimaging radiologist.

      2. Materials and methods

      2.1 Study design

      The study was conducted at four radiology departments in Riyadh, Saudi Arabia. Ethical approval was obtained from the Institutional Review Board of King Saud University Medical City. The study design was cross-sectional and observational, where the researchers sought consecutive patients with posterior strokes and PCC cases that required interpretation and definition by expert neuroradiologists.

      2.2 Patient selection and review of radiology reports

      Four plain CT-scanned brain cases with positive PCC and one CT-scanned brain with negative posterior circulation calcification were selected and defined by two senior expert radiologists at our center. The cases were uploaded into the (PACS) of four different centers as outsider cases. These cases were introduced into routine radiology cases and randomly assigned to fifteen GRs and fifteen SRRs for daily reporting. All the participating radiologists were blinded to all cases and unaware of the previous reports. Their reports were collected, and the participants were later interviewed regarding the missing diagnosis of posterior circulation calcification.

      2.3 Degree of PCC in the preselected cases

      The degrees of PCC for the preselected cases were 1) “severe” for the first case without posterior ischemic stroke, 2) “mild” for the second case without the presence of ischemic stroke, 3) “moderate” for the third case with the presence of posterior cerebral stroke, and 4) “severe” for the fourth case with posterior cerebral stroke. The fifth case was free of calcification and ischemic stroke.

      2.4 Scan technique and interpretation

      The brain imaging was undertaken on either a 64-detector CT scanner (Discovery CT750 HD; GE Healthcare, Chicago, IL) using a conventional CT technique (120 kV, 90 mAs, and 0.5 slice thickness) or a dual-source CT scanner (Somatom Definition Flash; Siemens; Germany) using a similar conventional CT technique (120 kV, 250 mAs, and 0.5 slice thickness).

      2.5 Statistical analysis

      All statistical analyses were carried out using SPSS software, Version 27.0.0.0 (IBM Corp., Armonk, NY). A descriptive analysis of the experience of the participating groups was provided. The percentage of participants missing the diagnoses among both groups was also provided. To determine interobserver agreement for the detection of PCC, a weighted kappa statistic was used for the analysis of interobserver agreement between the SRRs and GRs. All weighted kappa value assessments were presented based on the agreement of a misdiagnosis and a correct diagnosis.

      3. Results

      The SRRs group had a three-year mean for radiological experience, and the GRs group included board-certified radiologists with a seven-year mean for radiological experience. Of all the participating radiologists, 76.7% missed diagnosing and reporting PCC, 16.7% reported only one case with posterior calcification out of four cases, and 6.7% reported three cases with PCC out of the four cases (see Table 1).
      Table 1Number of participating radiologists.
      Detected CalcificationNo. of Participating RadiologistsPercentage
      02376.7
      1516.7
      326.7
      Total30100
      The detection rate was slightly higher for the GRs group: 73.3% missed reporting calcification, 20% detected only one case, and 6.7% detected three cases. Of the SRRs, 80% missed detecting and reporting PCC, and 13.3% reported a single case. Like the GR, 6.7% of the SRRs detected three cases with PCC (p = 0.003; see Table 2).
      Table 2Comparison of the detection rate of PCC between GRs and SRRs.
      Targeted GroupMissed (PCC)Single Detected CaseThree Detected CasesTotal
      General RadiologistsCount113115
      Percentage73.3%20%6.7%100%
      Senior Radiology ResidentsCount122115
      Percentage80%13.3%6.7%100%
      Note: P = 0.003
      The inter-rater agreements for the PCC cases with different degrees of calcification and positive and negative posterior ischemic stroke detection are provided in Table 3. There was 1) poor agreement for PCC detection and severe PCC in the negative stroke case (agreement for misdiagnosis, k = 0.93; correct diagnosis, k = 0.00); 2) poor interobserver agreement for mild posterior cerebral calcification in the negative stroke case (agreement for misdiagnosis, k = 0.93; correct diagnosis, k = 0.00); 3) poor interobserver agreement for moderate posterior cerebral calcification in the positive posterior stroke case (agreement for misdiagnosis, k = 0.92; correct diagnosis, k = 0.00); and 4) poor interobserver agreement for severe posterior cerebral calcification in the positive posterior cerebral stroke case (agreement for misdiagnosis, k = 0.846; correct diagnosis, k = 0.00). In contrast, the SRR and GR showed 100% agreement in reporting the normal CT brain case free of calcification.
      Table 3Inter-rater agreement between SRRs and GRs.
      General RadiologistsTotalWeighted kappa
      Missed DiagnosisCorrect Diagnosis
      Missed DiagnosisCorrect Diagnosis
      CASE 1SENIOR RESIDENTSMissed DiagnosisN131140.9290.000
      %92.9%100.0%93.3%
      Correct DiagnosisN101
      %7.1%0.0%6.7%
      CASE 2SENIOR RESIDENTSMissed DiagnosisN131140.9290.000
      %92.9%100.0%93.3%
      Correct DiagnosisN101
      %7.1%0.0%6.7%
      CASE 3SENIOR RESIDENTSMissed DiagnosisN122140.9230.000
      %92.3%100.0%93.3%
      Correct DiagnosisN101
      %7.7%0.0%6.7%
      CASE 4SENIOR RESIDENTSMissed DiagnosisN112130.8460.000
      %84.6%100.0%86.7%
      Correct DiagnosisN202
      %15.4%0.0%13.3%
      CASE 5SENIOR RESIDENTSMissed DiagnosisN015150.0001.000
      %0.0%100%100%
      Correct DiagnosisN000
      %0.0%0.0%0.0%

      4. Discussion

      This study highlighted that several degrees of PCC severity can be misdiagnosed or underreported by SRRs, and GRs based on CT brain scans. The goal of this study was to investigate the PCC detection rate by GRs and SRRs for comparison with an expert neuroradiologist’s readings. This study found low PCC detection rates for both groups, and the misdiagnosis rate was more than 93%. Posterior circulation calcification was occasionally reported in cases that involved posterior cerebral stroke, which increases the severity of PCC, and the detection rates for calcification were slightly higher in the GRs group compared to the SRRs group.
      An error rate of 4% was previously reported from daily practice radiology reports [
      • Caranci F.
      • Tedeschi E.
      • Leone G.
      • Reginelli A.
      • Gatta G.
      • Pinto A.
      • Squillaci E.
      • Briganti F.
      • Brunese L.
      Errors in neuroradiology.
      Radiol. Med. 2015; 120: 795-801https://doi.org/10.1007/S11547-015-0564-7
      ]. The impact of misinterpreting or misidentifying PCC is significant and causes delays in medical and surgical treatment plans. Radiological errors increase the mortality and morbidity of patients with stroke, especially in neuroradiological emergency cases [
      • Cirillo L.
      • Leonardi M.
      • Dall’Olio M.
      • Princiotta C.
      • Stafa A.
      • Simonetti L.
      • Toni F.
      • Agati R.
      Complications in the treatment of intracranial aneurysms with silk stents: an analysis of 30 consecutive patients.
      Interv. Neuroradiol. 2012; 18: 413-425
      ]. Most previous studies have summarized causes for radiologists’ errors made when identifying and interpreting PCC. These include the patient’s clinical history, the availability of previous studies, the environment of the reporting room, and the radiologist’s level of vigilance [
      • Brady A.
      • Laoide R.O.
      • McCarthy P.
      • McDermott R.
      Discrepancy and error in radiology: concepts, causes and consequences.
      Ulst. Med. J. 2012; 81: 3-9
      ].
      In our study, the participating radiologists were interviewed to clarify the reasons behind the missed diagnoses of PCC. Due to heavy workloads, the participants stated that they only reported the main exam findings. Another reason for not reporting calcification was a lack of clinical significance. The minority of the participants believed PCC was common in most patients, especially after the sixth decade of life, so there was no reason to report it. Calcification was reported exceptionally well in CT angiography cases, where stenosed arteries can be illustrated. In contrast, PCC was successfully noticed by some of the participants. Most of them were aware of the negative consequences of posterior calcifications and confirmed that calcification can be the main reason for posterior cerebral stroke. They also believed that posterior circulation calcifications can be associated with other vascular calcifications, and thus the patient can be referred for further investigations.
      Several factors have been proposed in the literature to reduce PCC neuroimaging errors, including improving radiologists’ knowledge, well set-up systems and working conditions, turnaround times for reporting, double reporting techniques, and good communication between referring physicians and reporting radiologists [
      • Grassi R.
      • Rea G.
      • Scaglione M.
      • Brunese L.
      • Scialpi M.
      Imaging of tracheocele: report of three cases and review of the literature.
      Radiol. Med. 2000; 100: 285-287
      ]. The following are recommended to reduce errors in PCC detection and diagnosis: the use of PACS facilities to assess posterior fossa vasculature with proper adjustment of window width and level, as well as radiologists paying more attention to brain parenchyma in posterior circulation territory that are affected by ischemic change. Posterior fossa ischemia cases must include the presence or absence of vasculature calcifications. A scoring system for posterior brain cerebral circulation calcification should be established to accurately grade the severity of calcification.
      Our study included some limitations. First, we analyzed the detection of PCC in limited cases. Other important morphological variables were not considered, such as the location of the calcification, the degree of stenosis, and parenchyma enhancement. We included only five cases; however, our aim was to measure the feasibility of PCC detection by several SRR and GR. Therefore, we limited our study to five preselected cases with cerine pathology. Furthermore, CT angiography was not provided for the sample cases, as most emergency cases are diagnosed based on plain CT brain scans. To the best of our knowledge, this is the first study in Saudi Arabia that has assessed inter-rater agreement between a large number of SRRs and GRs for the radiologic description of PCC. We recommend future longitudinal studies to identify potential sources of error at multiple centers with more morphological variable data.

      5. Conclusion

      Our study concluded that there was a significant misdiagnosis of PCC by both the GRs and SRRs groups. PCC was frequently missed, even in the case of positive posterior ischemic stroke. There was no interobserver agreement for the detection of PCC compared to an expert reader among both groups. The GRs showed fair agreement compared to the SRRs. Therefore, it is important to increase the awareness of PCC and clarify the negative impacts of such vascular calcifications. Furthermore, interobserver agreement could be improved by establishing a diagnostic criterion for PCC and double reporting technique with an expert neuroradiologist. This would reduce the number of misdiagnosis cases.

      Funding

      None to declare.

      CRediT authorship contribution statement

      MJA and MAF contributed to study conceptualization and review final manuscript. MA, FKA, DMA, AMA and AA contributed to study design and data collection. MJA contributed to data interpretation and analysis. MA and MJA contributed to manuscript writing, editing and preparation. The presenting work was check and submitted by MJA. All authors reviewed the final draft.

      Conflict of interest

      The authors disclose no conflict of interests, and the work was not supported or funded by any medical company.

      Acknowledgments

      The authors are grateful to the Medical Imaging Department Administration and PACS team at KSMC. The authors would like to thank all Medical and supporting staff of the Radiology Department at KSMC for their help. In addition, the authors would like to thank the Deanship of Scientific Research at Prince Sattam bin Abdulaziz University for the support by providing statistical packages for analysis.

      Ethics approval and consent to participate

      This study was conducted in accordance with the declaration of Helsinki. The studies involving human participants were reviewed and approved by the Ethics Committee at king Saud Medical city (ref no. H1RI-09-Mar21-01). Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

      References

        • Venkat A.
        • et al.
        Factors associated with stroke misdiagnosis in the emergency department: a retrospective case-control study.
        Neuroepidemiology. 2018; 51 (123–27)
        View in Article
        • Nouh A.
        • Remke J.
        • Ruland S.
        Ischemic posterior circulation stroke: a review of anatomy, clinical presentations, diagnosis, and current management.
        Front. Neurol. 2014; 5: 1-16
        View in Article
        • Banerjee G.
        • Stone S.P.
        • Werring D.J.
        Posterior circulation ischaemic stroke.
        BMJ. 2018; 361: 1-7
        View in Article
        • Gökçal E.
        • et al.
        The association of vertebrobasilar calcification with etiological subtypes, stroke recurrence and outcome in acute brainstem ischemic stroke.
        Neurol. Neurochir. Pol. 2018; 52 (188–93)
        View in Article
        • Van Der Toorn J.E.
        • Engelkes S.R.
        • Kamran Ikram M.
        • Arfan Ikram M.
        • Vernooij M.W.
        • Kavousi M.
        • Bos D.
        Vertebrobasilar artery calcification: prevalence and risk factors in the general population.
        Atherosclerosis. 2019; : 46-52https://doi.org/10.1016/J.Atherosclerosis.2019.05.001
        View in Article
        • Jensen-Kondering U.
        Hyperdense artery sign on computed tomography in acute ischemic stroke.
        World J. Radiol. 2010; 2: 354
        View in Article
        • Caranci F.
        • Tedeschi E.
        • Leone G.
        • Reginelli A.
        • Gatta G.
        • Pinto A.
        • Squillaci E.
        • Briganti F.
        • Brunese L.
        Errors in neuroradiology.
        Radiol. Med. 2015; 120: 795-801https://doi.org/10.1007/S11547-015-0564-7
        View in Article
        • Cirillo L.
        • Leonardi M.
        • Dall’Olio M.
        • Princiotta C.
        • Stafa A.
        • Simonetti L.
        • Toni F.
        • Agati R.
        Complications in the treatment of intracranial aneurysms with silk stents: an analysis of 30 consecutive patients.
        Interv. Neuroradiol. 2012; 18: 413-425
        View in Article
        • Brady A.
        • Laoide R.O.
        • McCarthy P.
        • McDermott R.
        Discrepancy and error in radiology: concepts, causes and consequences.
        Ulst. Med. J. 2012; 81: 3-9
        View in Article
        • Grassi R.
        • Rea G.
        • Scaglione M.
        • Brunese L.
        • Scialpi M.
        Imaging of tracheocele: report of three cases and review of the literature.
        Radiol. Med. 2000; 100: 285-287
        View in Article

      Article info

      Publication history

      Published online: December 08, 2022
      Accepted: December 7, 2022
      Received: June 14, 2022

      Identification

      DOI: https://doi.org/10.1016/j.ejro.2022.100461

      Copyright

      © 2022 The Authors. Published by Elsevier Ltd.

      User license

      Creative Commons Attribution – NonCommercial – NoDerivs (CC BY-NC-ND 4.0) |
      How you can reuse Information Icon
      Creative Commons Attribution – NonCommercial – NoDerivs (CC BY-NC-ND 4.0)

      Permitted

      For non-commercial purposes:

      • Read, print & download
      • Redistribute or republish the final article
      • Text & data mine
      • Translate the article (private use only, not for distribution)
      • Reuse portions or extracts from the article in other works

      Not Permitted

      • Sell or re-use for commercial purposes
      • Distribute translations or adaptations of the article

      Elsevier's open access license policy

      ScienceDirect

      Access this article on ScienceDirect
      We use cookies to help provide and enhance our service and tailor content. To update your cookie settings, please visit the for this site.
      Copyright © 2023 Elsevier Inc. except certain content provided by third parties. The content on this site is intended for healthcare professionals.


      RELX