Advertisement

Low-contrast detectability and potential for radiation dose reduction using deep learning image reconstruction—A 20-reader study on a semi-anthropomorphic liver phantom

Open AccessPublished:April 02, 2022DOI:https://doi.org/10.1016/j.ejro.2022.100418

      Highlights

      • Deep learning image reconstruction improves low-contrast detectability in CT.
      • Performance improved with increasing strength of deep learning image reconstruction.
      • Results suggest potential for CT radiation dose reduction.

      Abstract

      Background

      A novel deep learning image reconstruction (DLIR) algorithm for CT has recently been clinically approved.

      Purpose

      To assess low-contrast detectability and dose reduction potential for CT images reconstructed with the DLIR algorithm and compare with filtered back projection (FBP) and hybrid iterative reconstruction (IR).

      Material and methods

      A customized upper-abdomen phantom containing four cylindrical liver inserts with low-contrast lesions was scanned at CT dose indexes of 5, 10, 15, 20 and 25 mGy. Images were reconstructed with FBP, 50% hybrid IR (IR50), and DLIR of low strength (DLL), medium strength (DLM) and high strength (DLH). Detectability was assessed by 20 independent readers using a two-alternative forced choice approach. Dose reduction potential was estimated separately for each strength of DLIR using a fitted model, with the detectability performance of FBP and IR50 as reference.

      Results

      For the investigated dose levels of 5 and 10 mGy, DLM improved detectability compared to FBP by 5.8 and 6.9 percentage points (p.p.), and DLH improved detectability by 9.6 and 12.3 p.p., respectively (all p < .007). With IR50 as reference, DLH improved detectability by 5.2 and 9.8 p.p. for the 5 and 10 mGy dose level, respectively (p < .03). With respect to this low-contrast detectability task, average dose reduction potential relative to FBP was estimated to 39% for DLM and 55% for DLH. Relative to IR50, average dose reduction potential was estimated to 21% for DLM and 42% for DLH.

      Conclusions:

      Low-contrast detectability performance is improved when applying a DLIR algorithm, with potential for radiation dose reduction.

      Abbreviations:

      DLIR (deep learning image reconstruction), DLL (deep learning image reconstruction of low strength), DLM (deep learning image reconstruction of medium strength), DLH (deep learning image reconstruction of high strength), FBP (filtered back projection), IR (iterative reconstruction)

      Keywords

      1. Introduction

      Computed Tomography (CT) has become an essential tool in modern clinical medicine [
      • Brenner D.J.
      • Hall E.J.
      Computed tomography—an increasing source of radiation exposure.
      ,
      • Novelline R.A.
      • Rhea J.T.
      • Rao P.M.
      • Stuk J.L.
      Helical CT in emergency radiology.
      ]. With widespread availability, a rapid increase in the use of CT imaging has been observed over the last decades [

      OECD , Computed tomography (CT) exams (indicator), 2020 .https://doi.org/10.1787/3c994537-en (Accessed 10 March 2021).

      ]. With the associated increase in radiation exposure, the potential increased risk for radiation-induced malignancy has become a public health concern [
      • Berrington de Gonzalez A.
      • Mahesh M.
      • Kim K.P.
      • Bhargavan M.
      • Lewis R.
      • Mettler F.
      • Land C.
      Projected cancer risks from computed tomographic scans performed in the United States in 2007.
      ]. In general, the benefit of dose reduction is offset by deterioration of image quality. Thus, technological advances to reduce radiation dose without compromising image quality are aspired in clinical practice.
      In CT-image reconstruction, filtered back projection (FBP) has been the dominant image reconstruction technique since the early 1970s, complemented by the first commercial iterative reconstruction (IR) algorithms in 2009 [
      • Pan X.
      • Sidky E.Y.
      • Vannier M.
      Why do commercial CT scanners still employ traditional, filtered back-projection for image reconstruction?.
      ,
      • Beister M.
      • Kolditz D.
      • Kalender W.A.
      Iterative reconstruction methods in X-ray CT.
      ]. Although demonstrated potential for dose reduction [
      • Willemink M.J.
      • de Jong P.A.
      • Leiner T.
      • de Heer L.M.
      • Nievelstein R.A.
      • Budde R.P.
      • Schilham A.M.
      Iterative reconstruction techniques for computed tomography Part 1: technical principles.
      ,
      • Geyer L.L.
      • Schoepf U.J.
      • Meinel F.G.
      • Nance Jr., J.W.
      • Bastarrika G.
      • Leipsic J.A.
      • Paul N.S.
      • Rengo M.
      • Laghi A.
      • Cecco C.N.De
      State of the art: iterative CT reconstruction techniques.
      ,
      • Solomon J.
      • Mileto A.
      • Ramirez-Giraldo J.C.
      • Samei E.
      Diagnostic performance of an advanced modeled iterative reconstruction algorithm for low-contrast detectability with a third-generation dual-source multidetector CT scanner: potential for radiation dose reduction in a multireader study.
      ], recent concerns have been made that rigorous application of IR may cause a decline in low-contrast detectability due to a shift in image noise texture, particularly when radiation dose is reduced below a certain threshold [
      • Mileto A.
      • Guimaraes L.S.
      • McCollough C.H.
      • Fletcher J.G.
      • Yu L.
      State of the art in abdominal CT: the limits of iterative reconstruction algorithms.
      ]. Multireader phantom studies have demonstrated that IR algorithms can preserve low-contrast detectability for only modest levels of dose reduction (up to approximately 25%) [
      • Euler A.
      • Solomon J.
      • Marin D.
      • Nelson R.C.
      • Samei E.
      A third-generation adaptive statistical iterative reconstruction technique: phantom study of image noise, spatial resolution, lesion detectability, and dose reduction potential.
      ,
      • Mileto A.
      • Zamora D.A.
      • Alessio A.M.
      • Pereira C.
      • Liu J.
      • Bhargava P.
      • Carnell J.
      • Cowan S.M.
      • Dighe M.K.
      • Gunn M.L.
      • Kim S.
      • Kolokythas O.
      • Lee J.H.
      • Maki J.H.
      • Moshiri M.
      • Nasrullah A.
      • O’Malley R.B.
      • Schmiedl U.P.
      • Soloff E.V.
      • Toia G.V.
      • Wang C.L.
      • Kanal K.M.
      CT detectability of small low-contrast hypoattenuating focal lesions: iterative reconstructions versus filtered back projection.
      ,
      • McCollough C.H.
      • Yu L.
      • Kofler J.M.
      • Leng S.
      • Zhang Y.
      • Li Z.
      • Carter R.E.
      Degradation of CT low-contrast spatial resolution due to the use of iterative reconstruction and reduced dose levels.
      ,
      • Goenka A.H.
      • Herts B.R.
      • Obuchowski N.A.
      • Primak A.N.
      • Dong F.
      • Karim W.
      • Baker M.E.
      Effect of reduced radiation exposure and iterative reconstruction on detection of low-contrast low-attenuation lesions in an anthropomorphic liver phantom: an 18-reader study.
      ]. In a clinical setting, patient studies have shown that iterative reconstruction does not improve performance at moderate levels of dose reduction and that performance is deteriorated at higher levels of radiation dose reduction [
      • Fletcher J.G.
      • Fidler J.L.
      • Venkatesh S.K.
      • Hough D.M.
      • Takahashi N.
      • Yu L.
      • Johnson M.
      • Leng S.
      • Holmes 3rd, D.R.
      • Carter R.
      • McCollough C.H.
      Observer performance with varying radiation dose and reconstruction methods for detection of hepatic metastases.
      ,
      • Jensen C.T.
      • Wagner-Bartak N.A.
      • Vu L.N.
      • Liu X.
      • Raval B.
      • Martinez D.
      • Wei W.
      • Cheng Y.
      • Samei E.
      • Gupta S.
      Detection of colorectal hepatic metastases is superior at standard radiation dose CT versus reduced dose CT.
      ].
      A novel deep learning image reconstruction (DLIR) algorithm received clinical approval in 2019 (TrueFidelity, GE Healthcare, Milwaukee, WI). Other vendor-specific algorithms for deep learning image reconstruction are also developed (AiCE, Canon Medical Systems, Otawara, Japan). As explained by a technical white paper [

      J. Hsieh, E. Liu, B. Nett, J. Tang, J.-B. Thibault, S. Sahney, A new era of image reconstruction: TrueFidelity – technical white paper on deep learning image reconstruction. 〈https://www.gehealthcare.com/-/jssmedia/040dd213fa89463287155151fdb01922.pdf〉, 2019 (Accessed 10 March 2021).

      ], having been trained with high-dose and low-dose FBP datasets across phantom and patient cases, the DLIR algorithm strives to suppress image noise without compromising image quality. The use of deep learning image reconstruction has demonstrated potential for improved image quality [
      • Akagi M.
      • Nakamura Y.
      • Higaki T.
      • Narita K.
      • Honda Y.
      • Zhou J.
      • Yu Z.
      • Akino N.
      • Awai K.
      Deep learning reconstruction improves image quality of abdominal ultra-high-resolution CT.
      ,
      • Jensen C.T.
      • Liu X.
      • Tamm E.P.
      • Chandler A.G.
      • Sun J.
      • Morani A.C.
      • Javadi S.
      • Wagner-Bartak N.A.
      Image quality assessment of abdominal CT by use of new deep learning image reconstruction: initial experience.
      ,
      • Njølstad T.
      • Schulz A.
      • Godt J.C.
      • Brøgger H.M.
      • Johansen C.K.
      • Andersen H.K.
      • Martinsen A.C.T.
      Improved image quality in abdominal computed tomography reconstructed with a novel Deep Learning Image Reconstruction technique–initial clinical experience.
      ] and image noise reduction without shifting noise texture [
      • Solomon J.
      • Lyu P.
      • Marin D.
      • Samei E.
      Noise and spatial resolution properties of a commercially available deep-learning based CT reconstruction algorithm.
      ,
      • Greffier J.
      • Hamard A.
      • Pereira F.
      • Barrau C.
      • Pasquier H.
      • Beregi J.P.
      • Frandon J.
      Image quality and dose reduction opportunity of deep learning image reconstruction algorithm for CT: a phantom study.
      ,
      • Brady S.L.
      • Trout A.T.
      • Somasundaram E.
      • Anton C.G.
      • Li Y.
      • Dillman J.R.
      Improving image quality and reducing radiation dose for pediatric CT by using Deep Learning Reconstruction.
      ]. Although mathematical observer models suggest improved object detection accuracy using deep learning reconstruction [
      • Brady S.L.
      • Trout A.T.
      • Somasundaram E.
      • Anton C.G.
      • Li Y.
      • Dillman J.R.
      Improving image quality and reducing radiation dose for pediatric CT by using Deep Learning Reconstruction.
      ,
      • Racine D.
      • Brat H.G.
      • Dufour B.
      • Steity J.M.
      • Hussenot M.
      • Rizk B.
      • Fournier D.
      • Zanca F.
      Image texture, low contrast liver lesion detectability and impact on dose: Deep learning algorithm compared to partial model-based iterative reconstruction.
      ], the human observer diagnostic performance on low-contrast detection across dose levels has, to our knowledge, yet to be explored in a controlled phantom setting.
      On this basis, the purpose of this study was to assess the low-contrast diagnostic performance and potential for radiation dose reduction when applying a DLIR algorithm using a customized semi-anthropomorphic upper-abdomen phantom.

      2. Materials and methods

      Institutional review board oversight was not required in this phantom-only study.

      2.1 Phantom design

      A customized semi-anthropomorphic upper-abdomen phantom (The Phantom Laboratory, Salem, NY) with a water equivalent diameter of 29.8 cm [
      • McCollough C.
      • Bakalyar D.M.
      • Bostani M.
      • Brady S.
      • Boedeker K.
      • Boone J.M.
      • Chen-Mayer H.H.
      • Christianson O.I.
      • Leng S.
      • Li B.
      • McNitt-Gray M.F.
      • Nilsen R.A.
      • Supanich M.P.
      • Wang J.
      Use of water equivalent diameter for calculating patient size and size-specific dose estimates (SSDE) in CT: the report of AAPM task group 220.
      ] containing four epoxy inserts with low-contrast lesions of different size and density compared to surrounding background material was applied in this study (Fig. 1).
      Fig. 1
      Fig. 1Photograph (A) and schematic (B) of semi-anthropomorphic upper-abdomen phantom applied in the study.

      2.2 Image acquisition and reconstruction

      The phantom was scanned on a 16-cm multidetector CT scanner (GE Revolution; GE Healthcare; Milwaukee, WI), with scan parameters listed in Table 1, applying a range of tube currents considered clinically relevant [

      A. Widmark, E.G. Norwegian Radiation Protection Authority Report 2014. Language: Norwegian. Available at 〈https://dsa.no/publikasjoner/stralevernrapport-6–2014-representative-doser-i-norge-2006–2009/StralevernRapport_06–2014.pdf〉 (Accessed 7 February 2022).

      ]. To increase data for statistical analysis, the phantom was scanned three times for each dose level where inserts were interchanged and rotated freely about the z-axis between these scans to avoid lesion location recall bias among readers. All series were reconstructed with FBP, 50% hybrid IR (IR50; ASiR-V 50%), and DLIR of low strength (DLL; TrueFidelity Low), medium strength (DLM; TrueFidelity Medium) and high strength (DLH; TrueFidelity High) with 2.5 mm slice thickness. For reconstruction of the DLIR images, raw data were sent to the DLIR vendor (GE Healthcare) as the study was initiated prior to clinical implementation of the DLIR algorithm which is now commercially available.
      Table 1Scan parameters applied in study.
      Scan parameterData
      Detector collimation (mm)80 (128 ×0.625 mm)
      Tube potential (kVp)120
      Pitch0.5
      Rotation speed (seconds)0.5
      Tube current-time product (mAs)75, 150, 225, 300 and 375
      CT dose index (mGy)5, 10, 15, 20 and 25
      Matrix512 × 512
      Scan field of viewLarge body
      Display field of view (mm)350
      Reconstruction kernelStandard kernel
      1 Tube potential of 120 kVp chosen based on phantom size and density.

      2.3 Reader interface and image interpretation

      Twenty readers with variable level of experience (range 1–25 years) voluntarily agreed to participate in reading sessions; ten radiologists, eight medical physicists and two radiographers. Readers were unaware of study design, reconstruction algorithm applied, number of lesions and lesion configuration (i.e., size, density and location). Readings were performed independently without time constraint.
      To assess low-contrast detectability, a series of consecutive two-alternative forced choice tests were performed applying a tailored script in the MATLAB environment (version 2018a; Mathworks, Natick, MA);(Fig. 2). Readers were presented with two cropped CT-images of a specific insert, one image containing hypo- or hyperdense lesions (the signal-present image) and one image containing background noise obtained from a homogenous part of the insert (the signal-absent image). Readers were then instructed to select the image most likely to contain lesions. Images were viewed on a diagnostic display in a clinical reading room with ambient light conditions and a constant window setting with window width of 150 HU and window level of 75 HU. The signal-present image was randomly selected to be on the left or right side of the screen, and all series were presented in a random order. Two different signal-present images were extracted from each scan. Thus, each reader was presented with a total of 600 pairwise image comparisons; 2 signal-present images x 3 scans x 4 inserts x 5 dose levels x 5 reconstruction algorithms. Finally, to reduce the potential effect of inhomogeneity in background noise mimicking a lesion, the signal-absent image was obtained from two separate z-locations of the homogenous part of the insert.
      Fig. 2
      Fig. 2Schematic of phantom insert (A) with corresponding CT image slice (B) of insert containing lesions (signal-present image) and homogenous part of insert (signal-absent image). Screenshot of two-alternative forced choice user interface presented to readers (C).

      2.4 Detectability and dose reduction potential

      Detectability was defined as the fraction of correctly selected images containing lesions, and was assessed separately for each dose level and reconstruction algorithm across readers. The dose reduction potential of the DLIR algorithm was defined as the reduction in dose level in which detectability was equal to that of FBP or IR50. To estimate detectability at dose levels not explicitly evaluated in the study setup, observer data was fitted to a mathematical model using the least squares approach in the Matlab software. With the assumption of detectability of 0.5 with dose level approaching zero (i.e., randomly guessing) and converging towards 1 with increasing dose level, the observer scores were modeled according to the equation:
      D=121+erfαdβ2
      (1)


      where D is detectability, erf is the error function, d is the dose level, and α and β are constant fitting parameters [
      • Solomon J.
      • Mileto A.
      • Ramirez-Giraldo J.C.
      • Samei E.
      Diagnostic performance of an advanced modeled iterative reconstruction algorithm for low-contrast detectability with a third-generation dual-source multidetector CT scanner: potential for radiation dose reduction in a multireader study.
      ]. In this mathematical model, detectability is modelled to 0.5 when dose level, d, approaches zero, as erf(0) = 0. With increasing dose level, the modelled detectability converges towards 1, as erf(∞) = 1. Dose reduction potential was estimated separately for each strength of DLIR, and estimated separately for each investigated dose level. Subgroup analyses were performed comparing radiologist to non-radiologist reader scores.

      2.5 Statistical analysis

      Comparisons of reader detectability scores across reconstruction algorithms were performed applying a pairwise student’s t-test at the 95% significance level under the null hypothesis that there was no difference in detectability between reconstruction algorithms. Adjusted odds ratios (OR) for correctly selecting the signal-present image (i.e., with low-contrast lesions) were estimated using mixed logistic regression. For this, reconstruction algorithm, dose level and lesion type were included as fixed effects whereas reader and the combination of scan number and CT image slice were random effects. Model estimates were computed using R statistical software (version 3.0.4, R Foundation for Statistical Computing, Vienna, Austria, (https://www.r-project.org/) applying the lme4 package [
      • Bates D.
      • Mächler M.
      • Bolker B.
      • Walker S.
      Fitting linear mixed-effects models using lme4.
      ].

      3. Results

      Example images of inserts across dose levels and reconstruction algorithms containing small and large lesions are presented in Fig. 3A and Fig. 3B, respectively. From the ensemble of 20 readers, a total of 12000 data points were obtained. One response was discarded due to reader misclassification, and thus 11,999 data points were included for the final analyses.
      Fig. 3
      Fig. 3Example CT-images of phantom insert with small ~4 mm lesions (A) and large ~8 mm lesions (B) for each reconstruction algorithm and dose level with schematic of ground truth.
      Detectability at an aggregate level for each reconstruction algorithm and subgroup analysis per insert type is presented in Fig. 4. Performance ranged from approximately 50% (i.e., random guess) for the insert with small lesions imaged at 5 mGy dose level to 100% for the insert with large lesions imaged at 25 mGy dose level. When comparing the DLIR algorithms to FBP, detectability was significantly improved for images reconstructed with DLM and DLH for all dose levels (all p < .003). The largest improvement in detectability was observed for the lower dose levels, where DLM improved detectability by 6.9 p.p. at the 5 mGy dose level (95% CI 2.1–11.7 p.p., p = .007) and 5.8 p.p. at the 10 mGy dose level (95% CI 2.3–9.4 p.p., p = .003). Correspondingly, DLH improved detectability by 9.6 p.p. at the 5 mGy dose level (95% CI 3.5–15.6 p.p., p = .004) and 12.3 p.p. at the 10 mGy dose level (95% CI 8.1–16.5 p.p., p < .001). Compared to IR50, detectability was significantly improved for images reconstructed with DLH for the 5, 10 and 15 mGy dose level (p < .03), with the highest observed difference at the 10 mGy dose level (+9.8 p.p. improvement in detectability, 95% CI 4.8–14.8 p.p., p = .001). An overview of differences in detectability per dose level and reconstruction algorithm is presented in Table 2.
      Fig. 4
      Fig. 4Results from reader sessions. Bar charts show average reader detectability scores at an aggregate level (A) and separately for each lesion type with 4 mm hyperdense lesions (B), 6 mm hyperdense lesions (C), 8 mm hyperdense lesions (D) and 8 mm hypodense lesions (E). Images were reconstructed with filtered back projection (FBP), 50% hybrid iterative reconstruction (IR50), and deep learning image reconstruction (DLIR) of low strength (DLL), medium strength (DLM) and high strength (DLH).
      Table 2Differences in detectability scores in percentage points (p.p.) for images reconstructed with DLIR of various strengths relative to hybrid IR and FBP.
      DLLDLMDLH
      ReferenceCT dose index, mGyDifference in detectability, p.p. (95% CI)p-valueDifference in detectability, p.p. (95% CI)p-valueDifference in detectability, p.p. (95% CI)p-value
      FBP5+ 3.3 (−2.9, +9.6).28+ 6.9 (+2.1, +11.7).007+ 9.6 (+3.5, +15.6).004
      10+ 1.5 (−2.8, +5.7).48+ 5.8 (+2.3, +9.4).003+ 12.3 (+8.1, +16.5)< 0.001
      15+ 1.3 (−2.0, +4.5).43+ 4.0 (+1.5, +6.5).003+ 5.8 (+3.2, +8.5)< 0.001
      20+ 6.3 (+2.7, +9.8).002+ 7.3 (+4.5, +10.1)< 0.001+ 8.3 (+4.8, +11.9)< 0.001
      25+ 1.3 (−0.3, +2.8).11+ 2.3 (+0.2, +4.3).03+ 2.3 (+0.6, +4.0).01
      IR505-1.0 (−5.7, +3.6).64+ 2.5 (−0.9, +5.9).14+ 5.2 (+0.7, +9.7).03
      10-1.0 (−5.3, +3.2).61+ 3.3 (−0.6, +7.3).09+ 9.8 (+4.8, +14.8).001
      15-0.6 (−3.3, +2.0).62+ 2.1 (−0.9, +5.1).16+ 4.0 (+1.7, +6.2).001
      20+ 0.8 (−2.0, +3.7).55+ 1.9 (−0.8, +4.6).17+ 2.9 (−0.8, +6.6).11
      25+ 1.0 (−0.8, +2.9).26+ 2.1 (−0.3, +4.5).09+ 2.1 (−0.3, +4.5).09
      P-values for difference by pairwise student’s t-test.
      DLH = deep learning image reconstruction of high strength, DLL = deep learning image reconstruction of low strength, DLM = deep learning image reconstruction of medium strength, FBP = filtered back projection, IR50 = 50% hybrid iterative reconstruction.
      Results from mixed logistic regression are presented in Table 3. Adjusted for lesion type and dose level, odds ratio (OR) for correctly selecting the signal-present CT image was significantly improved for all strengths of DLIR compared to FBP, with OR estimated to 1.41 for DLL (95% CI 1.18–1.75, p < .001), 2.05 for DLM (95% CI 1.66–2.52, p < .001) and 3.20 for DLH (95% CI 2.55–4.02, p < .001). Compared to IR50, OR for correctly selecting the signal-present CT image was significantly improved for the higher strengths of DLIR, with OR estimated to 1.42 for DLM (95% CI 1.15–1.77, p = .001) and 2.23 for DLH (95% CI 1.76–2.81, p < .001).
      Table 3Results from mixed logistic regression with estimated adjusted odds ratio (OR) for correctly selecting signal-present CT image.
      Fixed effects variableAdjusted OR with FBP as reference (95% CI)P-valueAdjusted OR with IR 50 as reference (95% CI)P-value
      Intercept0.44 (0.22–0.86).020.63 (0.32–1.24).18
      Dose level1.14 (1.10–1.18)< 0.0011.14 (1.10–1.18)< 0.001
      Lesion type
       4 mm hyperdense lesionsReferenceReference
       6 mm hyperdense lesions15.6 (8.06–30.23)< 0.00115.6 (8.06–30.23)< 0.001
       8 mm hyperdense lesions8.81 (4.67–16.62)< 0.0018.81 (4.67–16.62)< 0.001
       8 mm hypodense lesions4.07 (2.21–7.49)< 0.0014.07 (2.21–7.49)< 0.001
      Reconstruction algorithm
       FBPReference0.70 (0.57–0.85)< 0.001
       IR501.44 (1.18–1.75)< 0.001Reference
       DLL1.41 (1.15–1.71)< 0.0010.98 (0.80–1.20).83
       DLM2.05 (1.66–2.52)< 0.0011.42 (1.15–1.77).001
       DLH3.20 (2.55–4.02)< 0.0012.23 (1.76–2.81)< 0.001
      DLH = deep learning image reconstruction of high strength, DLL = deep learning image reconstruction of low strength, DLM = deep learning image reconstruction of medium strength, FBP = filtered back projection, IR50 = 50% hybrid iterative reconstruction.
      Dose reduction potential with FBP as reference was estimated to an average of 20% for DLL (range 17–22%), 39% for DLM (range 36–42%) and 55% for DLH (range 53–58%);(Table 4). With IR50 as reference, dose reduction potential was estimated to 21% for DLM (range 19–22%) and 42% for DLH (range 41–43%). Fig. 5 illustrates the dose reduction potential of DLH based on the detectability scores for DLH and IR50 across the investigated dose levels.
      Table 4Estimated dose reduction potential for hybrid IR and DLIR of various strengths relative to FBP.
      Dose level (mGy)Dose reduction potential
      Reference FBP dose (mGy)IR50DLLDLMDLHIR50DLLDLMDLH
      53.64.22.92.129%17%42%58%
      107.58.16.04.425%19%40%56%
      1511.712.09.36.822%20%38%55%
      2015.915.712.59.220%21%37%54%
      2520.319.515.911.719%22%36%53%
      Estimated dose level with comparable detectability performance and implied dose reduction potential for hybrid IR and DLIR of various strengths relative to FBP based on a mathematical model fitted to low-contrast detectability observer data.
      DLH = deep learning image reconstruction of high strength, DLL = deep learning image reconstruction of low strength, DLM = deep learning image reconstruction of medium strength, FBP = filtered back projection, IR50 = 50% hybrid iterative reconstruction.
      Fig. 5
      Fig. 5Plot of average low-contrast detectability scores for images reconstructed with deep learning image reconstruction of high strength (DLH) and filtered back projection (FBP);(A) and 50% hybrid iterative reconstruction (IR50);(B) for each investigated CT dose index (CTDIvol). Curves fitted to observer data allow for estimation of the potential reduction in dose level for DLH while maintaining comparable level of detectability to FBP and IR50 for each investigated dose level (dashed lines).
      When comparing radiologists (n = 10) to non-radiologists (n = 10), detectability scores were at an aggregate level slightly higher for radiologists (+2.9 p.p.). However, there was no trend for radiologists performing better for a specific dose level or reconstruction technique (Supplementary Table 1). Dose reduction potential using radiologist scores only and with FBP as reference was estimated to an average of 18%, 34% and 53% for DLL, DLM and DLH, respectively. With IR50 as reference, dose reduction potential was estimated to an average of 21% and 38% for DLM and DLH, respectively.

      4. Discussion

      This tailored phantom study demonstrates that CT images reconstructed with a DLIR algorithm can improve low-contrast detectability relative to conventional FBP and hybrid IR, and that this improvement in detectability is dependent on algorithm strength. Based on reader detectability scores, average dose reduction potential for DLIR of high strength was estimated to 55% compared to FBP and 42% compared to 50% hybrid IR.
      Fueled by increased availability, a rapid increase in the use of CT imaging has been observed over the last decades and the associated increase in radiation exposure has become a public health concern [

      OECD , Computed tomography (CT) exams (indicator), 2020 .https://doi.org/10.1787/3c994537-en (Accessed 10 March 2021).

      ,
      • Berrington de Gonzalez A.
      • Mahesh M.
      • Kim K.P.
      • Bhargavan M.
      • Lewis R.
      • Mettler F.
      • Land C.
      Projected cancer risks from computed tomographic scans performed in the United States in 2007.
      ]. Although CT provides tremendous benefit to patient care when used for appropriate indications, efforts to keep radiation dose delivered to the patient as low as reasonably achievable have been widely endorsed (i.e., the ALARA principle) [
      • McCollough C.H.
      • Primak A.N.
      • Braun N.
      • Kofler J.
      • Yu L.
      • Christner J.
      Strategies for reducing radiation dose in CT.
      ]. However, decreasing radiation dose is traditionally offset by a deterioration in image quality, driven by an increase in image noise [
      • Siegel M.J.
      • Schmidt B.
      • Bradley D.
      • Suess C.
      • Hildebolt C.
      Radiation dose and image quality in pediatric CT: effect of technical factors and phantom size and shape.
      ]. Detection and characterization of focal liver lesions can be a challenging diagnostic task in abdominal imaging, with decisive implication for patient care in an oncological setting [
      • Oliva M.R.
      • Saini S.
      Liver cancer imaging: role of CT, MRI, US and PET.
      ]. This low-contrast task is highly sensitive to image noise – especially if suboptimal dose levels have been applied during image acquisition [
      • Oliva M.R.
      • Saini S.
      Liver cancer imaging: role of CT, MRI, US and PET.
      ,
      • Kanal K.M.
      • Chung J.H.
      • Wang J.
      • Bhargava P.
      • Kohr J.R.
      • Shuman W.P.
      • Stewart B.K.
      Image noise and liver lesion detection with MDCT: a phantom study.
      ]. Recent reports have shed light on potential disadvantages of IR algorithms where dose reduction potential may be limited by deterioration in image quality primarily affecting low-contrast tasks, such as obscuring small liver lesions [
      • Mileto A.
      • Guimaraes L.S.
      • McCollough C.H.
      • Fletcher J.G.
      • Yu L.
      State of the art in abdominal CT: the limits of iterative reconstruction algorithms.
      ]. Thus, novel methods to reduce CT image noise are aspired in clinical practice in pursuit of dose reduction.
      Several studies have investigated the qualitative image quality of DLIR algorithms. Phantom studies have demonstrated that a DLIR algorithm can achieve robust noise reduction without deteriorating changes in image noise texture, although demonstrating somewhat degraded low-contrast spatial resolution [
      • Solomon J.
      • Lyu P.
      • Marin D.
      • Samei E.
      Noise and spatial resolution properties of a commercially available deep-learning based CT reconstruction algorithm.
      ,
      • Greffier J.
      • Hamard A.
      • Pereira F.
      • Barrau C.
      • Pasquier H.
      • Beregi J.P.
      • Frandon J.
      Image quality and dose reduction opportunity of deep learning image reconstruction algorithm for CT: a phantom study.
      ]. A recent phantom study by Racine al. investigating low-contrast detectability using channelized Hotelling observer as a surrogate marker for human observers estimated the dose reduction potential to 25% for DLM to and 33% for DLH compared to 60% hybrid IR [
      • Racine D.
      • Brat H.G.
      • Dufour B.
      • Steity J.M.
      • Hussenot M.
      • Rizk B.
      • Fournier D.
      • Zanca F.
      Image texture, low contrast liver lesion detectability and impact on dose: Deep learning algorithm compared to partial model-based iterative reconstruction.
      ]. The current study is thus an important supplement, demonstrating improved low-contrast detectability performance by human observers. In a clinical setting, the DLIR algorithm has demonstrated improved perceived overall image quality when compared to standard IR at standard dose abdominal CT [
      • Jensen C.T.
      • Liu X.
      • Tamm E.P.
      • Chandler A.G.
      • Sun J.
      • Morani A.C.
      • Javadi S.
      • Wagner-Bartak N.A.
      Image quality assessment of abdominal CT by use of new deep learning image reconstruction: initial experience.
      ,
      • Njølstad T.
      • Schulz A.
      • Godt J.C.
      • Brøgger H.M.
      • Johansen C.K.
      • Andersen H.K.
      • Martinsen A.C.T.
      Improved image quality in abdominal computed tomography reconstructed with a novel Deep Learning Image Reconstruction technique–initial clinical experience.
      ]. A study on images of pediatric patients reconstructed with a DLIR algorithm from a different vendor (AiCE, Canon Medical Systems) found that significant dose reduction was possible without sacrificing image quality [
      • Brady S.L.
      • Trout A.T.
      • Somasundaram E.
      • Anton C.G.
      • Li Y.
      • Dillman J.R.
      Improving image quality and reducing radiation dose for pediatric CT by using Deep Learning Reconstruction.
      ]. A study on low-dose abdominal CT applying the same vendor-specific algorithm found that deep learning reconstruction improves overall image quality and lesion detection compared to FBP and IR images [
      • Singh R.
      • Digumarthy S.R.
      • Muse V.V.
      • Kambadakone A.R.
      • Blake M.A.
      • Tabari A.
      • Hoi Y.
      • Akino N.
      • Angel E.
      • Madan R.
      • Kalra M.K.
      Image quality and lesion detection on deep learning reconstruction and iterative reconstruction of submillisievert chest and abdominal CT.
      ]. It should be noted that the AiCE algorithm is trained with model-based IR images, and results are not necessarily transferrable across vendors. Thus, it is important to take into account how the different algorithms are trained when evaluating and comparing vendor-specific DLIR algorithms.
      Interestingly, radiologist detectability scores were significantly, albeit only slightly, higher than non-radiologists. Subgroup analysis of results based in radiologist scores were important as radiologists assess the images in clinical practice. However, selecting a signal-present image in a two-alternative forced choice approach does not take into account the diagnostic considerations a radiologist is expected to take when assessing focal liver lesions. Thus, studies incorporating detectability in combination with assessment of diagnostic confidence in a clinical setting with clinicopathological correlation are important supplements to this study in further exploring the dose reduction potential of deep learning based CT image reconstruction algorithms.
      This study is not without limitations. First, only one specific scanner and one vendor-specific DLIR algorithm was applied, and results are not necessarily transferrable across vendors. Second, this study only compared one level of IR where several blends of hybrid-IR could have been explored. However, there is a practical balance between the number of metrics explored in such a human observer setting. This study was designed to explore different strengths of DLIR, and compare with standardly applied hybrid IR at our institution in addition to FBP as this vendor-specific DLIR algorithm is trained on FBP datasets. Third, this study lacks validation in a clinical setting, although the use of a semi-anthropomorphic upper-abdomen phantom containing lesions with a variation in size arguably mitigates this limitation.
      In conclusion, this study demonstrates that low-contrast detectability is improved when applying a novel DLIR algorithm, and that this improvement is dependent on algorithm strength. The dose reduction potential based on detectability performance for DLIR of high strength was estimated to 55% relative to FBP and 42% relative to hybrid IR. The results in this study can serve as a basis for clinical studies to further investigate the diagnostic performance and dose reduction potential of DLIR.

      Funding sources

      This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

      CRediT authorship contribution statement

      Tormund Njølstad: Conceptualization, Methodology, Investigation, Writing – original draft. Kristin Jensen: Data Curation, Investigation, Writing – review & editing. Anniken Dybwad; Data Curation, Investigation, Writing – review & editing. Øyvind Salvesen; Investigation, Writing – review & editing. Hilde K. Andersen: Investigation, Supervision, Writing – review & editing. Anselm Schulz: Conceptualization, Writing – review & editing, Supervision, Project administration.

      Declaration of Competing Interest

      The authors declare the following financial interests/personal relationships which may be considered as potential competing interests:This study is part of ongoing research at the Oslo University Hospital CT Research and Technology Group. Oslo University Hospital has institutional research agreements with GE Healthcare and The Phantom Laboratory, among others. The authors of this article had complete control of data that might have presented a conflict of interest throughout the study period, and the decision to publish has been at the sole discretion of the authors. The authors state no individual conflicts of interests.

      Acknowledgements

      The authors thank Johann Baptist Dormagen, Mats Engelhardtsen, Caroline Enevoldsen, Jan Glittum, Johannes Godt, Ingrid Helen Ryste Hauge, Ole Einar Heieren, Trine Hjørnevik, Cathrine K. Johansen, Nicolas Sogge, Hanne Solheim, Jarle Rambo Sølie, Vera Helene Tormodsrud, Bartek Trela, Bjørn Helge Østerås, Øyvind Tenden Øverbø and Kjetil Øye for participating in the reading session.

      Appendix A. Supplementary material

      References

        • Brenner D.J.
        • Hall E.J.
        Computed tomography—an increasing source of radiation exposure.
        New Engl. J. Med. 2007; 357: 2277-2284https://doi.org/10.1056/NEJMra072149
        • Novelline R.A.
        • Rhea J.T.
        • Rao P.M.
        • Stuk J.L.
        Helical CT in emergency radiology.
        Radiology. 1999; 213: 321-339https://doi.org/10.1148/radiology.213.2.r99nv01321
      1. OECD , Computed tomography (CT) exams (indicator), 2020 .https://doi.org/10.1787/3c994537-en (Accessed 10 March 2021).

        • Berrington de Gonzalez A.
        • Mahesh M.
        • Kim K.P.
        • Bhargavan M.
        • Lewis R.
        • Mettler F.
        • Land C.
        Projected cancer risks from computed tomographic scans performed in the United States in 2007.
        Arch. Intern. Med. 2009; 169: 2071-2077https://doi.org/10.1001/archinternmed.2009.440
        • Pan X.
        • Sidky E.Y.
        • Vannier M.
        Why do commercial CT scanners still employ traditional, filtered back-projection for image reconstruction?.
        Inverse Probl. 2009; 25123009https://doi.org/10.1088/0266-5611/25/12/123009
        • Beister M.
        • Kolditz D.
        • Kalender W.A.
        Iterative reconstruction methods in X-ray CT.
        Phys. Med. 2012; 28: 94-108https://doi.org/10.1016/j.ejmp.2012.01.003
        • Willemink M.J.
        • de Jong P.A.
        • Leiner T.
        • de Heer L.M.
        • Nievelstein R.A.
        • Budde R.P.
        • Schilham A.M.
        Iterative reconstruction techniques for computed tomography Part 1: technical principles.
        Eur. Radiol. 2013; 23: 1623-1631https://doi.org/10.1007/s00330-012-2765-y
        • Geyer L.L.
        • Schoepf U.J.
        • Meinel F.G.
        • Nance Jr., J.W.
        • Bastarrika G.
        • Leipsic J.A.
        • Paul N.S.
        • Rengo M.
        • Laghi A.
        • Cecco C.N.De
        State of the art: iterative CT reconstruction techniques.
        Radiology. 2015; 276: 339-357https://doi.org/10.1148/radiol.2015132766
        • Solomon J.
        • Mileto A.
        • Ramirez-Giraldo J.C.
        • Samei E.
        Diagnostic performance of an advanced modeled iterative reconstruction algorithm for low-contrast detectability with a third-generation dual-source multidetector CT scanner: potential for radiation dose reduction in a multireader study.
        Radiology. 2015; 275: 735-745https://doi.org/10.1148/radiol.15142005
        • Mileto A.
        • Guimaraes L.S.
        • McCollough C.H.
        • Fletcher J.G.
        • Yu L.
        State of the art in abdominal CT: the limits of iterative reconstruction algorithms.
        Radiology. 2019; 191422https://doi.org/10.1148/radiol.2019191422
        • Euler A.
        • Solomon J.
        • Marin D.
        • Nelson R.C.
        • Samei E.
        A third-generation adaptive statistical iterative reconstruction technique: phantom study of image noise, spatial resolution, lesion detectability, and dose reduction potential.
        AJR Am. J. Roentgenol. 2018; : 1301-1308https://doi.org/10.2214/ajr.17.19102
        • Mileto A.
        • Zamora D.A.
        • Alessio A.M.
        • Pereira C.
        • Liu J.
        • Bhargava P.
        • Carnell J.
        • Cowan S.M.
        • Dighe M.K.
        • Gunn M.L.
        • Kim S.
        • Kolokythas O.
        • Lee J.H.
        • Maki J.H.
        • Moshiri M.
        • Nasrullah A.
        • O’Malley R.B.
        • Schmiedl U.P.
        • Soloff E.V.
        • Toia G.V.
        • Wang C.L.
        • Kanal K.M.
        CT detectability of small low-contrast hypoattenuating focal lesions: iterative reconstructions versus filtered back projection.
        Radiology. 2018; : 443-454https://doi.org/10.1148/radiol.2018180137
        • McCollough C.H.
        • Yu L.
        • Kofler J.M.
        • Leng S.
        • Zhang Y.
        • Li Z.
        • Carter R.E.
        Degradation of CT low-contrast spatial resolution due to the use of iterative reconstruction and reduced dose levels.
        Radiology. 2015; : 499-506https://doi.org/10.1148/radiol.15142047
        • Goenka A.H.
        • Herts B.R.
        • Obuchowski N.A.
        • Primak A.N.
        • Dong F.
        • Karim W.
        • Baker M.E.
        Effect of reduced radiation exposure and iterative reconstruction on detection of low-contrast low-attenuation lesions in an anthropomorphic liver phantom: an 18-reader study.
        Radiology. 2014; : 154-163https://doi.org/10.1148/radiol.14131928
        • Fletcher J.G.
        • Fidler J.L.
        • Venkatesh S.K.
        • Hough D.M.
        • Takahashi N.
        • Yu L.
        • Johnson M.
        • Leng S.
        • Holmes 3rd, D.R.
        • Carter R.
        • McCollough C.H.
        Observer performance with varying radiation dose and reconstruction methods for detection of hepatic metastases.
        Radiology. 2018; 289: 455-464https://doi.org/10.1148/radiol.2018180125
        • Jensen C.T.
        • Wagner-Bartak N.A.
        • Vu L.N.
        • Liu X.
        • Raval B.
        • Martinez D.
        • Wei W.
        • Cheng Y.
        • Samei E.
        • Gupta S.
        Detection of colorectal hepatic metastases is superior at standard radiation dose CT versus reduced dose CT.
        Radiology. 2019; : 400-409https://doi.org/10.1148/radiol.2018181657
      2. J. Hsieh, E. Liu, B. Nett, J. Tang, J.-B. Thibault, S. Sahney, A new era of image reconstruction: TrueFidelity – technical white paper on deep learning image reconstruction. 〈https://www.gehealthcare.com/-/jssmedia/040dd213fa89463287155151fdb01922.pdf〉, 2019 (Accessed 10 March 2021).

        • Akagi M.
        • Nakamura Y.
        • Higaki T.
        • Narita K.
        • Honda Y.
        • Zhou J.
        • Yu Z.
        • Akino N.
        • Awai K.
        Deep learning reconstruction improves image quality of abdominal ultra-high-resolution CT.
        Eur. Radiol. 2019; 29: 6163-6171https://doi.org/10.1007/s00330-019-06170-3
        • Jensen C.T.
        • Liu X.
        • Tamm E.P.
        • Chandler A.G.
        • Sun J.
        • Morani A.C.
        • Javadi S.
        • Wagner-Bartak N.A.
        Image quality assessment of abdominal CT by use of new deep learning image reconstruction: initial experience.
        AJR Am. J. Roentgenol. 2020; : 1-8https://doi.org/10.2214/ajr.19.22332
        • Njølstad T.
        • Schulz A.
        • Godt J.C.
        • Brøgger H.M.
        • Johansen C.K.
        • Andersen H.K.
        • Martinsen A.C.T.
        Improved image quality in abdominal computed tomography reconstructed with a novel Deep Learning Image Reconstruction technique–initial clinical experience.
        Acta Radiol. Open. 2021; 10https://doi.org/10.1177/20584601211008391
        • Solomon J.
        • Lyu P.
        • Marin D.
        • Samei E.
        Noise and spatial resolution properties of a commercially available deep-learning based CT reconstruction algorithm.
        Med. Phys. 2020; https://doi.org/10.1002/mp.14319
        • Greffier J.
        • Hamard A.
        • Pereira F.
        • Barrau C.
        • Pasquier H.
        • Beregi J.P.
        • Frandon J.
        Image quality and dose reduction opportunity of deep learning image reconstruction algorithm for CT: a phantom study.
        Eur. Radiol. 2020; : 1-9https://doi.org/10.1007/s00330-020-06724-w
        • Brady S.L.
        • Trout A.T.
        • Somasundaram E.
        • Anton C.G.
        • Li Y.
        • Dillman J.R.
        Improving image quality and reducing radiation dose for pediatric CT by using Deep Learning Reconstruction.
        Radiology. 2021; 298: 180-188https://doi.org/10.1148/radiol.2020202317
        • Racine D.
        • Brat H.G.
        • Dufour B.
        • Steity J.M.
        • Hussenot M.
        • Rizk B.
        • Fournier D.
        • Zanca F.
        Image texture, low contrast liver lesion detectability and impact on dose: Deep learning algorithm compared to partial model-based iterative reconstruction.
        Eur. J. Radiol. 2021; 141109808https://doi.org/10.1016/j.ejrad.2021.109808
        • McCollough C.
        • Bakalyar D.M.
        • Bostani M.
        • Brady S.
        • Boedeker K.
        • Boone J.M.
        • Chen-Mayer H.H.
        • Christianson O.I.
        • Leng S.
        • Li B.
        • McNitt-Gray M.F.
        • Nilsen R.A.
        • Supanich M.P.
        • Wang J.
        Use of water equivalent diameter for calculating patient size and size-specific dose estimates (SSDE) in CT: the report of AAPM task group 220.
        AAPM Rep. 2014; 2014: 6-23
      3. A. Widmark, E.G. Norwegian Radiation Protection Authority Report 2014. Language: Norwegian. Available at 〈https://dsa.no/publikasjoner/stralevernrapport-6–2014-representative-doser-i-norge-2006–2009/StralevernRapport_06–2014.pdf〉 (Accessed 7 February 2022).

        • Bates D.
        • Mächler M.
        • Bolker B.
        • Walker S.
        Fitting linear mixed-effects models using lme4.
        J Stat. Softw. 2015; 67: 1-48https://doi.org/10.18637/jss.v067.i01
        • McCollough C.H.
        • Primak A.N.
        • Braun N.
        • Kofler J.
        • Yu L.
        • Christner J.
        Strategies for reducing radiation dose in CT.
        Radiol. Clin. 2009; 47: 27-40
        • Siegel M.J.
        • Schmidt B.
        • Bradley D.
        • Suess C.
        • Hildebolt C.
        Radiation dose and image quality in pediatric CT: effect of technical factors and phantom size and shape.
        Radiology. 2004; 233: 515-522
        • Oliva M.R.
        • Saini S.
        Liver cancer imaging: role of CT, MRI, US and PET.
        Cancer Imaging Off. Publ. Int. Cancer Imaging Soc. 2004; 4: S42https://doi.org/10.1102/1470-7330.2004.0011
        • Kanal K.M.
        • Chung J.H.
        • Wang J.
        • Bhargava P.
        • Kohr J.R.
        • Shuman W.P.
        • Stewart B.K.
        Image noise and liver lesion detection with MDCT: a phantom study.
        AJR Am. J. Roentgenol. 2011; 197: 437-441https://doi.org/10.2214/ajr.10.5726
        • Singh R.
        • Digumarthy S.R.
        • Muse V.V.
        • Kambadakone A.R.
        • Blake M.A.
        • Tabari A.
        • Hoi Y.
        • Akino N.
        • Angel E.
        • Madan R.
        • Kalra M.K.
        Image quality and lesion detection on deep learning reconstruction and iterative reconstruction of submillisievert chest and abdominal CT.
        AJR Am. J. Roentgenol. 2020; : 566-573https://doi.org/10.2214/ajr.19.21809