Advertisement

Noninvasively predict the micro-vascular invasion and histopathological grade of hepatocellular carcinoma with CT-derived radiomics

  • Xu Tong
    Affiliations
    Department of Radiotherapy, the Third Affiliated Hospital of Qiqihar Medical University, Qiqihar City, Heilongjiang Province, China
    Search for articles by this author
  • Jing Li
    Correspondence
    Correspondence to: No. 27 Taishun Street, Tiefeng District, Qiqihar City, Heilongjiang Province 161000, China.
    Affiliations
    Department of Radiotherapy, the Third Affiliated Hospital of Qiqihar Medical University, Qiqihar City, Heilongjiang Province, China
    Search for articles by this author
Open AccessPublished:May 16, 2022DOI:https://doi.org/10.1016/j.ejro.2022.100424

      Abstract

      Objectives

      This research aims to predict the micro-vascular invasion and histopathologic grade of hepatocellular carcinoma with the CT-derived radiomics.

      Methods

      The clinical and image data of 82 patients were accessed from the TCGA-LIHC collection in The Cancer Imaging Archive. Then the radiomics features were extracted from the CT images. For obtaining the appropriate feature subset, the redundant features were removed by means of intra-class agreement analysis, the Student t test, LASSO-regression and support vector machine (SVM) Recursive feature elimination (SVM-RFE). Then several machine-learning-based classifiers including SVM and random forest (RF) were established. To accurately evaluate the tumor grade and MVI with the integration of the Radiomics and clinical insights, the nomogram-based clinical models were constructed. The diagnostic performance was evaluated with ROC analysis.

      Results

      7 and 10 radiomics features were selected via LASSO regression and SVM-RFE for identifying the tumor grade with regard to 13 and 10 features selected via LASSO regression and SVM-RFE for evaluating the MVI. The combination of the classifier—RF and the selection strategy of SVM-RFE yielded the best performance for grading HCC (AUC: 0.898). Differently, the combination of the classifier—RF and the selection strategy of LASSO regression resulted in the best performance for identifying MVI (AUC: 0.876). Finally, two nomograms were constructed with radiomics score (Rscore) and clinical risk factors, which showed excellent predictive value for both tumor grade (AUC: 0.928) and MVI (AUC: 0.945).

      Conclusion

      CT-derived radiomics were valuable for noninvasively assessing the micro-vascular invasion and histopathologic grade of hepatocellular carcinoma.

      Keywords

      1. Introduction

      Hepatocellular carcinoma (HCC) has been a tremendous threat to human health for a very long time because of the notoriously high incidence together with the high mortality [
      • Craig A.J.
      • Von Felden J.
      • Garcia-Lezana T.
      • Sarcognato S.
      • Villanueva A.
      Tumour evolution in hepatocellular carcinoma.
      ]. At present, the first-line treatment options for different types of HCC contain the surgical resection, radiofrequency ablation, Transhepatic Arterial Chem therapy And Embolization (TACE) and so forth [
      • Lencioni R.
      • Cioni D.
      • Donati F.
      • Bartolozzi C.
      Combination of interventional therapies in hepatocellular carcinoma.
      ,
      • Liver EAFTSOT
      EASL clinical practice guidelines: management of hepatocellular carcinoma.
      ,
      • Heimbach J.K.
      • Kulik L.M.
      • Finn R.S.
      • et al.
      AASLD guidelines for the treatment of hepatocellular carcinoma.
      ]. However, the poor prognosis is broadly regarded as the huge challenge. Accurate prognostic prediction and evaluation may, to some extent, guide the clinical management of HCC [
      • Xu R.-h
      • Wei W.
      • Krawczyk M.
      • et al.
      Circulating tumour DNA methylation markers for diagnosis and prognosis of hepatocellular carcinoma.
      ]. Currently, evaluating the prognosis-related histopathological markers have been accepted as the effective approach for prognostic prediction. For example, high histopathological grade and the presence of micro-vascular invasion indicate the high probability of recurrence, lymphatic metastasis, strong tumor invasion and metastasis [
      • An C.
      • Park M.-S.
      • Jeon H.-M.
      • et al.
      Prediction of the histopathological grade of hepatocellular carcinoma using qualitative diffusion-weighted, dynamic, and hepatobiliary phase MRI.
      ,
      • Ameli S.
      • Shaghaghi M.
      • Ghasabeh M.A.
      • et al.
      Role of baseline volumetric functional MRI in predicting histopathologic grade and patients’ survival in hepatocellular carcinoma.
      ,
      • Jiang T.
      • Xu J.
      • Zou Y.
      • et al.
      Diffusion-weighted imaging (DWI) of hepatocellular carcinomas: a retrospective analysis of the correlation between qualitative and quantitative DWI and tumour grade.
      ,
      • Jhaveri K.S.
      • Cleary S.P.
      • Fischer S.
      • et al.
      Blood oxygen level‐dependent liver MRI: Can It predict microvascular invasion in HCC?.
      ]. Assessing the prognosis related histopathological factors such as tumor grade [
      • Pawlik T.M.
      • Gleisner A.L.
      • Anders R.A.
      • Assumpcao L.
      • Maley W.
      • Choti M.A.
      Preoperative assessment of hepatocellular carcinoma tumor grade using needle biopsy: implications for transplant eligibility.
      ], pathological stage [
      • Shen X.
      • Li N.
      • Li H.
      • Zhang T.
      • Wang F.
      • Li Q.
      Increased prevalence of regulatory T cells in the tumor microenvironment and its correlation with TNM stage of hepatocellular carcinoma.
      ], micro-vascular invasion [
      • Jhaveri K.S.
      • Cleary S.P.
      • Fischer S.
      • et al.
      Blood oxygen level‐dependent liver MRI: Can It predict microvascular invasion in HCC?.
      ], the expression of some histopathological markers containing Ki67 [
      • Cao Y.
      • Ke R.
      • Wang S.
      • et al.
      DNA topoisomerase IIα and Ki67 are prognostic factors in patients with hepatocellular carcinoma.
      ], CK-19 [
      • Feng J.
      • Zhu R.
      • Chang C.
      • et al.
      CK19 and glypican 3 expression profiling in the prognostic indication for patients with HCC after surgical resection.
      ] have drawn innumerable attention. Nevertheless, current gold standard for evaluating the prognostic markers, histopathological examination, is with many disadvantages containing invasiveness, time-consumption and potential sampling bias. Novel approaches with complementary advantages are urgently required.
      During the past few years, growing attention has been paid to image-based prognostic prediction. Various prognosis related histopathological markers of HCC including the tumor grade, micro-vascular invasion, capsule formation, and the expression of Ki67 and CK-19 have been broadly assessed through exploring the representative image predictors of cancer [
      • An C.
      • Park M.-S.
      • Jeon H.-M.
      • et al.
      Prediction of the histopathological grade of hepatocellular carcinoma using qualitative diffusion-weighted, dynamic, and hepatobiliary phase MRI.
      ,
      • Jhaveri K.S.
      • Cleary S.P.
      • Fischer S.
      • et al.
      Blood oxygen level‐dependent liver MRI: Can It predict microvascular invasion in HCC?.
      ,
      • Chen Y.
      • Qin X.
      • Long L.
      • et al.
      Diagnostic value of Gd‐EOB‐DTPA‐enhanced MRI for the expression of Ki67 and microvascular density in hepatocellular carcinoma.
      ]. With the core ideology of that images are more than pictures and they are data, radiomics have paved the unprecedented way for exploring the diagnostic markers and models from images [
      • Gillies R.J.
      • Kinahan P.E.
      • Hricak H.
      Radiomics: images are more than pictures, they are data.
      ]. Additionally, the integration of high-throughput radiomics features and robust artificial intelligence-based modes have been widely reported to yield extra clinical benefits in lesion discrimination, disease diagnosis and treatment efficacy prediction [
      • Scapicchio C.
      • Gabelloni M.
      • Barucci A.
      • Cioni D.
      • Saba L.
      • Neri E.
      A deep look into radiomics.
      ,
      • Song J.
      • Yin Y.
      • Wang H.
      • Chang Z.
      • Liu Z.
      • Cui L.
      A review of original articles published in the emerging field of radiomics.
      ,
      • Richardson M.L.
      • Garwood E.R.
      • Lee Y.
      • et al.
      Noninterpretive uses of artificial intelligence in radiology.
      ]. CT-based radiomics have shown great value in evaluating the prognostic markers of HCC [
      • Xu X.
      • Zhang H.-L.
      • Liu Q.-P.
      • et al.
      Radiomic analysis of contrast-enhanced CT predicts microvascular invasion and outcome in hepatocellular carcinoma.
      ,
      • Ma X.
      • Wei J.
      • Gu D.
      • et al.
      Preoperative radiomics nomogram for microvascular invasion prediction in hepatocellular carcinoma using contrast-enhanced CT.
      ]. Several previous studies also aimed to apply the CT-derived radiomics for characterizing the histopathological grade, micro-vascular invasion or other pathological markers of HCC [
      • Xu X.
      • Zhang H.-L.
      • Liu Q.-P.
      • et al.
      Radiomic analysis of contrast-enhanced CT predicts microvascular invasion and outcome in hepatocellular carcinoma.
      ,
      • Peng J.
      • Zhang J.
      • Zhang Q.
      • Xu Y.
      • Zhou J.
      • Liu L.
      A radiomics nomogram for preoperative prediction of microvascular invasion risk in hepatitis B virus-related hepatocellular carcinoma.
      ,
      • Mao B.
      • Zhang L.
      • Ning P.
      • et al.
      Preoperative prediction for pathological grade of hepatocellular carcinoma via machine learning–based radiomics.
      ]. However, limited number of studies aimed to apply the CT-derived radiomics for simultaneously predicting multiple prognostic markers of HCC. Besides, previous results varied for different studies and included cohorts, which demonstrated that more explorations and researches should be conducted. Therefore, this research aims to extract the radiomics features from CT images and then established the machine-learning-based diagnostic models for identifying the histopathological grade and micro-vascular invasion of HCC.

      2. Methods

      2.1 Patient cohort

      Both the image data and other clinical data were accessed from the TCGA-LIHC collection (https://wiki.cancerimagingarchive.net/display/Public/TCGA-LIHC) in Cancer Imaging Archive (TCIA). The local ethical approval (20-1574AB) and the written informed consents of all patient were successfully obtained, which is declared in the data source. In brief, the Cancer Genome Atlas Liver Hepatocellular Carcinoma (TCGA-LIHC) data collection provided a convenient community for researchers to investigate the hepatocellular carcinoma with insights of radiological findings, pathology, clinical outcome and genotype. Detailed multi-institution based data description of TCGA-LIHC can be found in the previous research [
      • Erickson B.
      • Kirk S.
      • Lee Y.
      • et al.
      Radiology data from the cancer genome atlas liver hepatocellular carcinoma [TCGA-LIHC] collection.
      ].
      In total, a dataset of 97 subjects were downloaded from the TCGA-LIHC collection. The available data were included according to the inclusion criterion and exclusion criterion as followings:

      2.1.1 Inclusion criterions

      • 1)
        Pathologically confirmed as HCC without preoperative treatment.

      2.1.2 Exclusion criterions

      • 1)
        The absence of CT image data.
      • 2)
        The absence of pathological results regarding the micro-vascular invasion or histopathological grade.
      • 3)
        The poor image quality of CT images.
      All subjects included into this research underwent abdominal multiphasic dynamic contrast-enhanced CT with the multi-detector row CT (MDCT) units (GE LightSpeed QX/I, GE Healthcare, USA or Siemens Sensation 16, Siemens, Germany). Detailed imaging parameters are listed as follows: 120 kV, auto tube current, field of view (FOV): 320–500 mm× 320–500 mm, scanning matrix: 512 × 512, reconstruction kernel: standard, scan type: helical, slice thickness 5 mm, slice gap: 5 mm, reconstructed section thickness 2 mm. The arterial-phase (AP), venous-phase (VP) and delay-phase (DP) CT were performed at 30–35 s, 65–70 s and 150–180 s after intravenous injection of contrast enhanced agent (Ultravist 370, Bayer Schering Pharma, Berlin, German, Dose: 1.5 mL/kg, injection rate: 3.0 mL/s).
      Pathological and clinical characteristics were likewise accessed from the TCGA-LIHC collection. Gender (0) and gender (1) respectively represent female and male. HCC was pathologically staged as IA (1), IB (2), II (3), IIIA (4), IIIB (5), IVA (6) and IVB (7).
      Percutaneous fine needle aspiration biopsy of liver lesion was performed in patients with local infiltration anesthesia. Then formalin-fixed paraffin-embedded biopsy specimens’ sections were stained with hematoxylin and eosin (H&E) for the following histopathological evaluation. All HCCs were classified into four grades (ES-1, ES-2, ES-3 and ES-4; Low Grade: ES-1 & ES-2, High Grade: ES-3 & ES-4) according to the Edmondson-Steiner grading guideline [
      • Edmondson H.A.
      • Steiner P.E.
      Primary carcinoma of the liver. A study of 100 cases among 48,900 necropsies.
      ].

      2.2 Radiomics and the diagnostic models

      The schematic flowchart of this study is shown in Fig. 1. The detailed processes were listed as the followings:

      2.2.1 Tumor segmentation

      The entire tumor in CT images were segmented via two abdominal radiologists with 17 years’ and 13 years’ experience, respectively, through an open-sourced software named as ITK-SNAP (http://www.itksnap.org/pmwiki/pmwiki.php). The lesions were segmented in the venous-phase (VP) CT images and then the volume of interests were copied to the other phases.

      2.2.2 Feature extraction

      The CT-derived radiomics features were extracted via an open-source Python-based software named Pyradiomics (https://pypi.python.org/pypi/pynetdicom). Radiomics features of 8 categories containing First Order Statistics (19 features), Shape-based (3D) (16 features), Shape-based (2D) (10 features), Gray Level Cooccurence Matrix (24 features), Gray Level Run Length Matrix (16 features), Gray Level Size Zone Matrix (16 features), Neighbouring Gray Tone Difference Matrix (5 features), Gray Level Dependence Matrix (14 features) were extracted. In total, 321 image features were obtained from the CT images (arterial phase, venous phase, delayed phase) of each patient.

      2.2.3 Feature reduction

      According to the statistical and algorithmic guidelines [

      Ying X. , An overview of overfitting and its solutions, J. Phys. Conf. Ser., 1168, 2019, 022022.

      ], redundant meaningfulness radiomics features will unnecessarily increase the model complexity and then holds potential risk of overfitting, which means the established models only have satisfying performance in training cohorts but have poor performance in validation cohorts. Therefore, it has been widely reported that well-designed feature selection strategies are necessary for establishing robust models [
      • Horvat N.
      • Veeraraghavan H.
      • Khan M.
      • et al.
      MR imaging of rectal cancer: radiomics analysis to assess treatment response after neoadjuvant therapy.
      ,
      • Homayounieh F.
      • Ebrahimian S.
      • Babaei R.
      • et al.
      CT radiomics, radiologists, and clinical information in predicting outcome of patients with COVID-19 pneumonia.
      ]. For this study, the image features were selected according to the following steps: (1) The intra-class coefficients (ICC) of each image feature were calculated to quantify the agreement and reproducibility. The image features with the ICC of less than 0.8 were removed. (2) The Student t test was utilized to screen the image features with significant differences between different subgroups (with micro-vascular invasion vs without micro-vascular invasion, high-grade HCC vs low-grade HCC). (3) Next, Least Absolute Shrinkage and Selection Operator (LASSO) regression or Support Vector Machine-Recursive Feature Elimination (SVM-RFE) was carried out to determine the ultimate feature subset.

      2.3 Radiomics-based diagnostic models

      Either for predicting the patients with micro-vascular invasion or the patients with high-grade HCC, machine-learning-based classifiers including the random forest (RF) and support vector machine (SVM) were established for achieving the diagnostic purpose with the different combination of feature subset by means of the different R packages including the randomForest and e1071. Consequently, a total of 8 models were constructed. 5-fold cross-validation was then used to select the best radiomics-based model with the highest area under the curve (AUC) of receiver operating characteristic (ROC) curves. To avoid the sampling bias, stratified sampling was performed in this study.

      2.4 Nomogram-based predictor

      The binary logistic regression model was firstly utilized to screen the independent clinical risk factors and establish the clinical model. The predictive probability of best radiomics model was determined as the Radiomics score (Rscore). Then, the nomogram-based predictors were constructed with Rscore and independent clinical risk factors.

      2.5 Statistical analysis

      ICC was calculated to quantify the intraclass agreement of the feature values given by two observers. The student t test was performed to explore the image features showing significant differences between different subgroups. Diagnostic performance of different models was evaluated by receiver operating characteristic (ROC) curve analysis. The detailed indexes of diagnostic performance included the sensitivity, specificity, area under the curves (AUC) and Youden index. It should be noted that in order to obtain the statistical results with reliability, the establishment, evaluation and comparison of radiomics model, clinical model and nomogram predictor were based on same 5-fold splitting data. P values of less than 0.05 were regarded as statistically different. All the statistical analysis were conducted with the SPSS 26.0 (SPSS, Chicago, IL, USA), R (R language 4.0.3, R Core Team, 2020) and Medcalc (MedCalc Software, Belgium).

      3. Results

      3.1 Patients cohort

      A total of 97 patients were accessed from the TCGA-LIHC collection (https://wiki.cancerimagingarchive.net/display/Public/TCGA-LIHC) in Cancer Imaging Archive (TCIA). 10 patients were excluded because of the absence of the complete CT images. 3 patients were excluded because of the unavailable histopathological results. 2 patients were excluded as the image quality was poor or incomplete images. Ultimately, 82 patients (Male: 54, Female: 28; Age: 61.8 ± 14.0, Min: 20, Max: 85;) were included. Detailed baseline clinical characteristics were listed in Table 1.
      Table 1Patients characteristics.
      CharacteristicsValuesp Values (p£/p§)
      Mean age (years)61.8 ± 14.0 (Min: 20, Max: 85)0.043£/0.021§
      GenderMen (54/65.9%)/Women (28/34.1%)0.039£/0.044§
      Mean Height (cm)167.1 ± 13.5 (Min: 64.0, Max: 188.0)0.872£/0.451§
      Mean Weight (kg)78.0 ± 20.9 (Min: 47.0, Max: 129.0)0.542£/0.770§
      RaceNA (6/7.3%)/Black (3/3.7%)/White (50/61.0%)/ Asia (23/28.0%)0.890£/0.718§
      Liver CirrhosisWith (59/72.0%)/Without (23/28.0%)0.072£/0.088§
      Tumor BurdenUnifocal (68/82.9%)/ Multifocal (14/17.1%)0.121£/0.205§
      No. of Lesions2.9 ± 2.3 (Min: 1.0, Max: 8.0)0.100£/0.151§
      Diameter (cm)5.9 ± 2.1 (Min: 1.2, Max: 14.2)0.080£/0.720§
      Segment LocationI-II (3/3.7%)/III (7/8.5%)/IV (18/22.0%)/V (13/15.9%)/VI (22/26.8%)/VII-VIII (19/23.2%)0.388£/0.659§
      EtiologyHepatitis B virus (25/30.5%)/Hepatitis C virus (34/41.5%)/Alcohol and other (23/28.0%)0.062£/0.215§
      MVINegative (25/30.5%)/ Positive (57/69.5%)0.040£
      GradeLow (53/64.6%))/High (29/35.4%)0.033§
      Alpha Fetoprotein7498.0 ± 13,787.0 (Min: 1.0, Max: 10,3900)0.022£/0.045§
      T StageT1 (35/42.7%)/T2 (16/19.5%)/T3 (7/8.5%)/T3a (9/11.0%)/T3b (8/9.8%)/T4 (7/8.5%)0.004£/0.002§
      N StageN0 (63/76.8%)/N1(3/3.7%)/NX (16/19.5%)0.101£/0.171§
      M StageM0 (68/82.9)/M1(4/4.9%)/MX (10/12.2%)0.235£/0.188§
      Child pugh classificationNA (12/14.6%)/A (53/64.6%)/B (17/20.7%)0.098£/0.297§
      ECOG Score0 (41/50.0%)/1 (32/39.0%)/2 (9/11.0%)0.132£/0.007§
      Note: 1) NA indicates the results are unknown or unavailable. 2) p£ and p§ represent the p values for histopathologic grade and micro-vascular invasion, respectively. The p values were calculated according to the Fisher's exact test for categorical variables and the p values were calculated according to the Mann-Whitney U test for continuous variables.

      3.2 Radiomics features and models

      A total of 321 Radiomics features were extracted from the arterial, venous and delayed phase of CT images of each included patient. 39 features were removed due to the low intra-class coefficient (ICC < 0.8). For identifying the histopathological grade and MVI, 217 and 251 features were then removed because there were no significant differences between the subgroups (high grade vs low grade and MVI (+) vs MVI (-)). Next, two feature selection strategies including LASSO Regression and SVM-RFE were respectively performed to further eliminate the redundant features. As Table 2 and Table 3 show, 7 and 10 radiomics features were selected via LASSO regression and SVM-RFE for assessing the tumor grade with regard to 13 and 10 features selected via LASSO regression and SVM-RFE for evaluating the MVI. Next, random combination of two feature selection methods and two machine-learning classifying algorithms resulted in four radiomics-based predictive models for either evaluating grade or evaluating MVI. The 5-fold diagnostic performance of above eight models were exhibited in Table 4, which suggested the combination of SVM-RFE and RF had the best performance for grading HCC (AUC values: Fold 1: 0.906, Fold 2: 0.917, Fold 3: 0.857, Fold 4: 0.938, Fold 5: 0.875, Mean: 0.898). Differently, the combination of LASSO regression and RF had the best performance for identifying the MVI (AUC values: Fold 1: 0.889, Fold 2:0.926, Fold 3: 0.889, Fold 4: 0.800, Fold 5: 0.875, Mean: 0.876). The ROC curves of the two best models are shown in Fig. 2. Consequently, the 10 features selected via SVM-RFE and 13 features selected via LASSO regression severed as the best feature subsets for evaluating the grade and MVI, respectively. Fig. 3 shows the categorical distribution of the aforementioned two feature subsets. Among the best feature subset for grading HCC, the number of features belonging to First-order features, GLCM Features, GLDM features, GLRLM features, GLSZM features, NGTDM features and shape features were 1 (10%), 2 (20%), 3 (40%), 0, 0, 1(10%) and 2 (20%), respectively. Additionally, among the best feature subset for assessing MVI, the number of features belonging to First-order features, GLCM Features, GLDM features, GLRLM features, GLSZM features, NGTDM features and shape features were 2 (15.4%), 3 (23.1%), 1 (7.7%), 1 (7.7%), 2 (15.4%), 1 (7.7%) and 3 (23.1%), respectively. Fig. 4 displayed the value distribution of selected features in different subgroups (high grade, low grade, MVI (+), MVI (-)).
      Table 2Selected features for grading HCC.
      Selection strategyRadiomics featuresCategoryEnhanced phase
      LASSO Regression (7 features)MedianFOVP
      AutocorrelationGLCMDP
      ContrastGLCMAP
      Joint EntropyGLCMAP
      Large Dependence High Gray Level EmphasisGLCMVP
      Low Gray Level Zone Emphasis (LGLZE)GLSZMVP
      ElongationShapeDP
      MedianFOAP
      SVM-RFE (Top 10 features)AutocorrelationGLCMVP
      ContrastGLCMVP
      Low Gray Level Emphasis (LGLE)GLDMDP
      Dependence Entropy (DE)GLDMAP
      Dependence Non-Uniformity (DN)GLDMVP
      Large Dependence Emphasis (LDE)GLDMDP
      CoarsenessNGTDMAP
      ElongationShapeDP
      FlatnessShapeDP
      Notes: 1) FO is the abbreviation of First-Order. 2) Enhanced phase indicates which phase of CT images the corresponding features were extracted from. 3) AP, VP, and DP represent the arterial-phase (AP), venous-phase (VP) and delay-phase (DP), respectively.
      Table 3Selected features for identifying the MVI status.
      Selection strategyRadiomics featuresCategoryEnhanced phase
      LASSO Regression

      (13 features)
      KurtosisFOAP
      Total EnergyFOVP
      AutocorrelationGLCMAP
      ContrastGLCMDP
      Difference EntropyGLCMVP
      Low Gray Level Emphasis (LGLE)GLDMVP
      Gray Level Non-Uniformity (GLN)GLRLMVP
      Low Gray Level Zone Emphasis (LGLZE)GLSZMAP
      Small Area Low Gray Level Emphasis (SALGLE)GLSZMVP
      CoarsenessNGTDMDP
      ElongationShapeVP
      FlatnessShapeAP
      Spherical DisproportionShapeDP
      SVM-RFE

      (Top 10 features)
      MeanFOVP
      Joint AverageGLCMAP
      AutocorrelationGLCMDP
      Cluster ShadeGLCMDP
      Difference EntropyGLCMVP
      Small Dependence Emphasis (SDE)GLDMVP
      Gray Level Non-Uniformity (GLN)GLRLMAP
      Low Gray Level Zone Emphasis (LGLZE)GLSZMVP
      Zone Percentage (ZP)GLSZMDP
      Spherical DisproportionShapeAP
      Notes: 1) FO is the abbreviation of First-Order. 2) Enhanced phase indicates which phase of CT images the corresponding features were extracted from. 3) AP, VP, and DP represent the arterial-phase (AP), venous-phase (VP) and delay-phase (DP), respectively.
      Table 4Diagnostic performance of radiomics-based models.
      Feature

      Selection
      ClassifierFold1Fold2Fold3Fold4Fold5Mean
      GradeLASSOSVM0.6250.6250.6860.6670.5860.638
      GradeSVM-RFESVM0.8000.6670.7000.7780.7290.735
      GradeLASSORF0.8060.6860.8150.7780.6390.745
      GradeSVM-RFERF0.9060.9170.8570.9380.8750.898
      MVILASSOSVM0.6110.6670.6670.8330.6250.681
      MVISVM-RFESVM0.8330.8330.8330.8330.7500.817
      MVILASSORF0.8890.9260.8890.8000.8750.876
      MVISVM-RFERF0.6240.7620.7150.780.8120.721
      Note: The values are the AUC values.
      Fig. 2
      Fig. 2Diagnostic performance of best radiomics-based models for evaluating the tumor grade and MVI.
      Fig. 3
      Fig. 3Categorical distribution of best feature subset for assessing tumor grade (first row) and MVI (second row).
      Fig. 4
      Fig. 4Value distribution of selected features in different subgroups (High Grade, Low Grade, MVI (+), MVI (-)). Note: 1) A grid in the longitudinal direction represents a patient. A grid on the horizontal represents a feature. 2) Only the best feature subsets for assessing the tumor grade and MVI are displayed.

      3.3 The construction of Nomograms with the integration of clinical Risk factor and Radiomics Score (Rscore)

      According to the binary logistic regression established with different clinical factors as independent variables and grade or MVI status as dependent variables, age, gender, alpha fetal protein (AFP) and tumor stage were identified as the independent risk factors of tumor grade (p < 0.05), and age, gender, AFP, tumor stage together with Eastern Cooperative Oncology Group (ECOG) score were identified as independent risk factors of MVI (p < 0.05) (Table 5). Therefore, the above risk factors and Rscore were utilized for constructing the Nomograms. The Nomograms utilized to assess the HCC grade and MVI status were displayed in Fig. 5 and Fig. 6. In addition, Fig. 5 and Fig. 6 also exhibited the diagnostic performance of different models including clinical models established with clinical factors, radiomics models established with radiomics features and nomogram predictors. The nomogram predictor possessed the best performance for predicting the tumor grade (AUC: 0.928) followed by radiomics model (AUC: 0.876) and the clinical model (AUC: 0.731). Similarly, the nomogram predictor also possessed the best performance for identifying the MVI status (AUC: 0.945) followed by the radiomics model (AUC: 0.890) and clinical model (AUC: 0.716) (Fig. 5, Fig. 6 and Table 6). As shown in Table 6, for predicting the HCC grade and MVI status, the diagnostic efficacy of the radiomics model was significantly higher than that of the clinical models. Furthermore, the results also indicated that the diagnostic performance of nomogram predictors for evaluating the grade as well as MVI is significantly better than not only clinical models but Radiomics model (p < 0.05).
      Table 5Determine independent clinical risk factor for histopathologic grade and micro-vascular invasion.
      Histopathologic Grade
      Clinical VariablesCoefficientsSDP values
      Age1.2560.4130.045
      Gender0.5230.3480.037
      AFP0.2740.0800.029
      Tumor Stage1.7670.6920.002
      MVI
      Clinical VariablesCoefficientsSDP values
      Age1.0750.6210.030
      Gender0.4810.2560.041
      AFP0.1880.0920.048
      Tumor Stage1.5830.8710.011
      Ecog Score1.9890.6740.005
      Fig. 5
      Fig. 5Nomogram-based predictor for grading HCC and the diagnostic performance comparison.
      Fig. 6
      Fig. 6Nomogram-based predictor for identifying MVI status and the diagnostic performance comparison.
      Table 6Diagnostic performance evaluation and comparison.
      Model Evaluation
      Sensitivity (%)Specificity (%)AUCYouden index
      Grade
      Clinical Model85.061.00.7310.460
      Radiomics Model90.080.50.8760.705
      Nomogram Predictor80.095.10.9280.751
      MVI
      Clinical Model95.056.10.7160.511
      Radiomics Model95.070.30.8900.657
      Nomogram Predictor95.080.50.9450.755
      Model Comparison
      GradeP Values
      Radiomics Model vs Clinical Model

      Nomogram predictor vs Clinical Model

      Nomogram predictor vs Radiomics Model

      MVI

      Radiomics Model vs Clinical Model

      Nomogram predictor vs Clinical Model

      Nomogram predictor vs Radiomics Model
      0.005
      0.002
      0.038
      0.002
      0.001
      0.045
      Note: The model highlighted by red color is the better model compared to the other model.

      4. Discussion

      The highlights of this research are as the followings: (1) The CT-derived radiomics features were utilized to construct the diagnostic models for predicting dual prognostic markers including the histopathological grade and MVI. Compared to a lot of previously-reported studies aiming to evaluate the single prognostic factor, more evaluation insights regarding the prognostic indicators will provide more comprehensive characterization of the tumor during clinical management. (2) The included patients in this study were from multi-centers. Besides, as displayed in Table 1, the patients in this study belonged to multiple races. The aforementioned data source will be conducive to prove the applicability of the strategy proposed in this research. (3) Integrating the clinical risk factors and radiomics features derived from CT images, the nomogram predictors showed excellent diagnostic efficacy for evaluating the histopathological grade and MVI.
      In this research, most features selected for predicting tumor grade and the MVI status were high-order texture features rather than widely-used first-order features. The results demonstrated that there were only 10.0% and 15.3% first-order features in the best feature subset, which was similar to plenty of previous researches [
      • Geng Z.
      • Zhang Y.
      • Wang S.
      • et al.
      Radiomics analysis of susceptibility weighted imaging for hepatocellular carcinoma: exploring the correlation between histopathology and radiomics features.
      ,
      • Yuan G.
      • Song Y.
      • Li Q.
      • et al.
      Development and validation of a contrast-enhanced CT-based radiomics nomogram for prediction of therapeutic efficacy of anti-PD-1 antibodies in advanced HCC patients.
      ,
      • Shu J.
      • Tang Y.
      • Cui J.
      • et al.
      Clear cell renal cell carcinoma: CT-based radiomics features for the prediction of Fuhrman grade.
      ]. During daily clinical practice, CT images based diagnostic conclusions are usually drawn by naked eyes. The diagnostic insights are essentially based on the first-order features such as the overall attenuation (mean, median value). Invisible to the naked eyes, a lot of high-order texture features are of great importance for clinical application [
      • She Y.
      • Zhang L.
      • Zhu H.
      • et al.
      The predictive value of CT-based radiomics in differentiating indolent from invasive lung adenocarcinoma in patients with pulmonary nodules.
      ,
      • Jiang C.
      • Luo Y.
      • Yuan J.
      • et al.
      CT-based radiomics and machine learning to predict spread through air space in lung adenocarcinoma.
      ,
      • Wang T.
      • Gong J.
      • Duan H.-H.
      • Wang L.-J.
      • Ye X.-D.
      • Nie S.-D.
      Correlation between CT based radiomics features and gene expression data in non-small cell lung cancer.
      ]. On the one hand, texture features are able to serve as the quantitative image markers for biomedical application. On the other hand, texture features can be utilized to construct the diagnostic models for various clinical applications such as tumor diagnosis, treatment efficacy evaluation and prognostic prediction. Moreover, the rapid development of artificial intelligence technology, including machine learning, deep learning, reinforcement learning and transfer learning, also brings unlimited possibilities for radiomics [
      • Lu C.-F.
      • Hsu F.-T.
      • Hsieh K.L.-C.
      • et al.
      Machine learning–based radiomics for molecular subtyping of gliomas.
      ,
      • Zheng X.
      • Yao Z.
      • Huang Y.
      • et al.
      Deep learning radiomics can predict axillary lymph node status in early-stage breast cancer.
      ]. Aforementioned issues further indicated the advantage of extracting CT-derived radiomics features for biomedical application.
      In detail, the best feature subset for grading HCC contained 10 features including Median, Autocorrelation, Contrast, Low Gray Level Emphasis (LGLE), Dependence Entropy (DE), Dependence Non-Uniformity (DN), Large Dependence Emphasis (LDE), Coarseness, Elongation and Flatness. Similarly, best feature subset for identifying the MVI status contained 13 features including Mean, Joint Average, Autocorrelation, Cluster Shade, Difference Entropy, Small Dependence Emphasis (SDE), Gray Level Non-Uniformity (GLN), Low Gray Level Zone Emphasis (LGLZE), Zone Percentage (ZP) and Spherical Disproportion. Above features are able to provide the characterization of tumor micro-structural heterogeneity in terms of gray level distribution, inhomogeneity of signal intensity, morphological differences and so forth. For example, Autocorrelation can be applied for quantifying the magnitude of the fineness and coarseness of texture. Tumors with high heterogeneity tend to have a coarser texture [
      • Cho H.-h
      • Lee S.-h
      • Kim J.
      • Park H.
      Classification of the glioma grading using radiomics analysis.
      ]. Contrast can be utilized to quantify the variation of local signal intensity [
      • Van Griethuysen J.J.
      • Fedorov A.
      • Parmar C.
      • et al.
      Computational radiomics system to decode the radiographic phenotype.
      ]. Elongation severs as the measure of irregularity of ROI shape [
      • Ahmed A.
      • Elmohr M.
      • Fuentes D.
      • et al.
      Radiomic mapping model for prediction of Ki-67 expression in adrenocortical carcinoma.
      ]. With the assistance of high-order features hidden under the naked eyes, different clinical models can be established to achieve different clinical goals.
      In this research, great diagnostic performance for evaluating the tumor grade and MVI were achieved with the radiomics based predictive model. The potential causes were as the followings: (1) Feature selection strategy was carefully designed. The feature selection in this study mainly contained 3 steps. Instable and meaningfulness features were firstly removed according to the ICC and Student t Test. Then, LASSO Regression and SVM-RFE were respectively performed. LASSO Regression and SVM-RFE are two machine learning-based feature selection strategies showing great potential in constructing the clinical predictive models [
      • Y-q Huang
      • C-h Liang
      • He L.
      • et al.
      Development and validation of a radiomics nomogram for preoperative prediction of lymph node metastasis in colorectal cancer.
      ,
      • Zhang X.
      • Xu X.
      • Tian Q.
      • et al.
      Radiomics assessment of bladder cancer grade using texture features from diffusion‐weighted imaging.
      ]. (2) Two machine learning classifiers including SVM and RF were then established. Compared to conventional linear classifiers such as regression-based models, through nonlinear transformation to high-dimensional feature space, SVM can construct a discriminant function in the high-dimensional feature space to realize the classification of samples, and cleverly avoids the problem of "dimension disaster" [
      • Noble W.S.
      What is a support vector machine?.
      ]. By means of integrating multiple classification tree, the random forest can achieve higher classification accuracy. In addition, due to the introduction of randomness, it has a certain anti-noise ability [
      • Qi Y.
      Random forest for bioinformatics.
      ]. (3) Rather than utilizing single feature selection approach and single classifier to construct single model for clinical application, the random combination of two feature selection approaches (LASSO-regression and SVM-RFE) and two classifiers (SVM and RF) altogether yielded 8 predictive models in this research, which was conducive to obtain the model with the best performance.
      To obtain the more powerful predictors, nomogram-based predictors were constructed with clinical risk factors and the radiomics model. Age, gender, tumor stage along with AFP were screened as the independent risk factors of HCC grade, and age, gender, AFP, tumor stage, and ECOG score were selected as independent risk factors of MVI. High tumor stage and higher expression of AFP were more common in the patients with MVI and high-grade HCC. Furthermore, higher ECOG score, in this study, also indicated a high probability of MVI. The above results were consistent with many previous researches [
      • Zhang Y.
      • Chen S.-w
      • Liu L.-l
      • Yang X.
      • Cai S.-h
      • Yun J.-p
      A model combining TNM stage and tumor size shows utility in predicting recurrence among patients with hepatocellular carcinoma after resection.
      ,
      • Ma W.-j
      • Wang H.-y
      • Teng L.-s
      Correlation analysis of preoperative serum alpha-fetoprotein (AFP) level and prognosis of hepatocellular carcinoma (HCC) after hepatectomy.
      ,
      • Choi H.J.
      • Cho B.C.
      • Sohn J.H.
      • et al.
      Brain metastases from hepatocellular carcinoma: prognostic factors and outcome.
      ]. Besides, our results also demonstrated that age and gender were also associated with the histopathologic grade and MVI of HCC, which corresponded to some previous findings that the age and gender also served as independent risk factors and then were incorporated into the nomogram-based predictors [
      • Zhou K.
      • Zhou Q.
      • Cai C.
      A genomic-clinicopathologic nomogram for predicting overall survival of hepatocellular carcinoma.
      ,
      • Li P.
      • Huang W.
      • Wang F.
      • et al.
      Nomograms based on inflammatory biomarkers for predicting tumor grade and micro-vascular invasion in stage I/II hepatocellular carcinoma.
      ,
      • Ma X.
      • Wei J.
      • Gu D.
      • et al.
      Preoperative radiomics nomogram for microvascular invasion prediction in hepatocellular carcinoma using contrast-enhanced CT.
      ]. Importantly, this study suggested that the integration of clinical indicators and radiomics resulted in fascinating diagnostic power for assessing the tumor grade and MVI (AUC > 0.900). The diagnostic efficacy of nomogram-based predictors was significantly better than that of either the radiomics model or clinical model. The above results revealed that CT-based radiomics can be applied for simultaneously predicting multiple important prognostic markers, which will be of great clinical potential for many other applications not limited to hepatic diseases but other cancers. The excellent predictive power may result from the following factors: (1) The combination of clinical risk factors and radiomics led to a comprehensive characterization of HCC from multiple perspectives. (2) radiomics-based model laid a solid foundation for the excellent performance of nomogram-based predictors.
      Several limitations should be acknowledged in this study. Firstly, although some approaches such as cross-validation have been carried out, the sample size of this study is not very large, which may hold potential risk for statistical bias. In the subsequent study, efforts need to be made to enroll more patients and further enhance the evidence. Secondly, no patient cohort was utilized as an external validation group. Thirdly, only two prognostic factors including MVI and grade were incorporated as the predictive target. More important markers should be incorporated to evaluate the feasibility of applying the radiomics-based model for predicting multiple markers.

      5. Conclusion

      This research indicated that CT-derived high-throughput radiomics features can serve as the quantitative biomarkers for characterizing hepatocellular carcinoma. Furthermore, with the assistance of machine learning, accurate and non-invasive prediction of histopathological grade as well as micro-vascular invasion can be achieved, which holds great potential for guiding the clinical management and predicting the prognosis of patients with HCC.

      Ethics approval and consent to participate

      The local ethical approval (20-1574AB) was obtained from the Institutional Review Board of Qiqihar Medical University. The written informed consents of all patient were successfully obtained, which is declared in the data source.

      Funding

      This project was supported by the Project of Heilongjiang Provincial Health Commission (Grant No. 2020-441 ).

      CRediT authorship contribution statement

      Jing Li: Conceptualization, Writing – review & editing. Xu Tong: Software, Data curation, Visualization, Investigation, Validation, Supervision, Writing – original draft, Methodology.

      Acknowledgements

      None.

      Declarations of interest

      None.

      References

        • Craig A.J.
        • Von Felden J.
        • Garcia-Lezana T.
        • Sarcognato S.
        • Villanueva A.
        Tumour evolution in hepatocellular carcinoma.
        Nat. Rev. Gastroenterol. Hepatol. 2020; 17: 139-152
        • Lencioni R.
        • Cioni D.
        • Donati F.
        • Bartolozzi C.
        Combination of interventional therapies in hepatocellular carcinoma.
        Hepato Gastroenterol. 2001; 48: 8-14
        • Liver EAFTSOT
        EASL clinical practice guidelines: management of hepatocellular carcinoma.
        J. Hepatol. 2018; 69: 182-236
        • Heimbach J.K.
        • Kulik L.M.
        • Finn R.S.
        • et al.
        AASLD guidelines for the treatment of hepatocellular carcinoma.
        Hepatology. 2018; 67: 358-380
        • Xu R.-h
        • Wei W.
        • Krawczyk M.
        • et al.
        Circulating tumour DNA methylation markers for diagnosis and prognosis of hepatocellular carcinoma.
        Nat. Mater. 2017; 16: 1155-1161
        • An C.
        • Park M.-S.
        • Jeon H.-M.
        • et al.
        Prediction of the histopathological grade of hepatocellular carcinoma using qualitative diffusion-weighted, dynamic, and hepatobiliary phase MRI.
        Eur. Radiol. 2012; 22: 1701-1708
        • Ameli S.
        • Shaghaghi M.
        • Ghasabeh M.A.
        • et al.
        Role of baseline volumetric functional MRI in predicting histopathologic grade and patients’ survival in hepatocellular carcinoma.
        Eur. Radiol. 2020; 30: 3748-3758
        • Jiang T.
        • Xu J.
        • Zou Y.
        • et al.
        Diffusion-weighted imaging (DWI) of hepatocellular carcinomas: a retrospective analysis of the correlation between qualitative and quantitative DWI and tumour grade.
        Clin. Radiol. 2017; 72: 465-472
        • Jhaveri K.S.
        • Cleary S.P.
        • Fischer S.
        • et al.
        Blood oxygen level‐dependent liver MRI: Can It predict microvascular invasion in HCC?.
        J. Magn. Reson. Imaging. 2013; 37: 692-699
        • Pawlik T.M.
        • Gleisner A.L.
        • Anders R.A.
        • Assumpcao L.
        • Maley W.
        • Choti M.A.
        Preoperative assessment of hepatocellular carcinoma tumor grade using needle biopsy: implications for transplant eligibility.
        Ann. Surg. 2007; 245: 435
        • Shen X.
        • Li N.
        • Li H.
        • Zhang T.
        • Wang F.
        • Li Q.
        Increased prevalence of regulatory T cells in the tumor microenvironment and its correlation with TNM stage of hepatocellular carcinoma.
        J. Cancer Res. Clin. 2010; 136: 1745-1754
        • Cao Y.
        • Ke R.
        • Wang S.
        • et al.
        DNA topoisomerase IIα and Ki67 are prognostic factors in patients with hepatocellular carcinoma.
        Oncol. Lett. 2017; 13: 4109-4116
        • Feng J.
        • Zhu R.
        • Chang C.
        • et al.
        CK19 and glypican 3 expression profiling in the prognostic indication for patients with HCC after surgical resection.
        PLoS One. 2016; 11e0151501
        • Chen Y.
        • Qin X.
        • Long L.
        • et al.
        Diagnostic value of Gd‐EOB‐DTPA‐enhanced MRI for the expression of Ki67 and microvascular density in hepatocellular carcinoma.
        J. Magn. Reson. Imaging. 2020; 51: 1755-1763
        • Gillies R.J.
        • Kinahan P.E.
        • Hricak H.
        Radiomics: images are more than pictures, they are data.
        Radiology. 2016; 278: 563-577
        • Scapicchio C.
        • Gabelloni M.
        • Barucci A.
        • Cioni D.
        • Saba L.
        • Neri E.
        A deep look into radiomics.
        La Radiol. Med. 2021; 126: 1296-1311
        • Song J.
        • Yin Y.
        • Wang H.
        • Chang Z.
        • Liu Z.
        • Cui L.
        A review of original articles published in the emerging field of radiomics.
        Eur. J. Radiol. 2020; 127108991
        • Richardson M.L.
        • Garwood E.R.
        • Lee Y.
        • et al.
        Noninterpretive uses of artificial intelligence in radiology.
        Acad. Radiol. 2021; 28: 1225-1235
        • Xu X.
        • Zhang H.-L.
        • Liu Q.-P.
        • et al.
        Radiomic analysis of contrast-enhanced CT predicts microvascular invasion and outcome in hepatocellular carcinoma.
        J. Hepatol. 2019; 70: 1133-1144
        • Ma X.
        • Wei J.
        • Gu D.
        • et al.
        Preoperative radiomics nomogram for microvascular invasion prediction in hepatocellular carcinoma using contrast-enhanced CT.
        Eur. Radiol. 2019; 29: 3595-3605
        • Peng J.
        • Zhang J.
        • Zhang Q.
        • Xu Y.
        • Zhou J.
        • Liu L.
        A radiomics nomogram for preoperative prediction of microvascular invasion risk in hepatitis B virus-related hepatocellular carcinoma.
        Diagn. Interv. Radiol. 2018; 24: 121
        • Mao B.
        • Zhang L.
        • Ning P.
        • et al.
        Preoperative prediction for pathological grade of hepatocellular carcinoma via machine learning–based radiomics.
        Eur. Radiol. 2020; 30: 6924-6932
        • Erickson B.
        • Kirk S.
        • Lee Y.
        • et al.
        Radiology data from the cancer genome atlas liver hepatocellular carcinoma [TCGA-LIHC] collection.
        Cancer Imaging Arch. 2016;
        • Edmondson H.A.
        • Steiner P.E.
        Primary carcinoma of the liver. A study of 100 cases among 48,900 necropsies.
        Cancer. 1954; 7: 462-503
      1. Ying X. , An overview of overfitting and its solutions, J. Phys. Conf. Ser., 1168, 2019, 022022.

        • Horvat N.
        • Veeraraghavan H.
        • Khan M.
        • et al.
        MR imaging of rectal cancer: radiomics analysis to assess treatment response after neoadjuvant therapy.
        Radiology. 2018; 287: 833-843
        • Homayounieh F.
        • Ebrahimian S.
        • Babaei R.
        • et al.
        CT radiomics, radiologists, and clinical information in predicting outcome of patients with COVID-19 pneumonia.
        Radiol. Cardiothorac. Imaging. 2020; 2e200322
        • Geng Z.
        • Zhang Y.
        • Wang S.
        • et al.
        Radiomics analysis of susceptibility weighted imaging for hepatocellular carcinoma: exploring the correlation between histopathology and radiomics features.
        Magn. Reson. Med. Sci. 2020; (mp. 2020-0060)
        • Yuan G.
        • Song Y.
        • Li Q.
        • et al.
        Development and validation of a contrast-enhanced CT-based radiomics nomogram for prediction of therapeutic efficacy of anti-PD-1 antibodies in advanced HCC patients.
        Front. Immunol. 2020; 11
        • Shu J.
        • Tang Y.
        • Cui J.
        • et al.
        Clear cell renal cell carcinoma: CT-based radiomics features for the prediction of Fuhrman grade.
        Eur. J. Radiol. 2018; 109: 8-12
        • She Y.
        • Zhang L.
        • Zhu H.
        • et al.
        The predictive value of CT-based radiomics in differentiating indolent from invasive lung adenocarcinoma in patients with pulmonary nodules.
        Eur. Radiol. 2018; 28: 5121-5128
        • Jiang C.
        • Luo Y.
        • Yuan J.
        • et al.
        CT-based radiomics and machine learning to predict spread through air space in lung adenocarcinoma.
        Eur. Radiol. 2020; 30: 4050-4057
        • Wang T.
        • Gong J.
        • Duan H.-H.
        • Wang L.-J.
        • Ye X.-D.
        • Nie S.-D.
        Correlation between CT based radiomics features and gene expression data in non-small cell lung cancer.
        J. X-Ray Sci. Technol. 2019; 27: 773-803
        • Lu C.-F.
        • Hsu F.-T.
        • Hsieh K.L.-C.
        • et al.
        Machine learning–based radiomics for molecular subtyping of gliomas.
        Clin. Cancer Res. 2018; 24: 4429-4436
        • Zheng X.
        • Yao Z.
        • Huang Y.
        • et al.
        Deep learning radiomics can predict axillary lymph node status in early-stage breast cancer.
        Nat. Commun. 2020; 11: 1-9
        • Cho H.-h
        • Lee S.-h
        • Kim J.
        • Park H.
        Classification of the glioma grading using radiomics analysis.
        PeerJ. 2018; 6e5982
        • Van Griethuysen J.J.
        • Fedorov A.
        • Parmar C.
        • et al.
        Computational radiomics system to decode the radiographic phenotype.
        Cancer Res. 2017; 77: e104-e107
        • Ahmed A.
        • Elmohr M.
        • Fuentes D.
        • et al.
        Radiomic mapping model for prediction of Ki-67 expression in adrenocortical carcinoma.
        Clin. Radiol. 2020; 75: 479. e417-479. e422
        • Y-q Huang
        • C-h Liang
        • He L.
        • et al.
        Development and validation of a radiomics nomogram for preoperative prediction of lymph node metastasis in colorectal cancer.
        J. Clin. Oncol. 2016; 34: 2157-2164
        • Zhang X.
        • Xu X.
        • Tian Q.
        • et al.
        Radiomics assessment of bladder cancer grade using texture features from diffusion‐weighted imaging.
        J. Magn. Reson. Imaging. 2017; 46: 1281-1288
        • Noble W.S.
        What is a support vector machine?.
        Nat. Biotechnol. 2006; 24: 1565-1567
        • Qi Y.
        Random forest for bioinformatics.
        Ensemble Machine Learning. Springer, 2012: 307-323
        • Zhang Y.
        • Chen S.-w
        • Liu L.-l
        • Yang X.
        • Cai S.-h
        • Yun J.-p
        A model combining TNM stage and tumor size shows utility in predicting recurrence among patients with hepatocellular carcinoma after resection.
        Cancer Manag. Res. 2018; 10: 3707
        • Ma W.-j
        • Wang H.-y
        • Teng L.-s
        Correlation analysis of preoperative serum alpha-fetoprotein (AFP) level and prognosis of hepatocellular carcinoma (HCC) after hepatectomy.
        World J. Surg. Oncol. 2013; 11: 1-7
        • Choi H.J.
        • Cho B.C.
        • Sohn J.H.
        • et al.
        Brain metastases from hepatocellular carcinoma: prognostic factors and outcome.
        J. Neuro Oncol. 2009; 91: 307-313
        • Zhou K.
        • Zhou Q.
        • Cai C.
        A genomic-clinicopathologic nomogram for predicting overall survival of hepatocellular carcinoma.
        BMC Cancer. 2020; 20: 1-10
        • Li P.
        • Huang W.
        • Wang F.
        • et al.
        Nomograms based on inflammatory biomarkers for predicting tumor grade and micro-vascular invasion in stage I/II hepatocellular carcinoma.
        Biosci. Rep. 2018; 38
        • Ma X.
        • Wei J.
        • Gu D.
        • et al.
        Preoperative radiomics nomogram for microvascular invasion prediction in hepatocellular carcinoma using contrast-enhanced CT.
        Eur. Radiol. 2019;