Korean Journal of Radiology

The Korean Society of Radiology (대한영상의학회)
권호 표지
DOI QR코드

DOI QR Code

Interpretation of Complete Tumor Response on MRI Following Chemoradiotherapy of Rectal Cancer: Inter-Reader Agreement and Associated Factors in Multi-Center Clinical Practice

  • Hae Young Kim (Department of Radiology, Seoul National University Bundang Hospital) ;
  • Seung Hyun Cho (Department of Radiology, Kyungpook National University Chilgok Hospital, School of Medicine, Kyungpook National University) ;
  • Jong Keon Jang (Department of Radiology and Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center) ;
  • Bohyun Kim (Department of Radiology, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea) ;
  • Chul-min Lee (Department of Radiology, Hanyang University College of Medicine) ;
  • Joon Seok Lim (Department of Radiology, Severance Hospital, Yonsei University College of Medicine) ;
  • Sung Kyoung Moon (Department of Radiology, Kyung Hee University Hospital) ;
  • Soon Nam Oh (Department of Radiology, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea) ;
  • Nieun Seo (Department of Radiology, Severance Hospital, Yonsei University College of Medicine) ;
  • Seong Ho Park (Department of Radiology and Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center)
  • Received : 2023.12.06
  • Accepted : 2024.02.03
  • Published : 2024.04.01
Download PDF
⟨ Previous Next ⟩

Abstract

Objective: To measure inter-reader agreement and identify associated factors in interpreting complete response (CR) on magnetic resonance imaging (MRI) following chemoradiotherapy (CRT) for rectal cancer. Materials and Methods: This retrospective study involved 10 readers from seven hospitals with experience of 80-10210 cases, and 149 patients who underwent surgery after CRT for rectal cancer. Using MRI-based tumor regression grading (mrTRG) and methods employed in daily practice, the readers independently assessed mrTRG, CR on T2-weighted images (T2WI) denoted as mrCRT2W, and CR on all images including diffusion-weighted images (DWI) denoted as mrCRoverall. The readers described their interpretation patterns and how they utilized DWI. Inter-reader agreement was measured using multi-rater kappa, and associated factors were analyzed using multivariable regression. Correlation between sensitivity and specificity of each reader was analyzed using Spearman coefficient. Results: The mrCRT2W and mrCRoverall rates varied widely among the readers, ranging 18.8%-40.3% and 18.1%-34.9%, respectively. Nine readers used DWI as a supplement sequence, which modified interpretations on T2WI in 2.7% of cases (36/1341 [149 patients × 9 readers]) and mostly (33/36) changed mrCRT2W to non-mrCRoverall. The kappa values for mrTRG, mrCRT2W, and mrCRoverall were 0.56 (95% confidence interval: 0.49, 0.62), 0.55 (0.52, 0.57), and 0.54 (0.51, 0.57), respectively. No use of rectal gel, larger initial tumor size, and higher initial cT stage exhibited significant association with a higher interreader agreement for assessing mrCRoverall (P ≤ 0.042). Strong negative correlations were observed between the sensitivity and specificity of individual readers (coefficient, -0.718 to -0.963; P ≤ 0.019). Conclusion: Inter-reader agreement was moderate for assessing CR on post-CRT MRI. Readers' varying standards on MRI interpretation (i.e., threshold effect), along with the use of rectal gel, initial tumor size, and initial cT stage, were significant factors associated with inter-reader agreement.

Keywords

Acknowledgement

This study was supported by a research fund from the Korean Society of Radiology through Radiology Imaging Network of Korea for Clinical Research (RINK-CR) and by the Korean Society of Abdominal Radiology.

References

  1. van der Valk MJM, Hilling DE, Bastiaannet E, Meershoek-Klein Kranenbarg E, Beets GL, Figueiredo NL, et al. Long-term outcomes of clinical complete responders after neoadjuvant treatment for rectal cancer in the International Watch & Wait Database (IWWD): an international multicentre registry study. Lancet 2018;391:2537-2545 
  2. Smith FM, Cresswell K, Myint AS, Renehan AG. Is "watch-and-wait" after chemoradiotherapy safe in patients with rectal cancer? BMJ 2018;363:k4472 
  3. Maas M, Nelemans PJ, Valentini V, Das P, Rodel C, Kuo LJ, et al. Long-term outcome in patients with a pathological complete response after chemoradiation for rectal cancer: a pooled analysis of individual patient data. Lancet Oncol 2010;11:835-844 
  4. Jayaprakasam VS, Alvarez J, Omer DM, Gollub MJ, Smith JJ, Petkovska I. Watch-and-wait approach to rectal cancer: the role of imaging. Radiology 2023;307:e221529 
  5. Jang JK, Choi SH, Park SH, Kim KW, Kim HJ, Lee JS, et al. MR tumor regression grade for pathological complete response in rectal cancer post neoadjuvant chemoradiotherapy: a systematic review and meta-analysis for accuracy. Eur Radiol 2020;30:2312-2323 
  6. Park SH, Cho SH, Choi SH, Jang JK, Kim MJ, Kim SH, et al. MRI assessment of complete response to preoperative chemoradiation therapy for rectal cancer: 2020 guide for practice from the Korean Society of Abdominal Radiology. Korean J Radiol 2020;21:812-828 
  7. Benson AB, Venook AP, Al-Hawary MM, Azad N, Chen YJ, Ciombor KK, et al. Rectal cancer, version 2.2022, NCCN clinical practice guidelines in oncology. J Natl Compr Canc Netw 2022;20:1139-1167 
  8. Bates DDB, Shaish H, Gollub MJ, Harisinghani M, Lall C, Sheedy SP. Multi-practice survey on MR imaging practice patterns in rectal cancer in the United States. Abdom Radiol (NY) 2022;47:28-37 
  9. Hong CW, Chernyak V, Choi JY, Lee S, Potu C, Delgado T, et al. A multicenter assessment of interreader reliability of LI-RADS version 2018 for MRI and CT. Radiology 2023;307:e222855 
  10. Hsieh SS, Cook DA, Inoue A, Gong H, Sudhir Pillai P, Johnson MP, et al. Understanding reader variability: a 25-radiologist study on liver metastasis detection at CT. Radiology 2023;306:e220266 
  11. Min JH, Lee MW, Park HS, Lee DH, Park HJ, Lim S, et al. Interobserver variability and diagnostic performance of gadoxetic acid-enhanced MRI for predicting microvascular invasion in hepatocellular carcinoma. Radiology 2020;297:573-581 
  12. El Khababi N, Beets-Tan RGH, Tissier R, Lahaye MJ, Maas M, Curvo-Semedo L, et al. Comparison of MRI response evaluation methods in rectal cancer: a multicentre and multireader validation study. Eur Radiol 2023;33:4367-4377 
  13. Yuval JB, Patil S, Gangai N, Omer DM, Akselrod DG, Fung A, et al. MRI assessment of rectal cancer response to neoadjuvant therapy: a multireader study. Eur Radiol 2023;33:5761-5768 
  14. Aker M, Boone D, Chandramohan A, Sizer B, Motson R, Arulampalam T. Diagnostic accuracy of MRI in assessing tumor regression and identifying complete response in patients with locally advanced rectal cancer after neoadjuvant treatment. Abdom Radiol (NY) 2018;43:3213-3219 
  15. Lee MA, Cho SH, Seo AN, Kim HJ, Shin KM, Kim SH, et al. Modified 3-point MRI-based tumor regression grade incorporating DWI for locally advanced rectal cancer. AJR Am J Roentgenol 2017;209:1247-1255 
  16. Sclafani F, Brown G, Cunningham D, Wotherspoon A, Mendes LST, Balyasnikova S, et al. Comparison between MRI and pathology in the assessment of tumour regression grade in rectal cancer. Br J Cancer 2017;117:1478-1485 
  17. Siddiqui MR, Gormly KL, Bhoday J, Balyansikova S, Battersby NJ, Chand M, et al. Interobserver agreement of radiologists assessing the response of rectal cancers to preoperative chemoradiation using the MRI tumour regression grading (mrTRG). Clin Radiol 2016;71:854-862 
  18. van den Broek JJ, van der Wolf FS, Lahaye MJ, Heijnen LA, Meischl C, Heitbrink MA, et al. Accuracy of MRI in restaging locally advanced rectal cancer after preoperative chemoradiation. Dis Colon Rectum 2017;60:274-283 
  19. Voogt ELK, Nordkamp S, van Zoggel DMGI, Daniels-Gooszen AW, Nieuwenhuijzen GAP, Bloemen JG, et al. MRI tumour regression grade in locally recurrent rectal cancer. BJS Open 2022;6:zrac033 
  20. Yoen H, Park HE, Kim SH, Yoon JH, Hur BY, Bae JS, et al. Prognostic value of tumor regression grade on MR in rectal cancer: a large-scale, single-center experience. Korean J Radiol 2020;21:1065-1076 
  21. Jang JK, Lee JL, Park SH, Park HJ, Park IJ, Kim JH, et al. Magnetic resonance tumour regression grade and pathological correlates in patients with rectal cancer. Br J Surg 2018;105:1671-1679 
  22. Kottner J, Audige L, Brorson S, Donner A, Gajewski BJ, Hrobjartsson A, et al. Guidelines for reporting reliability and agreement studies (GRRAS) were proposed. J Clin Epidemiol 2011;64:96-106 
  23. Kim SH, Chang HJ, Kim DY, Park JW, Baek JY, Kim SY, et al. What is the ideal tumor regression grading system in rectal cancer patients after preoperative chemoradiotherapy? Cancer Res Treat 2016;48:998-1009 
  24. Gormly KL. High-resolution T2-weighted MRI to evaluate rectal cancer: why variations matter. Korean J Radiol 2021;22:1475-1480 
  25. Royal College of Radiologists. Recommendations for cross-sectional imaging in cancer management, second edition [accessed on November 10, 2023]. Available at: https://www.rcr.ac.uk/media/mv1hidxf/rcr-publications_recommendations-for-cross-sectional-imaging-in-cancer-management-second-edition-12-colon-rectum-and-anal-canal-cancer_april-2022.pdf
  26. Cancer Council Australia. Clinical practice guidelines for the prevention, early detection and management of colorectal cancer [accessed on November 10, 2023]. Available at: https://www.cancer.org.au/clinical-guidelines/bowel-cancer/colorectal-cancer 
  27. Beets-Tan RGH, Lambregts DMJ, Maas M, Bipat S, Barbaro B, Curvo-Semedo L, et al. Magnetic resonance imaging for clinical management of rectal cancer: updated recommendations from the 2016 European Society of Gastrointestinal and Abdominal Radiology (ESGAR) consensus meeting. Eur Radiol 2018;28:1465-1475 
  28. Gollub MJ, Arya S, Beets-Tan RG, dePrisco G, Gonen M, Jhaveri K, et al. Use of magnetic resonance imaging in rectal cancer patients: Society of Abdominal Radiology (SAR) rectal cancer disease-focused panel (DFP) recommendations 2017. Abdom Radiol (NY) 2018;43:2893-2902 
  29. Shin Y, Kim KW, Lee AJ, Sung YS, Ahn S, Koo JH, et al. A good practice-compliant clinical trial imaging management system for multicenter clinical trials: development and validation study. JMIR Med Inform 2019;7:e14310 
  30. Bhoday J, Smith F, Siddiqui MR, Balyasnikova S, Swift RI, Perez R, et al. Magnetic resonance tumor regression grade and residual mucosal abnormality as predictors for pathological complete response in rectal cancer postneoadjuvant chemoradiotherapy. Dis Colon Rectum 2016;59:925-933 
  31. Santiago I, Barata M, Figueiredo N, Pares O, Henriques V, Galzerano A, et al. The split scar sign as an indicator of sustained complete response after neoadjuvant therapy in rectal cancer. Eur Radiol 2020;30:224-238 
  32. Patel UB, Brown G, Rutten H, West N, Sebag-Montefiore D, Glynne-Jones R, et al. Comparison of magnetic resonance imaging and histopathological response to chemoradiotherapy in locally advanced rectal cancer. Ann Surg Oncol 2012;19:2842-2852 
  33. Quarfoot D, Levine RA. How robust are multirater interrater reliability indices to changes in frequency distribution? Am Stat 2016;70:373-384 
  34. Merkel S, Mansmann U, Siassi M, Papadopoulos T, Hohenberger W, Hermanek P. The prognostic inhomogeneity in pT3 rectal carcinomas. Int J Colorectal Dis 2001;16:298-304 
  35. Leeflang MM, Deeks JJ, Gatsonis C, Bossuyt PM. Systematic reviews of diagnostic test accuracy. Ann Intern Med 2008;149:889-897 
  36. Tan Y, Fu D, Li D, Kong X, Jiang K, Chen L, et al. Predictors and risk factors of pathologic complete response following neoadjuvant chemoradiotherapy for rectal cancer: a population-based analysis. Front Oncol 2019;9:497 
  37. Lorimer PD, Motz BM, Kirks RC, Boselli DM, Walsh KK, Prabhu RS, et al. Pathologic complete response rates after neoadjuvant treatment in rectal cancer: an analysis of the national cancer database. Ann Surg Oncol 2017;24:2095-2103 
  38. Lee S, Kassam Z, Baheti AD, Hope TA, Chang KJ, Korngold EK, et al. Rectal cancer lexicon 2023 revised and updated consensus statement from the Society of Abdominal Radiology Colorectal and Anal Cancer Disease-Focused Panel. Abdom Radiol (NY) 2023;48:2792-2806 
  39. Fokas E, Appelt A, Glynne-Jones R, Beets G, Perez R, Garcia-Aguilar J, et al. International consensus recommendations on key outcome measures for organ preservation after (chemo) radiotherapy in patients with rectal cancer. Nat Rev Clin Oncol 2021;18:805-816 
  40. Jang JK, Lee CM, Park SH, Kim JH, Kim J, Lim SB, et al. How to combine diffusion-weighted and T2-weighted imaging for MRI assessment of pathologic complete response to neoadjuvant chemoradiotherapy in patients with rectal cancer? Korean J Radiol 2021;22:1451-1461 
  41. Lambregts DMJ, Delli Pizzi A, Lahaye MJ, van Griethuysen JJM, Maas M, Beets GL, et al. A pattern-based approach combining tumor morphology on MRI with distinct signal patterns on diffusion-weighted imaging to assess response of rectal tumors after chemoradiotherapy. Dis Colon Rectum 2018;61:328-337 
  42. Maas M, Lambregts DM, Nelemans PJ, Heijnen LA, Martens MH, Leijtens JW, et al. Assessment of clinical complete response after chemoradiation for rectal cancer with digital rectal examination, endoscopy, and MRI: selection for organ-saving treatment. Ann Surg Oncol 2015;22:3873-3880 
  43. Hall WA, Li J, You YN, Gollub MJ, Grajo JR, Rosen M, et al. Prospective correlation of magnetic resonance tumor regression grade with pathologic outcomes in total neoadjuvant therapy for rectal adenocarcinoma. J Clin Oncol 2023;41:4643-4651 
  44. Chandramohan A, Siddiqi UM, Mittal R, Eapen A, Jesudason MR, Ram TS, et al. Diffusion weighted imaging improves diagnostic ability of MRI for determining complete response to neoadjuvant therapy in locally advanced rectal cancer. Eur J Radiol Open 2020;7:100223