arXiv:2611.03109v1 Announce Type: cross
Abstract: Accurate preoperative prediction of lung adenocarcinoma invasiveness is important for surgical planning and individualized treatment. However, overlapping CT appearances make it challenging to distinguish invasive adenocarcinoma (IAD) from Pre-IAD lesions. We propose a multimodal prompt-learning framework with dual-granularity image-text alignment for 3D CT-based IAD classification. A 3D ResNet-CBAM encoder extracts global and spatial visual features. A frozen RadBERT backbone is adapted with separate learnable prompts for report-level and keyword-level text encoding. These prompts are initialized using semantic representations derived from real-world clinical reports. Global contrastive alignment connects whole-volume CT representations with reports generated from clinician-annotated radiological attributes to provide overall semantic supervision. A multiple instance learning-based local alignment module associates spatial CT features with vascular-focused radiological keywords, to capture local imaging characteristics potentially associated with tumor invasion. The framework jointly optimizes classification and alignment objectives, while inference requires only CT images. Experiments included 1,593 pathologically confirmed nodules. On the test set, the proposed method achieved an accuracy of 87.92%, F1-score of 87.78%, and AUC of 0.9499. Its AUC significantly exceeded that of the image-only Res-CBAM baseline with the same visual backbone (0.9297; DeLong test, p = 0.0078). Ablation analyses supported the complementary contributions of global and local alignment and the benefit of report-based prompt initialization. Qualitative visualizations showed activation within nodules and adjacent vascular structures. These findings suggest that dual-granularity radiological supervision can enhance CT-based invasiveness prediction while preserving a CT-only inference workflow.
arxiv
Multimodal Prompt Learning with Dual-Granularity Contrastive Alignment for Preoperative Prediction of Lung Adenocarcinoma Invasiveness Using 3D CT
This paper presents a multimodal prompt-learning framework that combines 3D CT images with text prompts derived from clinical reports to predict lung adenocarcinoma invasiveness. The method uses a 3D ResNet-CBAM encoder and a frozen RadBERT with learnable prompts, plus global and local contrastive alignment modules. On a test set of 1,593 nodules, it achieved 87.92% accuracy, 87.78% F1-score, and 0.9499 AUC, significantly outperforming an image-only baseline. The framework requires only CT images at inference, preserving clinical usability.
The paper reports a concrete, recent AI development with strong quantitative results and a practical CT-only inference workflow, offering clear value for medical imaging and multimodal learning readers.
Comments & Questions (0)
No comments yet
Be the first to comment!