A Python Pipeline for Cancer Detection in Medical Images

In shortLoad and audit the data, split by patient, preprocess for the modality, fine-tune a pretrained model, evaluate with sensitivity and specificity, and explain predictions with Grad-CAM. Version everything.

For research and education only — not for clinical use.

1. Tools

Python with NumPy and pandas; pydicom or SimpleITK for medical formats; OpenCV and Pillow for images; PyTorch or TensorFlow/Keras for models; scikit-learn for metrics; Matplotlib for figures.

2. Audit the data

Count images per class, per patient and per source. Look at samples from each class. Check for duplicates, text burned into images and label noise.

3. Split correctly

Create train, validation and test sets at patient level. Keep the test set untouched until the end.

4. Preprocess for the modality

• Histopathology: tile large slides into patches, apply stain normalisation.
• Mammography: keep high resolution, handle left/right orientation.
• CT: window intensities (lung, soft tissue) before scaling.
• Dermoscopy: colour constancy, hair removal if needed.

5. Train

Start with a pretrained CNN, replace the classifier head, train with class-weighted loss if classes are imbalanced, and use early stopping on validation loss.

6. Evaluate

Report sensitivity (recall), specificity, precision, F1 and ROC-AUC with confidence intervals. Add a confusion matrix and calibration plot.

7. Explain

Generate Grad-CAM maps for correct and incorrect predictions. Discuss failure cases — reviewers value honesty.

8. Reproduce

Fix random seeds, save the environment (requirements.txt), and keep a notebook that runs end to end.

Related: brain tumor MRI classification and how much data you need.

Want this done for your dataset? See cancer detection in medical images with Python.