Principal Investigator: Montse Pardàs
Email: montse.pardas@upc.edu
Web: www.imatge.upc.edu
Brief Theme Descriptions
The selected candidate will work on generative AI for medical imaging. The synthesis of medical images has a wide range of uses, including data augmentation for training datasets, enhancing image quality, noise reduction, and anomaly detection. These applications are critical for improving diagnostic accuracy and developing robust medical imaging technologies.
Currently, multi-modal foundation models for natural images are widely available, having been trained on millions of pairs of natural images and text captions. However, the development of similar models for medical imaging faces significant challenges. Ethical and legal constraints on sharing medical data, combined with the diversity and scarcity of such data, limit progress in this field. These barriers necessitate innovative approaches to adapt and expand existing models to meet the specific needs of medical imaging.
This research will aim to extend the representation capabilities of large pretrained foundation models to generate domain-specific images. Techniques such as LoRA fine-tuning and textual inversion for diffusion models will be explored to achieve this goal. Two specific domains of focus, due to ongoing participation in European and national projects, are prostate MRI and histology images. These domains provide a unique opportunity to address pressing challenges in medical imaging while contributing to impactful healthcare solutions.
Available Infrastructures: GPU Computing Services and UPC Facilities.
Possible Secondments: Several possibilities, to be agreed with the candidate.
Keywords: Medical Imaging; Deep Learning; Multi-Modal Foundation Models.
