Navigating the Multimodal Map: Insights into Foundation Models

Venue

Online training course run by the NextGen Data Scientists, AstraZeneca

Trainers

Sylwia Majchrowska, Ricardo Mokhtari

Course structure and links

Day	Title	Activity	Materials
0	Troubleshooting software installations	preparation	Introduction and installations
1	SAM Concept Cove	Session	Materials
2	Multimodal data handling	Session	Materials

References

1. LangSAM Code
2. Grounding DiNO Code Paper
3. SAM Code Paper
4. Attention illustrated blog
5. Attention video
6. Another attention video
7. Cross attention
8. Visualise a transformer
9. The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification
10. MMML Tutorial - ICML 2023
11. Multimodal data fusion – analysis
12. Fusion of Multi-Modal Data Stream for Clinical Event Prediction - Imon Banerjee, PhD
13. Data-Efficient Multimodal Fusion on a Single GPU
14. Integrated multimodal artificial intelligence framework for healthcare applications
15. Inferring multimodal latent topics from electronic health records
16. Multimodal Risk Prediction with Physiological Signals, Medical Images and Clinical Notes