Events

Do Foundation Models See The Same Sky?

Foundation models promise to unify astronomy’s fragmented multimodal data, but building one and putting it to scientific use involves solving a puzzle of interconnected pieces. We will discuss some of these pieces, starting with the Multimodal Universe, which contains 80+ TB of data from over 30 surveys, recently converted to the HATS format and uploaded onto Hugging Face, enabling fast crossmatching from your laptop. We will then discuss an example astro-foundation model: AstroPT, an autoregressive transformer that we found scales with compute analagously to a language model. Finally, we will discuss how we can use crossmatched images and spectra of the same galaxies to test the Platonic Representation Hypothesis. We found that better performing models converge to a more similar local embedding space, but do not agree on where to embed galaxys globally. Taken holistically, our PU results suggest that astronomy may be able to build on general-purpose pretrained models instead of pre-training from scratch.

Monday, November 2, 2026 9:30 AM

Inperson
Speaker will be in person – BHI Conference Room

Virtual
Zoom