Point Spread Function Modelling with Neural Fields and a Differentiable Optical Model

Full description

Context. Weak gravitational lensing [1] is a powerful probe of the Large Scale Structure of our Universe. Cosmologists use weak lensing to study the nature of dark matter and its spatial distribution. Weak lensing missions require highly accurate shape measurements of galaxy images. The telescope’s instrumental response, or point spread function (PSF), produces a deformation of the observed images. This deformation can be mistaken for the effects of weak lensing in the galaxy images, thus being one of the primary sources of systematic error when doing weak lensing science. Therefore, estimating a reliable and accurate PSF model is crucial for the success of any weak lensing mission. The PSF field can be interpreted as a convolutional kernel that affects each of our observations of interest that varies spatially, spectrally, and temporally. The PSF model needs to be able to cope with each of these variations. We use specific stars considered as point sources in the field-of-view to constrain our PSF model. These stars, which are unresolved objects, provide us with degraded samples of the PSF field. The observations go through different types of degradations depending on the properties of the telescope. Some of these degradations include undersampling, an integration over the instrument’s passband, and additive noise. We finally build the PSF model using these degraded observations and then use the model to infer the PSF at the position of galaxies. This procedure constitutes the ill-posed inverse problem of PSF modelling. See [2] for a recent review on PSF modelling.
The recently launched Euclid survey represents one of the most complex challenges for PSF modelling. Because of the very broad passband of Euclid’s visible imager (VIS) ranging from 550nm to 900nm, PSF models need to capture not only the PSF field spatial variations but also its chromatic variations. Each star observation is inte- grated with the object’s spectral energy distribution (SED) over the whole VIS passband. As the observations are undersampled, a super-resolution step is also required. A recent model coined WaveDiff [3] was proposed to tackle the PSF modelling problem for Euclid and is based on a differentiable optical model. WaveDiff achieved state-of-the-art performance and is currently being implemented into Euclid’s data processing pipelines.
Goals. The recently introduced neural fields [4] have shown impressive performance in computer vision tasks. Neural fields are coordinate-based neural networks that parametrise physical properties of scenes or objects across space (and time). These networks gained particular visibility by tackling the problem of 3D scene reconstruction [5] from several 2D images of the scene.
If we consider the (x,y) focal plane coordinates as viewing directions, the PSF modelling problem can be considered as a scene reconstruction from a fixed set of 2D images (the observed stars). One goal of the internship is to adapt these ideas in recent neural fields works and combine them with the WaveDiff model. The next goal is to build a new PSF model that would allow us to better capture the spatial variations of the PSF field. One last goal is to include spurious spectral variations in the neural field and condition it on existing simulations.
The candidate
The candidate must be pursuing a Master 2 degree (or equivalent) and should have a background in signal/image processing. The candidate should be comfortable with the Python programming language, and ideally with a deep learning framework (e.g. TensorFlow, PyTorch, JAX) and open-source and collaborative development tools (e.g. GitHub). Knowledge of machine learning, Fourier optics and experience processing astronomical images is not required but is beneficial.
The candidate will acquire expertise in astronomical image processing, Fourier optics, machine learning and deep learning. In addition, the intern will learn to work in a collaborative development environment and contribute to existing software packages. The knowledge acquired during the internship applies to a wide range of applications in various fields, e.g., biomedical imaging, and astrophysics.

Contact
The internship will take place in the LILAS lab (CEA Saclay), which is working on signal (and image) processing and machine learning applied to physics applications.

References
[1] R. Mandelbaum. “Weak Lensing for Precision Cosmology”. In: Annual Review of Astronomy and Astro- physics 56 (2018), pp. 393–433. doi: 10.1146/annurev-astro-081817-051928. arXiv: 1710.03235.
[2] T. Liaudat, J.-L. Starck, M. Kilbinger, and P.-A. Frugier. “Point spread function modelling for astronomical
telescopes: a review focused on weak gravitational lensing studies”. In: arXiv e-prints, arXiv:2306.07996
(June 2023), arXiv:2306.07996. doi: 10.48550/arXiv.2306.07996. arXiv: 2306.07996 [astro-ph.IM].
[3] T. Liaudat, J.-L. Starck, M. Kilbinger, and P.-A. Frugier. “Rethinking data-driven point spread function modeling with a differentiable optical model”. In: Inverse Problems 39.3 (Feb. 2023), p. 035008. doi:
10.1088/1361-6420/acb664.
[4] Y. Xie, T. Takikawa, S. Saito, O. Litany, S. Yan, N. Khan, F. Tombari, J. Tompkin, V. Sitzmann, and
S. Sridhar. “Neural Fields in Visual Computing and Beyond”. In: arXiv e-prints, arXiv:2111.11426 (Nov.
2021), arXiv:2111.11426. doi: 10.48550/arXiv.2111.11426. arXiv: 2111.11426 [cs.CV].
[5] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng. “NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis”. In: arXiv e-prints, arXiv:2003.08934 (Mar. 2020),
arXiv:2003.08934. doi: 10.48550/arXiv.2003.08934. arXiv: 2003.08934 [cs.CV].
2

Location

Saclay

Internship conditions

• Internship duration: 6 months
• Level of study: Bac+5
• Training: Master 2
• Continuation in PhD thesis: Yes
• Application deadline: 1 juillet 2024

Useful methods and technics:
Machine learning

Supervisor

Tobias LIAUDAT
Phone: +33 1 69 08
Email :

Head of the laboratory DEDIP / LILAS

Phone: