*This jupyter notebook is part of a collection of notebooks on various topics discussed during the Time Domain Astrophysics course delivered by Stefano Covino at the [Università dell'Insubria](https://www.uninsubria.eu/) in Como (Italy). Please direct questions and suggestions to [stefano.covino@inaf.it](mailto:stefano.covino@inaf.it).*
- The exression "big data" in modern astronomy is not just a “hot keyword”.
- The expression "big data" in modern astronomy is not just a “hot keyword”.
- Observational astronomy today is a considerable enterprise with billions of dollars supporting ∼20,000 scientists producing ∼15,000 refereed papers annually.
- Not to mention the theoretical efforts, often based on profitable synergies with other fields.
- Astrostatistics is playing an increasing role in the analysis of astronomical observations and linking data to astrophysical theory.
%% Cell type:markdown id:5994ee22 tags:
## Open problems in astrostatistics
***
### Galaxy clustering and large-scale structure
***
- Galaxy clustering: the distribution of galaxies in space proves to be surprising complex from the viewpoint of spatial point processes.
- Many statistical studies of large-scale structure rely on isotropic two- and three-point correlation functions as well as Fourier power spectra.
- Other studies seek to locate particular clusters, filaments or voids.
%% Cell type:markdown id:c6ac7962 tags:
### The photo-z conundrum
***
- Photometric redshift (photo-z ) estimation has become a vital tool in the extragalactic astronomy and observational cosmology.
- The challenge of photo-z accuracy then depends on the statistical procedures used to calibrate photometric measurements to spectroscopic redshifts.
%% Cell type:markdown id:62e3110b tags:
### Bayesian modeling
***
- Bayesian modeling has become a standard practice in many fields.
- Sampling algorithms, theoretical advancements related to prior selections, are all active research areas.
- Likelihood-free modeling: two main forms of statistical models can be distinguished: those describe by probability distributions for which an explicit likelihood can be written, and implicit or generative models.
%% Cell type:markdown id:30ada84c tags:
### Challenges in signal analysis
***
- The study of variable objects in the sky − time domain astronomy − is burgeoning with more than 2000 studies annually.
- Gravitational wave detection: the statistical challenge with is to detect short-lived chirp-like events in a continuous time series where noise is dominated by instrumental effects that can be continuous (perhaps caused by vibrations in the mirror structures) or transient (perhaps caused by minor Earth tremors).
%% Cell type:markdown id:ce0c21cc tags:
### Machine learning techniques
***
- This is literally an “exploding” field.
- Just a couple of instances: photometry of blended galaxies and the accelerated expansion of the universe.
- Apart from the folklore around the subject, large-langage model technologies can have a strong impact on our researches, although still to largely be evaluated.
%% Cell type:markdown id:2d9d81e6 tags:
## Final remarks
***
- Contemporary astronomical data analysis often elude the capabilities of classical statistical techniques, and inevitably requires the use and development of sophisticated, and sometimes novel, statistical tools.
- Astronomy requires expertise in vast fields of statistics and information science: nonparametric and parametric inference (especially Bayesian), high-dimensional nonlinear regression, censoring and truncation, measurement error theory, spatial point processes, image analysis, time series analysis, multivariate analysis, clustering and classification, and many other forms of machine learning.
> ## Take all of this very seriously!
%% Cell type:markdown id:aab2b827 tags:
## <p style="text-align:center;">A forza di guardare il cielo e di respirare a pieni polmoni l’aria fresca della notte, mi sembrava di riempirmi di stelle</p>
## <p style="text-align:center;">Tiziano Terzani
</p>
%% Cell type:markdown id:ffe137ba tags:
## Reference & Material
-[Feigelson et al. (2021) - "21st Century Statistical and Computational Challenges in Astrophysics"](https://ui.adsabs.harvard.edu/abs/2021AnRSA...8..493F/abstract)
%% Cell type:markdown id:dff2c952 tags:
## Further Material
Papers for examining more closely some of the discussed topics.
-[Nguyen et al. (2023) - "AstroLLaMA: Towards Specialized Foundation Models in Astronomy"](https://ui.adsabs.harvard.edu/abs/2023arXiv230906126D/abstract)
This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the examples, unless obtained from other properly quoted sources, under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Stefano Covino, Time Domain Astrophysics - Lecture notes featuring computational examples, 2024*.
This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the examples, unless obtained from other properly quoted sources, under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Stefano Covino, Time Domain Astrophysics - Lecture notes featuring computational examples, 2025*.
