This notebooks demonstrates how to use the AGILE truth catalog in order to reduce the simulated images and create photometric catalogs using the LSST Science pipelines
We use imSim (https://lsstdesc.org/imSim/index.html) to simulate the images. The purpose of this notebook is not to doucment the feature of imSim, but only to demonstrate how to pass our truth catalog as input to imSim, and how to run software using the tools available in AGILE. Note that imSim only simulates raw LSSTCam data (incl. e.g. sky and instrumental noise), which are not science ready products. The raw images are processed in subsequent steps by using the LSST Science pipelines.
Below, we provide helpful links and resources to further understand the LSST Science Pipelines and the butler:
The following notebook may be accessed [here](https://www.ict.inaf.it/gitlab/akke.viitanen/lsst_inaf_agile/-/blob/a619e4244f56e731d11724f8a70dd5e7d6e1fda4/docs/notebooks/create_photometric_catalog.ipynb)
The following notebook may be accessed [here](https://www.ict.inaf.it/gitlab/akke.viitanen/lsst_inaf_agile/-/blob/feature_documentation/docs/notebooks/create_photometric_catalog.ipynb)
Butler repositories have a limitation of working only in single-threaded applications. To circumvent this, it is suggested that any large-scale production does not call pipetask and/or butler directly, but through some driver software instead. BPS is one solution for these, but others exist, as well. Refer to: https://pipelines.lsst.io/modules/lsst.ctrl.bps/quickstart.html and references therein.
This notebooks demonstrates the basic usage of the truth catalog library, and generates an extermely small truth catalog for testing purposes.
In AGILE, each individual class of source (e.g. AGN, galaxy, star) is first contained in its own catalog. The final step in the truth catalog creation is to simply combine these individual source catalogs into one combined catalog that can be used for future image simulations.
The different classes of objects considered are:
1. Galaxies
2. AGNs
3. Stars
4. Binary stars
The following notebook may be accessed [here](https://www.ict.inaf.it/gitlab/akke.viitanen/lsst_inaf_agile/-/blob/a619e4244f56e731d11724f8a70dd5e7d6e1fda4/docs/notebooks/create_truth_catalog.ipynb)
The following notebook may be accessed [here](https://www.ict.inaf.it/gitlab/akke.viitanen/lsst_inaf_agile/-/blob/feature_documentation/docs/notebooks/create_photometric_catalog.ipynb)
Here we list all the available columns in the truth catalog. The meaning of these columns is described in the appendix of Viitanen+2026.
It is important to note that the truth catalog contains ONE row per object. That is, a single row corresponds to either an AGN, galaxy, or a star. Consequently, not all the different classes of objects have all the different columns available. For example, (host) galaxy stellar mass ('M') is only available for galaxies (and AGNs).
The explanation of the catalog files is as follows:
+ egg.fits -- the EGG galaxy catalog
+ agn.fits -- the AGN catalog
+ stars.fits -- the full star catalog
+ binaries.fits -- the full binary star catalog
+ catalog.fits -- the combined truth catalog
In addition to:
+ egg-seds* -- databse files needed to generate EGG SEDs (refer to EGG documentation)
+ seds/ -- AGN SEDs in EGG format (refer to EGG documentation)
+ lightcurves/ -- stored lightcurves (see the AGN catalog section above)
Note that by default, all galaxy SEDs only reside in the EGG database. These can be stored on disk on demand, but essentially doubles the disk space used which can be problematic on larger catalogs.
Also, light curves are only generated by request. In case images are simulated, then a subset of light curves will be estimated automatically for the sources within the region of interest.
This notebooks demonstrates how to use the AGILE truth catalog in order to simulate synthetic LSST images.
We use imSim (https://lsstdesc.org/imSim/index.html) to simulate the images. The purpose of this notebook is not to doucment the feature of imSim, but only to demonstrate how to pass our truth catalog as input to imSim, and how to run software using the tools available in AGILE. Note that imSim only simulates raw LSSTCam data (incl. e.g. sky and instrumental noise), which are not science ready products. The raw images are processed in subsequent steps by using the LSST Science pipelines.
Below, we provide helpful links and resources to further understand the LSST survey, LSSTCam, and the survey strategy.
+ VRO survey strategy: https://survey-strategy.lsst.io/
The following notebook may be accessed [here](https://www.ict.inaf.it/gitlab/akke.viitanen/lsst_inaf_agile/-/blob/a619e4244f56e731d11724f8a70dd5e7d6e1fda4/docs/notebooks/simulate_images.ipynb)
The following notebook may be accessed [here](https://www.ict.inaf.it/gitlab/akke.viitanen/lsst_inaf_agile/-/blob/feature_documentation/docs/notebooks/create_photometric_catalog.ipynb)
Simulating any large dataset is a computationally expensive task. To simulate any significantly large dataset (such as the AGILE DR1), one is adviced to wrap the `image_simulator.simulate_image` into a callable script which could be executed with multiple cores and/or threads. In AGILE DR1, this orchestration was done with slurm (https://slurm.schedmd.com/documentation.html).