Commit 02ed6d93 authored by Andrea Giannetti's avatar Andrea Giannetti
Browse files

Documented pipeline at high level.

parent a5bd8d68
Loading
Loading
Loading
Loading
+108 −3
Original line number Diff line number Diff line
# SAK documentation

## Scrum board

The SAK Scrum board is digital, and can be found here:
https://miro.com/app/board/uXjVPMBghtU=/
If you don't have access, request it writing to the Scrum Master: andrea.giannetti@inaf.it.

## GDrive

The GDrive was created here: https://drive.google.com/drive/u/1/folders/1cr91sIz9t4r0_4DtyNf61iV4PsnJa9Q8

It is organized as follows:

* Product scopes [volume_density_probe]
    * Raw data [Data]
    * Scripts and pipelines [ETL]
    * Activity reports [Reports]

## Repository

The project remote repository is:
https://www.ict.inaf.it/gitlab/andrea.giannetti/swiss_army_knife/-/tree/master

## Pipeline

The SAK non-LTE, toy model pipeline uses three main layers:

1. **Staging layer:** In the staging layer (`etl/stg`), the `stg_radmc_input_generator.py` file takes care of preparing
   the input files for RADMC, and saves them in the `etl/mdl/radmc_files` folder.

   The `etl/stg/config/config.yml` file contains the default values of the parameters used to prepare the RADMC files
   for the postprocessing. All of them are described in detail in the following.

2. **Model layer:** The model layer (`etl/mdl`) takes care of preparing the RADMC command according to the configuration
   in the `etl/mdl/config/config.yml` file. This is done by the `mdl_prepare_radmc_command.py` script, that
   creates `radmc3d_postprocessing.sh`.

   The results can then be converted to fits cubes via the `mdl_save_results_as_fits.py` script. The cubes are saved by
   default into `prs/fits/cubes`, for later processing.

3. **Presentation layer:** In the presentation layer (`etl/prs`) moment-0 maps and line-ratio maps are computed
   executing the `prs_compute_integrated_fluxes_and_ratios.py` script. At the moment, the integration limits cannot be
   specified, and the entire cube is collapsed. *WARNING:* Pay attention to the presence of secondary modeled lines in
   the simulated spectra.

   The script `prs_inspect_results.py` reduces the ratio maps to single points and produces an image of the ratio values
   as a function of gas number density and temperature.

The entire ETL pipeline is executed by the `main.py` script, where it is possible to define overrides for the default
values in the specific stage configuration file (so that it's possible to specify an entire grid of models). These
overrides are included into the `etl/config/config.yml` configuration file.

The result inspection (`prs_inspect_results.py`) is not included in the pipeline.

### Configuration files parameters

In this paragraph we describe in more detail the parameters that can be set in the different configuration files, and
their meaning.

#### The staging configuration file (`etl/stg/config/config.yml`)

The staging config file has three main categories:
* **grid:**
  * grid_type: regular or spherical
  * coordinate_system: cartesian or polar
  * central_density: the central gas number density of the toy model. If you are using a power-law distribution, it corresponds also to the maximum possible density in the grid. 
  * density_unit: the units in which the number density is expressed, e.g. 'cm^-3'
  * density_powerlaw_idx: the power-law index of the density distribution
  * density_value_at_reference: the gas number density at a reference value, e.g. at 1 pc
  * distance_reference: the reference radius for scaling the power-law
  * distance_reference_unit: the units of the reference radius, e.g. 'pc', 'AU', 'cm'
  * dust_temperature: the dust temperature value
  * dust_temperature_unit: the dust temperature unit
  * dust_temperature_powerlaw_idx: the dust temperature power-law index
  * microturbulence: the unresolved turbulent velocity of the gas
  * microturbulence_unit: the units of the microturbulence value, e.g. 'km/s'
  * dimN: a dictionary-like parameter, it should include the 'size', 'size_unit', 'shape', and 'refpix' keys. E.g. {"size":1, "size_units": "pc", "shape": 3, "refpix": 1}. If only dim1 is provided, the grid is cubic.
    * size: the grid size in physical units
    * size_unit: the units in which the grid size is expressed, e.g. 'pc'
    * shape: the number of cells in the grid for this axis
    * refpix: the reference pixel that corresponds to the grid "centre" from which the distance are computed for power-law models, for instance
  * velocity_field: velocity field to apply to the gas, it can only be 'solid' at the moment (the gas is assumed to rotate around the y axis). Power-law is in principle also supported, but the wiring is still to be implemented.  
  * velocity_gradient: the value of the velocity gradient for solid-body roation.
  * velocity_gradient_unit: the unit of the velocity gradient, e.g. "km/s/pc"

* **lines:**
  * species_to_include: the list of molecular species to include in the RADMC postprocessing, e.g. ['e-ch3oh']
  * molecular_abundances: a dict-like parameter, containing the species name and the corresponding fractional abundance, e.g. {"e-ch3oh": 1e-8, "p-h2": 1}
  * lines_mode: the line transfer mode. It can be 'lte', 'lvg', 'optically_thin_non_lte', 'user_defined_populations' (see the RADMC documentation if in doubt)
  * collision_partners: the list of collision partners to be used; it must appear in the correct order as in the molecule_{molname}.inp file of the molecule to be simulated, e.g. ['p-h2']

* **radmc:**
  * nphotons: the number of photons to use for the postprocessing
  * scattering_mode_max: override the scattering settings in the dust opacity files; 0 excludes scattering, 1 treats it in an isotropic way (if defined), 2 includes anisotropic scattering (if defined)
  * iranfreqmode: 1
  * tgas_eq_tdust: whether the gas temperature is equal to the dust temperature (if not, it must be specified or computed separately!)

#### The model configuration file (`etl/mdl/config/config.yml`)

The model configuration file has only the "radmc" category:

* **radmc:**
  * inclination: the inclination wrt. the line-of-sight in degrees
  * position_angle: the position angle wrt. the observer
  * imolspec: the index of the species to be modeled; defaults to the first
  * iline: the line identifier for the line to be modeled, according to the molecule_{molname}.inp
  * width_kms: the range in km/s to be modeled around the line
  * nchannels: the number of channels to be considered
  * npix: the number of pixels in the final image

#### The global configuration file (`etl/config/config.yml`)

The global configuration file has only the "overrides" category:

* **overrides:**
    dust_temperature_grid_type: the spacing in the dust temperature grid, can be: 'linear', 'log'
    dust_temperature_limits: the limits in the dust temperature grid, e.g. [10, 30]; the last point is excluded
    dust_temperature_step: the step size in the grid; if the spacing is logarithmic, it represents the increase factor, e.g. 10 means steps of one order of magnitude
    gas_density_grid_type: the spacing in the gas number density grid, can be: 'linear', 'log'
    gas_density_limits: the limits in the gas number density grid, e.g. [1e4, 1e8]; the last point is excluded
    gas_density_step: the step size in the grid; if the spacing is logarithmic, it represents the increase factor, e.g. 10 means steps of one order of magnitude
    gas_density_unit: the units in which the gas number density is expressed, e.g. 'cm^-3'
    lines_to_process: the list of line identifiers to process, according to the molecule_{molname}.inp file, e.g. ['87', '86']