Loading documentation/documentation.md +24 −13 Original line number Diff line number Diff line Loading @@ -51,6 +51,8 @@ The SAK non-LTE, toy model pipeline uses three main layers: as a function of gas number density and temperature. Be aware that at the moment the query is hardcoded and works for spherical models only. The script `prs_prepare_backup.py` compressed and copies the output of a model run for sharing. The entire ETL pipeline is executed by the `main.py` script, where it is possible to define overrides for the default values in the specific stage configuration file (so that it's possible to specify an entire grid of models). These overrides are included into the `etl/config/config.yml` configuration file. Loading @@ -64,6 +66,12 @@ The pipeline is now dockerized. To run it clone the repository and in bash run: from the root project directory. Docker compose will bring up a local database for your runs, with a persistent storage, so that all the results can be found and inspected. Similarly, a local volume is mounted, so that intermediate files ( radmc files, cubes, moment zero- and ratio images) can be found in the project directory structure, after a run. The main routine accepts the following parameters as input: * --run_id: the run id of the grid to process; if not provided, generates a new run_id; * --cleanup_scratches: whether to empty the mld/scratches directory; * --distributed: whether the grid is processed in a distributed environment, in which case it uses a queue from the database to process all models in the grid, otherwise, executes with multiprocessing. To reset the database, use the command: Loading Loading @@ -163,13 +171,16 @@ The global configuration file has two categories, "computation" and "overrides": * **overrides:** * dust_temperature_grid_type: the spacing in the dust temperature grid, can be: 'linear', 'log' * dust_temperature_limits: the limits in the dust temperature grid, e.g. [10, 30]; the last point is excluded * dust_temperature_step: the step size in the grid; if the spacing is logarithmic, it represents the increase factor, * dust_temperature_step: the step size in the grid; if the spacing is logarithmic, it represents the increase factor, e.g. 10 means steps of one order of magnitude * gas_density_grid_type: the spacing in the gas number density grid, can be: 'linear', 'log' * gas_density_limits: the limits in the gas number density grid, e.g. [1e4, 1e8]; the last point is excluded * gas_density_step: the step size in the grid; if the spacing is logarithmic, it represents the increase factor, e.g. 10 * gas_density_step: the step size in the grid; if the spacing is logarithmic, it represents the increase factor, e.g. 10 means steps of one order of magnitude * gas_density_unit: the units in which the gas number density is expressed, e.g. 'cm^-3' * lines_to_process: the list of line identifiers to process, according to the molecule_{molname}.inp file, e.g. [['87', '86'], ['88', '87']]. The parameter is gives as a list of lists, so that the program knows which ratios e.g. [['87', '86'], ['88', '87']]. The parameter is gives as a list of lists, so that the program knows which ratios to compute. etl/main.py +1 −1 Original line number Diff line number Diff line Loading @@ -240,7 +240,7 @@ if __name__ == '__main__': if remaining_models == 0: logger.info('All grid points processed. Summarizing results.') _cleanup = validate_parameter(args.cleanup_scratches, default=True) default='true').lower() == 'true' main_presentation_step(run_id=run_id, cleanup_scratches=_cleanup, results_dict=results) Loading
documentation/documentation.md +24 −13 Original line number Diff line number Diff line Loading @@ -51,6 +51,8 @@ The SAK non-LTE, toy model pipeline uses three main layers: as a function of gas number density and temperature. Be aware that at the moment the query is hardcoded and works for spherical models only. The script `prs_prepare_backup.py` compressed and copies the output of a model run for sharing. The entire ETL pipeline is executed by the `main.py` script, where it is possible to define overrides for the default values in the specific stage configuration file (so that it's possible to specify an entire grid of models). These overrides are included into the `etl/config/config.yml` configuration file. Loading @@ -64,6 +66,12 @@ The pipeline is now dockerized. To run it clone the repository and in bash run: from the root project directory. Docker compose will bring up a local database for your runs, with a persistent storage, so that all the results can be found and inspected. Similarly, a local volume is mounted, so that intermediate files ( radmc files, cubes, moment zero- and ratio images) can be found in the project directory structure, after a run. The main routine accepts the following parameters as input: * --run_id: the run id of the grid to process; if not provided, generates a new run_id; * --cleanup_scratches: whether to empty the mld/scratches directory; * --distributed: whether the grid is processed in a distributed environment, in which case it uses a queue from the database to process all models in the grid, otherwise, executes with multiprocessing. To reset the database, use the command: Loading Loading @@ -163,13 +171,16 @@ The global configuration file has two categories, "computation" and "overrides": * **overrides:** * dust_temperature_grid_type: the spacing in the dust temperature grid, can be: 'linear', 'log' * dust_temperature_limits: the limits in the dust temperature grid, e.g. [10, 30]; the last point is excluded * dust_temperature_step: the step size in the grid; if the spacing is logarithmic, it represents the increase factor, * dust_temperature_step: the step size in the grid; if the spacing is logarithmic, it represents the increase factor, e.g. 10 means steps of one order of magnitude * gas_density_grid_type: the spacing in the gas number density grid, can be: 'linear', 'log' * gas_density_limits: the limits in the gas number density grid, e.g. [1e4, 1e8]; the last point is excluded * gas_density_step: the step size in the grid; if the spacing is logarithmic, it represents the increase factor, e.g. 10 * gas_density_step: the step size in the grid; if the spacing is logarithmic, it represents the increase factor, e.g. 10 means steps of one order of magnitude * gas_density_unit: the units in which the gas number density is expressed, e.g. 'cm^-3' * lines_to_process: the list of line identifiers to process, according to the molecule_{molname}.inp file, e.g. [['87', '86'], ['88', '87']]. The parameter is gives as a list of lists, so that the program knows which ratios e.g. [['87', '86'], ['88', '87']]. The parameter is gives as a list of lists, so that the program knows which ratios to compute.
etl/main.py +1 −1 Original line number Diff line number Diff line Loading @@ -240,7 +240,7 @@ if __name__ == '__main__': if remaining_models == 0: logger.info('All grid points processed. Summarizing results.') _cleanup = validate_parameter(args.cleanup_scratches, default=True) default='true').lower() == 'true' main_presentation_step(run_id=run_id, cleanup_scratches=_cleanup, results_dict=results)