Commit faa8590f authored by Andrea Giannetti's avatar Andrea Giannetti
Browse files

Added reset script; added pre-flight check list; fixed creation of run_id file.

parent 3c969b54

documentation/run_preflight_checklist.md

0 → 100644
+9 −0
Original line number Diff line number Diff line 
* update the code to the last version

* remove fits files, output files, and archive files via the `reset_output.sh` script

* check 

  * `config/config.yml` for the grid configuration

  * `stg/config/config.yml` for the power-law indices and the grid pixels

  * `mdl/config/config.yml` for the image pixels (must be a multiple of the number of grid pixels)

* _if the run is distributed_ check

  * `sbatch_command.sh` for the job name and log file name

  * `slurm_template.sh` for the folder of the run (and possibly for a fixed run_id)

etl/main.py

+26 −24
Original line number Diff line number Diff line
@@ -49,6 +49,8 @@ def compute_full_grid(tdust, nh2, line, density_keyword, dust_temperature_keywor 
def initialize_queue(engine: sqlalchemy.engine,

                     run_id: str,

                     run_arguments: Iterator):

    is_initialized = engine.execute(f"select count(*) from tmp_execution_queue where run_id='{run_id}'").first()[0] != 0

    if is_initialized is False:

        for arguments in run_arguments:

            raw_insert_entry = {'run_id': run_id,

                                'dust_temperature': arguments[0],
@@ -142,6 +144,8 @@ def initialize_run(): 
        run_id = str(uuid.uuid4())

    compute_grid_elements(run_id=run_id)

    sys.stdout.write(run_id)

    with open('run_id.txt', 'w') as run_id_file:

        run_id_file.write(f'{run_id}\n')

    return run_id
@@ -237,8 +241,6 @@ if __name__ == '__main__': 
    assert run_id is not None

    _distributed = validate_parameter(args.distributed, default='false').lower() == 'true'

    results, remaining_models = process_models(distributed=_distributed)

    with open('run_id.txt', 'w') as run_id_file:

        run_id_file.write(f'{run_id}\n')

    if remaining_models == 0:

        logger.info('All grid points processed. Summarizing results.')

        _cleanup = validate_parameter(args.cleanup_scratches,

etl/reset_output.sh

0 → 100644
+16 −0
Original line number Diff line number Diff line 
#!/bin/bash



read -p "This will delete all of the results, are you sure? " -n 1 -r

echo



if [[ $REPLY =~ ^[Yy]$ ]]

then

    folder_list=("prs/fits/cubes" "prs/fits/moments" "prs/fits/ratios" "prs/fits/grids")

    for folder_name in $folder_list

    do

        rm $folder_name/*.fits

    done

    rm prs/output/*.png

    rm stg/archive/*.zip

    rm full_dataset*.csv

fi