diff --git a/docs/source/aspis.rst b/docs/source/aspis.rst index dcf12b8a678317f1e3b41c2c6aad7f8afcfc4724..677785711fb76ab554ee3edaee63a327fda751e9 100644 --- a/docs/source/aspis.rst +++ b/docs/source/aspis.rst @@ -21,7 +21,7 @@ integrated approach. This encompasses the whole chain of phenomena from the Sun to other planets. In this endeavour, it will be crucial to exploit a variety of products (datasets, codes, models), which ultimately populate the ASPIS system. ASPIS unifies multiple SWE resources in a flexible architecture to allow users to perform advanced scientific studies, both through a web interface, and a dedicated -SW package called ASPIS.py. A keyword, metadata-based system enhances search capabilities, especially as +SW package called ASPISpy. A keyword, metadata-based system enhances search capabilities, especially as data volume grows with recorded events. The ASPIS system is open-source, allowing researchers to study problems that span across disciplinary boundaries. It creates an environment where scientists can discover, understand, and model the connection between solar phenomena, interplanetary disturbances, and diff --git a/docs/source/aspis_py.rst b/docs/source/aspis_py.rst index ad37817f0152b962b785b158632d271c0c5108ab..199ac691a81d7abd832288d0db48f6d110e5c7b3 100644 --- a/docs/source/aspis_py.rst +++ b/docs/source/aspis_py.rst @@ -1,25 +1,25 @@ -ASPIS.py package +ASPISpy package ---------------- Intro ^^^^^ -ASPIS.py is a package of code that allows access to the data contained in ASPIS database and other +ASPISpy is a package of code that allows access to the data contained in ASPIS database and other various community databases, to perform client-side specific analyses of this data and to visualize it. The data available are of two types: data in the internal database produced by the CAESAR community and information on external data in other databases. Along with access to the data, the -ASPIS.py package contains code packages that can be imported as package objects, with which to perform +ASPISpy package contains code packages that can be imported as package objects, with which to perform analysis and/or visualization of the data. -The criterion of ASPIS.py is to act as a collector for the data and analysis software of the Space Weather (SWx) Italian +The criterion of ASPISpy is to act as a collector for the data and analysis software of the Space Weather (SWx) Italian community and to foster cooperation within the community. In fact, one can use the data and analysis methods developed by the community and, likewise, share one’s own data and methods. Indeed, we -have defined a standard template in ASPIS.py for generic data analysis code, and provide a how-to for +have defined a standard template in ASPISpy for generic data analysis code, and provide a how-to for developers in the SWx community to convert their own code into the standard template, making it -easily usable for the entire ASPIS.py community in a standard way. In this way ASPIS.py’s primary goal +easily usable for the entire ASPISpy community in a standard way. In this way ASPISpy’s primary goal is to promote collaboration and cooperation within the SWx community. It facilitates the sharing of data and analysis methods among researchers, enabling them to benefit from each other’s work. In -summary, ASPIS.py is a tool designed to streamline the access, analysis, and sharing of +summary, ASPISpy is a tool designed to streamline the access, analysis, and sharing of data and analysis methods within the SWx community. It aims to enhance collaboration and standardization, making it easier for researchers to work with and build upon each other’s work in the field of SWX and astrophysics in general. @@ -27,35 +27,35 @@ field of SWX and astrophysics in general. Functions ^^^^^^^^^ -The functions of the ASPIS.py module are conceived to implement the following key components of the +The functions of the ASPISpy module are conceived to implement the following key components of the ASPIS service: -1. Access for Advanced Users: ASPIS.py is designed to cater to advanced users in the SWx +1. Access for Advanced Users: ASPISpy is designed to cater to advanced users in the SWx community. It offers a set of functions that can be executed programmatically through a Python interpreter. This capability allows users to perform more complex computations and analysis tasks, surpassing the limitations of a web application. -2. Basic Visualization Tools: While ASPIS.py provides basic visualization tools/functions, it also en- +2. Basic Visualization Tools: While ASPISpy provides basic visualization tools/functions, it also en- courages users to leverage other freely available modules like Sunpy and Astropy for more ad- vanced visualization needs. This approach ensures compatibility and flexibility. -3. Ready-to-Use Python Snippets: ASPIS.py intends to provide ready-to-use Python code snippets. +3. Ready-to-Use Python Snippets: ASPISpy intends to provide ready-to-use Python code snippets. These snippets are designed to assist entry-level users, making it easier for them to get started with the module. Advanced users can further customize these code snippets to suit their specific research requirements. -4. Integration with User Programs: The expectation is that advanced users of ASPIS.py will seam- +4. Integration with User Programs: The expectation is that advanced users of ASPISpy will seam- lessly integrate the functions provided by the module into their own analysis and visualization pro- grams. This promotes flexibility and allows users to tailor their solutions to their research needs. -5. Structured Information in the Database: ASPIS.py relies on structured information within the database. +5. Structured Information in the Database: ASPISpy relies on structured information within the database. This information likely includes details about various SWx entities such as phenomena, data descriptions, and chains, which are essential for conducting meaningful analyses. -6. Data Outside the Database: ASPIS.py is designed to work with data that exists outside the database +6. Data Outside the Database: ASPISpy is designed to work with data that exists outside the database but corresponds to the descriptions available within the database. This flexibility allows researchers to combine data from various sources to enhance their analysis. -In summary, ASPIS.py is a versatile tool that caters to both entry-level and advanced users in the +In summary, ASPISpy is a versatile tool that caters to both entry-level and advanced users in the SWx community. It provides functions for querying, visualizing, analyzing, and modeling astro- physical data. The inclusion of code snippets and use case examples simplifies the usage for a broad range of researchers, and advanced users can extend and customize the functionality to meet their spe- @@ -64,10 +64,13 @@ cific research objectives. Installation ^^^^^^^^^^^^ -To install ASPIS.py, a user typically needs to follow these general steps. Please note that the specific -installation process may vary depending on the availability and distribution method of ASPIS.py. The user -has to ensure that Python is installed on the system before proceeding. The user can install ASPIS.py -using pip, which is the Python package manager: +**ASPISpy needs python 3.9 and higher to run.** + +To install ASPISpy, a user typically needs to follow these general steps. Please note that the specific +installation process may vary depending on the availability and distribution method of ASPISpy. The user +has to ensure that Python is installed on the system before proceeding. + +The user can install ASPISpy using pip, which is the Python package manager: ``mkdir aspispy`` @@ -94,7 +97,7 @@ the provided Wheel file. Note that version number may have changed. Use case example ^^^^^^^^^^^^^^^^ -Here we show an example of the way a user can use a specific product of the ASPIS.py package. +Here we show an example of the way a user can use a specific product of the ASPISpy package. from aspis import products as aps from datetime import datetime @@ -141,13 +144,13 @@ their usage and any additional requirements for proper execution. Code snippets and standardization ................................. -The idea of ASPIS.py is that code developed by different members of the Space Weather community in contiguous +The idea of ASPISpy is that code developed by different members of the Space Weather community in contiguous fields can be made available to each other and shared in exactly the same way as is usually done with data. In the area of data, standardization of formats and thus the use of databases in which to store information in a consistent and shared manner has been adopted for years. As for software, the situation is more complex because codes have very heterogeneous characteristics among themselves and code groupings still have strong limitations. Inspired by some recent experiences aimed at standardizing the -use of codes and comparing the performance of different methods on the same data, ASPIS.py aims to +use of codes and comparing the performance of different methods on the same data, ASPISpy aims to provide a structural scheme with the ability to make the use of code developed by different members of the Space Weather community for even different purposes available to the entire community. diff --git a/docs/source/index.rst b/docs/source/index.rst index a1a19db88ce38a3a36f2b160504861a0052b4b66..1dc03345fce3b7edd9b94fc8253c95cb18df0104 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -1,7 +1,7 @@ Welcome to ASPIS documentation! =================================== -This is the **ASPIS** service live documentation. Here you can learn how to use **ASPIS** webapp, aspis.py library and have a look about the informations of the data products hosted in **ASPIS** systems. +This is the **ASPIS** service live documentation. Here you can learn how to use **ASPIS** webapp, ASPISpy package and have a look about the informations of the data products hosted in **ASPIS** systems. .. image:: assets/images/logo_ASPIS_prtscrn.png :height: 320 diff --git a/docs/source/products/data_products.rst b/docs/source/products/data_products.rst index 7210268e2feef815c0ef52a710b191865f10af27..af4f7c23e85bf0a12471e29ea72a307542c08d7f 100644 --- a/docs/source/products/data_products.rst +++ b/docs/source/products/data_products.rst @@ -7,8 +7,8 @@ Atmosphere filerst_ATMO/* -Earth image/map ---------------- +Earth +----- .. toctree:: :titlesonly: @@ -61,8 +61,8 @@ Data Acquired at Lagrange 1 position filerst_L1/* -Mercury image/map ------------------ +Mercury +------- .. toctree:: :titlesonly: @@ -70,7 +70,7 @@ Mercury image/map filerst_Mercury/* -Sun image/map +Sun ------------- .. toctree:: @@ -79,7 +79,7 @@ Sun image/map filerst_SUN/* -Venus image/map +Venus --------------- .. toctree:: diff --git a/docs/source/products/filerst_IP/unitov_icme_tableset.rst b/docs/source/products/filerst_IP/unitov_icme_tableset.rst index c8b0204eab7cf0c34ea1c9204dfdbb5aea87147a..8fc1b9991012bee38988bd521e57f1b33f950f64 100644 --- a/docs/source/products/filerst_IP/unitov_icme_tableset.rst +++ b/docs/source/products/filerst_IP/unitov_icme_tableset.rst @@ -55,7 +55,7 @@ Dataset summary * - **observable:** - * - **latest update:** - - 2024-03-28 05:08:17 + - 2024-05-13 07:34:22 Columns specification ~~~~~~~~~~~~~~~~~~~~~ @@ -230,3 +230,136 @@ Columns specification - char - char - string (CME_0111_param_inv.txt ) - name of the file containing PDBM information from the statistical inversion procedure + +Code example +~~~~~~~~~~~~ + +``"""`` + +``This is an example script for accessing client-side ASPIS datasets.`` + +``The client is provided by the ASPISpy package and in particular the connection is resolved by the CaesarAPI class.`` + +``This example code allows the user to fetch a product from the ASPIS DB and`` + +``save it in a ready-to-use pandas.DataFrame structure.`` + +``To use this code to fetch another product, simply change`` + +``the product name and possibly the time range and columns of the dataset`` + +``(for the filtered_search method).`` + +``Notes:`` + +``- The CaesarAPI class provides a complete interface to the ASPIS DB through several methods,`` + +``each of which corresponds to a different API endpoint.`` + +``- Once CaesarAPI has been instantiated through authentication, the user can call the different methods`` + +``(e.g., CaesarAPI.download_data, CaesarAPI.filtered_search, etc. etc.).`` + +``- The import and initialization are the same for each code fragment.`` + +``"""`` + +``#### The following lines of code should be executed only once`` + +``#### during session initialization. If they are re-executed,`` + +``#### the session in ASPIS DB is recreated from scratch.`` + +``#### Once the session is initialized, the user can use of all`` + +``#### the methods provided by the CaesarAPI class.`` + +``from aspis.CaesarAPIwrapper.request_caesar import CaesarAPI`` + +``import pandas as pd`` + +``# The following lines to disable SSL certificate warning.`` + +``# Note: In this prototype version, the certificate is not available`` + +``import requests`` + +``from requests.packages.urllib3.exceptions import InsecureRequestWarning`` + +``requests.packages.urllib3.disable_warnings(InsecureRequestWarning)`` + +``# Initialize the CaesarAPI connector`` + +``api = CaesarAPI(credentials={'email': '','password': ''})`` + +``# [INFO] user authenticated`` + +``# Define the product to be fetched`` + +``product_name = "unitov_icme_tableset"`` + +``# FETCH THE FULL DATASET`` + +``r, df = api.get_product_detail(product_name)`` + +``# r is the full response of the request, df is the pandas.DataFrame`` + +``# which contains the dataset.`` + +``# > print(df.head())`` + +``# id time HPC_Tx HPC_Ty HPC_distance CME_num ...`` + +``# 0 0 1997-01-06T15:10:42 25.210143 -3.221796 0.983319 1 ...`` + +``# 1 1 1997-02-07T00:30:05 53.055793 -7.993757 0.986300 2 ...`` + +``# 2 2 1997-04-07T14:27:44 14.917489 -25.972924 1.001209 3 ...`` + +``# 3 3 1997-05-12T05:30:05 6.568649 21.393300 1.010308 4 ...`` + +``# 4 4 1997-05-21T21:00:53 -63.993601 -35.996400 1.012279 5 ...`` + +``# FETCH A SUBSET of `` + +``df = api.filtered_search(prod_id=product_name,filters=['time < 2005-01-01','time > 2000-01-01'],cols=['time','bz','dst','v_r'],return_dataframe=True)`` + +``# [INFO] Executing the following query:`` + +``# query MyQuery {`` + +``# filtered_search(filters: [{field: "time", operator: "lt", value: "2005-01-01"}{field: "time", operator: "gt", value: "2000-01-01"}]) {`` + +``# ... on UnitovIcmeTablesetType {`` + +``# time`` + +``# bz`` + +``# dst`` + +``# v_r`` + +``# }`` + +``# }`` + +``# }`` + +``# > print(df.head())`` + +``# time bz dst v_r`` + +``# 0 2000-01-18T17:54:05 16 -97 1132.60`` + +``# 1 2000-02-08T09:30:05 7 -25 1364.00`` + +``# 2 2000-02-10T02:30:05 13 -133 1097.20`` + +``# 3 2000-02-12T04:31:20 5 -67 1286.90`` + +``# 4 2000-02-17T19:31:23 15 -26 566.07`` + +`` +`` + diff --git a/docs/source/webapp.rst b/docs/source/webapp.rst index 4b1e8ab3a64d4875b8b7bfe6a7a0e1d966adfffc..30107bddb5e22eac51607eb2178f13a734463a3d 100644 --- a/docs/source/webapp.rst +++ b/docs/source/webapp.rst @@ -9,12 +9,12 @@ ASPIS webapp aims to provide an easy, user friendly, access to the data containe ASPIS webapp will be suited with tools able to easy access and visualize data. Technically, ASPIS web-app will access to database using a set of dedicated API (GET) who will act as middleware to the Database queries. A key feature will be the function to visualize several data-set in the same page. The -feature, strongly related to the casual-chain [ref. 1] concept is designed to take into account the standard +feature, strongly related to the casual-chain concept is designed to take into account the standard work-flow of the Space Weather community while they are inspecting correlation and liked phenomena inside different products. A small and protected amount of server-side and/or client side models should be run over visualized data for simple and light computation, notwithstanding the intention of the web-app is to facilitate the data exploration and cross-visualization. Users will be encouraged to implement heavy -computations and advanced data handling using ASPIS.py [ref. 9] libraries. +computations and advanced data handling using ASPISpy package. Functionalities ^^^^^^^^^^^^^^^ @@ -38,7 +38,7 @@ Advanced functionalities will be protected and will be accessible only to user i • Run internal, server-side, ASPIS models over an identified data-set • (optional) Light computation of model and data comparison -The Access Control List (ACL) will be provided by the ASPIS machines (Fig. 5) +The Access Control List (ACL) will be provided by the ASPIS machines. Time-based multi-plot visualization ................................... @@ -54,6 +54,7 @@ Data-set are generated by instruments spatially referenced by acquisition site ( physically mounted on a Spacecraft or on Earth ), by target position (ex. The instrument target is the Sun) and by line of sight (ex. the instrument acquire 20 square deg in a direction of the sky). This info can be used to facilitate the user to visualize: + • Instruments positions in a specified time span • Targets positions in a specified time span • Targets coverage in instruments line of sights. @@ -62,17 +63,27 @@ can be used to facilitate the user to visualize: Event-chain visualization ......................... -Causal-chains (ref. 1) will get a special section in the webapp. Causal-chains will take advantage from the +Causal-chains will get a special section in the webapp. Causal-chains will take advantage from the other basic visualization plots and can be associated here as a pre-defined organization of data subsets plots. Web-app dedicated section will be able to list and search all causal-chains. Web-app will be able to advise the user if, in a nominal visualization plot, the data subset is part of a know causal-chain and propose a suitable visualization of such a causal-chain. -Data proximity suggestions -.......................... +Data Discovery +.............. + +Aspis Data Discovery section is organized in four thematic sub-panels: + +• Phenomena +• Targets +• Scientific Topics +• Smart Discovery -User would be prompt by the system in case other data-sets, respect to ones already visualized, will -have a "proximity" (in terms of observed target, data coherence, data meaning, spatial nearness ) and -are available in the ASPIS Database system for the observed time-span or in a relevant subsequent time -(to consider propagation of the phenomena to other instruments). +The last selection method is strongly related to the causal chain concept. +The concept is designed to consider several possible workflows of the Space Weather community while searching for correlations or linked phenomena inside different products. +This criterion is called "smart discovery" and takes inspiration from the casual chain concept. +This discovery panel tries to represent a semantic generalization of the "casual chains" contained in the DB, presenting to the user a selection of data products that are linked for their 'conceptual proximity', i.e. they either are often associated by causality or correlation in Space Weather studies or are alternative data sources. +A few smart discovery associations are offered to the user (e.g., interplanetary disturbances studies, flare AR studies, geoeffective disturbances studies), possibly updating the existing associations and adding others, following the researchers' experience and interests. +Of course, jumping from one smart discovery representation to another is possible since some data products are present in more than one representation and can act as pivot points. +On the right size of the Data Discovery panel the list of the user selected products will be shown.