Skip to content
  1. Mar 26, 2021
  2. Oct 05, 2020
  3. Aug 21, 2020
  4. Aug 18, 2020
  5. Jan 29, 2020
    • Grégory Mantelet's avatar
      [TAP,UWS] Fix backup restoration. When restoring upload only, the error message: · 3f246d19
      Grégory Mantelet authored
      ```
      RESTORATION Incorrect JSON format for the serialization of an uploaded file!
      Caused by a org.json.JSONException: JSONObject["length"] not a string.
      [...]
      
      RESTORATION Incorrect JSON format for the DALIUpload labelled "xxx" of the job "xxxxxxxxx": "xxxxxxxx" is not pointing a job parameter representing a file!
      [...]
      
      RESTORATION Unexpected error while restoring the UWS!
      Caused by a java.lang.NullPointerException: Missing UploadFile! => Can not build a DaliUpload instance.
      [...]
      ```
      
      Now, in case of grave error while restoring backup files, it will be just
      disabled, instead of preventing start-up of the TAP service.
      In case of non-grave error while restoring a job or a user, the failed job or
      error won't be restored and then the restoration process will go on with the
      other jobs/users.
      3f246d19
  6. Jul 02, 2019
  7. May 10, 2019
    • Grégory Mantelet's avatar
      [ADQL,TAP] New parser for ADQL-2.1. · 89418d13
      Grégory Mantelet authored
      - Now, `ADQLParserFactory.createParser(...)` should be used to create a parser
      - Only the new function `LOWER` is supported for the moment
      - Not yet possible to manage the optional features _(next dev to come)_
      => 1st step for ADQL-Lib v2.0
      
      - TAP adapted so that using the last stable version of the ADQL language
        (i.e. 2.0 for the moment)
        - but not yet possible to set the ADQL version to use in the configuration
          file
      89418d13
  8. Apr 11, 2019
  9. Mar 20, 2019
  10. Mar 06, 2019
  11. Nov 13, 2018
  12. Nov 07, 2018
    • Grégory Mantelet's avatar
      [TAP] Fix memory issues for the 'text/plain' format. · 3e65794d
      Grégory Mantelet authored
      The previous text formatting process was storing the entire table in memory....
      hence OutOfMemoryError when dealing with large table.
      
      Now, this process is done entirely in memory only for a table having less than
      1000 lines. For a larger table, its content is stored in a temporary file. This
      file is deleted after usage or in case of error.
      
      This formatting process has been tested under JVM monitoring (both JConsole and
      VisualVM) and tables larger than 3,000,000 rows, with success.
      
      This commit fixes the GitHub issue #40
      3e65794d
  13. Nov 02, 2018
    • Grégory Mantelet's avatar
      [TAP] Fix cancellation of a text formatting process. · f912c20a
      Grégory Mantelet authored
      Before this fix, cancelling a TAP job (async or not) which was formatting the
      result in ASCII may failed, especially for large results. This was due to a
      non interruptible alignment process. This process is now checking whether a
      cancellation has been requested before formatting a new result line/row ; if so,
      the process is immediately stopped and the job can be cleaning declared as
      aborted.
      f912c20a
  14. Oct 26, 2018
    • Grégory Mantelet's avatar
      fb4cbcfd
    • Grégory Mantelet's avatar
      [TAP] Fix the TAP_SCHEMA mapping. · a5406513
      Grégory Mantelet authored
      When defining in the configuration file a different name for TAP_SCHEMA content,
      the service implementor was also forced to define the same mapping in the
      database with the column `dbName`.
      
      This is no longer necessary. From now on, the `dbName` column will be ignored
      for all standard TAP_SCHEMA content. Instead, the name specified in the
      configuration file (if any) will be used instead. This way, the mapping for
      standard TAP_SCHEMA content is only specified once and at only one place:
      the configuration file.
      
      _This commit resolves the GitHub issue #98_
      a5406513
  15. Sep 05, 2018
    • Grégory Mantelet's avatar
      [UWS,TAP] Fix the configuration file for the UPLOAD feature. · 7bd91a1c
      Grégory Mantelet authored
      The property `upload_default_db_limit` has been deprecated. Indeed, in the
      current state of the TAP protocol, this makes no sense: the user can not change
      the limit size (in bytes or rows) for uploaded tables.
      
      The property `upload_max_file_size` has been deprecated. It is actually
      duplicated: `upload_max_db_limit`, if expressed in bytes already lets put a
      limit on the maximum size of an uploaded table/file.
      
      The property `upload_max_request_size` has been added. It lets set a maximum
      size for a whole HTTP Multipart Request. By default it is set to 250MB.
      
      The default value of `upload_max_db_size` is now 1 million rows.
      
      The UPLOAD feature is still disabled by default (i.e. `upload_enabled=false`).
      7bd91a1c
  16. Aug 21, 2018
    • Grégory Mantelet's avatar
      [UWS,TAP] Replacement of com.oreilly.servlet by Apache Commons File Upload. · a0ec16fc
      Grégory Mantelet authored
      Thus, the UPLOAD management is much more stable and relies on a well documented
      and maintained library (but at the cost of the additionnal size of the final
      JAR). The most important point is the capability to efficiently limit the size
      of the incoming file and on the disk/memory management when uploading the
      file(s) and in case of error during this process (i.e. files must be deleted).
      a0ec16fc
  17. Aug 10, 2018
    • Grégory Mantelet's avatar
      [TAP] Return a clear error message in case of duplicated items in UPLOADs. · e447a487
      Grégory Mantelet authored
      Before correction, if two uploaded tables have been submitted by the user with
      the same name, or if one uploaded table contained duplicated column names, an
      obscure error message coming from the database was returned to the user.
      
      Now, duplicated items (tables and columns) are searched before ingestion in the
      database. When one is detected, an error is immediately returned to the user and
      the query is aborted.
      e447a487
    • Grégory Mantelet's avatar
      [TAP] Fix disk space consumption with UPLOAD for synchronous jobs. · 25f373f6
      Grégory Mantelet authored
      Files uploaded by the user when creating/executing a synchronous job were never
      deleted after the job execution.
      
      The same problem applied for the tables already uploaded in the database (in
      `TAP_UPLOAD`) when an error occurred before the end of the UPLOAD process.
      
      Now, in case of error when uploading one or more files, or in case of success of
      the job, all uploaded files and their corresponding database tables are deleted
      after the end of the job.
      25f373f6
  18. Aug 07, 2018
  19. Apr 23, 2018
  20. Apr 09, 2018
  21. Mar 07, 2018
    • gmantele's avatar
      Fix UDF parsing from the configuration file. · 96df1d5a
      gmantele authored
      The end of the description of a UDF was not detected when this UDF was followed
      by another UDF definition. This was due to an incorrect double quote escape in
      the regular expression of a UDF's definition.
      96df1d5a
  22. Feb 26, 2018
    • gmantele's avatar
      [UWS] Add 3 job destruction policies: ALWAYS_DELETE (default), ARCHIVE_ON_DATE · f3954c71
      gmantele authored
      and ALWAYS_ARCHIVE.
      
      When archiving a job, its former phase is stored in jobInfo under the name
      'oldPhase' if no jobInfo is already set.
      
      Archiving a job means that all input files and results are destroyed ; the
      error summary and jobInfo (even if it is a file) are kept.
      
      Each archive operation ends with a log message ; in ERROR if failed or in
      INFO if successful.
      
      This commit also includes the following things:
      
          - reformat on 80 characters width the Javadoc of all modified classes
      
      	- fix a bug with the phase transitions: since it is not possible any more
      	  to go from PENDING to EXECUTING directly, UWSJob.start(...) must first
      	  ensure to be on QUEUED phase. This bug has also been fixed in TAPJob.
      	  Similarly, before going into ARCHIVED phase the job must be set into
      	  ABORTED phase if not already in a final phase.
      f3954c71
    • gmantele's avatar
      [UWS,TAP] Add the property 'creationTime' to a UWS (and so TAP as well) job. · a8d98f64
      gmantele authored
      This property is displayed in the XML and JSON serialization of a job item.
      It is also backuped and restored from a UWS backup file.
      
      /!\ WARNING: Small modification of the API! Since creationTime must be
                   set by the UWS service at creation of a job, this date must also
                   be imported at backup restoration. Hence the modification of the
                   interface uws.service.UWSFactory and its implementations
                   (particularly tap.AbstractTAPFactory). Similarly the class
                   uws.job.UWSJob and tap.job.TAPJob have underwent exactly the same
                   modification in one of their constructors.
      
      This commit also adds the new optional jobRef items: creationTime, runID and
      ownerID in the XML and JSON serializations.
      a8d98f64
    • gmantele's avatar
      [TAP] Add an optional parameter to a UDF property: the UDF description. · f6a089c1
      gmantele authored
      Although the Java code allowed the specification of a description of a User
      Defined Function, it was not possible to set one in the UDFs listed in the
      configuration file.
      f6a089c1
  23. Feb 22, 2018
  24. Feb 08, 2018
  25. Jan 31, 2018
  26. Jan 12, 2018
  27. Jan 11, 2018