Skip to content
  1. Oct 19, 2020
  2. Aug 21, 2020
  3. Jul 02, 2019
  4. May 10, 2019
    • Grégory Mantelet's avatar
      [ADQL,TAP] New parser for ADQL-2.1. · 89418d13
      Grégory Mantelet authored
      - Now, `ADQLParserFactory.createParser(...)` should be used to create a parser
      - Only the new function `LOWER` is supported for the moment
      - Not yet possible to manage the optional features _(next dev to come)_
      => 1st step for ADQL-Lib v2.0
      
      - TAP adapted so that using the last stable version of the ADQL language
        (i.e. 2.0 for the moment)
        - but not yet possible to set the ADQL version to use in the configuration
          file
      89418d13
  5. Mar 13, 2019
    • Grégory Mantelet's avatar
      [ADQL] Add to the parser a function attempting to quickly fix an ADQL query. · 15cd5944
      Grégory Mantelet authored
      This new function - ADQLParser.tryQuickFix(...) - fixes the most common issues
      with ADQL queries:
      
      - replace Unicode confusable characters by their ASCII/UTF-8 version,
      - double-quote SQL reserved words/terms (e.g. `public`, `year`, `date`),
      - double-quote ADQL function names used a column name/alias (e.g. `distance`,
        `min`, `avg`),
      - double-quote invalid regular identifiers (e.g. `_RAJ2000`, `2mass`).
      
      The last point is far from being perfect but should work at least for
      identifiers starting with a digit or an underscore, or an identifier including
      one of the following character: `?`, `!`, `$`, `@`, `#`, `{`, `}`, `[`, `]`,
      `~`, `^` and '`'.
      
      It should also been noted that double-quoting a column/table name will make it
      case-sensitive. Then, it is possible that the query does not pass even after the
      double-quote operation ; the case would have to be checked by the user.
      
      Finally, there is no attempt to fix column and table names (i.e. case
      sensitivity and/or typos) using tables/columns list/metadata. That could be a
      possible evolution of this function or an additional feature to implement in the
      parser.
      15cd5944
  6. Mar 06, 2019
    • Grégory Mantelet's avatar
      [TAP] Support the datatype `unsigned byte` in `SQLServerTranslator`. · 41a46294
      Grégory Mantelet authored
      _This commit fixes the GitHub issue #97 ._
      41a46294
    • Grégory Mantelet's avatar
      [ADQL] Fix the SQL translation of ORDER BY's column identifiers of aliased · e6e8fd69
      Grégory Mantelet authored
      tables.
      
      Since commit 3d96c9d9 aliases put on a table
      without double quotes are put in lower case and then double quoted. This
      modification was not effective for ORDER BY's column identifiers.
      e6e8fd69
    • Grégory Mantelet's avatar
      [ADQL] Abort previous commit (09fac540). · 0df3cab6
      Grégory Mantelet authored
      The previous commit broke some JUnit tests. It was indeed not any more possible
      to prefix a column with the fully qualified table name
      (e.g. `"aSchema"."aTable"`) if the table was automatically aliased. If an alias
      is set on a table, this table should be referenced only by this alias. The
      previous commit was not able to replace fully qualified table names with their
      corresponding alias....and doing that would probably introduce other unexpected
      parsing/checking behaviors.
      
      So, it is better to keep this part of the parsing as simple as now. Just for few
      DB error messages showing real database names (and only if they are different
      from the ADQL ones), it does not worth _yet_ making the parsing more complex.
      
      _This commit ends the issue #108 ._
      0df3cab6
  7. Mar 05, 2019
  8. Jan 12, 2018
    • gmantele's avatar
      [ADQL] Fix the parsing and translation of a concatenation expression. · e4f38c95
      gmantele authored
      * The parsing did not allow unsigned numerics and SQL SET functions as
        specified in the ADQL 2.0 grammar
      
      * It was even forbidden to put a column whose the type is not String.
      
      * The translation of a concatenation expression was always prefixed by the
        ADQLList's name: CONCAT_STR. Of course, no database likes that...
      
      Regarding this last point, this commit fixes the GitHub issue #54
      e4f38c95
  9. Jan 11, 2018
  10. Jan 08, 2018
  11. Nov 30, 2017
  12. Nov 10, 2017
    • gmantele's avatar
      [ADQL] Fix escaping of double quotes in delimited identifiers. · 239c7178
      gmantele authored
      A delimited identifier is any sequence of characters between a pair of
      double quotes. For instance: "123 I am a delimited identifier!".
      
      It is of course possible to have double quotes inside this kind of identifier,
      but they have to be doubled in order to not be mistaken with the end of the
      identifier. For instance: "Cool ""identifier""".
      
      However, this escape option was not taken into account by the ADQL library,
      though the same mechanism was already in place for string contants.
      239c7178
  13. Sep 13, 2017
    • gmantele's avatar
      [ADQL] Also append an HINT message in the ParseException message when a SQL · fe4c3e97
      gmantele authored
      reserved word is encountered instead of a column/table/schema name/alias.
      
      On the contrary to the previous commit, this time a list of SQL reserved words
      has been added into the ADQL grammar. In this way, the parser will ensure that
      no word of this list is used in an ADQL query. The raised error is then enriched
      of an HINT message stating that this word is part of SQL, is not supported
      by ADQL and must be written between double quotes if used as an identifier.
      
      The list of SQL reserved words comes from the ADQL-2.0 standard, after removal
      of all potentially used ADQL words, in order to avoid a conflict with the
      already existing tokens in the ADQL grammar.
      fe4c3e97
    • gmantele's avatar
      [ADQL] Append an HINT message in the ParseException message when an ADQL · db0dfdad
      gmantele authored
      reserved word is encountered instead of a column/table/schema name/alias.
      
      No list of ADQL reserved words has been added into the ADQL grammar.
      
      However, the ADQL grammar has been slightly changed in order to provide a more
      precise location of the REAL wrong part of the query.
      
      Before this commit, if an ADQL reserved word (e.g. 'point') was encountered
      outside of its normal syntax (e.g. 'point' no followed by an opening
      parenthesis), the next token was highlighted instead of this one. Hence a
      confusing error message.
      
      For instance, the following ADQL query:
      
      ```sql
      SELECT point
      FROM aTable
      ```
      
      returned the following error message:
      
      > Encountered "FROM". Was expecting: "("
      
      Now, it will return the following one:
      
      > Encountered "point". Was expecting one of: "*" <QUANTIFIER> "TOP" [...]
      > (HINT: "point" is a reserved ADQL word. To use it as a column/table/schema name/alias, write it between double quotes.)
      
      This error message highlights exactly the source of the problem and even provide
      to the user a clear explanation of why the query did not parse and how it could
      be solved.
      db0dfdad
    • gmantele's avatar
  14. Sep 11, 2017
  15. Sep 08, 2017
    • gmantele's avatar
      [ADQL] Fix the transformation of NATURAL JOIN and JOIN...USING of MS-SQLServer. · e03e5725
      gmantele authored
      In the resulting SQL query, if there are an alias on the joined tables, these
      aliases must be used in the ON clause (instead of the full table name).
      
      For instance, the following ADQL query:
      
      ```sql
        SELECT *
        FROM tableA AS a NATURAL JOIN tableB AS b;
      ```
      
      should be translated into the following SQL:
      
      ```sql
        SELECT *
        FROM tableA AS a
          INNER JOIN tableB AS b
            ON a.id = b.id
      ```
      
      This commit complete the resolution of the Pull Request #16
      (more details about the issue can be got in there)
      e03e5725
    • gmantele's avatar
      [ADQL] Fix return type of some math. functions for MS-SQLServer. · 73630130
      gmantele authored
      In MS-SQLServer, the following functions return a value of the same
      type as the given argument:
        - abs
        - ceiling
        - degrees
        - floor
        - radians
        - round.
      
      Since ADQL requires that these functions return a DOUBLE
      (i.e. a float in MS-SQLServer), it must be ensured that the given
      argument is a float.
      
      This commit partially fixes the Pull Request #16
      73630130
    • gmantele's avatar
      [ADQL] Throwing a ParseException instead of an Error · a382b251
      gmantele authored
      when an incorrect character that can not be interpreted by
      the JavaCC Token Manager is encountered.
      
      Actually, the TokenMgrError thrown by JavaCC is caught by all
      ADQLParser.parseQuery(...) functions, wrapped inside a ParseException
      which is finally thrown instead of the TokenMgrError. In this way,
      ADQL-Lib users just have to care about a single Throwable:
      ParseException.
      
      Besides the error message has been slightly modified from:
      
      > Lexical error at line 1, column 10.  Encountered: "\u00e9" (233), after : \"\"
      
      to:
      
      > Incorrect character encountered at l.1, c.10: \"\\u00e9\" ('é'), after : \"\"
      
      Thus, the error is more user-friendly, more easy to understand by users.
      Additionally, the incorrect character is displayed, as before, in its unicode
      expression, but also in its character form (instead of an integer value that
      nobody can really understand).
      
      This commit fixes the GitHub issue #17
      a382b251
  16. Aug 02, 2017
    • gmantele's avatar
      [ADQL,TAP] Basic MySQL support. · cf721ff2
      gmantele authored
      On the contrary to other DBMS, MySQL supports schemas, but as databases.
      Which means that JDBCConnection gets from the JDBC driver that MySQL does
      not support schemas and so it tries to access TAP_SCHEMA tables with a
      'TAP_SCHEMA_' prefix instead of 'TAP_SCHEMA.' one. Since MySQL does
      behave like it supports schemas, JDBCConnection.supportsSchema must be
      set to 'true' if the DBMS is MySQL.
      
      Besides, since no schemas are returned by the JDBC driver
      (cf DatabaseMetadata.listSchema(...)), the function
      JDBCConnection.isSchemaExisting(String, DatabaseMetaData) must always assume
      that the specified schema exists if the DBMS is MySQL. This is particularly
      important when the existence of 'TAP_UPLOAD' must be checked, because if the
      function returns 'false' the library will attempt to create the database/schema
      'TAP_UPLOAD' and will obviously fail due to a lack of permissions. Of course,
      it means that the TAP implementor must create manually the schema/database
      'TAP_UPLOAD' him-/her-self.
      
      The second particularity of MySQL is the quote character for identifiers.
      It is a back-quote (`) instead of a double quote ("). To reflect this
      difference, JDBCTranslator.appendIdentifier(...) has been overwritten in a new
      JDBCTranslator extension: MySQLTranslator.
      
      The translation of all SQL types and mathematical functions have been adapted to
      MySQL according to the online documentation. Few tests have been done locally
      with a tiny database. This seems to work but further testing should be
      performed in order to ensure the stability of this implementation.
      cf721ff2
  17. Jun 01, 2017
    • gmantele's avatar
      [ADQL] Fix nasty infinite loop when wrapping matches with SimpleReplaceHandler. · 66304427
      gmantele authored
      This infinite loop occured only when the replacement object is just
      a wrapping of the matching object ; after replacement, the new object was
      inspected for matching objects.
      
      Example: infinite loop if we want to wrap all foo(...) functions with
               the function ROUND in the following query:
          SELECT foo(foo(123)) FROM myTable
      	     Expected result:
          SELECT ROUND(foo(ROUND(foo(123)))) FROM myTable
      	     But generated result was:
          SELECT ROUND(ROUND(ROUND(......foo(foo(123))))) FROM myTable
      66304427
  18. May 10, 2017
  19. Apr 20, 2017
  20. Apr 04, 2017
    • gmantele's avatar
      [ADQL] Complete commit "Re-Fix GROUP BY's columns handling" · 8e2fa9ff
      gmantele authored
      (https://github.com/gmantele/taplib/commit/7a70c6038cef460ab169682bed391bb5ae1de1e9)
      
      It was not possible to use a GROUP BY with a qualified column name.
      So finally, now, a GROUP BY is a ClauseADQL<ADQLColumn> instead of
      a ClauseADQL<ColumnReference>. Indeed, according to the ADQL's BNF,
      GROUP BY items are only columns as they would appear in the SELECT
      clause (i.e. qualified or not, delimited or not). On the other
      hand an ORDER BY accepts ONLY column index or non-qualified column
      name/alias.
      
      The class ColumnReference is kept for backward compatibility (or in
      case the next version of the ADQL grammar make items of GROUP BY and
      ORDER BY of the same type: index or qualified column). Besides, this
      class is still inherited for the ORDER BY clause items
      (see adql.query.ADQLOrder).
      8e2fa9ff
  21. Apr 03, 2017
  22. Mar 29, 2017
  23. Mar 10, 2017
  24. Mar 08, 2017
    • gmantele's avatar
      [ADQL] Recompilation of the ADQLParser classes using JavaCC 6.0. · 005fc622
      gmantele authored
      Two classes have been modified after compilation:
          - ADQLParser - a simple cast for one of the automatically generated
                         constructor.
          - ParseException - the token position has been stored for better
                             syntax error messages.
      A note has been added at the top comment for both files to highlight
      the modified parts and how to restore them after re-generation.
      005fc622
  25. Mar 02, 2017