Commit cf721ff2 authored by gmantele's avatar gmantele
Browse files

[ADQL,TAP] Basic MySQL support. 

On the contrary to other DBMS, MySQL supports schemas, but as databases.
Which means that JDBCConnection gets from the JDBC driver that MySQL does
not support schemas and so it tries to access TAP_SCHEMA tables with a
'TAP_SCHEMA_' prefix instead of 'TAP_SCHEMA.' one. Since MySQL does
behave like it supports schemas, JDBCConnection.supportsSchema must be
set to 'true' if the DBMS is MySQL.

Besides, since no schemas are returned by the JDBC driver
(cf DatabaseMetadata.listSchema(...)), the function
JDBCConnection.isSchemaExisting(String, DatabaseMetaData) must always assume
that the specified schema exists if the DBMS is MySQL. This is particularly
important when the existence of 'TAP_UPLOAD' must be checked, because if the
function returns 'false' the library will attempt to create the database/schema
'TAP_UPLOAD' and will obviously fail due to a lack of permissions. Of course,
it means that the TAP implementor must create manually the schema/database
'TAP_UPLOAD' him-/her-self.

The second particularity of MySQL is the quote character for identifiers.
It is a back-quote (`) instead of a double quote ("). To reflect this
difference, JDBCTranslator.appendIdentifier(...) has been overwritten in a new
JDBCTranslator extension: MySQLTranslator.

The translation of all SQL types and mathematical functions have been adapted to
MySQL according to the online documentation. Few tests have been done locally
with a tiny database. This seems to work but further testing should be
performed in order to ensure the stability of this implementation.
parent f7989175

src/adql/translator/MySQLTranslator.java

0 → 100644
+304 −0
Original line number Diff line number Diff line 
package adql.translator;



import adql.db.DBType;

import adql.db.DBType.DBDatatype;

import adql.db.STCS.Region;

import adql.parser.ParseException;

import adql.query.IdentifierField;

import adql.query.operand.function.geometry.AreaFunction;

import adql.query.operand.function.geometry.BoxFunction;

import adql.query.operand.function.geometry.CentroidFunction;

import adql.query.operand.function.geometry.CircleFunction;

import adql.query.operand.function.geometry.ContainsFunction;

import adql.query.operand.function.geometry.DistanceFunction;

import adql.query.operand.function.geometry.ExtractCoord;

import adql.query.operand.function.geometry.ExtractCoordSys;

import adql.query.operand.function.geometry.IntersectsFunction;

import adql.query.operand.function.geometry.PointFunction;

import adql.query.operand.function.geometry.PolygonFunction;

import adql.query.operand.function.geometry.RegionFunction;



/*

 * This file is part of ADQLLibrary.

 *

 * ADQLLibrary is free software: you can redistribute it and/or modify

 * it under the terms of the GNU Lesser General Public License as published by

 * the Free Software Foundation, either version 3 of the License, or

 * (at your option) any later version.

 *

 * ADQLLibrary is distributed in the hope that it will be useful,

 * but WITHOUT ANY WARRANTY; without even the implied warranty of

 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

 * GNU Lesser General Public License for more details.

 *

 * You should have received a copy of the GNU Lesser General Public License

 * along with ADQLLibrary.  If not, see <http://www.gnu.org/licenses/>.

 *

 * Copyright 2017 - Astronomisches Rechen Institut (ARI)

 */



/**

 * <p>Translates all ADQL objects into an SQL interrogation query designed for MySQL.</p>

 *

 * <p><i><b>Important</b>:

 * 	The geometrical functions are translated exactly as in ADQL.

 * 	You will probably need to extend this translator to correctly manage the geometrical functions.

 * </i></p>

 *

 * @author Gr&eacute;gory Mantelet (ARI)

 * @version 2.1 (08/2017)

 * @since 2.1

 */

public class MySQLTranslator extends JDBCTranslator {



	/** MySQL requires a length for variable-length types such as CHAR, VARCHAR,

	 * BINARY and VARBINARY. This static attributes is the default value set

	 * by this translator if no length is specified. */

	public static int DEFAULT_VARIABLE_LENGTH = 200;



	/** Indicate the case sensitivity to apply to each SQL identifier

	 * (only SCHEMA, TABLE and COLUMN).

	 * <p><i>Note:

	 * 	In this implementation, this field is set by the constructor and never

	 * 	modified elsewhere. It would be better to never modify it after the

	 * construction in order to keep a certain consistency.

	 * </i></p>

	 */

	protected byte caseSensitivity = 0x00;



	/**

	 * Build a MySQLTranslator which always translates in SQL all identifiers

	 * (schema, table and column) in a case sensitive manner ; in other words,

	 * schema, table and column names will be surrounded by back-quotes in the

	 * SQL translation.

	 */

	public MySQLTranslator(){

		caseSensitivity = 0x0F;

	}



	/**

	 * Build a MySQLTranslator which will always translate in SQL identifiers

	 * with the defined case sensitivity.

	 *

	 * @param catalog	<i>true</i> to translate catalog names with back-quotes

	 *               	(case sensitive in the DBMS), <i>false</i> otherwise.

	 * @param schema	<i>true</i> to translate schema names with back-quotes

	 *              	(case sensitive in the DBMS), <i>false</i> otherwise.

	 * @param table		<i>true</i> to translate table names with back-quotes

	 *             		(case sensitive in the DBMS), <i>false</i> otherwise.

	 * @param column	<i>true</i> to translate column names with back-quotes

	 *              	(case sensitive in the DBMS), <i>false</i> otherwise.

	 */

	public MySQLTranslator(final boolean allCaseSensitive){

		caseSensitivity = allCaseSensitive ? (byte)0x0F : (byte)0x00;

	}



	/**

	 * Build a MySQLTranslator which always translates in SQL all identifiers

	 * (schema, table and column) in the specified case sensitivity ; in other

	 * words, schema, table and column names will all be surrounded or not by

	 * back-quotes in the SQL translation.

	 *

	 * @param allCaseSensitive	<i>true</i> to translate all identifiers in a

	 *                        	case sensitive manner

	 *                        	(surrounded by back-quotes),

	 *                        	<i>false</i> for case insensitivity.

	 */

	public MySQLTranslator(final boolean catalog, final boolean schema, final boolean table, final boolean column){

		caseSensitivity = IdentifierField.CATALOG.setCaseSensitive(caseSensitivity, catalog);

		caseSensitivity = IdentifierField.SCHEMA.setCaseSensitive(caseSensitivity, schema);

		caseSensitivity = IdentifierField.TABLE.setCaseSensitive(caseSensitivity, table);

		caseSensitivity = IdentifierField.COLUMN.setCaseSensitive(caseSensitivity, column);

	}



	@Override

	public boolean isCaseSensitive(final IdentifierField field){

		return field == null ? false : field.isCaseSensitive(caseSensitivity);

	}



	@Override

	public StringBuffer appendIdentifier(final StringBuffer str, final String id, final boolean caseSensitive){

		/* Note: In MySQL the identifier quoting character is a back-quote. */

		if (caseSensitive && !id.matches("\"[^\"]*\""))

			return str.append('`').append(id).append('`');

		else

			return str.append(id);

	}



	/* ********************************************************************** */

	/* *                                                                    * */

	/* * TYPE MANAGEMENT                                                    * */

	/* *                                                                    * */

	/* ********************************************************************** */



	@Override

	public DBType convertTypeFromDB(final int dbmsType, final String rawDbmsTypeName, String dbmsTypeName, final String[] params){

		// If no type is provided return VARCHAR:

		if (dbmsTypeName == null || dbmsTypeName.trim().length() == 0)

			return null;



		// Put the dbmsTypeName in lower case for the following comparisons:

		dbmsTypeName = dbmsTypeName.toLowerCase();



		// Extract the length parameter (always the first one):

		int lengthParam = DBType.NO_LENGTH;

		if (params != null && params.length > 0){

			try{

				lengthParam = Integer.parseInt(params[0]);

			}catch(NumberFormatException nfe){}

		}



		// SMALLINT

		if (dbmsTypeName.equals("smallint") || dbmsTypeName.equals("tinyint") || dbmsTypeName.equals("bool") || dbmsTypeName.equals("boolean"))

			return new DBType(DBDatatype.SMALLINT);

		// INTEGER

		else if (dbmsTypeName.equals("integer") || dbmsTypeName.equals("int") || dbmsTypeName.equals("mediumint"))

			return new DBType(DBDatatype.INTEGER);

		// BIGINT

		else if (dbmsTypeName.equals("bigint"))

			return new DBType(DBDatatype.BIGINT);

		// REAL

		else if (dbmsTypeName.equals("float") || dbmsTypeName.equals("real"))

			return new DBType(DBDatatype.REAL);

		// DOUBLE

		else if (dbmsTypeName.equals("double") || dbmsTypeName.equals("double precision") || dbmsTypeName.equals("dec") || dbmsTypeName.equals("decimal") || dbmsTypeName.equals("numeric") || dbmsTypeName.equals("fixed"))

			return new DBType(DBDatatype.DOUBLE);

		// BINARY

		else if (dbmsTypeName.equals("bit") || dbmsTypeName.equals("binary") || dbmsTypeName.equals("char byte"))

			return new DBType(DBDatatype.BINARY, lengthParam);

		// VARBINARY

		else if (dbmsTypeName.equals("varbinary"))

			return new DBType(DBDatatype.VARBINARY, lengthParam);

		// CHAR

		else if (dbmsTypeName.equals("char") || dbmsTypeName.equals("character") || dbmsTypeName.equals("nchar") || dbmsTypeName.equals("national char"))

			return new DBType(DBDatatype.CHAR, lengthParam);

		// VARCHAR

		else if (dbmsTypeName.equals("varchar") || dbmsTypeName.equals("character varying") || dbmsTypeName.equals("nvarchar") || dbmsTypeName.equals("national varchar"))

			return new DBType(DBDatatype.VARCHAR, lengthParam);

		// BLOB

		else if (dbmsTypeName.equals("blob") || dbmsTypeName.equals("tinyblob") || dbmsTypeName.equals("mediumblob") || dbmsTypeName.equals("longblob"))

			return new DBType(DBDatatype.BLOB);

		// CLOB

		else if (dbmsTypeName.equals("text") || dbmsTypeName.equals("tinytext") || dbmsTypeName.equals("mediumtext") || dbmsTypeName.equals("longtext"))

			return new DBType(DBDatatype.CLOB);

		// TIMESTAMP

		else if (dbmsTypeName.equals("timestamp") || dbmsTypeName.equals("date") || dbmsTypeName.equals("datetime") || dbmsTypeName.equals("time") || dbmsTypeName.equals("year"))

			return new DBType(DBDatatype.TIMESTAMP);

		// Default:

		else

			return null;

	}



	@Override

	public String convertTypeToDB(final DBType type){

		if (type == null)

			return "VARCHAR(" + DEFAULT_VARIABLE_LENGTH + ")";



		switch(type.type){



			case SMALLINT:

			case INTEGER:

			case REAL:

			case BIGINT:

			case TIMESTAMP:

				return type.type.toString();



			case DOUBLE:

				return "DOUBLE PRECISION";



			case CHAR:

			case VARCHAR:

			case BINARY:

			case VARBINARY:

				return type.type.toString() + "(" + (type.length > 0 ? type.length : DEFAULT_VARIABLE_LENGTH) + ")";



			case BLOB:

				return "BLOB";



			case CLOB:

				return "TEXT";



			case POINT:

			case REGION:

			default:

				return "VARCHAR(" + DEFAULT_VARIABLE_LENGTH + ")";

		}

	}



	@Override

	public Region translateGeometryFromDB(final Object jdbcColValue) throws ParseException{

		throw new ParseException("Unsupported geometrical value! The value \"" + jdbcColValue + "\" can not be parsed as a region.");

	}



	@Override

	public Object translateGeometryToDB(final Region region) throws ParseException{

		throw new ParseException("Geometries can not be uploaded in the database in this implementation!");

	}



	/* ********************************************************************** */

	/* *                                                                    * */

	/* * SPATIAL FUNCTIONS TRANSLATION                                      * */

	/* *                                                                    * */

	/* ********************************************************************** */



	@Override

	public String translate(ExtractCoord extractCoord) throws TranslationException{

		return getDefaultADQLFunction(extractCoord);

	}



	@Override

	public String translate(ExtractCoordSys extractCoordSys) throws TranslationException{

		return getDefaultADQLFunction(extractCoordSys);

	}



	@Override

	public String translate(AreaFunction areaFunction) throws TranslationException{

		return getDefaultADQLFunction(areaFunction);

	}



	@Override

	public String translate(CentroidFunction centroidFunction) throws TranslationException{

		return getDefaultADQLFunction(centroidFunction);

	}



	@Override

	public String translate(DistanceFunction fct) throws TranslationException{

		return getDefaultADQLFunction(fct);

	}



	@Override

	public String translate(ContainsFunction fct) throws TranslationException{

		return getDefaultADQLFunction(fct);

	}



	@Override

	public String translate(IntersectsFunction fct) throws TranslationException{

		return getDefaultADQLFunction(fct);

	}



	@Override

	public String translate(BoxFunction box) throws TranslationException{

		return getDefaultADQLFunction(box);

	}



	@Override

	public String translate(CircleFunction circle) throws TranslationException{

		return getDefaultADQLFunction(circle);

	}



	@Override

	public String translate(PointFunction point) throws TranslationException{

		return getDefaultADQLFunction(point);

	}



	@Override

	public String translate(PolygonFunction polygon) throws TranslationException{

		return getDefaultADQLFunction(polygon);

	}



	@Override

	public String translate(RegionFunction region) throws TranslationException{

		return getDefaultADQLFunction(region);

	}



}

src/tap/config/ConfigurableTAPFactory.java

+29 −17
Original line number Diff line number Diff line
@@ -16,7 +16,7 @@ package tap.config; 
 * You should have received a copy of the GNU Lesser General Public License

 * along with TAPLibrary.  If not, see <http://www.gnu.org/licenses/>.

 *

 * Copyright 2016 - Astronomisches Rechen Institut (ARI)

 * Copyright 2016-2017 - Astronomisches Rechen Institut (ARI)

 */



import static tap.config.TAPConfiguration.DEFAULT_BACKUP_BY_USER;
@@ -33,9 +33,11 @@ import static tap.config.TAPConfiguration.KEY_SQL_TRANSLATOR; 
import static tap.config.TAPConfiguration.VALUE_JDBC;

import static tap.config.TAPConfiguration.VALUE_JDBC_DRIVERS;

import static tap.config.TAPConfiguration.VALUE_JNDI;

import static tap.config.TAPConfiguration.VALUE_MYSQL;

import static tap.config.TAPConfiguration.VALUE_NEVER;

import static tap.config.TAPConfiguration.VALUE_PGSPHERE;

import static tap.config.TAPConfiguration.VALUE_POSTGRESQL;

import static tap.config.TAPConfiguration.VALUE_SQLSERVER;

import static tap.config.TAPConfiguration.VALUE_USER_ACTION;

import static tap.config.TAPConfiguration.getProperty;
@@ -50,8 +52,10 @@ import javax.naming.NamingException; 
import javax.sql.DataSource;



import adql.translator.JDBCTranslator;

import adql.translator.MySQLTranslator;

import adql.translator.PgSphereTranslator;

import adql.translator.PostgreSQLTranslator;

import adql.translator.SQLServerTranslator;

import tap.AbstractTAPFactory;

import tap.ServiceConnection;

import tap.TAPException;
@@ -74,7 +78,7 @@ import uws.service.log.UWSLog.LogLevel; 
 * </p>

 *

 * @author Gr&eacute;gory Mantelet (ARI)

 * @version 2.1 (04/2016)

 * @version 2.1 (08/2017)

 * @since 2.0

 */

public class ConfigurableTAPFactory extends AbstractTAPFactory {
@@ -200,11 +204,19 @@ public class ConfigurableTAPFactory extends AbstractTAPFactory { 
		else if (sqlTranslator.equalsIgnoreCase(VALUE_PGSPHERE))

			translator = PgSphereTranslator.class;



		// case d: a client defined ADQLTranslator (with the provided class name)

		// case d: SQLServer translator

		else if (sqlTranslator.equalsIgnoreCase(VALUE_SQLSERVER))

			translator = SQLServerTranslator.class;



		// case e: MySQL translator

		else if (sqlTranslator.equalsIgnoreCase(VALUE_MYSQL))

			translator = MySQLTranslator.class;



		// case f: a client defined ADQLTranslator (with the provided class name)

		else if (TAPConfiguration.isClassName(sqlTranslator))

			translator = TAPConfiguration.fetchClass(sqlTranslator, KEY_SQL_TRANSLATOR, JDBCTranslator.class);



		// case e: unsupported value

		// case g: unsupported value

		else

			throw new TAPException("Unsupported value for the property " + KEY_SQL_TRANSLATOR + ": \"" + sqlTranslator + "\" !");

src/tap/config/TAPConfiguration.java

+39 −33
Original line number Diff line number Diff line
@@ -36,7 +36,7 @@ import tap.backup.DefaultTAPBackupManager; 
 * and it must be used only thanks to its static classes and attributes.</i></p>

 *

 * @author Gr&eacute;gory Mantelet (ARI)

 * @version 2.1 (03/2017)

 * @version 2.1 (08/2017)

 * @since 2.0

 */

public final class TAPConfiguration {
@@ -134,6 +134,12 @@ public final class TAPConfiguration { 
	public final static String VALUE_POSTGRESQL = "postgres";

	/** Value of the property {@link #KEY_SQL_TRANSLATOR} to select a PgSphere translator. */

	public final static String VALUE_PGSPHERE = "pgsphere";

	/** Value of the property {@link #KEY_SQL_TRANSLATOR} to select an SQLServer translator.

	 * @since 2.1*/

	public final static String VALUE_SQLSERVER = "sqlserver";

	/** Value of the property {@link #KEY_SQL_TRANSLATOR} to select a MySQL translator.

	 * @since 2.1*/

	public final static String VALUE_MYSQL = "mysql";

	/** Name/Key of the property specifying by how many rows the library should fetch a query result from the database.

	 * This is the fetch size for to apply for synchronous queries. */

	public final static String KEY_SYNC_FETCH_SIZE = "sync_fetch_size";

src/tap/config/tap_configuration_file.html

+2 −2
Original line number Diff line number Diff line
@@ -167,10 +167,10 @@ 
				<td>text</td>

				<td>

					<p>The translator to use in order to translate ADQL to a SQL compatible with the used DBMS and its spatial extension.</p>

					<p>The TAP library supports only Postgresql (without spatial extension) and PgSphere for the moment. But you can provide your own SQL translator

					<p>The TAP library supports only Postgresql (no spatial extension), PostgreSQL+PgSphere, SQLServer (no spatial extension) and MySQL (no spatial extension) for the moment. But you can provide your own SQL translator

					(even if it does not have spatial features), by providing the name of a class (within brackets: {...}) that implements ADQLTranslator and which have at least an empty constructor.</p>

				</td>

				<td><ul><li>postgres</li><li>pgsphere</li><li>{apackage.MyADQLTranslator}</li></ul></td>

				<td><ul><li>postgres</li><li>pgsphere</li><li>sqlserver</li><li>mysql</li><li>{apackage.MyADQLTranslator}</li></ul></td>

			</tr>

			<tr class="optional">

				<td class="done">sync_fetch_size</td>

src/tap/config/tap_full.properties

+3 −3
Original line number Diff line number Diff line
@@ -2,7 +2,7 @@ 
#             FULL TAP CONFIGURATION FILE                #

#                                                        #

# TAP Version: 2.1                                       #

# Date: 03 Feb. 2017                                     #

# Date: 02 Aug. 2017                                     #

# Author: Gregory Mantelet (ARI)                         #

#                                                        #

##########################################################
@@ -62,11 +62,11 @@ database_access = 
# [MANDATORY]

# The translator to use in order to translate ADQL to a SQL compatible with the used DBMS and its spatial extension.

# 

# The TAP library supports only Postgresql (without spatial extension) and PgSphere for the moment. But you can provide your own SQL translator

# The TAP library supports only Postgresql (no spatial extension), PostgreSQL+PgSphere, SQLServer (no spatial extension) and MySQL (no spatial extension) for the moment. But you can provide your own SQL translator

# (even if it does not have spatial features), by providing the name of a class (within brackets: {...}) that implements ADQLTranslator (for instance: {apackage.MyADQLTranslator})

# and which have at least an empty constructor.

# 

# Allowed values: postgres, pgsphere, a class name

# Allowed values: postgres, pgsphere, sqlserver, mysql, a class name

sql_translator = postgres



# [OPTIONAL]