BigQuery returns "Unparseable query parameter `` in type `TYPE_INT64`" when the query projects a literal that is a parameter of a prepared statement - jdbc

BigQuery returns the error Unparseable query parameter `` in type ``TYPE_INT64``" when executing a query that meets all these conditions:
I run the query as a prepared statement.
The query has a literal of type String in the SELECT.
The value of this literal is passed as a parameter. For example SELECT ? AS field_1....
For example, this code:
import java.sql.*;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
public class TestBigQueryPreparedStatement {
private static final String CONNECTION_URI =
"jdbc:bigquery://<connection_uri>";
private static final String QUERY = "SELECT ? AS `_SOURCE_TABLE`, field1 FROM test_view";
public static void main(String[] args) throws ClassNotFoundException {
Class.forName("com.simba.googlebigquery.jdbc42.Driver");
try (Connection connection = DriverManager.getConnection(CONNECTION_URI)) {
try (PreparedStatement ps = connection.prepareStatement(QUERY)) {
// The problem also occurs if I replace this line with "ps.setObject(...)
ps.setString(1, "SOURCE_TABLE");
try (ResultSet rs = ps.executeQuery()) {
while (rs.next()) {
System.out.println(rs.getString(1) + " " + rs.getInt(2));
}
}
}
} catch (SQLException e) {
e.printStackTrace();
}
}
}
Produces this error:
java.sql.SQLException: [Simba][BigQueryJDBCDriver](100032) Error executing query job. Message: Unparseable query parameter `` in type `TYPE_INT64`, Bad int64 value: SOURCE_TABLE value: 'SOURCE_TABLE'
at com.simba.googlebigquery.googlebigquery.client.requests.jobs.JobsInsertRequest.throwException(Unknown Source)
at com.simba.googlebigquery.googlebigquery.client.requests.AbstractRequestWithRetry.executeWithRetry(Unknown Source)
at com.simba.googlebigquery.googlebigquery.client.queryclient.JobsInsertClient.executeQuery(Unknown Source)
at com.simba.googlebigquery.googlebigquery.client.BQClient.executeQuery(Unknown Source)
at com.simba.googlebigquery.googlebigquery.dataengine.BQAbstractExecutor.execute(Unknown Source)
at com.simba.googlebigquery.googlebigquery.dataengine.BQSQLExecutor.execute(Unknown Source)
at com.simba.googlebigquery.jdbc.common.SPreparedStatement.executeWithParams(Unknown Source)
at com.simba.googlebigquery.jdbc.common.SPreparedStatement.executeQuery(Unknown Source)
at TestBigQueryPreparedStatement.main(TestBigQueryPreparedStatement.java:20)
Caused by: com.simba.googlebigquery.googlebigquery.client.exceptions.JobExecutionErrorException: [Simba][BigQueryJDBCDriver](100032) Error executing query job. Message: Unparseable query parameter `` in type `TYPE_INT64`, Bad int64 value: SOURCE_TABLE value: 'SOURCE_TABLE'
... 9 more
Caused by: java.lang.Exception: Unparseable query parameter `` in type `TYPE_INT64`, Bad int64 value: SOURCE_TABLE value: 'SOURCE_TABLE'
at com.simba.googlebigquery.googlebigquery.client.requests.jobs.JobsInsertRequest.execute(Unknown Source)
at com.simba.googlebigquery.googlebigquery.client.requests.jobs.JobsInsertRequest.execute(Unknown Source)
... 8 more
Reproducible with the latest JDBC driver of BigQuery (Simba v1.3.0 1001).
This error also occurs with PreparedStatement.setDate(...) and setFloat(...) but works fine with setDecimal(...), setInt(...) (I have not checked the output with all the setXXX methods of PreparedStatement)
Is it possible to execute in BigQuery queries that have a literal in the SELECT and to execute the query as a prepared statement?
This is a sample scenario. My application has an execution engine that runs SQL queries on any database and it always does so with prepared statements. Occasionally, the query will have a literal in the SELECT and with BigQuery, I get this error above (it works with any other database). I can do certain changes specifically for the query generator of BigQuery but it would be very difficult to change the code so the literals in the SELECT clause are passed as literals, not parameters of the prepared statement.

You cannot use positional query parameters in the select or from part of the query. Positional parameters can only occur in the where clause of the query.
SELECT word, word_count
FROM `bigquery-public-data.samples.shakespeare`
WHERE corpus = ?
AND word_count >= ?
ORDER BY word_count DESC;";
See also https://cloud.google.com/bigquery/docs/samples/bigquery-query-params-positional

Related

Athena throws Simba JDBC 11300 with ALTER TABLE ADD PARTITION

Running an ALTER TABLE ADD PARTITION using the Athena JDBC driver is throwing an Exception that is proving hard to understand, this is the code:
package none;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.Statement;
import java.util.Properties;
public class test {
public static void main(String[] args) throws Exception {
Properties info = new Properties();
info.put("User","myUser");
info.put("Password","myPass");
info.put("S3OutputLocation", "s3://my-bucket/output");
Class.forName("com.simba.athena.jdbc.Driver");
Connection connection = DriverManager.getConnection("jdbc:awsathena://AwsRegion=us-east-1;",info);
Statement statement = connection.createStatement();
ResultSet queryResults = statement.executeQuery("ALTER TABLE test01 ADD PARTITION (col2 = 'b') LOCATION 's3://my-bucket/col2=b/'");
}
}
This is the error that the code above throws, note that the query runs OK in the Athena console if I copy paste it from the error message:
Exception in thread "main" java.sql.SQLDataException: [Simba][JDBC](11300) A ResultSet was expected but not generated from query "ALTER TABLE test01 ADD PARTITION (col2 = 'b') LOCATION 's3://my-bucket/col2=b/'". Query not executed.
at com.simba.athena.exceptions.ExceptionConverter.toSQLException(Unknown Source)
at com.simba.athena.jdbc.common.SStatement.checkCondition(Unknown Source)
at com.simba.athena.jdbc.common.SStatement.executeNoParams(Unknown Source)
at com.simba.athena.jdbc.common.SStatement.executeNoParams(Unknown Source)
at com.simba.athena.jdbc.common.SStatement.executeQuery(Unknown Source)
at none.test.main(test.java:22)
However, if I replace the statement with a simple SELECT, it works fine:
//this works fine
ResultSet queryResults = statement.executeQuery("SELECT * FROM test01");
queryResults.next();
System.out.println(queryResults.getString("col1") + " -> " + queryResults.getString("col2"));
Here's the table definition:
CREATE EXTERNAL TABLE test01 (
`col1` string
) PARTITIONED BY (
col2 string
)
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
WITH SERDEPROPERTIES (
'serialization.format' = ',',
'field.delim' = ','
) LOCATION 's3://my-bucket/'
TBLPROPERTIES ('has_encrypted_data'='false');
The driver version used is the one provided on the Athena docs: https://s3.amazonaws.com/athena-downloads/drivers/JDBC/SimbaAthenaJDBC_2.0.9/AthenaJDBC42_2.0.9.jar
Any thoughts on how to address this? Thanks!
You appear to be using statement.executeQuery(...) to execute a SQL statement that isn't a query.
Try using statement.execute(...) instead:
statement.execute("ALTER TABLE test01 ADD PARTITION (col2 = 'b') LOCATION 's3://my-bucket/col2=b/'");

spark jdbc api can't use built-in function

I want to get subquery from impala table as one dataset.
Code like this:
String subQuery = "(select to_timestamp(unix_timestamp(now())) as ts from my_table) t"
Dataset<Row> ds = spark.read().jdbc(myImpalaUrl, subQuery, prop);
But result is error:
Caused by: java.sql.SQLDataException: [Cloudera][JDBC](10140) Error converting value to Timestamp.
I can use unix_timestamp function,but to_timestmap failed, why?
I found code in org.apache.spark.sql.execution.datasources.jdbc.JDBC.compute() exists some problem:
sqlText = s"SELECT $columnList FROM ${options.table} $myWhereClause"
$columList contains " like "col_name" , when I delete " it work fine.
I solve this problem by add dialect, default dialect will add "" to column name,
JdbcDialect ImpalaDialect = new JdbcDialect(){
#Override
public boolean canHandle(String url) {
return url.startsWith("jdbc:impala") || url.contains("impala");
}
#Override
public String quoteIdentifier(String colName) {
return colName;
}
};
JdbcDialects.registerDialect(ImpalaDialect);

Problem in updating a data column in spring

I have a Database table called ProgramData. their i have a data column called Id and executed. id set to be as auto increment.
Table structure is like this.
What i want is according to id executed column need to be updated. following is my code segment.
public void saveDtvProgDataExecuted()
{
ProgramData programeData = new ProgramData();
String SQL = "UPDATE program_data SET executed=1 WHERE programeData.id = ?";
this.jdbcTemplate.update(SQL);
}
If i run this code this gives me error like bad SQL grammar [UPDATE program_data SET executed=1 WHERE programeData.id = ?]; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '?' at line 1
Problem is you’re not passing the ID value to the jdbctemplate.
You should use
this.jdbctemplate.update(SQL, id);
Where id is the id of the record you’re updating.
Please refer to the documentation for more information:
http://static.springsource.org/spring/docs/3.1.x/spring-framework-reference/htmlsingle/spring-framework-reference.html#jdbc-updates
TRY THIS statement while you are passing ? in your sql query it need to be set while execution.
String SQL = "UPDATE program_data SET executed=1 WHERE programeData.id = ?";
this.jdbcTemplate.update(SQL,new PreparedStatementCallback<Boolean>(){
#Override
public Boolean doInPreparedStatement(PreparedStatement ps)
throws SQLException, DataAccessException {
ps.setInt(1,"here you need to pass value of programeData.id);
return ps.execute();
}
});

To execute a dynamic query which using DB function

Requirement :
Having a query stored in DB with in a query there is a where condition in that its calling a database function.
Using spring MVC I need to get the query, pass the parameter and get the return value.
This is the query:
SELECT COUNT(*)
FROM IncidentHdr ih, IncidentUser iu
WHERE ih.incidentId = iu.incidentHdr.incidentId
AND get_response_team_access (ih.incidentId, :perscode)
Here get_response_team_access is a DB function which returns an integer. Query works fine as we tested in DB using dummy data.
What I tried So far :
#PersistenceContext
private EntityManager em;
#Override
public Long getAlertCount(String queryString, long persCode) throws DataAccessException {
Query q = em.createQuery(queryString);
q.setParameter("perscode", persCode);
return (long) q.getSingleResult();
}
Throws Exception:
ERROR org.hibernate.hql.internal.ast.ErrorCounter - <AST>:1:293: unexpected AST node: (
antlr.NoViableAltException: unexpected AST node: (
To call DB function from JPQL you have to use FUNCTION keyword.
SELECT COUNT(*) FROM IncidentHdr ih,IncidentUser iu
WHERE ih.incidentId = iu.incidentHdr.incidentId
AND FUNCTION('get_response_team_access',ih.incidentId, :perscode)
Use FUNCTION (formerly FUNC) to call database specific functions from
JPQL
Usage:
You can use FUNCTION to call database functions that are not supported
directly in JPQL and to call user or library specific functions.
Source: http://www.eclipse.org/eclipselink/documentation/2.4/jpa/extensions/j_func.htm

How to run JDBC update query without knowing column type

I have to run a jdbc update query using preparedstatement/statement without knowing the column type.
I have a query say ' update table set status=? where id=? '
and i am getting a map of values like {("status"="123"), ("id"="546")}
Now I don't know the column type, is there any generic way to run this query using jdbc?
Instead of running - ps.setString(1,map.get(("status"));
beacause i don't know the column type of status field in DB (it may be a int also)
Please help me in solving this without using spring jdbc templates.
ps.setString(1,map.get(("status")) will work for integer also, you just has to take care that value you are putting in integer column is of int type.
Following code explains that:
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.SQLException;
public class SOJDBC
{
public static void main(String rgs[])
{
Connection con = DBConnection.getConnection("TEMP");
try
{
PreparedStatement pstmt = con.prepareStatement("insert into STUDENT(ROLLNO,NAME,AGE) values(?,?,?)");
pstmt.setString(1, "1"); //column type is integer, will work because the value is of int type
pstmt.setString(2, "Bhushan");
pstmt.setString(3, "25"); //column type is integer, will work because the value is of int type
pstmt.executeUpdate();
}
catch (SQLException e)
{
e.printStackTrace();
}
}
}

Resources