I have a Spring Batch application in which i have to export a partition based on a config table, using Oracle Data Dump. In the config table I have information like days to run the export, name of the table to export and so on.
This is my query provider
private PagingQueryProvider queryProviderStep1() throws Exception {
SqlPagingQueryProviderFactoryBean queryProvider = new SqlPagingQueryProviderFactoryBean();
queryProvider.setDataSource(infraConfig.dataSourceLocal());
queryProvider.setDatabaseType("ORACLE");
queryProvider
.setSelectClause("SELECT part.partition_name, config.ID_CONFIG_DATA_EXPORT, config.SERVIDOR_ORIGEM, "
+ "config.SERVIDOR_DESTINO, config.CAMINHO_DESTINO, config.NOME_TABELA, config.TEMPO_RETENCAO_BD, "
+ "config.TEMPO_RETENCAO_TAPELIBRARY, config.TEMPO_DELAY, config.FREQUENCIA_EXECUCAO ");
queryProvider.setFromClause("FROM user_tab_partitions#CDRONE_RAC part "
+ "INNER JOIN CONFIG_DATA_EXPORT config ON config.NOME_TABELA = part.TABLE_NAME "
+ "LEFT JOIN CONFIG_DATA_EXPORT_LOG exlog ON config.ID_CONFIG_DATA_EXPORT = exlog.ID_CONFIG_DATA_EXPORT ");
queryProvider.setWhereClause(" WHERE exlog.ID_CONFIG_DATA_EXPORT IS NULL");
queryProvider.setSortKey("config.id_config_data_export");
return queryProvider.getObject();
The problem I have is when Spring generates the 'remainingPagesSql' which it includes ' AND ((config.id_config_data_export > ?))' at the end of the query. I was expecting that Spring would automatically use the sortKey as the parameter, but it does not and gives me the error:
org.springframework.jdbc.UncategorizedSQLException: PreparedStatementCallback; uncategorized SQLException for SQL [SELECT * FROM (SELECT part.partition_name, config.ID_CONFIG_DATA_EXPORT, config.SERVIDOR_ORIGEM, config.SERVIDOR_DESTINO, config.CAMINHO_DESTINO, config.NOME_TABELA, config.TEMPO_RETENCAO_BD, config.TEMPO_RETENCAO_TAPELIBRARY, config.TEMPO_DELAY, config.FREQUENCIA_EXECUCAO FROM user_tab_partitions#CDRONE_RAC part INNER JOIN CONFIG_DATA_EXPORT config ON config.NOME_TABELA = part.TABLE_NAME LEFT JOIN CONFIG_DATA_EXPORT_LOG exlog ON config.ID_CONFIG_DATA_EXPORT = exlog.ID_CONFIG_DATA_EXPORT WHERE exlog.ID_CONFIG_DATA_EXPORT IS NULL ORDER BY config.id_config_data_export ASC) WHERE ROWNUM <= 10 AND ((config.id_config_data_export > ?))]; SQL state [99999]; error code [17041]; Missing IN or OUT parameter at index:: 1; nested exception is java.sql.SQLException: Missing IN or OUT parameter at index:: 1
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:84) ~[spring-jdbc-4.3.10.RELEASE.jar:4.3.10.RELEASE]
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81) ~[spring-jdbc-4.3.10.RELEASE.jar:4.3.10.RELEASE]
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81) ~[spring-jdbc-4.3.10.RELEASE.jar:4.3.10.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:649) ~[spring-jdbc-4.3.10.RELEASE.jar:4.3.10.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:684) ~[spring-jdbc-4.3.10.RELEASE.jar:4.3.10.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:716) ~[spring-jdbc-4.3.10.RELEASE.jar:4.3.10.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:726) ~[spring-jdbc-4.3.10.RELEASE.jar:4.3.10.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:776) ~[spring-jdbc-4.3.10.RELEASE.jar:4.3.10.RELEASE]
at org.springframework.batch.item.database.JdbcPagingItemReader.doReadPage(JdbcPagingItemReader.java:222) ~[spring-batch-infrastructure-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.item.database.AbstractPagingItemReader.doRead(AbstractPagingItemReader.java:108) ~[spring-batch-infrastructure-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.read(AbstractItemCountingItemStreamItemReader.java:88) ~[spring-batch-infrastructure-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProvider.doRead(SimpleChunkProvider.java:91) ~[spring-batch-core-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProvider.read(SimpleChunkProvider.java:157) ~[spring-batch-core-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProvider$1.doInIteration(SimpleChunkProvider.java:116) ~[spring-batch-core-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:374) ~[spring-batch-infrastructure-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215) ~[spring-batch-infrastructure-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144) ~[spring-batch-infrastructure-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProvider.provide(SimpleChunkProvider.java:110) ~[spring-batch-core-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:69) ~[spring-batch-core-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:406) ~[spring-batch-core-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:330) ~[spring-batch-core-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133) ~[spring-tx-4.3.10.RELEASE.jar:4.3.10.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:272) ~[spring-batch-core-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:81) ~[spring-batch-core-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.repeat.support.TaskExecutorRepeatTemplate$ExecutingRunnable.run(TaskExecutorRepeatTemplate.java:262) ~[spring-batch-infrastructure-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_121]
Caused by: java.sql.SQLException: Missing IN or OUT parameter at index:: 1
I've tried to map a parameter but I don't have or don't know how to use this parameter as it's based on the PAGE_SIZE, so in each time the query is executed, the value changes.
FIRST EDIT
I've updated the sortKey to "id_config_data_export", without the "config.". But the error still happens.
Finally made it work! Instead of writing the query directly in Java, I created a view with the query and it worked. I still don't know why it was not working but that's the workaround.
This is the queryProvider now:
private PagingQueryProvider queryProviderStep1() throws Exception {
SqlPagingQueryProviderFactoryBean queryProvider = new SqlPagingQueryProviderFactoryBean();
queryProvider.setDataSource(infraConfig.dataSourceLocal());
queryProvider.setDatabaseType("ORACLE");
queryProvider.setSelectClause("SELECT * ");
queryProvider.setFromClause("FROM VW_CONFIG_EXPORT_PART_CDRONE ");
queryProvider.setSortKey("id_config_data_export");
return queryProvider.getObject();
Related
I have two entities Vendor and Goods with one-to-many relation, the relation looks like:
I am using mybatis with annotation, the mapper:
GoodsMapper
public interface GoodsMapper {
#Select("select * from goods where id=#{goodsId}")
#Results({
#Result(id = true, column = "id", property = "id"),
#Result(column = "name", property = "name"),
#Result(column = "vendor_id", property = "vendor",
one = #One(select = "com.xxx.server.mapper.VendorMapper.getVendor"))
})
Goods getGoods(#Param("goodsId") String goodsId);
}
VendorMapper
public interface VendorMapper {
#Select("select * from vendor where id=#{vendorId}")
Vendor getVendor(#Param("vendorId") String vendorId);
}
ignore the entity code & others...
when I invoked goodsMapper.getGoods(goodsId), I caught the following exception :
Caused by: org.apache.ibatis.exceptions.PersistenceException:
### Error querying database. Cause: java.lang.IllegalArgumentException: Mapped Statements collection does not contain value for com.xxx.server.mapper.VendorMapper.getVendor
### The error may exist in com/xxx/server/mapper/GoodsMapper.java (best guess)
### The error may involve com.xxx.server.mapper.GoodsMapper.getGoods
### The error occurred while handling results
### SQL: select * from goods where id=?
### Cause: java.lang.IllegalArgumentException: Mapped Statements collection does not contain value for com.xxx.server.mapper.VendorMapper.getVendor
at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30)
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:150)
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:141)
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectOne(DefaultSqlSession.java:77)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:433)
... 117 more
Caused by: java.lang.IllegalArgumentException: Mapped Statements collection does not contain value for com.xxx.server.mapper.VendorMapper.getVendor
at org.apache.ibatis.session.Configuration$StrictMap.get(Configuration.java:933)
at org.apache.ibatis.session.Configuration.getMappedStatement(Configuration.java:726)
at org.apache.ibatis.session.Configuration.getMappedStatement(Configuration.java:719)
at org.apache.ibatis.executor.resultset.DefaultResultSetHandler.getNestedQueryMappingValue(DefaultResultSetHandler.java:740)
at org.apache.ibatis.executor.resultset.DefaultResultSetHandler.getPropertyMappingValue(DefaultResultSetHandler.java:465)
at org.apache.ibatis.executor.resultset.DefaultResultSetHandler.applyPropertyMappings(DefaultResultSetHandler.java:441)
I have checked the class path com.xxx.server.mapper.VendorMapper.getVendor for the select of #One, it is correct.
Appreciate any kind help~
In my case this was caused by referenced collection not being initialized by Spring yet.
Solution is to add #DependsOn annotation to the "parent" mapper.
#DependsOn("VendorMapper")
public interface GoodsMapper{
...
}
#Repository("VendorMapper")
public interface VendorMapper {
...
}
When I am trying to use PreparedStatement with Statement.RETURN_GENERATED_KEYS, I don't know why PreparedStatement object is requiring an extra parameter. I have already googled every possible string but didn't get any response.
Below is the code that I am using for inserting a record.
StringBuilder headerSql = new StringBuilder();
headerSql.append("insert into TABLE_XXX(subscriber_id,company_id,stg_inv_id,inv_loc_id,count_date,count_status,count_type,creation_user,creation_date,update_user,update_date) \n");
headerSql.append("(select ?,?,XXXX_PK.NEXTVAL,?,sysdate,0,0,?,sysdate,?,sysdate from dual \n");
headerSql.append("where not exists(select 1 from TABLE_XXX where subscriber_id=? and company_id=? and inv_loc_id=? and count_status=0 and count_type=0)) \n");
String stgInvId=null;
PreparedStatement ps = null;
ResultSet rs = null;
try {
ps = con.prepareStatement(headerSql.toString(), Statement.RETURN_GENERATED_KEYS);
ps.setString(1,Integer.toString(IDObj.getSubscriberID()));
ps.setString(2,Integer.toString(IDObj.getCompanyID()));
ps.setString(3,locId);
ps.setString(4,IDObj.getUserLogin());
ps.setString(5,IDObj.getUserLogin());
ps.setString(6,Integer.toString(IDObj.getSubscriberID()));
ps.setString(7,Integer.toString(IDObj.getCompanyID()));
ps.setString(8,locId);
ps.executeUpdate();
rs = ps.getGeneratedKeys();
rs.next();
stgInvId=rs.getString("stg_inv_id");
} catch (Exception e) {
System.out.println(headerSql.toString());
e.printStackTrace();
}
Below is the list of parameter that is getting bound :
As you can see above, I don't now from where it is adding the last parameter and what value I need to pass here. Please guide me guys.
Below is the Stacktrace :
java.sql.SQLException: ORA-00933: SQL command not properly ended
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:111)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:330)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:287)
at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:742)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:212)
at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:951)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1159)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3284)
at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:3367)
at com.birchstreet.synergy.countmanagement.dao.CountManagementDAOImpl.inserStgHeader(CountManagementDAOImpl.java:325)
at com.birchstreet.synergy.countmanagement.dao.CountManagementDAOImpl.inserStgRecord(CountManagementDAOImpl.java:280)
at com.birchstreet.webservices.synergy.WSCountManagement.setLocationsInStartInventory(WSCountManagement.java:138)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:143)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:160)
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:158)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:97)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102)
at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:303)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:286)
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1072)
at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:399)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:381)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:344)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:221)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:624)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:799)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1455)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
In oracle DB RETURN_GENERATED_KEYS works in that way.
1) RETURN_GENERATED_KEYS flag
PreparedStatement ps = con.prepareStatement("INSERT INTO XXX( A, B) VALUES ( xxx_s.nextval, 'aaa')",Statement.RETURN_GENERATED_KEYS);
ps.executeUpdate();
ResultSet rs = ps.getGeneratedKeys();
while(rs.next()) {
System.out.println(rs.getString(1));
}
Oracle doesn't know which column was auto-generated and it is returning rowid. returning rowid into :? is appended to the insert
2) List of columns.
PreparedStatement ps = con.prepareStatement("INSERT INTO XXX( A, B) VALUES ( xxx_s.nextval, 'aaa')",new String[]{"A"});
ps.executeUpdate();
ResultSet rs = ps.getGeneratedKeys();
while(rs.next()) {
System.out.println(rs.getString(1));
}
con.close();
In this case we explicitly specify auto generated column. returning a into :? is appended to the insert
In both case you to access generated column you have to use positional access. Access by column name is not supported.
However you are trying to do insert as select . Statment is tranformated to
insert into xxxy select * from xxxy returning rowid into :var ;
And this Statement is not supported by oracle db. And exception ORA-00933: SQL command not properly ended is raised.
I'm doing integration testing in a Hsqldb. My production database is an Oracle Database.
Versions
Hibernate : 4.1.3.final.
Hsqldb : 2.3.3. ( I can't use the 2.3.4 because it can't run all my junit tests in one click.)
My problem
To create my test database, I followed this tutorial with minor modifications. Everything works fine except for the methods that use the TO_CHAR(integer) function from Oracle. For those methods, I got an unexpected token: )
This is the code that causes the exception
select ="select p.name, to_char(p.id) "
+ " from t_player p " +
"inner join t_job job on j.id=p.id_job ";
This is a part of my unit test
public void testFindPlayer() throws ClassNotFoundException, SQLException {
Class.forName("org.hsqldb.jdbcDriver");
Connection connection = DriverManager.getConnection("jdbc:hsqldb:mem:DB", "sa", "");
String syntax_ora = "SET DATABASE SQL SYNTAX ORA TRUE";
PreparedStatement ps_ora = connection.prepareStatement(syntax_ora);
ps_ora.execute();
ps_ora.close();
connection.close();
List<String[]> actual_player = Player.findPlayer("Name");
List<String[]> expected_player = new ArrayList<String[]>();
//etc..
This is the exception
Caused by: org.hsqldb.HsqlException: unexpected token: )
at org.hsqldb.error.Error.parseError(Unknown Source)
at org.hsqldb.ParserBase.unexpectedToken(Unknown Source)
at org.hsqldb.ParserDQL.readExpression(Unknown Source)
at org.hsqldb.ParserDQL.readSQLFunction(Unknown Source)
at org.hsqldb.ParserDQL.readColumnOrFunctionExpression(Unknown Source)
at org.hsqldb.ParserDQL.XreadSimpleValueExpressionPrimary(Unknown Source)
at org.hsqldb.ParserDQL.XreadAllTypesValueExpressionPrimary(Unknown Source)
at org.hsqldb.ParserDQL.XreadAllTypesPrimary(Unknown Source)
at org.hsqldb.ParserDQL.XreadAllTypesFactor(Unknown Source)
at org.hsqldb.ParserDQL.XreadAllTypesTerm(Unknown Source)
at org.hsqldb.ParserDQL.XreadAllTypesCommonValueExpression(Unknown Source)
at org.hsqldb.ParserDQL.XreadValueExpression(Unknown Source)
at org.hsqldb.ParserDQL.XreadSelect(Unknown Source)
at org.hsqldb.ParserDQL.XreadQuerySpecification(Unknown Source)
at org.hsqldb.ParserDQL.XreadSimpleTable(Unknown Source)
at org.hsqldb.ParserDQL.XreadQueryPrimary(Unknown Source)
at org.hsqldb.ParserDQL.XreadQueryTerm(Unknown Source)
at org.hsqldb.ParserDQL.XreadQueryExpressionBody(Unknown Source)
at org.hsqldb.ParserDQL.XreadQueryExpression(Unknown Source)
at org.hsqldb.ParserDQL.compileCursorSpecification(Unknown Source)
at org.hsqldb.ParserCommand.compilePart(Unknown Source)
at org.hsqldb.ParserCommand.compileStatement(Unknown Source)
at org.hsqldb.Session.compileStatement(Unknown Source)
at org.hsqldb.StatementManager.compile(Unknown Source)
at org.hsqldb.Session.execute(Unknown Source)
... 41 more
How I tried to fix it
1) I try another function from Oracle that is to_char(number,'format').
Basically, I change the select part with the following:
select ="select p.name, to_char(p.id,'999999') "
+ " from t_player p " +
"inner join t_job job on j.id=p.id_job ";
But then, I got this exception : incompatible data type in operation
2) I put the SET DATABASE SQL OSYNTAX part into comments
The same exceptions are raised.
Do you have an idea on how to fix this issue please?
Thank you for your reply.
HSQLDB's built-in TO_CHAR function supports only date and timestamp arguments.
You can create a user-defined TO_CHAR function for numeric values. For example:
CREATE FUNCTION TO_CHAR(param DECIMAL) RETURNS VARCHAR(20)
RETURN CAST(param AS VARCHAR(20))
CREATE FUNCTION TO_CHAR(param DECIMAL, format VARCHAR(20)) RETURNS VARCHAR(20)
RETURN CAST(param AS VARCHAR(20))
Now TO_CHAR(p.id) will return the id as a string and the two-arge version also returns the same.
I have the same error using the CAST function, like this:
cast(id as VARCHAR)
But the correct is:
cast(id as VARCHAR(20))
I am facing issue when I try to get some products from hive table and process/apply rools in spark.
//function which return products from Hive table
def getProductsList(hiveContext: org.apache.spark.sql.hive.HiveContext): scala.collection.mutable.MutableList[Product] = {
val products = scala.collection.mutable.MutableList[Product]()
val results = hiveContext.sql("select item_id,value from details where type_id=12");
val collection = results.collect();
var i = 0;
results.collect.foreach(t => {
val product = new Product(collection(i)(0).asInstanceOf[Long], collection(i)(1).asInstanceOf[String]);
i = i+ 1;
products += product
})
products
}
Calling getProductsList function and applying drools rools on products.
val randomProducts = this.getProductsList(hiveContext)
val rdd = ssc.sparkContext.parallelize(randomProducts)
val evaluatedProducts = rdd.mapPartitions(incomingProducts => {
print("Hello");
rulesExecutor.evalRules(incomingProducts) })
val productdf = hiveContext.applySchema(evaluatedProducts, classOf[Product])
})
As showin in above rdd mapPartitions iteration not happening and it is throwing following error. But I am sure rdd is not empty.
Exception in thread "main" java.lang.NullPointerException
at org.spark-project.guava.reflect.TypeToken.method(TypeToken.java:465)
at org.apache.spark.sql.catalyst.JavaTypeInference$$anonfun$2.apply(JavaTypeInference.scala:103)
at org.apache.spark.sql.catalyst.JavaTypeInference$$anonfun$2.apply(JavaTypeInference.scala:102)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:108)
at org.apache.spark.sql.catalyst.JavaTypeInference$.org$apache$spark$sql$catalyst$JavaTypeInference$$inferDataType(JavaTypeInference.scala:102)
at org.apache.spark.sql.catalyst.JavaTypeInference$.inferDataType(JavaTypeInference.scala:47)
at org.apache.spark.sql.SQLContext.getSchema(SQLContext.scala:995)
at org.apache.spark.sql.SQLContext.createDataFrame(SQLContext.scala:488)
at org.apache.spark.sql.SQLContext.applySchema(SQLContext.scala:1028)
at com.cloudera.sprue.ValidateEan$.main(ValidateEan.scala:70)
at com.cloudera.sprue.ValidateEan.main(ValidateEan.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:672)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
16/05/05 07:44:48 INFO SparkContext: Invoking stop() from shutdown hook
Please help me out to resolve this issue.
As we need the final result as DataFrame, Let's use SchemaRDD which returned from hiveContext.sql().
//defining schema
case class Product(id: Long, value: String)
//loading data from Hive table
val results: DataSet[Row] = hiveContext.sql("select item_id,value from details where type_id=12")
//convert ROW type to Product type then pass it to rulesExecutor.evalRules()
val evaluatedProducts = results.map(productRow => rulesExecutor.evalRules(Product(productRow.getLong(0), productRow.getString(1)))).toDF()
I'd assume rulesExecutor.evalRules() will aceeptProduct type. If not we can go head with Row type (without explicitly converting in map()).
I am using Spring and my class is annotated with #Transactional.
I am using the SimpleJdbcInsert but I am getting the following warning:
TableMetaDataProvider: - Unable to locate table meta data for
'data.data_insert' -- column names must be provided
I have three tables and all the three are having the relationship such that:
primary key of table1 is the foreign key in table 2 and the primary key in table 2 is the foreign key in table 3.
Showing table 1 insert code:
java.sql.Timestamp timestamp = getCurrentJavaSqlTimestamp();
Map<String, Object> params = new HashMap<String, Object>();
params.put("notes", task.getNotes());
params.put("recording_time", timestamp);
params.put("end_user_id", 805);
SimpleJdbcInsert insertData = new SimpleJdbcInsert(dataSource).
withTableName("data.data_insert").
usingColumns("notes", "recording_time",
"end_user_id").usingGeneratedKeyColumns("data_id");
long dataId = insertData.executeAndReturnKey(params).longValue();
The error logs:
2015-09-29 14:10:27,133 WARN [http-8080-2] LegacyFlexJsonExceptionMessageConverter: - Generated Key Name(s) not specificed. Using the generated keys features requires specifying the name(s) of the generated column(s) for User ID: 805, Request ID: f8da3bb5-0613-4a74-9ca8-95a6ab4f1692, clientIP: 127.0.0.1 uri: /admin/dataInsert, Request Parameters:
org.springframework.dao.InvalidDataAccessApiUsageException: Generated Key Name(s) not specificed. Using the generated keys features requires specifying the name(s) of the generated column(s)
at org.springframework.jdbc.core.simple.AbstractJdbcInsert.prepareStatementForGeneratedKeys(AbstractJdbcInsert.java:530)
at org.springframework.jdbc.core.simple.AbstractJdbcInsert.access$0(AbstractJdbcInsert.java:528)
at org.springframework.jdbc.core.simple.AbstractJdbcInsert$1.createPreparedStatement(AbstractJdbcInsert.java:448)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:581)
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:843)
at org.springframework.jdbc.core.simple.AbstractJdbcInsert.executeInsertAndReturnKeyHolderInternal(AbstractJdbcInsert.java:445)
at org.springframework.jdbc.core.simple.AbstractJdbcInsert.executeInsertAndReturnKeyInternal(AbstractJdbcInsert.java:426)
at org.springframework.jdbc.core.simple.AbstractJdbcInsert.doExecuteAndReturnKey(AbstractJdbcInsert.java:380)
at org.springframework.jdbc.core.simple.SimpleJdbcInsert.executeAndReturnKey(SimpleJdbcInsert.java:122)
at com.gridpoint.energy.datamodel.impl.PGDataFixBackUpManagerBean.backupDataInRange(PGDataFixBackUpManagerBean.java:79)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:319)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
The correct one:
java.sql.Timestamp timestamp = getCurrentJavaSqlTimestamp();
Map<String, Object> params = new HashMap<String, Object>();
params.put("notes", task.getNotes());
params.put("recording_time", timestamp);
params.put("end_user_id", 805);
SimpleJdbcInsert insertData = new
SimpleJdbcInsert(dataSource).withSchemaName("data").
withTableName("data_insert")
usingColumns("notes", "recording_time",
"end_user_id").usingGeneratedKeyColumns("data_id");
long dataId = insertData.executeAndReturnKey(params).longValue();
So just needed a schemaName using withSchemaName.