all. I met a strange error when I use hive udf through jdbc client.
I have a udf to help me convert a string into time stamp format called reformat_date. I firstly execute ADD JAR and CREATE TEMPORARY FUNCTION, both work fine.
The SQL also can be explained in hive cli mode, and can be executed. But when use jdbc client, I got errors:
Query returned non-zero code: 10, cause:
FAILED: Error in semantic analysis: Line 1:283 Wrong arguments ''20121201000000'':
org.apache.hadoop.hive.ql.metadata.HiveException:
Unable to execute method public org.apache.hadoop.io.Text com.aa.datawarehouse.hive.udf.ReformatDate.evaluate(org.apache.hadoop.io.Text) on object com.aa.datawarehouse.hive.udf.ReformatDate#4557e3e8 of class com.aa.datawarehouse.hive.udf.ReformatDate with arguments {20121201000000:org.apache.hadoop.io.Text} of size 1:
at com.aa.statistic.dal.impl.TjLoginDalImpl.selectAwakenedUserCount(TjLoginDalImpl.java:258)
at com.aa.statistic.backtask.service.impl.UserBehaviorAnalysisServiceImpl.recordAwakenedUser(UserBehaviorAnalysisServiceImpl.java:326)
at com.aa.statistic.backtask.controller.BackstatisticController$21.execute(BackstatisticController.java:773)
at com.aa.statistic.backtask.controller.BackstatisticController$DailyExecutor.execute(BackstatisticController.java:823)
My SQL is
select count(distinct a.user_id) as cnt from ( select user_id, user_kind, login_date, login_time from tj_login_hive where p_month = '2012_12' and login_date = '20121201' and user_kind = '0' ) a join ( select user_id from tj_login_hive where p_month <= '2012_12' and datediff(to_date(reformat_date(concat('20121201', '000000'))), to_date(reformat_date(concat(login_date, '000000')))) >= 90 ) b on a.user_id = b.user_id
Thanks.
i think your udf threw exception.
if reformat_date function is that you make, you should check your logic.
if not, you should check the udf's specification.
Related
I'm trying to use parameters for column and table name to SQL query over JNDY, which doesn't work, while parameters for values work fine.
I defined 3 simple parameter components with default values:
paramTable = 'dummyTable'
paramColumn = 'dummyColumn'
paramValue = 'dummyValue'
Then I defined an SQL query over JNDI, that looks like:
select * from ${paramTable} where ${paramColumn} = ${paramValue};
... which does not work, because parameter values of paramTable and paramColumn don't seem to apply properly, which causes:
Caused by: java.sql.SQLSyntaxErrorException: ORA-00903: invalid table name
Here are some query modifications, and their results really disappoint me:
select * from dummyTable where dummyColumn = ${paramValue};
-- works fine and delivers 1 row
select * from ${paramTable} where dummyColumn = 'dummyValue';
-- causes 'invalid table name'
select * from dummyTable where ${paramColumn} = 'dummyValue';
-- does not cause any errors, but if I console.log() the result set of the query,
it is empty (1 row is expected though)
Do I miss something?
In hive,
I am trying to select the entry with the minimum timestamp, however it's throwing the following error, not sure what is the reason.
select * from sales where partition_batch_ts = (select max(partition_batch_ts) from sales);
Error
Error while compiling statement: FAILED: ParseException line 1:91 cannot recognize input near 'select' 'max' '(' in expression specification
I think you need to use proper table alias. Also, IN must be used instead of =
SELECT s1.*
FROM sales s1
WHERE s1.partition_batch_ts IN
(SELECT MAX(partition_batch_ts)
FROM sales s2);
From Hive manual, SUBQUERIES :
As of Hive 0.13 some types of subqueries are supported in the WHERE
clause.
I am using Spring JPA with DB2, when i use paging repository and queries for second page it throws error.
This is the generated query
SELECT *
FROM (SELECT inner2_.*,
ROWNUMBER()
OVER(
ORDER BY ORDER OF inner2_) AS rownumber_
FROM (SELECT db2DATAa0_.c_type AS col_0_0_,
db2DATAa0_.h_proc AS col_1_0_,
db2DATAa0_.n_vin AS col_2_0_,
db2DATAa0_.i_cust AS col_3_0_
FROM dcu.v_rpt_data_hist db2DATAa0_
WHERE db2DATAa0_.reportid = '0H000488089'
AND ( db2DATAa0_.c_type = 'S'
OR db2DATAa0_.c_type = 'N'
OR db2DATAa0_.c_type = 'A'
OR db2DATAa0_.c_type = 'T' )
ORDER BY db2DATAa0_.h_proc desc
FETCH first 30 ROWS only) AS inner2_) AS inner1_
WHERE rownumber_ > 15
ORDER BY rownumber_
Error:
2719372 [2016-10-21 16:29:02,040] [RxCachedThreadScheduler-13] WARN org.hibern
ate.engine.jdbc.spi.SqlExceptionHelper - SQL Error: -199, SQLState: 42601
2719379 [2016-10-21 16:29:02,047] [RxCachedThreadScheduler-13] ERROR org.hibern
ate.engine.jdbc.spi.SqlExceptionHelper - DB2 SQL Error: SQLCODE=-199, SQLSTATE=
42601, SQLERRMC=OF;??( [ DESC ASC NULLS RANGE CONCAT || / MICROSECONDS MICROSECO
ND, DRIVER=3.57.82
Any idea?
Your error states the ILLEGAL USE OF KEYWORD OF. TOKEN [DESC ASC NULLS RANGE CONCAT] WAS EXPECTED.
I identified this as the critical part of the query:
ORDER BY ORDER OF inner2_
DB2 expects one of DESC, ASC, NULLS, RANGE, CONCAT after the second ORDER keyword.
This issue can be resolve by change dialect.
Change dialect in configuration or property file to DB2ZOSDialect
I have a query as below which is returning expected records when run from the SQL Developer
SELECT *
FROM MY_TABLE WHERE ( CRT_TS > TO_DATE('25-Aug-2016 15:08:18', 'DD-MON-YYYY HH24:MI:SS')
or UPD_TS > TO_DATE('25-Aug-2016 15:08:18', 'DD-MON-YYYY HH24:MI:SS'));
I think that we will not need to apply TO_DATE when we are passing java.util.Date object as date parameters but the below code snippet is silently returning me 0 records.
My SQL query in Java class is as below:
SELECT *
FROM MY_TABLE WHERE ( CRT_TS > :lastSuccessfulReplicationTimeStamp1
or UPD_TS > :lastSuccessfulReplicationTimeStamp2);
The code which executes the above query is as below but the below code snippet is silently returning me 0 records:
parameters.put("lastSuccessfulReplicationTimeStamp1", new java.sql.Date(outputFileMetaData.getLastSuccessfulReplicationTimeStamp().getTime()));
parameters.put("lastSuccessfulReplicationTimeStamp2", new java.sql.Date(outputFileMetaData.getLastSuccessfulReplicationTimeStamp().getTime()));
list = namedParameterJdbcTemplateOracle.query(sql, parameters, myTabRowMapper);
Please advise.
I guess you already found the answer but if anybody else needs it, here's what I've found:
java.sql.Date doesn't have time, just the date fields. Either use java.sql.Timestamp or java.util.Date. Both seems to be working for me with NamedParameterJdbcTemplate.
A little variation to above solution can be when your input(lastSuccessfulReplicationTimeStamp1/lastSuccessfulReplicationTimeStamp2) is a String instead of Date/TimeStamp (which is what i was looking for and found at this link -> may be it can help someone):
MapSqlParameterSource parameters = new MapSqlParameterSource();
parameters.addValue("lastSuccessfulReplicationTimeStamp1", lastSuccessfulReplicationTimeStamp1, Types.TIMESTAMP);
parameters.addValue("lastSuccessfulReplicationTimeStamp2", lastSuccessfulReplicationTimeStamp2, Types.TIMESTAMP);
list = namedParameterJdbcTemplateOracle.query(sql, parameters, myTabRowMapper);
I am trying to execute a sql script from file using ScriptUtils.readScript method:
sql = ScriptUtils.readScript(fileReader,
ScriptUtils.DEFAULT_COMMENT_PREFIX,
ScriptUtils.DEFAULT_STATEMENT_SEPARATOR);
getJdbcTemplate().update(sql);
But I get the error org.springframework.jdbc.BadSqlGrammarException: StatementCallback; bad SQL grammar and from the logs I see that semicolon * in the sql statement is not ignored even I am using ScriptUtils.DEFAULT_STATEMENT_SEPARATOR why it isn't working? What's wrong here?
Edit: I know that I can solve this by using:
getJdbcTemplate().update(sql.replace(";", ""));
but maybe there is another solution?
Edit2: Here is example of sql that I need to execute:
INSERT
INTO MYTABLE
(
ID,
MYNUMBER,
MYVALUE
)
SELECT
ID,
0,
B.MYVALUE
FROM ATABLE A,
BTABLE B
WHERE A.ID = B.ID
AND NOT EXISTS
(SELECT 1 FROM MYTABLE M WHERE M.ID = A.ID
);
I don't think you're using ScriptUtils.readScript the right way. The javadocs themselves state:
Mainly for internal use within the framework.
Looking at the source code, it seems that all this function does is load all the lines from a file into a single string, with some logic around comments. The use of the separator in this method is minor and appears only to be relevant if there is a whitespace at the end of it.
If you want to ignore the separator, you'll need to remove it the way that you suggested (with a replace).