How to insert into a snowflake variant field using a DAO? - dao

I have the following code:
#RegisterMapper(MyEntity.ResultMapper.class)
#UseStringTemplate3StatementLocator
public interface MyDao {
#Transaction(TransactionIsolationLevel.SERIALIZABLE)
#SqlBatch("INSERT INTO mySchema.myTable (" +
" id, entity_type, entity_id, flags " +
" ) VALUES " +
"(" +
" :stepId , :entityType , :entityId,parse_json(:flags) " +
")")
#BatchChunkSize(500)
Object create( #BindBean List<MyEntity> entities );
}
As you can see, I am bulk inserting a list of entities into my Snowflake table using this DAO.
The issue is that I am unable to insert into the flags columns, which is a variant. I have tried to_variant(:flags) and currently parse_json(:flags), but the JDBI keeps throwing the following error:
net.snowflake.client.jdbc.SnowflakeSQLException: SQL
compilation error:
Invalid expression [PARSE_JSON(?)] in VALUES clause
[statement:"INSERT INTO mySchema.myTable ( id, entity_type,
entity_id, flags ) VALUES ( :stepId , :entityType , :entityId,
parse_json(:flags) )", located:"null", rewritten:"null",
arguments:{ positional:{}, named:{timeStamp:'null',
entityType:MYENTITY,
flags:'{"client":"myClient","flow":"myFlow"}',stepId:null,
entityId:'189643357241513', class:class myOrg.MyEntity}, finder:[]}]
How should I pass the value in the flags column ? Has anyone attempted this before? The flags field in MyEntity is in my control, I can keep it as a POJO or a String, whichever helps me resolve this issue.

See the comment by Jiansheng Huang for answer:
INSERT INTO T SELECT parse_json(:flag);

Related

Query - String Literal Too Long ORA Error

I am working in Cold Fusion 11* and I am getting the following error:
ORA-01704: string literal too long.
First off, I received this error and corrected it after looking over several sites by changing my code from:
<cfquery datasource="#dsn#">
update paragraphs
set paragraph_text = #input#
where paragraph_id=#rs_d.paragraph_id#
</cfquery>
To:
<cfquery datasource="#dsn#">
update paragraphs
set paragraph_text = <cfqueryparam cfsqltype="CF_SQL_CLOB" value=#input#>
where paragraph_id=#rs_d.paragraph_id#
</cfquery>
That fix worked perfectly. Now I am getting the same error, but instead of using cfquery, I am building the sql query into a string before working with it. So this is how the code looks:
sql = "insert into log (LOG_ENTRY_ID, program_id, paragraph_id, action, userid,";
sql = sql & " paragraph_text_old, paragraph_text_new, comment_id, current_program_status, new_program_status)";
sql = sql & " values (1 ," & program_id & ",";
if (paragraph_id neq ""){
sql = sql & paragraph_id & ",";
}
else{
sql = sql & " null,";
}
sql = sql & "'" & action & "',";
sql = sql & userid & ", '";
sql = sql & DoubleSingleQuotes(paragraph_text_old) & "','";
sql = sql & DoubleSingleQuotes(paragraph_text_new) & "',";
if (comment_id neq ""){
sql = sql & comment_id & ",";
}
else{
sql = sql & " null,";
}
if (current_program_status neq ""){
sql = sql & "'" & current_program_status & "',";
}
else{
sql = sql & " null,";
}
if (new_program_status neq ""){
sql = sql & "'" & new_program_status & "'";
}
else{
sql = sql & " null";
}
sql = sql & ")";
cfstmt(sql);
---End function
<cffunction name="cfstmt">
<cfargument name="sql">
<cfquery name="rs" datasource="#dsn#">
#PreserveSingleQuotes(sql)#
</cfquery>
</cffunction>
The cause of the error is old_paragraph_text and new_paragraph_text. I was wondering if it is possible to include a cfqueryparam type solution to this problem like in the previous problem/solution I included. I tried including it straight like the first one but I am getting compilation errors in the code. Any thoughts or tips would be helpful, thanks.
(Too long for comments)
wanted to limit the code so instead of determining the null value
outside the query, I added the case statements
Honestly, it does not save much, since the code is essentially doing an if/else either way. Except now the work is done on the db side, instead of the app server where it belongs.
Having worked with a lot of legacy apps, I realize they often incorporate what you might euphemistically call "questionable" code ;-) However, you should never use raw client variables in SQL. Unless you have a very good reason to do otherwise, always use cfqueryparam.
Internally, cfqueryparam uses bind variables. Two of the most important benefits are:
Bind variables help protect against sql injection, by preventing the literal values from being executed as SQL commands. This protects the database from values containing maliciously crafted SQL.
For queries executed multiple times, bind variables also boost performance by encouraging the database to reuse execution plans. Otherwise, the database may choose to generate a new execution plan when the query parameters change, which is costly.
CFQueryparam also has some other nice features, such as the "null" attribute. It can be used to submit a null value when certain conditions are met.
Finally, in terms of best practices, it is also a good idea to fully scope all variables. So for example, if the variables are submitted in the FORM scope, the final query might look something like this. (Modify the cfsqltypes as needed.)
<cfquery datasource="#variables.dsn#">
INSERT INTO into log (
program_id
, paragraph_id
, userid
, action
, paragraph_text_old
, paragraph_text_new
)
VALUES
(
<cfqueryparam value="#FORM.program_id#" cfsqltype="CF_SQL_INTEGER">
, <cfqueryparam value="#FORM.paragraph_id#" cfsqltype="CF_SQL_INTEGER" null="#NOT IsNumeric(FORM.paragraph_id)#">
, <cfqueryparam value="#Session.userid#" cfsqltype="CF_SQL_INTEGER">
, 'Paragraph Updated'
, <cfqueryparam cfsqltype="CF_SQL_CLOB" value="#rs_d.paragraph_text#">
, <cfqueryparam cfsqltype="CF_SQL_CLOB" value="#FORM.input#">
)
</cfquery>
NB: Unless a column has a different default value assigned, NULL will be inserted automatically when that column is omitted from the INSERT list.
I suggest re-arranging your logic to determine field null variables which then get used in the null attribute of cfqueryparam. Something like this:
<cfscript>
fieldOneNull = conditionForNull ? true : false;
fieldTwoNull = conditionForNull ? true : false;
etc
</cfscript>
<cfquery>
insert into table
(field1, field2, etc)
values
(
<cfqueryparam cfsqltype="cf_sql_whatever" value="something" null="#fieldOneNull#">
, <cfqueryparam cfsqltype="cf_sql_whatever" value="something" null="#fieldTwoNull#">
, etc
)
Thanks for the suggestions. Instead of modifying the existing function which is used in different areas of the application, I added a new insert statement in the position where the function was being called to deal with the clobs. To deal with the null fields I used a case statement.
<cfquery datasource="#dsn#">
insert into log (
LOG_ENTRY_ID, program_id, paragraph_id, userid, action, paragraph_text_old
, paragraph_text_new, comment_id, current_program_status, new_program_status
)
values (
null ,#program_id#,(case when #paragraph_id# = '' then null else #paragraph_id# end)
, #Session.userid# , 'Paragraph Updated'
, <cfqueryparam cfsqltype="CF_SQL_CLOB" value=#rs_d.paragraph_text#>
, <cfqueryparam cfsqltype="CF_SQL_CLOB" value=#input#>
, null, null , null
)
</cfquery>

How to use SQL query to define table in dbtable?

In JDBC To Other Databases I found the following explanation of dbtable parameter:
The JDBC table that should be read. Note that anything that is valid in a FROM clause of a SQL query can be used. For example, instead of a full table you could also use a subquery in parentheses.
When I use the code:
CREATE TEMPORARY TABLE jdbcTable
USING org.apache.spark.sql.jdbc
OPTIONS (
url "jdbc:postgresql:dbserver",
dbtable "mytable"
)
everything works great, but the following:
dbtable "SELECT * FROM mytable"
leads to the error:
What is wrong?
Since dbtable is used as a source for the SELECT statement it has be in a form which would be valid for normal SQL query. If you want to use subquery you should pass a query in parentheses and provide an alias:
CREATE TEMPORARY TABLE jdbcTable
USING org.apache.spark.sql.jdbc
OPTIONS (
url "jdbc:postgresql:dbserver",
dbtable "(SELECT * FROM mytable) tmp"
);
It will be passed to the database as:
SELECT * FROM (SELECT * FROM mytable) tmp WHERE 1=0
Code In Scala
val checkQuery = "(SELECT * FROM " + inputTableName + " ORDER BY " + columnName + " DESC LIMIT 1) AS timetable"
val timeStampDf = spark.read.format("jdbc").option("url", url).option("dbtable", checkQuery).load()
Adding an alias is also necessary after the query in parenthesis.

simple join syntax not properly working

I'd like to show the number of records in the history table grouped by name of service
service(code,name)
history(id, code,....)
Please note that there is no relationship between the two table history and service, history stores the activity independently from the other tables
I have tested this sql query and it returns the expected result:
select s.name, count(*) from history c
join service s
on c.code=s.code
where c.state='INITIALE'
group by s.name
Actually, I'd like to write it in jpql, I did alike
Query query =entityManager.createQuery(" select s.name, count(*) from ServiceEntity s join"
+ " HistoryEntity c "
+ " where c.code=s.code and c.state='INITIALE'"
+ " group by c.name order by c.name"
);
I got this error : Path expected for join!....
Invalid path: 'c.code'....right-hand operand of a binary operator was null....unexpected end of subtree
Try this
Query query = entityManager.createQuery("select s.name, count(s) from ServiceEntity s, HistoryEntity c "
+ " where c.code = s.code and c.state = 'INITIALE'"
+ " group by s.name order by s.name"
);

Fetch dates (by month or year) stored in TIMESTAMP using JPA

I am facing a problem and I would like you to help me.
It turns out I have one table in my Oracle 11g database where I store failures of one electronic device. The table definition is following:
CREATE TABLE failure
( failure_id NUMERIC NOT NULL
, fecha TIMESTAMP NOT NULL
, module_id NUMERIC NOT NULL
, code NUMERIC
, PRIMARY KEY(failure_id)
);
Where 'fecha' means 'date'.
I need to fetch failures by YEAR or by MONTH for one specific module but I can't. My ORM maps the TIMESTAMP type to java.sql.Date but I don't know how to compare the month in the JPQL sentence. I have tried to use ORACLE functions with native queries but I front with another issue: to cast the results.
I am using JPA 2.0 with Eclipselink 2.3.2.
My doubts are:
Can I use Oracle functions with this version of Eclipselink library? My experience say no.
Query query = entityManager.createQuery("SELECT f FROM Failure f "
+ "WHERE EXTRACT(YEAR FROM f.fecha) = ?1 "
+ "AND f.moduleId.moduleId = ?2");
query.setParameter(1, year);
query.setParameter(2, idModule);
I get this error: Unexpected token [(]
Can I use Eclipselink functions? My experience say no.
Query query = entityManager.createQuery("SELECT f FROM Failure f "
+ "WHERE EXTRACT('YEAR', f.fecha) = ?1 "
+ "AND f.moduleId.moduleId = ?2");
query.setParameter(1, year);
query.setParameter(2, idModule);
Same error.
Do you know a simple way to fetch this data using only one query?
I know I can fetch one module and then check failures with loops but I think it is not the best performing solution.
Thanks.
My sources:
Eclipselink JPA functions link
Eclipselink Query Enhancements link
A native query is written in the SQL dialect of your DB so can use DB specific functionality see the createNativeQuery methods of the EntityManager.
However there is another solution, test the timestamp against a lower and upper value:
WHERE f.fecha >= '2012-9-1' AND f.fecha < '2012-10-1'
The syntax in EclipseLink 2.4 for EXTRACT is,
EXTRACT(YEAR, f.fecha)
http://wiki.eclipse.org/EclipseLink/UserGuide/JPA/Basic_JPA_Development/Querying/JPQL#Functions
I used Eclipselink v 2.4 functions and I am getting values using this:
Query query = entityManager.createQuery("SELECT f FROM Failure f "
+ "WHERE SQL('EXTRACT (YEAR FROM ?)', f.fecha) = ?1 "
+ "AND f.moduleId.moduleId = ?2 ");
Extracting year from date stored in database avoids to me one comparison between two dates.
Use
Query query = entityManager.createQuery("SELECT f FROM Failure f "
+ "WHERE EXTRACT(YEAR from f.fecha) = ?1 "
+ "AND f.moduleId.moduleId = ?2");
I was facing the same problem. I only checked for the correct syntax of that EXTRACT function for oracle and it worked for me! (Notice the FROM clause into the function syntax.
#NamedQuery(name = "Registros.findByFechacaptura", query = "SELECT s FROM Registros s WHERE EXTRACT(YEAR FROM s.fechacaptura) = EXTRACT(YEAR FROM :fechacaptura)")

jdbc result set metadata: getting physical column names on aliased columns

I'm using jdbc to execute query statements (in jruby)
# made-up example
sql = "select " +
"c.type as cartype, " +
"o.id as ownerid, " +
"o.type as ownertype " +
"from cars c " +
"inner join owners o " +
"on c.vin = o.vin"
# 'stmt' gotten with jdbc-connection.create_statement()
result_set = stmt.execute_query(sql)
meta_data = result_set.get_meta_data()
col_count = result_set.get_column_count()
I can query the various column aliases (get_column_name) and tables (get_table_name) for each column through the column indexes, but I also need the actual/physical names of the columns, un-aliased.
How do I get the physical/actual name of column, as it is defined in the schema ("ownerid" column alias is column "id", for instance)?
From tests with other database types, it looks as though this is database+driver specific. Using mysql get_column_name returns the actual/physical column name while get_column_label returns the alias. As an aside, both database types (mysql and sqlite) return the physical table name through get_table_name.

Resources