I'm trying to create a flow in Mule that allow queries in a table with a dynamic where-statement.
The idea is to send a map of parameters with zero or more entries and then build a query string from that.
I use hsqldb for testing and get some strange errors about unexpected token.
Here is the flow:
<script:transformer name="extractParameters">
<script:script file="extractParameters.groovy" engine="groovy"/>
</script:transformer>
<flow name="datafetch.flow">
<vm:inbound-endpoint address="vm://datafetch" exchange-pattern="request-response"/>
<enricher target="#[variable:where-statement]">
<transformer ref="extractParameters"/>
</enricher>
<jdbc:outbound-endpoint connector-ref="datasource"
queryKey="parameterized-select"
exchange-pattern="request-response"
mimeType="text/plain">
<jdbc:query key="parameterized-select"
value="select * from ${datasource.table} #[variable:where-statement]"/>
</jdbc:outbound-endpoint>
<json:object-to-json-transformer name="CaseInsensitiveHashMapToJson"/>
</flow>
The enricher is a groovy script that converts a json-structure to the where-statement string:
import groovy.json.JsonSlurper
def input = new JsonSlurper().parseText(payload)
def parameters = input?.get("parameters")
def result = ""
if(parameters==null) return result
def where = parameters.inject([]) { list, entry -> list << "${entry.key}=${entry.value}"}.join(" AND ")
if (where.isEmpty()) return result
result = "where " + where
return result
Sending empty parameters results in the enricher producing an empty string, and the errors are:
1. unexpected token: ? (org.hsqldb.HsqlException)
org.hsqldb.error.Error:-1 (null)
2. unexpected token: ?(SQL Code: -5581, SQL State: + 42581) (java.sql.SQLSyntaxErrorException)
org.hsqldb.jdbc.Util:-1 (null)
3. unexpected token: ? Query: select * from TABLE1 ? Parameters: [](SQL Code: -5581, SQL State: + 42581) (java.sql.SQLException)
org.apache.commons.dbutils.QueryRunner:540 (null)
Sending one parameter (C1=0) results in the enricher producing the string "where C1=0", and the errors are:
1. unexpected token: ? (org.hsqldb.HsqlException)
org.hsqldb.error.Error:-1 (null)
2. unexpected token: ?(SQL Code: -5581, SQL State: + 42581) (java.sql.SQLSyntaxErrorException)
org.hsqldb.jdbc.Util:-1 (null)
3. unexpected token: ? Query: select * from TABLE1 ? Parameters: [where C1=0](SQL Code: -5581, SQL State: + 42581) (java.sql.SQLException)
org.apache.commons.dbutils.QueryRunner:540 (null)
So, it looks like the query expects something else than a string where I have written #[variable:where-statement]
So, what do I need to feed into the where-statement variable to make this work?
Or, is there some other way to specify this?
I tryed this way and it works:
<scripting:transformer doc:name="Groovy">
<scripting:script engine="Groovy"><![CDATA[def q ="select * from myTable where 1=1 ";
if(payload.containsKey('param1')) q = q + " And mycolumn = " + payload.param1;
payload.put('dynamicWhere', q);
return payload;]]></scripting:script>
</scripting:transformer>
<outbound-endpoint exchange-pattern="request-response" connector-ref="ConnectorDb" doc:name="Database" address="jdbc://#[payload.dynamicWhere]" />
...
Related
I am trying to add a parameter (Period) to change part of the SQL statement but I am getting the "Token Comma Expected" error:
let
Period = Excel.CurrentWorkbook(){[Name="Period"]}[Content]{0}[Period],
Source = Sql.Database("FVC-HO-SQL-02", "FVCHLD", [Query="SELECT * FROM OPENQUERY ([NAMIBIA], 'SELECT * FROM PAS19NFPM23.LedgerTransactions where GDC = ''D'' and PPeriod = "Period"')#(lf)union all#(lf)SELECT * FROM OPENQUERY ([NAMIBIA], 'SELECT * FROM PAS19FREWB23.LedgerTransactions where GDC = ''D'' and PPeriod = Period')"])
Details above, unsure how to proceed.
Wherever it used to say period=xxx use period="&Period&"
I am facing an issue with some SQL scripts where I use parameter passing.
I call a script like this:
DEFINE sowner = "SOWNER"
DEFINE auser = "AUSER"
DEFINE tbsp = "TAB"
DEFINE insp = "IND"
#R1/do_your_job.sql $owner $auser $tbsp $insp
And inside R1/do_your_job :
define sowner = $1
define auser = $2
define tbsp = $3
define insp = $4
alter session set current_schema=$sowner;
And I get: Ora 00911: invalid character marking my $ sign
Why is it not decoding $sowner to "SOWNER"?
My file is marked with "set define $" anyway...
Thank you.
Use '&variable' while passing and '&1', '&2' etc while reading.
DEFINE sowner = "SOWNER"
DEFINE auser = "AUSER"
DEFINE tbsp = "TAB"
DEFINE insp = "IND"
#R1/do_your_job.sql &sowner &auser &tbsp &insp
Your do_your_job.sql may use it as
select '&1','&2','&3','&4' from dual;
Result
#R1/do_your_job.sql &sowner &auser &tbsp &inspSQL> SQL> SQL> SQL> SQL>
old 1: select '&1','&2','&3','&4' from dual
new 1: select 'SOWNER','AUSER','TAB','IND' from dual
'SOWNE 'AUSE 'TA 'IN
------ ----- --- ---
SOWNER AUSER TAB IND
I'm trying to create a jar to run on snappy-job shell with streaming.
I have aggregation function and it works in windows perfectly. But I need to have a table with one value for each key. Base on a example from github a create a jar file and now I have problem with put into sql command.
My code for aggregation:
val resultStream: SchemaDStream = snsc.registerCQ("select publisher, cast(sum(bid)as int) as bidCount from " +
"AggrStream window (duration 1 seconds, slide 1 seconds) group by publisher")
val conf = new ConnectionConfBuilder(snsc.snappySession).build()
resultStream.foreachDataFrame(df => {
df.write.insertInto("windowsAgg")
println("Data received in streaming window")
df.show()
println("Updating table updateTable")
val conn = ConnectionUtil.getConnection(conf)
val result = df.collect()
val stmt = conn.prepareStatement("put into updateTable (publisher, bidCount) values " +
"(?,?+(nvl((select bidCount from updateTable where publisher = ?),0)))")
result.foreach(row => {
println("row" + row)
val publisher = row.getString(0)
println("publisher " + publisher)
val bidCount = row.getInt(1)
println("bidcount : " + bidCount)
stmt.setString(1, publisher)
stmt.setInt(2, bidCount)
stmt.setString(3, publisher)
println("Prepared Statement after bind variables set: " + stmt.toString())
stmt.addBatch()
}
)
stmt.executeBatch()
conn.close()
})
snsc.start()
snsc.awaitTermination()
}
I have to update or insert to table updateTable, but during update command the current value have to added to the one from stream.
And now :
What I see when I execute the code:
select * from updateTable;
PUBLISHER |BIDCOUNT
--------------------------------------------
publisher333 |10
Then I sent message to kafka:
1488487984048,publisher333,adv1,web1,geo1,11,c1
and again select from updateTable:
select * from updateTable;
PUBLISHER |BIDCOUNT
--------------------------------------------
publisher333 |11
the Bidcount value is overwritten instead of added.
But when I execute the put into command from snappy-sql shell it works perfectly:
put into updateTable (publisher, bidcount) values ('publisher333',4+
(nvl((select bidCount from updateTable where publisher =
'publisher333'),0)));
1 row inserted/updated/deleted
snappy> select * from updateTable;
PUBLISHER |BIDCOUNT
--------------------------------------------
publisher333 |15
Could you help me with this case? Mayby someone has other solution for insert or update value using snappydata ?
Thank you in advanced.
bidCount value is read from tomi_update table in case of streaming but it's getting read from updateTable in case of snappy-sql. Is this intentional? May be you wanted to use updateTable in both the cases ?
Any calls using jdbcTemplate.queryForList returns a list of Maps which have NULL values for all columns. The columns should've had string values.
I do get the correct number of rows when compared to the result I get when I run the same query in a native SQL client.
I am using the JDBC ODBC bridge and the database is MS SQL server 2008.
I have the following code in my DAO:
public List internalCodeDescriptions(String listID) {
List rows = jdbcTemplate.queryForList("select CODE, DESCRIPTION from CODE_DESCRIPTIONS where LIST_ID=? order by sort_order asc", new Object[] {listID});
//debugcode start
try {
Connection conn1 = jdbcTemplate.getDataSource().getConnection();
Statement stat = conn1.createStatement();
boolean sok = stat.execute("select code, description from code_descriptions where list_id='TRIGGER' order by sort_order asc");
if(sok) {
ResultSet rs = stat.getResultSet();
ResultSetMetaData rsmd = rs.getMetaData();
String columnname1=rsmd.getColumnName(1);
String columnname2=rsmd.getColumnName(2);
int type1 = rsmd.getColumnType(1);
int type2 = rsmd.getColumnType(2);
String tn1 = rsmd.getColumnTypeName(1);
String tn2 = rsmd.getColumnTypeName(2);
log.debug("Testquery gave resultset with:");
log.debug("Column 1 -name:" + columnname1 + " -typeID:"+type1 + " -typeName:"+tn1);
log.debug("Column 2 -name:" + columnname2 + " -typeID:"+type2 + " -typeName:"+tn2);
int i=1;
while(rs.next()) {
String cd=rs.getString(1);
String desc=rs.getString(2);
log.debug("Row #"+i+": CODE='"+cd+"' DESCRIPTION='"+desc+"'");
i++;
}
} else {
log.debug("Query execution returned false");
}
} catch(SQLException se) {
log.debug("Something went haywire in the debug code:" + se.toString());
}
log.debug("Original jdbcTemplate list result gave:");
Iterator<Map<String, Object>> it1= rows.iterator();
while(it1.hasNext()) {
Map mm = (Map)it1.next();
log.debug("Map:"+mm);
String code=(String)mm.get("CODE");
String desc=(String)mm.get("description");
log.debug("CODE:"+code+" : "+desc);
}
//debugcode end
return rows;
}
As you can see I've added some debugging code to list the results from the queryForList and I also obtain the connection from the jdbcTemplate object and uses that to sent the same query using the basic jdbc methods (listID='TRIGGER').
What is puzzling me is that the log outputs something like this:
Testquery gave resultset with:
Column 1 -name:code -typeID:-9 -typeName:nvarchar
Column 2 -name:decription -typeID:-9 -typeName:nvarchar
Row #1: CODE='C1' DESCRIPTION='BlodoverxF8rin eller bruk av blodprodukter'
Row #2: CODE='C2' DESCRIPTION='Kodetilfelle, hjertestans/respirasjonstans'
Row #3: CODE='C3' DESCRIPTION='Akutt dialyse'
...
Row #58: CODE='S14' DESCRIPTION='Forekomst av hvilken som helst komplikasjon'
...
Original jdbcTemplate list result gave:
Map:(CODE=null, DESCRIPTION=null)
CODE:null : null
Map:(CODE=null, DESCRIPTION=null)
CODE:null : null
...
58 repetitions total.
Why does the result from the queryForList method return NULL in all columns for every row? How can I get the result I want using jdbcTemplate.queryForList?
The xF8 should be the letter ø so I have some encoding issues, but I can't see how that may cause all values - also strings not containing any strange letters (se row#2) - to turn into NULL values in the list of maps returned from the jdbcTemplate.queryForList method.
The same code ran fine on another server against a MySQL Server 5.5 database using the jdbc driver for MySQL.
The issue was resolved by using the MS SQL Server jdbc driver rather than using the JDBC ODBC bridge. I don't know why it didn't work with the bridge though.
I have a question about the SAP Function Module "http_post".
I just want to post a short message (msg) out of the SAP to a Push Notification Server (pushd-Github-Projekt) I installed before. Now I'm not sure how to pass the message.
I tested the FM with the test-symbol:
CALL FUNCTION 'HTTP_POST'
exporting
ABSOLUTE_URI = uri " Uniform Resource Identifier (RFC 1945)
* REQUEST_ENTITY_BODY_LENGTH = 14 "request_entity_body_length
* RFC_DESTINATION = " RFC Destination
* PROXY = " HTTP Proxy Rechner
* PROXY_USER = " Benutzername auf dem Proxy Rechner
* PROXY_PASSWORD = " Passwort auf dem Proxy Rechner
* USER = " Benutzername auf dem HTTP Server
* PASSWORD = " Passwort auf dem HTTP Server
* BLANKSTOCRLF = " Blanks in CRLF konvertieren im Entity Body
* importing
* STATUS_CODE = " Statuscode ( 2xx = OK )
* STATUS_TEXT = " Text zum Statuscode
* RESPONSE_ENTITY_BODY_LENGTH = " Länge vom Response-Entity-Body
tables
REQUEST_ENTITY_BODY = '{"msg":"test"}' "request_entity_body
RESPONSE_ENTITY_BODY = '' " Response-Entity-Body Daten
RESPONSE_HEADERS = '' " Header Zeilen vom Response
REQUEST_HEADERS = 'Content-Type: application/json' "request_headers
* exceptions
* CONNECT_FAILED = 1
* TIMEOUT = 2
* INTERNAL_ERROR = 3
* TCPIP_ERROR = 4
* SYSTEM_FAILURE = 5
* COMMUNICATION_FAILURE = 6
* OTHERS = 7
.
I know my values are no tables, but I tested it with the test-symbol, where you can write the values directly in a table.
When I start the FM, I get an Bad Request error in the SAP
and this error at the push notification server:
SyntaxError: Unexpected token
at Object.parse (native)
at IncomingMessage.<anonymous> ...Path from the pushd..
express\node_modules\connect\lib\middleware\json.js:76:27
at incomingMessage.EventEmitter.emit events.js:92:17
at _stream:readable.js:919:16
at process._tickCallback <node.js:419:13>
Can anyone help me how to pass the request to the FM HTTP-Post? It has to be sth. with msg, because otherwise the Push-Notification-Server can't handle it.
In SAP_BASIS release 731 or higher, I would strongly recommend to use the class CL_HTTP_CLIENTfor doing HTTP requests. See here an example report on how to do it. Replace the dummy string http:1.2.3.4:80/testjon/by your URL in question.
report z_test_http_post.
start-of-selection.
perform start.
* ---
form start.
data: lv_status type i,
lv_error_occurred type flag,
lv_error_msg type string,
lv_response_body type string.
perform send_json using
'http://1.2.3.4:80/testjson/' " Use your URL here
'{"hello":"world"}' " Use your JSON here
changing lv_status lv_response_body
lv_error_occurred
lv_error_msg.
* Show result
format color col_heading.
write: / 'Response status:', lv_status.
if lv_error_occurred = 'X'.
format color col_negative.
write: / 'Error occurred:', lv_error_msg.
endif.
format color col_normal.
write: / 'Response:', lv_response_body.
endform. "start
form send_json using iv_url type string
iv_json_data type string
changing cv_status type i
cv_response_body type string
cv_error_occurred type flag
cv_error_msg type string.
data: lo_client type ref to if_http_client.
clear: cv_error_msg,
cv_status,
cv_error_occurred,
cv_error_msg.
if iv_url is initial.
* No URL passed
message e349(sbds) into cv_error_msg.
cv_error_occurred = 'X'.
return.
endif.
call method cl_http_client=>create_by_url
exporting
url = iv_url
importing
client = lo_client
exceptions
argument_not_found = 1
plugin_not_active = 2
internal_error = 3
others = 4.
if sy-subrc ne 0.
message id sy-msgid type sy-msgty number sy-msgno
with sy-msgv1 sy-msgv2 sy-msgv3 sy-msgv4
into cv_error_msg.
cv_error_occurred = 'X'.
return.
endif.
lo_client->request->set_cdata( iv_json_data ).
lo_client->request->set_content_type( 'application/json' ).
lo_client->request->set_method( 'POST' ).
call method lo_client->send
exceptions
http_communication_failure = 1
http_invalid_state = 2
http_processing_failed = 3
others = 4.
if sy-subrc ne 0.
lo_client->get_last_error( importing message = cv_error_msg ).
cv_error_occurred = 'X'.
return.
endif.
lo_client->receive( exceptions others = 1 ).
if sy-subrc ne 0.
lo_client->get_last_error( importing message = cv_error_msg ).
cv_error_occurred = 'X'.
return.
endif.
cv_response_body = lo_client->response->get_cdata( ).
lo_client->response->get_status( importing code = cv_status ).
endform.