We have a int-jms:message-driven-channel-adapter --> a transformer --> a filter --> another transformer --> int-jdbc:outbound-channel-adapter (to insert in table_1)
(considering --> as channels)
I want to change this flow to insert into 2 tables instead of 1 but for table_2 I want to insert only if data corresponding to some fields in the message is already not present in the table i.e. insert if not present.
One thing I figured is that I will now need a pub-sub-channel with ignore-failures=false to split the flow into 2 tables.
My question is that what component should I use to select the data to check if data exists in table_2? I first thought inbound-channel-adapter is the right choice but couldn't figure out how to slide it between 2 components i.e. say a transformer and outbound-channel-adapter.
Some things I can think of are:
1. Use a filter, passing it jdbcTemplate so that the filter itself can fire a JDBC query to accept if record doesn't exist.
2. Use a outbound-channel-adapter and the insert query should have the check for data existence also, something like insert-if. I am not sure if Oracle has something like this. I am researching.
Please point me to an example or documentation or tell me the best way.
Thanks
Actually you can use
<chain>
<header-enricher>
<header name="original" expression="payload"/>
</header-enricher>
<int-jdbc:outbound-gateway query="SELECT count(*) from TABLE where ..."/>
<filter expression="payload == 0"/>
<transformer expression="headers.original"/>
</chain>
From other side <filter> with JdbcTemplate direct usage is good choice too.
Re. insert-if. It can work too if you have a unique constraint on the table. In this case an Exception will be thrown. And if you have <publish-subscribe-channel ignore-failures=false>, it would work too.
Related
I have a database table on which a sorted query needs to be done.
To do the sorting a join on another table is requiered. The problem is that this other table does not exist in the database because we read the required data on the services startup from a CSV file and keep it as an in-memory list.
Is it possible to somehow inject this list as a kind of virtual database into Spring Data? So that it could use this list to make the required join and sorting.
As far as I know, the only other options I have would be to create a real database table from this in-memory list or load the whole table and do the sorting in the service itself.
You can add a special order by expression through e.g. Spring Data Specification, but that is going to be very ugly. In HQL it looks like this:
case rootAlias.attribute when 'value1' then 1 when 'value2' then 2 ... else null end
which will return some integer value by which you can sort ascending or descending, based on the mapping you have.
Even if you have lots of values, I would rather recommend you don't do a join at all, and instead try to make this attribute of your main table sortable, so that you don't need this mapping. You can maybe create a trigger that maintains a column based on the mapping, which can be used for sorting directly. If you do all your changes through JPA/Hibernate, you could also use a #PreUpdate/#PrePersist listener to handle the maintenance of this column.
My Springboot application has orm.xml file in my WEB-INF folder.
I have few named native queries in the orm.xml file.
I want to add a where condition in the query based on the values in the parameter to the query.
I am looking for some thing like this:
<named-native-query name="MyReport.countInventory">
<query>
SELECT COUNT(*) AS COUNT
FROM
Inventory
WHERE
<if :date>
add_date = :date
</if>
</query>
</named-native-query>
This is not possible with JPA.
It just allows straight-forward SQL statements without any conditional processing.
In many cases, one can actually implement the required conditional logic in SQL itself as Turo described in his/her comment.
Since you are seeming to be using Spring Data JPA you can also use Specifications to dynamically create conditions.
Another alternative would be to use MyBatis which allows constructing SQL statements in a way similar to what you outline in your question.
The following example is taken from http://www.mybatis.org/mybatis-3/dynamic-sql.html
<select id="findActiveBlogWithTitleLike"
resultType="Blog">
SELECT * FROM BLOG
WHERE state = ‘ACTIVE’
<if test="title != null">
AND title like #{title}
</if>
</select>
I have an XML file containing Client id and addresses which I need to load into relational tables in Oracle database.
<Clients>
<Client id="100">
<name>ABC Corportation</name>
<Address>
<Addr1>1 Pine Street</Addr1>
<City>Chennai</City>
<State>Tamil Nadu</State>
<Country>India</Country>
<Postalcode>6000000</Postalcode>
</Address>
<Address>
<Addr1>1 Apple Street</Addr1>
<City>Coimbatore</City>
<State>Tamil Nadu</State>
<Country>India</Country>
<Postalcode>6000101</Postalcode>
</Address>
</Client>
<Client id="101">
....
....
</Client>
I have 2 relational tables defined as below-
Client
CLIENT_ID (Unique Key)
CLIENT_NAME
Client_Location
CLIENT_ID
ADDR1
CITY
STATE
COUNTRY
POSTAL_CODE
Updates to client address at source will sent in the XML file everyday. The ETL designed in a way that it requires a unique key on the table based on which it will identify the change coming in the XML as INSERT or UPDATE and accordingly sync the table to the XML. Identifying DELETEs is not really necessary.
Question: What should be defined as the unique key for Client_Location to process incremental changes coming everyday in the XML file? There is no identifier for address in the XML file. I was thinking about creating an additional hashing column (using ORA_HASH function) based on the 3 columns (STATE, COUNTRY, POSTAL_CODE). The unique key for the table would (CLIENT_ID, <>) which the ETL will use.. The idea is that it is not common for STATE/COUNTRY/POSTAL_CODE to change in an address. Ofcourse, this is a big assumption which I'm making. I would like to implement the below-
1) If there is any small change to ADDR1, I want the ETL to pick it up as a "valid" update at source and sync it to the table.
2) If there is a small change in the STATE/COUNTRY/POSTAL_CODE (eg: typo correction or case change like India to INDIA), then I don't want this to picked as a change because it would lead to INSERT (hashing value would change which is part of the unique key) and in turn duplicate rows in the table.
Does the idea of using a hashing column to define uniqueness make sense? Is there a better way to deal with this?
Is there a way to tweak ORA_HASH to produce results expected in #2 above?
If the client can have only one location reuse CLIENT_ID as primary key.
If there are more locations posible add SEQUENCEkey (sequence number 1..N) to the CLIENT_ID as a PK.
The simplest possibility to distinct and identify the locations is to use the feature of XML that the order of elements is well defined and has meaning. So the first ADDRESS element (pine street) becomes sequence 1, the second 2 and so on.
Please check the FOR ORDINALITY clause on XML table how to get this identification while parsing the XML.
You may also add TIMESTAMP (as a simple attribute - not a key) to keep the timestamp of the change and a STATUS column to identify deleted locations.
HASH may be usefull to quickly test a change if you have tons of columns, but for 5 columns it is probably an overkill (as you may simple compare the column values). I'd not recommend to used HASH as part of a key as it has no advantage to the proposed solution.
I´m using Spring Integration JDBC support to persist a message in one of several tables (>20) depending on a certain condition (stored in the message headers as "table"):
<int:channel id="cmTablesJdbcChannel"></int:channel>
<int-jdbc:outbound-channel-adapter channel="cmTablesJdbcChannel"
id="cmTableJdbcOutputAdaptor" data-source="datasource"
query="insert into TABLE_NAME values (int_id, parent_int_id, name) values (:headers[int_id],:headers[parent_int_id],:headers[name])">
</int-jdbc:outbound-channel-adapter>
I have tried to replace TABLE_NAME by several expressions but none worked:
${headers['table']}
#{headers['table']}
:[headers['table]}
I´m trying to avoid the usage of 20 different outbound channel adaptors and reuse a single one but dynamically setting the name of the table to be used. Do you know if it is possible?
There were similar questions but related to the parameters to be used: How can I create a dynamic query for Spring Integration JDBC outbound-channel-adapter?
No, it doesn't work now and TABLE_NAME can't be as a parameter.
Feel free to raise JIRA issue to consider something like query-expression to build the INSERT/UPDATE SQL at runtime against request message.
In meantime you should use NamedParameterJdbcTemplate from the some custom POJO to be used from the <outbound-channel-adapter> or like a complex expression:
<service-activator input-channel="cmTablesJdbcChannel" output-channel="nullChannel"
expression="#jdbcTemplate.update('insert into ' + headers.table + ' (int_id, parent_int_id, name) values (:int_id,:parent_int_id,:name)', headers)"/>
Note, don't use direct SQL expression building with the parameter values. The parametrized variant with : is preferable way for any RDBMS. It will be compiled on the server side (indexes, query plan etc.) and reused for all other upcoming executions.
I'm am getting irritated with iReports. Problem is that I have a data set returning data for multiple customers and I want to use the "Group Expression" against the customer ID and have the report lay out the Detail Tabs per customer.
I'm finding that, seeming randomly, where there is more that one data row for a customer iReports will generate two or more groupings (Sometimes it does what I expect and group all the customer data together), the field IDing the customer is the same and doesn't change.
Has anyone seen this before? To be honest I can't believe it is actually a bug, but something I've missed. Just much searching as yet to find a suitable result.
I think this is a data sorting problem.
The quote from iReport Ultimate Guide:
JasperReports groups records by evaluating the group expression. Every
time the expression's value changes, a new group instance is created.
The engine does not perform any record sorting (if not explicitly
requested), so when we define groups we should always take care of the
records sorting. That is, if we want to group a set of addresses by
country, the records we select for the report should already by
ordered by country. It is simple to sort data when using an SQL query
by using the ORDER BY clause. When this is not possible (that is, when
obtaining the records from an XML document), we can request that
JasperReports sort the data for us. This can be done using the sort
options available in the query window
You can sort data in these ways:
in case using of Database jdbc connection datasource type you can add ORDER BY customerId clause to the report's query, where customerId - column name of field with customer id
in case using of File csv connection or something like this you can organize data sorting by adding sortField property for field to the report's template (jrxml file):
<jasperReport ...>
...
<field name="customerId" class="java.lang.String"/>
<sortField name="customerId"/>
SQL Statement has ORDER BY?
iReport group is grouped by customer_id?