How to get the type of values from a JDBC query? - spring

I have a spring batch job that takes in a user query, executes that query to find the selected items, and then I want to insert those items into another database. The problem is that I have to convert elements like dates from the resulting query to insert them again. How can I tell the type of the values returned from the query?
This is what I use to read the items which works properly.
#Bean("querySelectiveItems")
#StepScope
public JdbcCursorItemReader querySelectiveItems(#Qualifier("selectiveSourceDatabase") DataSource dataSource,
#Value("#{jobExecutionContext[" + EtlConfiguration.JOB_PARM_MIGRATION_CONFIG + "]}") MigrationDefinition migrationDefinition
) {
JdbcCursorItemReader reader = new JdbcCursorItemReader<>();
reader.setSql(migrationDefinition.getMigrations().getTable().getQuery());
reader.setDataSource(dataSource);
reader.setRowMapper(new ColumnMapRowMapper());
log.info("Queried for items");
return reader;
}
The following is what I wrote to write to the destination database. The problem is that the values I have to insert are unknown because they are the result of a user query. For example if there is a datatype in my insert statement I must have a TO_DATE around the date value. Is there a way to do this?
#Component
#Lazy
class InsertSelectedItems implements ItemWriter<Map<String, Object>> {
private MigrationDefinition migrationDefinition;
private JdbcTemplate destinationTemplate;
public void setDestinationTemplate(JdbcTemplate destinationTemplate) {
this.destinationTemplate = destinationTemplate;
}
public void setMigrationDefinition(MigrationDefinition migrationDefinition) {
this.migrationDefinition = migrationDefinition;
}
#Override
public void write(List<? extends Map<String, Object>> items) throws Exception {
ArrayList<String> columns = new ArrayList<>();
ArrayList<String> values = new ArrayList<>();
System.out.println(items);
for (Map<String, Object> map : items) {
for (Map.Entry<String, Object> entry : map.entrySet()) {
columns.add(entry.getKey());
values.add(String.valueOf(entry.getValue()));
}
}
String sql = String.format("%s ( %s ) VALUES ( %s ) ",
migrationDefinition.getMigrations().getTable().getInsert(),
String.join(",", columns),
String.join(",", values));
log.info(sql);
destinationTemplate.update(sql);
}
}

Related

Spring Batch Dynamic Insert and Update with DB to DB

There is scenario where we want to load data from DB to DB .
But we want to check if data is already present in target system then update it otherwise insert it into DB.
We are using below approach:
#Bean
ItemWriter<Student> onosItemWriter1() {
JdbcBatchItemWriter<Student> databaseItemWriter = new JdbcBatchItemWriter<>();
databaseItemWriter.setDataSource(dataSource);
databaseItemWriter.setJdbcTemplate(namedParameterJdbcTemplate);
databaseItemWriter.setSql(INSERT_QUERY);
ItemPreparedStatementSetter<Student> valueSetter = new ItemPreparedStatementSetter<Student>() {
#Override
public void setValues(Student student, PreparedStatement statement) throws SQLException {
if (Student.getId() < 0) {
log.info("Inserting!");
databaseItemWriter.setSql(INSERT_QUERY);
statement.setString(1, student.getName());
statement.setString(2, student.getEmail());
} else {
log.info("updateing!!!!");
databaseItemWriter.setSql(UPDATE_QUERY);
statement.setString(1, student.getName());
statement.setString(2, student.getEmail());
}
}
};
databaseItemWriter.setItemPreparedStatementSetter(valueSetter);
return databaseItemWriter;
}
above its working only for insert. How I can do it with one JDBC batch Item writer to update it as well dynamically. if record is already present in chunks.
The code you shared won't work, because you are configuring the query on the item writer based on a runtime information from the item itself (Student.getId() < 0).
we want to check if data is already present in target system then update it otherwise insert it into DB
I would keep it simple and write a custom item writer like follows:
public class StudentItemWriter implements ItemWriter<Student> {
private static final String INSERT_QUERY = "insert into student ...";
private static final String UPDATE_QUERY = "update student set ... where id = ?";
private JdbcTemplate jdbcTemplate;
public StudentItemWriter(DataSource dataSource) {
this.jdbcTemplate = new JdbcTemplate(dataSource);
}
public void write(List<? extends Student> students) throws Exception {
for(Student student : students) {
int updated = jdbcTemplate.update(UPDATE_QUERY,...);
if(updated == 0) {
jdbcTemplate.update(INSERT_QUERY,...);
}
}
}
}

Spring Boot rest controller: how to return clean json

I've set this method to return a response from a Spring Boot rest controller:
public ResponseEntity<Map<String, Object>> get(#PathVariable("id") long id) {
try {
return new ResponseEntity<>(this.ReportDAO.read("dbuser1"), HttpStatus.OK);
} catch (Exception e) {
return new ResponseEntity<>(null, HttpStatus.INTERNAL_SERVER_ERROR);
}
}
And this is the DAO method:
#Autowired
private JdbcTemplate jdbcTemplate;
public Map<String, Object> read(String testParam) {
List<SqlParameter> parameters = Arrays.asList(new SqlParameter(Types.NVARCHAR));
CallableStatementCreator csc = new CallableStatementCreator() {
#Override
public CallableStatement createCallableStatement(Connection con) throws SQLException {
CallableStatement cs = con.prepareCall("{call test (?)}");
cs.setString(1, testParam);
return cs;
}
};
return jdbcTemplate.call(csc, parameters);
}
I'm successfully having a json object as response but in this format:
#result-set-1: [ {…}, {…} ]
while I'm expecting to have:
[ {…}, {…} ]
Why is the resultset inserted into #result-set-1 key? How can I modify this behaviour?
JdbcTemplate#call returns Map<String, Object> You can alter this behaviour by specifically extracting key from map using key #result-set-1.
This is how i have done it:
sql
CREATE TABLE `sample_log` (
`id` bigint NOT NULL AUTO_INCREMENT,
`message` varchar(255) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci
Insert statements:
insert into sample_log (message) values('West Country');
insert into sample_log (message) values('Welcome User');
Stored procedure:
CREATE PROCEDURE `fetch_sample_logs`(
in message_query varchar(30)
)
BEGIN
SELECT * FROM new_db.sample_log where message like message_query;
END
Controller
#RequestMapping("/logs")
#RestController
class SampleLogController {
private final JdbcTemplate jdbcTemplate;
#Autowired
SampleLogController(JdbcTemplate jdbcTemplate) {
this.jdbcTemplate = jdbcTemplate;
}
#GetMapping("/call")
public Object get() {
final Map<String, Object> call = jdbcTemplate.call(connection -> {
CallableStatement cs = connection.prepareCall("{call fetch_sample_logs (?)}");
cs.setString(1, "%wel%");
return cs;
}, Collections.singletonList(new SqlParameter(Types.VARCHAR)));
return Optional.of(call.getOrDefault("#result-set-1", Collections.emptyList()));
}
}
I am suggesting that you extract all the result-sets and concat them together. You could do as the other answer suggests and just get "#result-set-1" from the Map, but I would suggest at the very least converting the ResultSet to an application-represented object ("Thing" pojo) before returning from the dao method. I think that concatening the result-sets together is probably a more durable solution, unless someone can think of a reason as to why not.
#Autowired
private JdbcTemplate jdbcTemplate;
public List<Thing> read(String testParam) {
List<SqlParameter> parameters = Arrays.asList(new SqlParameter(Types.NVARCHAR));
CallableStatementCreator csc = new CallableStatementCreator() {
#Override
public CallableStatement createCallableStatement(Connection con) throws SQLException {
CallableStatement cs = con.prepareCall("{call test (?)}");
cs.setString(1, testParam);
return cs;
}
};
Map<String, Object> result = jdbcTemplate.call(csc, parameters);
return result.values().stream().map(o -> fromResultSet((ResultSet) o)
.flatMap(List::stream).collect(toList());
}
private List<Thing> fromResultSet(ResultSet resultSet) {
List<Thing> list = new ArrayList<>();
while (resultSet.next()) {
Thing thing = new Thing(resultSet.getString("resultCol1"), resultSet.getString("resultCol2")
list.add(user);
}
}
I modified some code from Resultset To List to actually parse the result set.

Convert to simplejdbccall resultset to java objects

I am calling DB2 procedure which takes a input parameter and returns a resultset.
How can i map the O/P to my pojo class.
I have to map the result to nexted pojo classes.
simpleJdbcCall = new SimpleJdbcCall(jdbctemplate)
.withSchemaName("myschema")
.withProcedureName("DB2-PROC")
.declareParameters(
new SqlParameter("1", Types.VARCHAR)
);
Map<String, Object> map = simpleJdbcCall.execute("2020-01-01");
for (Map.Entry<String, Object> entry : map.entrySet()) {
System.out.println("Entry value is " + entry.getValue() );
}
//my o/p
Entry value is [{Col_1=abc, col_2=abc,col_2=xyz, col_2=abc},....];
you can use returningResultSet(parameterName, rowMapper) method to map values to object.
Here some reference code:
SimpleJdbcCall procedureActor = new SimpleJdbcCall(dataSource)
.withSchemaName("myschema")
.withProcedureName("DB2-PROC")
.declareParameters(
new SqlParameter("1", Types.VARCHAR))
.returningResultSet("mapObjRefrence", new RowMapper<Contact>() {
#Override
public Contact mapRow(ResultSet rs, int rowNum) throws SQLException {
YourPojo pojo = new YourPojo();
pojo.setId(rs.getInt("col_1"));
pojo.setName(rs.getString("col_2"));
pojo.setEmail(rs.getString("col_2"));
pojo.setAddress(rs.getString("col_3"));
pojo.setTelephone(rs.getString("col_4"));
return contact;
}
});
Map<String, Object> out = procedureActor.execute("2020-01-01");
List<YourPojo> listPojos = (List<YourPojo>) out.get("mapObjRefrence");
Also you can check for multi table results: How to get multi table results using SimpleJDBCCall in spring?

Conditional IN from DynamoDb not working on java

I am working with DynamoDB with Spring Boot 2.1, and I'm facing an error when I need o user the clause IN during the conditional evaluation. Even with lines that fulfill the requirements, the query result is empty.
How can I return the lines from the table after explicit the result within the IN clause ?
public class DynamoRepository {
private final DynamoDBMapper dynamoDBMapper;
public Optional<List<USER>> query(String id) {
Map<String, String> ean = new HashMap<>();
ean.put("#status", "status");
Map<String, AttributeValue> eav = new HashMap<>();
eav.put(":id", new AttributeValue().withS(documento));
DynamoDBQueryExpression<USER> queryExpression = new DynamoDBQueryExpression<USER>()
.withKeyConditionExpression("id = :id")
.withFilterExpression("#status in (ACTIVE, PENDING)")
.withExpressionAttributeNames(ean)
.withExpressionAttributeValues(eav);
List<USER> query = dynamoDBMapper.query(USER.class, queryExpression);
return query.isEmpty() ? Optional.empty() : Optional.of(query);
}
}
After taking a while, my solution was to define the status' values as Expression Attribute Values like the code below
public class DynamoRepository {
private final DynamoDBMapper dynamoDBMapper;
public Optional<List<USER>> query(String id) {
Map<String, String> ean = new HashMap<>();
ean.put("#status", "status");
Map<String, AttributeValue> eav = new HashMap<>();
eav.put(":id", new AttributeValue().withS(documento));
eav.put(":active", new AttributeValue().withS("ACTIVE"));
eav.put(":pending", new AttributeValue().withS("PENDING"));
DynamoDBQueryExpression<USER> queryExpression = new DynamoDBQueryExpression<USER>()
.withKeyConditionExpression("id = :id")
.withFilterExpression("#status in (:active, :pending)")
.withExpressionAttributeNames(ean)
.withExpressionAttributeValues(eav);
List<USER> query = dynamoDBMapper.query(USER.class, queryExpression);
return query.isEmpty() ? Optional.empty() : Optional.of(query);
}
}

RowMapper returns the list , but execute returned values returns the list size as 1?

please find below my sample code.The Row mapper returns a list. When printed it give me the size in the DB but when i check
(List) employeeDaomap .get("allEmployees") i get the list size as 1 , and entire rows as one item? why what is the wrong in implementation
Also Spring doc says not to use rs.next(), how do we get the list of
values from the DB
public class MyTestDAO extends StoredProcedure {
/** The log. */
static Logger log = Logger.getLogger(MyTestDAO.class);
private static final String SPROC_NAME = "TestSchema.PKG_Test.prc_get_employee_list";
TestRowMapper mapper=new TestRowMapper();
public MyTestDAO(DataSource dataSource){
super(dataSource, SPROC_NAME);
declareParameter(new SqlOutParameter("allEmployees", OracleTypes.CURSOR, mapper));
compile();
}
/**
* Gets the myemplist data from the DB
*
*/
public List<EmployeeDAO> getEmployeeList()
throws Exception {
Map<String,Object> employeeDaomap =new HashMap<String,Object>();
employeeDaomap =execute();
log.info("employeeDaomap after execute ="+employeeDaomap);
log.info("employeeDaomap after execute size ="+employeeDaomap.size()); // expected 1
List<EmployeeDAO> list = (List<EmployeeDAO>) employeeDaomap .get("allEmployees");
log.info("size of the list ="+list.size()); // need to get the size of the list ,
return list;
}
private Map<String, Object> execute() {
return super.execute(new HashMap<String, Object>());
}
}
public class TestRowMapper implements RowMapper<List<EmployeeDAO>> {
static Logger log = Logger.getLogger(TestRowMapper.class);
#Override
public List<EmployeeDAO> mapRow(ResultSet rs, int rowNum)
throws SQLException {
// TODO Auto-generated method stub
rs.setFetchSize(3000);
List<EmployeeDAO> responseItems = new ArrayList<EmployeeDAO>();
EmployeeDAO responseItem = null;
log.info("row num "+rowNum);
while (rs.next()) {
responseItem = new EmployeeDAO();
responseItem.setID(rs.getString("id"));
responseItem.setName(rs.getString("name"));
responseItem.setDesc(rs.getString("desc"));
responseItems.add(responseItem);
}
log.info("TestRowMapper items ="+responseItems);
return responseItems;
}
}
The solution is to use the implements ResultSetExtractor instead of RowMapper and provide implementation for extractData.
public class TestRowMapper implements ResultSetExtractor<List<EmployeeDAO>> {
static Logger log = Logger.getLogger(TestRowMapper.class);
#Override
public List<EMAccountResponse> extractData(ResultSet rs)
throws SQLException, DataAccessException {
rs.setFetchSize(3000);
List<EmployeeDAO> responseItems = new ArrayList<EmployeeDAO>();
EmployeeDAO responseItem = null;
log.info("row num "+rowNum);
while (rs.next()) {
responseItem = new EmployeeDAO();
responseItem.setID(rs.getString("id"));
responseItem.setName(rs.getString("name"));
responseItem.setDesc(rs.getString("desc"));
responseItems.add(responseItem);
}
log.info("TestRowMapper items ="+responseItems);
return responseItems;
}
}

Resources