My requirement is to read CSV,XML,JSON, excel file format through file upload functionality using spring controller. Read the file transform and save into the database.
I want to use genericItem processor like Spring Batch ItemProcessor to read the above file formats. My transform logic will be common for all and then save it to the Database.
Is there any way by which i can use spring batch ItemProcessor in standalone way without creating the batch job or is there any open source tool which can read the file in above formats?
You can use apache camel to get the desired flow. There are lot of code examples out in internet to configure apache camel with spring-boot.
Here in this example, I assume that data is posted to /csv , /excel , /xml end points and data is routed to direct:records. From direct:records we have a custom processor to process the data. We can have multiple steps like that.
#Component
public class StudentRoute extends RouteBuilder {
#Override
public void configure() {
restConfiguration()
.component("servlet")
.bindingMode(RestBindingMode.json);
rest("/student").produces("application/json")
.post("/csv").to("direct:records")
.post("/excel").to("direct:records")
.post("/xml").to("direct:records");
from("direct:records")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
Object value = exchange.getIn().getBody(Object.class);
//write your logic here
}
});
}
}
Also follow this link to get more about Apache Camel. Spring Boot, Apache Camel, and Swagger UI
Related
I am new to spring integration framework. Currently i am working on a project which has a requirement to download the files to a local directory.
My goal is to complete the below task
1.Download the files by suing spring integration to a local directory
2.Trigger a batch job.It means to read the file and extract a specific column information.
I am able to connect to SFTP server.But facing difficulty how to use spring integration java DSL to download the files and trigger a batch job.
Below code to connect to SFTP Session Factory
#Bean
public SessionFactory<ChannelSftp.LsEntry> sftpSessionFactory() {
DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(true);
factory.setHost(sftpHost);
factory.setPort(sftpPort);
factory.setUser(sftpUser);
if (sftpPrivateKey != null) {
factory.setPrivateKey(sftpPrivateKey);
factory.setPrivateKeyPassphrase(privateKeyPassPhrase);
} else {
factory.setPassword("sftpPassword");
}
factory.setPassword("sftpPassword");
logger.info("Connecting to SFTP Server" + factory.getSession());
System.out.println("Connecting to SFTP Server" + factory.getSession());
factory.setAllowUnknownKeys(true);
return new CachingSessionFactory<ChannelSftp.LsEntry>(factory);
}
Below code to download the files from remote to local
#Bean
public IntegrationFlowBuilder integrationFlow() {
return IntegrationFlows.from(Sftp.inboundAdapter(sftpSessionFactory()));
}
I am using spring integration dsl. i am not able to get what to code here.
I am trying many possible ways to do this.But not able to get how to proceed with this requirement.
Can anyone one help me how to approach at this and if possible share me a sample code for reference?
The Sftp.inboundAdapter() produces messages with a File as a payload. So, having that IntegrationFlows.from(Sftp.inboundAdapter(sftpSessionFactory())) you can treat as a first task done.
Your problem from here that you don't make an integrationFlow, but rather return that IntegrationFlowBuilder and register it as a #Bean. That's where it doesn't work for you.
You need to continue a flow definition and call its get() in the end to return an integrationFlow instance which already has to be registered as a bean. If this code flow is confusing a bit, consider to implement an IntegrationFlowAdapter as a #Component.
To trigger a batch job you need consider to use a FileMessageToJobRequest in a .transform() EIP-method and then a JobLaunchingGateway in a .handle() EIP-method.
See more info in docs:
https://docs.spring.io/spring-integration/reference/html/dsl.html#java-dsl
https://docs.spring.io/spring-integration/reference/html/sftp.html#sftp-inbound
https://docs.spring.io/spring-batch/docs/4.3.x/reference/html/spring-batch-integration.html#spring-batch-integration-configuration
BTW, the last one has a flow sample exactly for your use-case.
Is there any way to refresh springboot configuration as soon as we change .properties file?
I came across spring-cloud-config and many articles/blogs suggested to use this for a distributed environment. I have many deployments of my springboot application but they are not related or dependent on one another. I also looked at few solutions where they suggested providing rest endpoints to refresh configs manually without restarting application. But I want to refresh configuration dynamically whenever I change .properties file without manual intervention.
Any guide/suggestion is much appreciated.
Can you just use the Spring Cloud Config "Server" and have it signal to your Spring Cloud client that the properties file changed. See this example:
https://spring.io/guides/gs/centralized-configuration/
Under the covers, it is doing a poll of the underlying resource and then broadcasts it to your client:
#Scheduled(fixedRateString = "${spring.cloud.config.server.monitor.fixedDelay:5000}")
public void poll() {
for (File file : filesFromEvents()) {
this.endpoint.notifyByPath(new HttpHeaders(), Collections
.<String, Object>singletonMap("path", file.getAbsolutePath()));
}
}
If you don't want to use the config server, in your own code, you could use a similar scheduled annotation and monitor your properties file:
#Component
public class MyRefresher {
#Autowired
private ContextRefresher contextRefresher;
#Scheduled(fixedDelay=5000)
public void myRefresher() {
// Code here could potentially look at the properties file
// to see if it changed, and conditionally call the next line...
contextRefresher.refresh();
}
}
I am building a new spring boot application deployable to bluemix (cloud foundry) which needs to do the following:
use spring-cloud-cloudfoundry-connector to discover user-provided "properties service": read the service URL and credentials from VCAP_APPLICATION env variable.
This step is completed.
connect to properties service via HTTP call, receive JSON response, parse individual property values and expose them as application properties (in Environment object?)
What would be the correct solution for this in spring-boot app?
In older non-boot Spring app, the property service call would be initiated early in Spring lifecycle by a class that extended PropertySourcesPlaceholderConfigurer and the property collection from the service would be handled inside postProcessBeanFactory() method call of the same class.
public class CustomPopertiesFactory
extends PropertySourcesPlaceholderConfigurer
implements EnvironmentAware {
private Properties properties;
getServiceCredentials() {
// parse VCAP_APPLICATION json
final String localVcapServices = System.getProperty("VCAP_SERVICES");
// extract url, username, pwd to connect to the service
}
connectToService () {
// via HTTP request using RestTemplate
// parse JSON response and add properties to this.properties
... this.properties.put("prop1", valueFromJson);
}
#Override
public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) throws BeansException {
getServiceCredentials();
connectToService();
// load values from properties service into app properties
setProperties(properties);
// continue with lifecycle and load properties from other sources
super.postProcessBeanFactory(beanFactory);
}
}
That was painful to maintain and switch between cloud and local spring profiles IMO and I am wondering if spring boot has a better way of handling external properties.
I ended up replacing "properties service" with spring-cloud-config-server
and using spring-cloud-config-client
in my spring boot application to consume properties from spring-cloud-config-server.
We are developing a RESTFUL services in spring framework and the log framework we are using is apache CXF.
We get the request in the form of JSON from the consumers of our services and we need to mask some of the contents of JSON before printing in the log files.
We are trying to have a custom interceptor to intercept the logger messages but the request never reaches the custom interceptor class. Can you please provide your thoughts to resolve the issue.
Below are the additional details :
public class CustomLogInterceptor extends LoggingInInterceptor {
public String transform(String originalLogString) {
// Custom logic will be here to mask the originalLogString
}
}
spring context XML changes:
The CustomLogInterceptor class is included in the spring context XML file.
Any help is greatly appreciated!
I have a requirement where by the application will need to be periodically refreshed to subscribe from different end points. I am using Apache Camel for orchestration and I am comfortable subscribing to an end point.
I have a routebuilder class as follows:
public class SampleRouteBuilder extends RouteBuilder {
#Override
public void configure() throws Exception {
String... subscriptionTopicUris = someService.getUris();
// Simple logic - read from URIs and write to topic
from(subscriptionTopicUris) //
.to(destinationUri) //
.routeId("SAMPLE_ROUTE_ID");
}
}
I run a scheduled job with a given time interval and I remove the route from Camel Context, stop the context and add it back and start the context. However, the configure method is not fired on starting the context and hence the service (someService) possibly returning a different list of URIs is never fired.
How do I reload the route?
I am using Spring, Apache Camel and annotation based approach and this is a web application running in jetty.
Thanks