I am using Spring Boot with Thymeleaf.
When user click on related button, it sends post requst and in the related controller method there is a function which takes 20 mins. This function does not return a value.
I just want to process this function in background. When application comes to the this function's line, it should send parameters to this function and keep processing without waiting a return.
What is the best practice for this case?
Many thanks in advance.
UPDATE
My config class
#Configuration
#EnableAsync
public class SpringAsyncConfig implements AsyncConfigurer{
#Bean(name = "ocrThread-")
public Executor threadPoolTaskExecutor() {
return new ThreadPoolTaskExecutor();
}
#Override
public Executor getAsyncExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(2);
executor.setMaxPoolSize(2);
executor.setQueueCapacity(10);
executor.initialize();
return executor;
}
#Override
public AsyncUncaughtExceptionHandler getAsyncUncaughtExceptionHandler() {
// TODO Auto-generated method stub
return null;
}
}
Service class
#Service
public class OcrService {
#Async
public String treadliOcr(List<String> liste, String kok) throws
InterruptedException, IOException, TesseractException {
.....
}
}
Controller
#RequestMapping(value="/aktar", method= RequestMethod.POST)
public String aktar(#RequestParam("belgeAdi") String belgeAdi,
#RequestParam("evrakTurId") String evrakTurId,
#RequestParam("kategoriId") String kategoriId,
#RequestParam("belgeTurId") String belgeTurId,
#RequestParam("firmaId") String firmaId,
#RequestParam("projeId") String projeId,
#RequestParam("aciklama") String aciklama) throws InterruptedException, IOException, TesseractException{
Integer b = null;
Integer p = null;
String klasor = getInitYol();
String belgeOnAd = belgeAdi.substring(0, 14);
BelgeIsimleri belgeIsimleri = new BelgeIsimleri();
List<String> seciliListe = belgeIsimleri.seciliBelgeleriFiltrele(klasor, belgeOnAd);
for(String s:seciliListe){
File file = new File (getInitYol()+s);
if(file.renameTo(new File("D:\\Done\\"+s))){
file.delete();
System.out.println(s+"yi sildi");
}
}
OcrService ocr = new OcrService();
String result=ocr.treadliOcr(seciliListe,getInitYol());
System.out.println("Ocr dan döndü");
Integer et = Integer.valueOf(evrakTurId);
Integer k = Integer.valueOf(kategoriId);
if(null==belgeTurId || "".equals(belgeTurId)){
}else{
b = Integer.valueOf(belgeTurId);
}
Integer f = Integer.valueOf(firmaId);
if(null==projeId || "".equals(projeId)){
}else{
p = Integer.valueOf(projeId);
}
belgeRepo.save(new BelgeEntity(et,k ,b , f ,p ,aciklama, result,belgeOnAd));
return "redirect:/verigiris";
}
Spring provides annotation support for asynchronous method execution via #Async and #EnableAsync: https://spring.io/guides/gs/async-method/
Related
I am using spring cloud function on AWS lambda. I am trying to achieve global exception handling like Spring Boot using #ExceptionHandler annotation. But this method is not getting executed and I am getting 500 for any type of exception.
Sample code is below-
#SpringBootApplication
public class App{
public static void main( String[] args ){
SpringApplication.run(App.class, args);
}
#Bean
public Function<Message<User>, User> getUser(){
return (message)->{
User u = message.getPayload();
if(u==null){
throw new ResponseStatusException(HttpStatus.BAD_REQUEST,"No user details provided");
}
return u;
}
}
#ExceptionHandler(ResponseStatusException.class)
public APIGatewayProxyResponseEvent handleException(ResponseStatusException e){
APIGatewayProxyResponseEvent response = new APIGatewayProxyResponseEvent();
response.setStatusCode(e.getRawStatusCode());
response.setBody(e.getMessage());
return response;
}
}
I am getting 500 in response instead Bad Request. Is there any way to achieve this scenario ?
You can provide your custom exceptionHandler while building SpringBootLambdaContainerHandler.
public class StreamLambdaHandler implements RequestStreamHandler {
private static SpringBootLambdaContainerHandler<AwsProxyRequest, AwsProxyResponse> handler;
static {
handler = new SpringBootProxyHandlerBuilder<AwsProxyRequest>()
.defaultProxy()
.exceptionHandler(***your customer handler here***)
// other methods are skipped
.buildAndInitialize();
}
}
If you are using spring cloud functions no need to use SpringBootLambdaContainerHandler, what you need to do is create a custom routing function and handle the exception thrown from your lambda function and return APIGatewayProxyResponseEvent. below shows how I achieved the desired result
public class CustomRoutingFunction implements Function<Message<?>, APIGatewayProxyResponseEvent> {
private final FunctionCatalog functionCatalog;
private final FunctionProperties functionProperties;
private final MessageRoutingCallback routingCallback;
public static final String DEFAULT_ROUTE_HANDLER = "defaultMessageRoutingHandler";
public CustomRoutingFunction(FunctionCatalog functionCatalog,
FunctionProperties functionProperties,
MessageRoutingCallback routingCallback) {
this.functionCatalog = functionCatalog;
this.functionProperties = functionProperties;
this.routingCallback = routingCallback;
}
#Override
public APIGatewayProxyResponseEvent apply(Message<?> input) {
ObjectMapper mapper = new ObjectMapper();
mapper.configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false);
try {
String functionDefinition = this.routingCallback.routingResult(input);
SimpleFunctionRegistry.FunctionInvocationWrapper function = functionCatalog.lookup(functionDefinition);
Object output = function.apply(input);
String payload = mapper.writeValueAsString(output);
return new APIGatewayProxyResponseEvent()
.withIsBase64Encoded(false)
.withBody(payload)
.withHeaders(Map.of(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE,
"statusCode", "200"))
.withStatusCode(200);
} catch (Exception e) {
return new APIGatewayProxyResponseEvent()
.withIsBase64Encoded(false)
.withHeaders(Map.of(HttpHeaders.CONTENT_TYPE,
MediaType.APPLICATION_JSON_VALUE,
"statusCode", String.valueOf(HttpStatus.BAD_REQUEST.value())))
.withBody(e.getMessage())
.withStatusCode(HttpStatus.BAD_REQUEST.value());
}
}
}
now you need to register your router function as a bean and pass it to spring.cloud.function.definition
#Bean
public CustomRoutingFunction customRoutingFunction(FunctionCatalog functionCatalog,
FunctionProperties functionProperties,
#Nullable MessageRoutingCallback routingCallback,
#Nullable DefaultMessageRoutingHandler defaultMessageRoutingHandler) {
if (defaultMessageRoutingHandler != null) {
FunctionRegistration functionRegistration = new FunctionRegistration(defaultMessageRoutingHandler, CustomRoutingFunction.DEFAULT_ROUTE_HANDLER);
functionRegistration.type(FunctionTypeUtils.consumerType(ResolvableType.forClassWithGenerics(Message.class, Object.class).getType()));
((FunctionRegistry) functionCatalog).register(functionRegistration);
}
return new CustomRoutingFunction(functionCatalog, functionProperties, routingCallback);
}
inside your application.yml file
spring:
cloud:
function:
definition: customRoutingFunction
I am trying out to write data to my local Elasticsearch Docker Container (7.4.2), for simplicity I used the AbstractReactiveElasticsearchConfiguration given from Spring also Overriding the entityMapper function. The I constructed my repository extending the ReactiveElasticsearchRepository
Then in the end I used my autowired repository to saveAll() my collection of elements containing the data. However Elasticsearch doesn't write any data. Also i have a REST controller which is starting my whole process returning nothing basicly, DeferredResult>
The REST method coming from my ApiDelegateImpl
#Override
public DeferredResult<ResponseEntity<Void>> openUsageExporterStartPost() {
final DeferredResult<ResponseEntity<Void>> deferredResult = new DeferredResult<>();
ForkJoinPool.commonPool().execute(() -> {
try {
openUsageExporterAdapter.startExport();
deferredResult.setResult(ResponseEntity.accepted().build());
} catch (Exception e) {
deferredResult.setErrorResult(e);
}
}
);
return deferredResult;
}
My Elasticsearch Configuration
#Configuration
public class ElasticSearchConfig extends AbstractReactiveElasticsearchConfiguration {
#Value("${spring.data.elasticsearch.client.reactive.endpoints}")
private String elasticSearchEndpoint;
#Bean
#Override
public EntityMapper entityMapper() {
final ElasticsearchEntityMapper entityMapper = new ElasticsearchEntityMapper(elasticsearchMappingContext(), new DefaultConversionService());
entityMapper.setConversions(elasticsearchCustomConversions());
return entityMapper;
}
#Override
public ReactiveElasticsearchClient reactiveElasticsearchClient() {
ClientConfiguration clientConfiguration = ClientConfiguration.builder()
.connectedTo(elasticSearchEndpoint)
.build();
return ReactiveRestClients.create(clientConfiguration);
}
}
My Repository
public interface OpenUsageRepository extends ReactiveElasticsearchRepository<OpenUsage, Long> {
}
My DTO
#Data
#Document(indexName = "open_usages", type = "open_usages")
#TypeAlias("OpenUsage")
public class OpenUsage {
#Field(name = "id")
#Id
private Long id;
......
}
My Adapter Implementation
#Autowired
private final OpenUsageRepository openUsageRepository;
...transform entity into OpenUsage...
public void doSomething(final List<OpenUsage> openUsages){
openUsageRepository.saveAll(openUsages)
}
And finally my IT test
#SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT)
#Testcontainers
#TestPropertySource(locations = {"classpath:application-it.properties"})
#ContextConfiguration(initializers = OpenUsageExporterApplicationIT.Initializer.class)
class OpenUsageExporterApplicationIT {
#LocalServerPort
private int port;
private final static String STARTCALL = "http://localhost:%s/open-usage-exporter/start/";
#Container
private static ElasticsearchContainer container = new ElasticsearchContainer("docker.elastic.co/elasticsearch/elasticsearch:6.8.4").withExposedPorts(9200);
static class Initializer implements ApplicationContextInitializer<ConfigurableApplicationContext> {
#Override
public void initialize(final ConfigurableApplicationContext configurableApplicationContext) {
final List<String> pairs = new ArrayList<>();
pairs.add("spring.data.elasticsearch.client.reactive.endpoints=" + container.getContainerIpAddress() + ":" + container.getFirstMappedPort());
pairs.add("spring.elasticsearch.rest.uris=http://" + container.getContainerIpAddress() + ":" + container.getFirstMappedPort());
TestPropertyValues.of(pairs).applyTo(configurableApplicationContext);
}
}
#Test
void testExportToES() throws IOException, InterruptedException {
final List<OpenUsageEntity> openUsageEntities = dbPreparator.insertTestData();
assertTrue(openUsageEntities.size() > 0);
final String result = executeRestCall(STARTCALL);
// Awaitility here tells me nothing is in ElasticSearch :(
}
private String executeRestCall(final String urlTemplate) throws IOException {
final String url = String.format(urlTemplate, port);
final HttpUriRequest request = new HttpPost(url);
final HttpResponse response = HttpClientBuilder.create().build().execute(request);
// Get the result.
return EntityUtils.toString(response.getEntity());
}
}
public void doSomething(final List<OpenUsage> openUsages){
openUsageRepository.saveAll(openUsages)
}
This lacks a semicolon at the end, so it should not compile.
But I assume this is just a typo, and there is a semicolon in reality.
Anyway, saveAll() returns a Flux. This Flux is just a recipe for saving your data, and it is not 'executed' until subscribe() is called by someone (or something like blockLast()). You just throw that Flux away, so the saving never gets executed.
How to fix this? One option is to add .blockLast() call:
openUsageRepository.saveAll(openUsages).blockLast();
But this will save the data in a blocking way effectively defeating the reactivity.
Another option is, if the code you are calling saveAll() from supports reactivity is just to return the Flux returned by saveAll(), but, as your doSomething() has void return type, this is doubtful.
It is not seen how your startExport() connects to doSomething() anyway. But it looks like your 'calling code' does not use any notion of reactivity, so a real solution would be to either rewrite the calling code to use reactivity (obtain a Publisher and subscribe() on it, then wait till the data arrives), or revert to using blocking API (ElasticsearchRepository instead of ReactiveElasticsearchRepository).
I have spring application which expose REST endpoint, lets name it "doAction". As the request it consumes object:
class Person{
private String name;
private String email;
}
Some clients can call this endpoint by passing data with different practice of writing words, like:
Peter_1
name = Peter
email = peter#gmail.com (lower case)
Mark_2
name = mark
email = MARK#gmail.com (upper case)
Julia_3
name = julia
email = JuliaToward#gmail.com (camel case)
Is there some approach to force all income data be parsed to lowercase(lets assume all fields are Strings)?
So as a result I desire to have:
Peter_1
name = peter
email = peter#gmail.com
Mark_2
name = mark
email = mark#gmail.com
Julia_3
name = julia
email = juliatoward#gmail.com
Solution for Jackson is appreciated.
Short answer Call toLower in the setter
Here is an example:
class Animal
{
private String name;
public void setName(final String newValue)
{
StringUtils.trimToNull(StringUtils.lowerCase(newValue));
}
}
I also recommend either trimToNUll or trimToEmpty.
If you are using Spring Data Rest with spring mvc and you want all incoming string data to be in lower case then define following
public class StringSerializer extends StdDeserializer<String>{
public StringSerializer() {
this(null);
}
public StringSerializer(Class<String> vc) {
super(vc);
}
#Override
public String deserialize(JsonParser p, DeserializationContext ctxt) throws IOException, JsonProcessingException {
JsonToken t = p.getCurrentToken();
if (t==JsonToken.VALUE_STRING){
String receivedValue = p.getText();
if (receivedValue == null)
return null;
else
return receivedValue.toLowerCase();
}else{
return null;
}
}
}
And following:
#Configuration
public class RestDataConfig extends RepositoryRestMvcConfiguration {
#Override
#Bean
public ObjectMapper halObjectMapper() {
ObjectMapper mapper = super.halObjectMapper();
SimpleModule module = new SimpleModule();
module.addDeserializer(String.class, new StringSerializer());
mapper.registerModule(module);
return mapper;
}
}
I'm trying to implement a rest api consuming excel file. I'm using spring-boot and code is available here.
Code works fine when using FileSystemResource for payload. But i'm not able to make the code work with ByteArrayResource in replacement of FileSystemResource:
RestApi.java:
#RestController
public class RestApi {
private static final Logger LOGGER = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
#PostMapping("/api/upload")
public ResponseEntity<?> uploadFile(#RequestParam("file") MultipartFile uploadfile) {
LOGGER.debug("Single file upload!");
try {
LOGGER.info("\n\n ****** File name: {}, type {}! ************", uploadfile.getOriginalFilename(), uploadfile.getContentType());
this.processExcelFile(uploadfile.getInputStream());
} catch (Exception e) {
return new ResponseEntity<>(HttpStatus.BAD_REQUEST);
}
return new ResponseEntity<>("Successfully uploaded - " + uploadfile.getOriginalFilename(), new HttpHeaders(), HttpStatus.OK);
}
private List<String> processExcelFile(InputStream stream) throws Exception {
List<String> result = new ArrayList<String>();
//Create Workbook instance holding reference to .xlsx file
try(XSSFWorkbook workbook = new XSSFWorkbook(stream);) {
//Get first/desired sheet from the workbook
XSSFSheet sheet = workbook.getSheetAt(0);
//Iterate through each rows one by one
Iterator<Row> rowIterator = sheet.iterator();
while (rowIterator.hasNext()) {
Row row = rowIterator.next();
String cellValue = row.getCell(0).getRichStringCellValue().toString();
result.add(cellValue);
LOGGER.info("\n\n ****** Cell value: {} ************", cellValue);
}
return result;
}
}
}
RestApiTest:
#RunWith(SpringRunner.class)
#SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT)
public class RestApiTest {
#Autowired
private TestRestTemplate restTemplate;
#Autowired
private ResourceLoader loader;
#Test
public void testUploadFile() throws Exception {
Resource resource = this.loader.getResource("classpath:test.xlsx");
MultiValueMap<String, Object> parts = new LinkedMultiValueMap<>();
// parts.add("file", new FileSystemResource(resource.getFile()));
parts.add("file", new ByteArrayResource(IOUtils.toByteArray(resource.getInputStream())));
String response = this.restTemplate.postForObject("/api/upload", parts, String.class);
Assertions.assertThat(response).containsIgnoringCase("success");
}
}
I'm getting following error when running test:
java.lang.AssertionError:
Expecting:
<"{"timestamp":1487852597527,"status":400,"error":"Bad Request","exception":"org.springframework.web.multipart.support.MissingServletRequestPartException","message":"Required request part 'file' is not present","path":"/api/upload"}">
to contain:
<"success">
(ignoring case)
Any idea?
when using loader.getResource(...) you must use resource itself as answered above. So you don't need ByteArrayResource. I got this problem, but I'm not using resource from classpath. So if someone really need to use ByteArrayResource, here is my workaround
public class FileNameAwareByteArrayResource extends ByteArrayResource {
private String fileName;
public FileNameAwareByteArrayResource(String fileName, byte[] byteArray, String description) {
super(byteArray, description);
this.fileName = fileName;
}
#Override
public String getFilename() {
return fileName;
}
}
and then use it
parts.add("file", new FileNameAwareByteArrayResource("filename", byteArray));
I'm trying to test if #Async annotation of Spring is working as expected on my project. But It doesn't.
I have this test:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(classes = GlobalConfiguration.class)
public class ActivityMessageListenerTest {
#Autowired
private ActivityMessageListener activityMessageListener;
private Long USER_ID = 1l;
private Long COMPANY_ID = 2l;
private Date DATE = new Date(10000000);
private String CLASSNAME = "className";
private Long CLASSPK = 14l;
private Integer TYPE = 22;
private String EXTRA_DATA = "extra";
private Long RECIVED_USER_ID = 99l;
#Before
public void setup() throws Exception {
}
#Test
public void testDoReceiveWithException() throws Exception {
System.out.println("Current thread " + Thread.currentThread().getName());
Map<String, Object> values = new HashMap();
values.put(ActivityMessageListener.PARAM_USER_ID, USER_ID);
values.put(ActivityMessageListener.PARAM_COMPANY_ID, COMPANY_ID);
values.put(ActivityMessageListener.PARAM_CREATE_DATE, DATE);
values.put(ActivityMessageListener.PARAM_CLASS_NAME, CLASSNAME);
values.put(ActivityMessageListener.PARAM_CLASS_PK, CLASSPK);
values.put(ActivityMessageListener.PARAM_TYPE, TYPE);
values.put(ActivityMessageListener.PARAM_EXTRA_DATA, EXTRA_DATA );
values.put(ActivityMessageListener.PARAM_RECEIVED_USER_ID, RECIVED_USER_ID);
Message message = new Message();
message.setValues(values);
MessageBusUtil.sendMessage(MKTDestinationNames.ACTIVITY_REGISTRY, message);
}
}
As you can see I'm printing the name of the current thread.
The class containing the #Async method is:
public class ActivityMessageListener extends BaseMessageListener {
public static final String PARAM_USER_ID = "userId";
public static final String PARAM_COMPANY_ID = "companyId";
public static final String PARAM_CREATE_DATE = "createDate";
public static final String PARAM_CLASS_NAME = "className";
public static final String PARAM_CLASS_PK = "classPK";
public static final String PARAM_TYPE = "type";
public static final String PARAM_EXTRA_DATA = "extraData";
public static final String PARAM_RECEIVED_USER_ID = "receiverUserId";
public ActivityMessageListener() {
MessageBusUtil.addQueue(MKTDestinationNames.ACTIVITY_REGISTRY, this);
}
#Override
#Async(value = "activityExecutor")
public void doReceive(Message message) throws Exception {
System.out.println("Current " + Thread.currentThread().getName());
if (1> 0)
throw new RuntimeException("lalal");
Map<String, Object> parameters = message.getValues();
Long userId = (Long)parameters.get(ActivityMessageListener.PARAM_USER_ID);
Long companyId = (Long)parameters.get(ActivityMessageListener.PARAM_COMPANY_ID);
Date createDate = (Date)parameters.get(ActivityMessageListener.PARAM_CREATE_DATE);
String className = (String)parameters.get(ActivityMessageListener.PARAM_CLASS_NAME);
Long classPK = (Long)parameters.get(ActivityMessageListener.PARAM_CLASS_PK);
Integer type = (Integer)parameters.get(ActivityMessageListener.PARAM_TYPE);
String extraData = (String)parameters.get(ActivityMessageListener.PARAM_EXTRA_DATA);
Long receiverUserId = (Long)parameters.get(ActivityMessageListener.PARAM_RECEIVED_USER_ID);
ActivityLocalServiceUtil.addActivity(userId, companyId, createDate, className, classPK, type, extraData, receiverUserId);
}
}
Here I'm printing the name of the current thread inside of the #Async method, and the name is the same as before, main. So it's not working.
The GlobalConfiguration is:
#Configuration
#EnableAspectJAutoProxy
#EnableTransactionManagement
#ComponentScan({
"com.shn.configurations",
...some packages...
})
public class GlobalConfiguration {...}
And inside one of the specified packages has the activityExecutor bean:
#Configuration
#EnableAsync(proxyTargetClass = true)
public class ExecutorConfiguration {
#Bean
public ActivityMessageListener activityMessageListener() {
return new ActivityMessageListener();
}
#Bean
public TaskExecutor activityExecutor()
{
ThreadPoolTaskExecutor threadPoolTaskExecutor =
new ThreadPoolTaskExecutor();
threadPoolTaskExecutor.setCorePoolSize(10);
threadPoolTaskExecutor.setMaxPoolSize(10);
threadPoolTaskExecutor.setQueueCapacity(100);
return threadPoolTaskExecutor;
}
}
What I'm doing wrong?
Tricky.
Asynchronous behavior is added through proxying.
Spring provides you with a proxy that wraps the actual object and performs the actual invocation in a separate thread.
It looks something like this (except most of this is done dynamically with CGLIB or JDK proxies and Spring handlers)
class ProxyListener extends ActivityMessageListener {
private ActivityMessageListener real;
public ProxyListener(ActivityMessageListener real) {
this.real = real;
}
TaskExecutor executor; // injected
#Override
public void doReceive(Message message) throws Exception {
executor.submit(() -> real.doReceive(message)); // in another thread
}
}
ActivityMessageListener real = new ActivityMessageListener();
ProxyListener proxy = new ProxyListener(real);
Now, in a Spring world, you'd have a reference to the proxy object, not to the ActivityMessageListener. That is
ActivityMessageListener proxy = applicationContext.getBean(ActivityMessageListener.class);
would return a reference to the ProxyListener. Then, through polymorphism, invoking doReceive would go to the overriden Proxy#doReceive method which would invoke ActivityMessageListener#doReceive through delegation and you'd get your asynchronous behavior.
However, you're in a half Spring world.
Here
public ActivityMessageListener() {
MessageBusUtil.addQueue(MKTDestinationNames.ACTIVITY_REGISTRY, this);
}
the reference this is actually referring to the real ActivityMessageListener, not to the proxy. So when, presumably, you send your message on the bus here
MessageBusUtil.sendMessage(MKTDestinationNames.ACTIVITY_REGISTRY, message);
you're sending it to the real object, which doesn't have the proxy asynchronous behavior.
The full Spring solution would be to have the MessabeBus (and/or its queue) be Spring beans in which you can inject the fully process (proxied, autowired, initialized) beans.
In reality, since CGLIB proxies are really just subclasses of your types, so the ProxyListener above would actually also add itself to the bus since the super constructor would be invoked. It would seem though that only one MessageListener can register itself with a key, like MKTDestinationNames.ACTIVITY_REGISTRY. If this isn't the case, you'd have to show more of that code for explanation.
In your test, if you do
activityMessageListener.doReceive(message);
you should see that asynchronous behavior since activityMessageListener should hold a reference to the proxy.