How to accept http requests after shutdown signal in Quarkus? - quarkus

I tried this:
void onShutdown(#Observes final ShutdownEvent event) throws InterruptedException {
log.infof("ShutdownEvent received, waiting for %s seconds before shutting down", shutdownWaitSeconds);
TimeUnit.SECONDS.sleep(shutdownWaitSeconds);
log.info("Continue shutting down");
}
But after receiving ShutdownEvent Quarkus already responds with 503 to http requests. Looks like this could be done with ShutdownListener in preShutdown method. I have implemented this listener but it does not get called yet. How do I register ShutdownListener?
Use case here is OpenShift sending requests to terminating pod.

Option 1: Create Quarkus extension
Instructions are here. ShutdownController is my own class implementing ShutdownListener where I have a sleep in preShutdown method.
class ShutdownControllerProcessor {
#BuildStep
FeatureBuildItem feature() {
return new FeatureBuildItem("shutdown-controller");
}
#BuildStep
ShutdownListenerBuildItem shutdownListener() {
// Called at build time. Default constructor will be called at runtime.
// Getting MethodNotFoundException when calling default constructor here.
return new ShutdownListenerBuildItem(new ShutdownController(10));
}
}
Option 2: Modify ShutdownRecorder private static final field
New shutdown listener can be added using reflection. This is a bit ugly solution.
registerIfNeeded() need to be called after Quarkus startup, for example with timer 1 second after #PostConstruct.
#ApplicationScoped
public class ListenerRegisterer {
public void registerIfNeeded() {
try {
tryToRegister();
} catch (NoSuchFieldException | IllegalAccessException e) {
throw new IllegalStateException(e);
}
}
private void tryToRegister() throws NoSuchFieldException, IllegalAccessException {
final var field = ShutdownRecorder.class.getDeclaredField("shutdownListeners");
field.setAccessible(true);
final var listeners = (List<ShutdownListener>) field.get(null);
if (listeners != null && !listeners.toString().contains("ShutdownController")) {
listeners.add(new ShutdownController(10));
setFinalStatic(field, listeners);
}
}
private static void setFinalStatic(final Field field, final Object newValue) throws NoSuchFieldException, IllegalAccessException {
field.setAccessible(true);
final var modifiersField = Field.class.getDeclaredField("modifiers");
modifiersField.setAccessible(true);
modifiersField.setInt(field, field.getModifiers() & ~Modifier.FINAL);
field.set(null, newValue);
}
}

Related

Micrometer - WebMvcTagsContributor not adding custom tags

I'm trying to add custom tags - the path variables and their values from each request - to each metric micrometer generates. I'm using spring-boot with java 16.
From my research i've found that creating a bean of type WebMvcTagsContributor alows me to do just that.
This is the code
public class CustomWebMvcTagsContributor implements WebMvcTagsContributor {
private static int PRINT_ERROR_COUNTER = 0;
#Override
public Iterable<Tag> getTags(HttpServletRequest request, HttpServletResponse response,
Object handler,
Throwable exception) {
return Tags.of(getAllTags(request));
}
private static List<Tag> getAllTags(HttpServletRequest request) {
Object attributesMapObject = request.getAttribute(View.PATH_VARIABLES);
if (isNull(attributesMapObject)) {
attributesMapObject = request.getAttribute(HandlerMapping.URI_TEMPLATE_VARIABLES_ATTRIBUTE);
if (isNull(attributesMapObject)) {
attributesMapObject = extractPathVariablesFromURI(request);
}
}
if (nonNull(attributesMapObject)) {
return getPathVariablesTags(attributesMapObject);
}
return List.of();
}
private static Object extractPathVariablesFromURI(HttpServletRequest request) {
Long currentUserId = SecurityUtils.getCurrentUserId().orElse(null);
try {
URI uri = new URI(request.getRequestURI());
String path = uri.getPath(); //get the path
UriTemplate uriTemplate = new UriTemplate((String) request.getAttribute(
HandlerMapping.BEST_MATCHING_PATTERN_ATTRIBUTE)); //create template
return uriTemplate.match(path); //extract values form template
} catch (Exception e) {
log.warn("[Error on 3rd attempt]", e);
}
return null;
}
private static List<Tag> getPathVariablesTags(Object attributesMapObject) {
try {
Long currentUserId = SecurityUtils.getCurrentUserId().orElse(null);
if (nonNull(attributesMapObject)) {
var attributesMap = (Map<String, Object>) attributesMapObject;
List<Tag> tags = attributesMap.entrySet().stream()
.map(stringObjectEntry -> Tag.of(stringObjectEntry.getKey(),
String.valueOf(stringObjectEntry.getValue())))
.toList();
log.warn("[CustomTags] [{}]", CommonUtils.toJson(tags));
return tags;
}
} catch (Exception e) {
if (PRINT_ERROR_COUNTER < 5) {
log.error("[Error while getting attributes map object]", e);
PRINT_ERROR_COUNTER++;
}
}
return List.of();
}
#Override
public Iterable<Tag> getLongRequestTags(HttpServletRequest request, Object handler) {
return null;
}
}
#Bean
public WebMvcTagsContributor webMvcTagsContributor() {
return new CustomWebMvcTagsContributor();
}
In order to test this, i've created a small spring boot app, added an endpoint to it. It works just fine.
The problem is when I add this code to the production app.
The metrics generates are the default ones and i can't figure out why.
What can I check to see why the tags are not added?
local test project
http_server_requests_seconds_count {exception="None", method="GET",id="123",outcome="Success",status="200",test="test",uri="/test/{id}/compute/{test}",)1.0
in prod - different (& bigger) app
http_server_requests_seconds_count {exception="None", method="GET",outcome="Success",status="200",uri="/api/{something}/test",)1.0
What i've tried and didn't work
Created a bean that implemented WebMvcTagsProvider - this one had an odd behaviour - it wasn't creating metrics for endpoints that had path variables in the path - though in my local test project it worked as expected
I added that log there in order to see what the extra tags are but doesn't seem to reach there as i don't see anything in the logs - i know, you might say that the current user id stops it, but it's not that.

Can not run few methods sequentially when Spring Boot starts

I have to run a few methods when Application starts, like the following:
#SpringBootApplication
public class Application implements CommandLineRunner {
private final MonitoringService monitoringService;
private final QrReaderServer qrReaderServer;
#Override
public void run(String... args) {
monitoringService.launchMonitoring();
qrReaderServer.launchServer();
}
However, only the first one is executed! And the application is started:
... Started Application in 5.21 seconds (JVM running for 6.336)
... START_MONITORING for folder: D:\results
The second one is always skipped!
If change the call order - the only the second one will be executed.
Could not find any solution for launching both at the beginning - tried #PostConstruct, ApplicationRunner, #EventListener(ApplicationReadyEvent.class)...
Looks like they are blocking each other somehow. Despite the fact that both have void type.
Monitoring launch implementation:
#Override
public void launchMonitoring() {
log.info("START_MONITORING for folder: {}", monitoringProperties.getFolder());
try {
WatchKey key;
while ((key = watchService.take()) != null) {
for (WatchEvent<?> event : key.pollEvents()) {
WatchEvent.Kind<?> kind = event.kind();
if (kind == ENTRY_CREATE) {
log.info("FILE_CREATED: {}", event.context());
// some delay for fully file upload
Thread.sleep(monitoringProperties.getFrequency());
String fullFileName = getFileName(event);
String fileName = FilenameUtils.removeExtension(fullFileName);
processResource(fullFileName, fileName);
}
}
key.reset();
}
} catch (InterruptedException e) {
log.error("interrupted exception for monitoring service", e);
} catch (IOException e) {
log.error("io exception while processing file", e);
}
}
QR Reader start (launch TCP server with Netty configuration):
#Override
public void launchServer() {
try {
ChannelFuture serverChannelFuture = serverBootstrap.bind(hostAddress).sync();
log.info("Server is STARTED : port {}", hostAddress.getPort());
serverChannel = serverChannelFuture.channel().closeFuture().sync().channel();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
} finally {
shutdownQuietly();
}
}
How to solve this issue?
Start launchMonitoring() asynchronously.
The easiest way to do this is to enable Async by adding #EnableAsync on your Application
and then annotate launchMonitoring() with #Async
Not sure if launchServer() should also be started asynchronously.
EDIT: completed Answer
No task executor bean found for async processing: no bean of type TaskExecutor and no bean named 'taskExecutor' either
By default Spring will create a SimpleAsyncTaskExecutor, but you can provide your taskExecutor
Example:
#EnableAsync
#Configuration
public class AsyncConfig implements AsyncConfigurer {
#Override
public Executor getAsyncExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.set... // your custom configs
executor.initialize();
return executor;
}
...
}

How to use a gRPC interceptor to attach/update logging MDC in a Spring-Boot app

Problem
I have a Spring-Boot application in which I am also starting a gRPC server/service. Both the servlet and gRPC code send requests to a common object to process the request. When the request comes in I want to update the logging to display a unique 'ID' so I can track the request through the system.
On the Spring side I have setup a 'Filter' which updates the logging MDC to add some data to the log request (see this example). this works fine
On the gRPC side I have created an 'ServerInterceptor' and added it to the service, while the interceptor gets called the code to update the MDC does not stick, so when a request comes through the gRPC service I do not get the ID printed in the log. I realize this has to do with the fact that I'm intercepting the call in one thread and it's being dispatched by gRPC in another, what I can't seem to figure out is how to either intercept the call in the thread doing the work or add the MDC information so it is properly propagated to the thread doing the work.
What I've tried
I have done a lot of searches and was quite surprised to not find this asked/answered, I can only assume my query skills are lacking :(
I'm fairly new to gRPC and this is the first Interceptor I'm writing. I've tried adding the interceptor several different ways (via ServerInterceptors.intercept, BindableService instance.intercept).
I've looked at LogNet's Spring Boot gRPC Starter, but I'm not sure this would solve the issue.
Here is the code I have added in my interceptor class
#Override
public <ReqT, RespT> ServerCall.Listener<ReqT> interceptCall(final ServerCall<ReqT, RespT> call, final Metadata headers, final ServerCallHandler<ReqT, RespT> next) {
try {
final String mdcData = String.format("[requestID=%s]",
UUID.randomUUID().toString());
MDC.put(MDC_DATA_KEY, mdcData);
return next.startCall(call, headers);
} finally {
MDC.clear();
}
}
Expected Result
When a request comes in via the RESTful API I see log output like this
2019-04-09 10:19:16.331 [requestID=380e28db-c8da-4e35-a097-4b8c90c006f4] INFO 87100 --- [nio-8080-exec-1] c.c.es.xxx: processing request step 1
2019-04-09 10:19:16.800 [requestID=380e28db-c8da-4e35-a097-4b8c90c006f4] INFO 87100 --- [nio-8080-exec-1] c.c.es.xxx: processing request step 2
2019-04-09 10:19:16.803 [requestID=380e28db-c8da-4e35-a097-4b8c90c006f4] INFO 87100 --- [nio-8080-exec-1] c.c.es.xxx: Processing request step 3
...
I'm hoping to get similar output when the request comes through the gRPC service.
Thanks
Since no one replied, I kept trying and came up with the following solution for my interceptCall function. I'm not 100% sure why this works, but it works for my use case.
private class LogInterceptor implements ServerInterceptor {
#Override
public <ReqT, RespT> ServerCall.Listener<ReqT> interceptCall(final ServerCall<ReqT, RespT> call,
final Metadata headers,
final ServerCallHandler<ReqT, RespT> next) {
Context context = Context.current();
final String requestId = UUID.randomUUID().toString();
return Contexts.interceptCall(context, call, headers, new ServerCallHandler<ReqT, RespT>() {
#Override
public ServerCall.Listener<ReqT> startCall(ServerCall<ReqT, RespT> call, Metadata headers) {
return new ForwardingServerCallListener.SimpleForwardingServerCallListener<ReqT>(next.startCall(call, headers)) {
/**
* The actual service call happens during onHalfClose().
*/
#Override
public void onHalfClose() {
try (final CloseableThreadContext.Instance ctc = CloseableThreadContext.put("requestID",
UUID.randomUUID().toString())) {
super.onHalfClose();
}
}
};
}
});
}
}
In my application.properties I added the following (which I already had)
logging.pattern.level=[%X] %-5level
The '%X' tells the logging system to print all of the CloseableThreadContext key/values.
Hopefully this may help someone else.
MDC stores data in ThreadLocal variable and you are right about - "I realize this has to do with the fact that I'm intercepting the call in one thread and it's being dispatched by gRPC in another". Check #Eric Anderson answer about the right way to use ThradLocal in the post -
https://stackoverflow.com/a/56842315/2478531
Here is a working example -
public class GrpcMDCInterceptor implements ServerInterceptor {
private static final String MDC_DATA_KEY = "Key";
#Override
public <R, S> ServerCall.Listener<R> interceptCall(
ServerCall<R, S> serverCall,
Metadata metadata,
ServerCallHandler<R, S> next
) {
log.info("Setting user context, metadata {}", metadata);
final String mdcData = String.format("[requestID=%s]", UUID.randomUUID().toString());
MDC.put(MDC_DATA_KEY, mdcData);
try {
return new WrappingListener<>(next.startCall(serverCall, metadata), mdcData);
} finally {
MDC.clear();
}
}
private static class WrappingListener<R>
extends ForwardingServerCallListener.SimpleForwardingServerCallListener<R> {
private final String mdcData;
public WrappingListener(ServerCall.Listener<R> delegate, String mdcData) {
super(delegate);
this.mdcData = mdcData;
}
#Override
public void onMessage(R message) {
MDC.put(MDC_DATA_KEY, mdcData);
try {
super.onMessage(message);
} finally {
MDC.clear();
}
}
#Override
public void onHalfClose() {
MDC.put(MDC_DATA_KEY, mdcData);
try {
super.onHalfClose();
} finally {
MDC.clear();
}
}
#Override
public void onCancel() {
MDC.put(MDC_DATA_KEY, mdcData);
try {
super.onCancel();
} finally {
MDC.clear();
}
}
#Override
public void onComplete() {
MDC.put(MDC_DATA_KEY, mdcData);
try {
super.onComplete();
} finally {
MDC.clear();
}
}
#Override
public void onReady() {
MDC.put(MDC_DATA_KEY, mdcData);
try {
super.onReady();
} finally {
MDC.clear();
}
}
}
}

Java 8 - Streams - exception handling is mandatory in stream processing? [duplicate]

I know how to create a reference to a method that has a String parameter and returns an int, it's:
Function<String, Integer>
However, this doesn't work if the function throws an exception, say it's defined as:
Integer myMethod(String s) throws IOException
How would I define this reference?
You'll need to do one of the following.
If it's your code, then define your own functional interface that declares the checked exception:
#FunctionalInterface
public interface CheckedFunction<T, R> {
R apply(T t) throws IOException;
}
and use it:
void foo (CheckedFunction f) { ... }
Otherwise, wrap Integer myMethod(String s) in a method that doesn't declare a checked exception:
public Integer myWrappedMethod(String s) {
try {
return myMethod(s);
}
catch(IOException e) {
throw new UncheckedIOException(e);
}
}
and then:
Function<String, Integer> f = (String t) -> myWrappedMethod(t);
or:
Function<String, Integer> f =
(String t) -> {
try {
return myMethod(t);
}
catch(IOException e) {
throw new UncheckedIOException(e);
}
};
You can actually extend Consumer (and Function etc.) with a new interface that handles exceptions -- using Java 8's default methods!
Consider this interface (extends Consumer):
#FunctionalInterface
public interface ThrowingConsumer<T> extends Consumer<T> {
#Override
default void accept(final T elem) {
try {
acceptThrows(elem);
} catch (final Exception e) {
// Implement your own exception handling logic here..
// For example:
System.out.println("handling an exception...");
// Or ...
throw new RuntimeException(e);
}
}
void acceptThrows(T elem) throws Exception;
}
Then, for example, if you have a list:
final List<String> list = Arrays.asList("A", "B", "C");
If you want to consume it (eg. with forEach) with some code that throws exceptions, you would traditionally have set up a try/catch block:
final Consumer<String> consumer = aps -> {
try {
// maybe some other code here...
throw new Exception("asdas");
} catch (final Exception ex) {
System.out.println("handling an exception...");
}
};
list.forEach(consumer);
But with this new interface, you can instantiate it with a lambda expression and the compiler will not complain:
final ThrowingConsumer<String> throwingConsumer = aps -> {
// maybe some other code here...
throw new Exception("asdas");
};
list.forEach(throwingConsumer);
Or even just cast it to be more succinct!:
list.forEach((ThrowingConsumer<String>) aps -> {
// maybe some other code here...
throw new Exception("asda");
});
Update
Looks like there's a very nice utility library part of Durian called Errors which can be used to solve this problem with a lot more flexibility. For example, in my implementation above I've explicitly defined the error handling policy (System.out... or throw RuntimeException), whereas Durian's Errors allow you to apply a policy on the fly via a large suite of utility methods. Thanks for sharing it, #NedTwigg!.
Sample usage:
list.forEach(Errors.rethrow().wrap(c -> somethingThatThrows(c)));
I think Durian's Errors class combines many of the pros of the various suggestions above.
Wrap a throwing function to a standard Java 8 functional interface.
Easily specify various policies for handling errors
When wrapping a method that returns a value, there is an important distinction between specifying a default value or rethrowing a RuntimeException.
Throwing versions of Java 8's functional interfaces
Similar to fge's answer
Standard interfaces for throwing specific exceptions
Which addresses Zoltán's concern
To include Durian in your project, you can either:
grab it from jcenter or maven central at com.diffplug.durian:durian:3.3.0
or just copy paste just two small classes into your code: Throwing.java and Errors.java
This is not specific to Java 8. You are trying to compile something equivalent to:
interface I {
void m();
}
class C implements I {
public void m() throws Exception {} //can't compile
}
Disclaimer: I haven't used Java 8 yet, only read about it.
Function<String, Integer> doesn't throw IOException, so you can't put any code in it that throws IOException. If you're calling a method that expects a Function<String, Integer>, then the lambda that you pass to that method can't throw IOException, period. You can either write a lambda like this (I think this is the lambda syntax, not sure):
(String s) -> {
try {
return myMethod(s);
} catch (IOException ex) {
throw new RuntimeException(ex);
// (Or do something else with it...)
}
}
Or, if the method you're passing the lambda to is one you wrote yourself, you can define a new functional interface and use that as the parameter type instead of Function<String, Integer>:
public interface FunctionThatThrowsIOException<I, O> {
O apply(I input) throws IOException;
}
If you don't mind to use a 3rd party lib (Vavr) you could write
CheckedFunction1<String, Integer> f = this::myMethod;
It also has the so-called Try monad which handles errors:
Try(() -> f.apply("test")) // results in a Success(Integer) or Failure(Throwable)
.map(i -> ...) // only executed on Success
...
Please read more here.
Disclaimer: I'm the creator of Vavr.
Sneaky throw idiom enables bypassing CheckedException of Lambda expression. Wrapping a CheckedException in a RuntimeException is not good for strict error handling.
It can be used as a Consumer function used in a Java collection.
Here is a simple and improved version of jib's answer.
import static Throwing.rethrow;
#Test
public void testRethrow() {
thrown.expect(IOException.class);
thrown.expectMessage("i=3");
Arrays.asList(1, 2, 3).forEach(rethrow(e -> {
int i = e.intValue();
if (i == 3) {
throw new IOException("i=" + i);
}
}));
}
This just wrapps the lambda in a rethrow. It makes CheckedException rethrow any Exception that was thrown in your lambda.
public final class Throwing {
private Throwing() {}
#Nonnull
public static <T> Consumer<T> rethrow(#Nonnull final ThrowingConsumer<T> consumer) {
return consumer;
}
/**
* The compiler sees the signature with the throws T inferred to a RuntimeException type, so it
* allows the unchecked exception to propagate.
*
* http://www.baeldung.com/java-sneaky-throws
*/
#SuppressWarnings("unchecked")
#Nonnull
public static <E extends Throwable> void sneakyThrow(#Nonnull Throwable ex) throws E {
throw (E) ex;
}
}
Find a complete code and unit tests here.
You could however create your own FunctionalInterface that throws as below..
#FunctionalInterface
public interface UseInstance<T, X extends Throwable> {
void accept(T instance) throws X;
}
then implement it using Lambdas or references as shown below.
import java.io.FileWriter;
import java.io.IOException;
//lambda expressions and the execute around method (EAM) pattern to
//manage resources
public class FileWriterEAM {
private final FileWriter writer;
private FileWriterEAM(final String fileName) throws IOException {
writer = new FileWriter(fileName);
}
private void close() throws IOException {
System.out.println("close called automatically...");
writer.close();
}
public void writeStuff(final String message) throws IOException {
writer.write(message);
}
//...
public static void use(final String fileName, final UseInstance<FileWriterEAM, IOException> block) throws IOException {
final FileWriterEAM writerEAM = new FileWriterEAM(fileName);
try {
block.accept(writerEAM);
} finally {
writerEAM.close();
}
}
public static void main(final String[] args) throws IOException {
FileWriterEAM.use("eam.txt", writerEAM -> writerEAM.writeStuff("sweet"));
FileWriterEAM.use("eam2.txt", writerEAM -> {
writerEAM.writeStuff("how");
writerEAM.writeStuff("sweet");
});
FileWriterEAM.use("eam3.txt", FileWriterEAM::writeIt);
}
void writeIt() throws IOException{
this.writeStuff("How ");
this.writeStuff("sweet ");
this.writeStuff("it is");
}
}
You can use unthrow wrapper
Function<String, Integer> func1 = s -> Unthrow.wrap(() -> myMethod(s));
or
Function<String, Integer> func2 = s1 -> Unthrow.wrap((s2) -> myMethod(s2), s1);
You can.
Extending #marcg 's UtilException and adding generic <E extends Exception> where necessary: this way, the compiler will force you again to add throw clauses and everything's as if you could throw checked exceptions natively on java 8's streams.
public final class LambdaExceptionUtil {
#FunctionalInterface
public interface Function_WithExceptions<T, R, E extends Exception> {
R apply(T t) throws E;
}
/**
* .map(rethrowFunction(name -> Class.forName(name))) or .map(rethrowFunction(Class::forName))
*/
public static <T, R, E extends Exception> Function<T, R> rethrowFunction(Function_WithExceptions<T, R, E> function) throws E {
return t -> {
try {
return function.apply(t);
} catch (Exception exception) {
throwActualException(exception);
return null;
}
};
}
#SuppressWarnings("unchecked")
private static <E extends Exception> void throwActualException(Exception exception) throws E {
throw (E) exception;
}
}
public class LambdaExceptionUtilTest {
#Test
public void testFunction() throws MyTestException {
List<Integer> sizes = Stream.of("ciao", "hello").<Integer>map(rethrowFunction(s -> transform(s))).collect(toList());
assertEquals(2, sizes.size());
assertEquals(4, sizes.get(0).intValue());
assertEquals(5, sizes.get(1).intValue());
}
private Integer transform(String value) throws MyTestException {
if(value==null) {
throw new MyTestException();
}
return value.length();
}
private static class MyTestException extends Exception { }
}
I had this problem with Class.forName and Class.newInstance inside a lambda, so I just did:
public Object uncheckedNewInstanceForName (String name) {
try {
return Class.forName(name).newInstance();
}
catch (ClassNotFoundException | InstantiationException | IllegalAccessException e) {
throw new RuntimeException(e);
}
}
Inside the lambda, instead of calling Class.forName("myClass").newInstance() I just called uncheckedNewInstanceForName ("myClass")
Create a custom return type that will propagate the checked exception. This is an alternative to creating a new interface that mirrors the existing functional interface with the slight modification of a "throws exception" on the functional interface's method.
Definition
CheckedValueSupplier
public static interface CheckedValueSupplier<V> {
public V get () throws Exception;
}
CheckedValue
public class CheckedValue<V> {
private final V v;
private final Optional<Exception> opt;
public Value (V v) {
this.v = v;
}
public Value (Exception e) {
this.opt = Optional.of(e);
}
public V get () throws Exception {
if (opt.isPresent()) {
throw opt.get();
}
return v;
}
public Optional<Exception> getException () {
return opt;
}
public static <T> CheckedValue<T> returns (T t) {
return new CheckedValue<T>(t);
}
public static <T> CheckedValue<T> rethrows (Exception e) {
return new CheckedValue<T>(e);
}
public static <V> CheckedValue<V> from (CheckedValueSupplier<V> sup) {
try {
return CheckedValue.returns(sup.get());
} catch (Exception e) {
return Result.rethrows(e);
}
}
public static <V> CheckedValue<V> escalates (CheckedValueSupplier<V> sup) {
try {
return CheckedValue.returns(sup.get());
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
Usage
// Don't use this pattern with FileReader, it's meant to be an
// example. FileReader is a Closeable resource and as such should
// be managed in a try-with-resources block or in another safe
// manner that will make sure it is closed properly.
// This will not compile as the FileReader constructor throws
// an IOException.
Function<String, FileReader> sToFr =
(fn) -> new FileReader(Paths.get(fn).toFile());
// Alternative, this will compile.
Function<String, CheckedValue<FileReader>> sToFr = (fn) -> {
return CheckedValue.from (
() -> new FileReader(Paths.get("/home/" + f).toFile()));
};
// Single record usage
// The call to get() will propagate the checked exception if it exists.
FileReader readMe = pToFr.apply("/home/README").get();
// List of records usage
List<String> paths = ...; //a list of paths to files
Collection<CheckedValue<FileReader>> frs =
paths.stream().map(pToFr).collect(Collectors.toList());
// Find out if creation of a file reader failed.
boolean anyErrors = frs.stream()
.filter(f -> f.getException().isPresent())
.findAny().isPresent();
What's going on?
A single functional interface that throws a checked exception is created (CheckedValueSupplier). This will be the only functional interface which allows checked exceptions. All other functional interfaces will leverage the CheckedValueSupplier to wrap any code that throws a checked exception.
The CheckedValue class will hold the result of executing any logic that throws a checked exception. This prevents propagation of a checked exception until the point at which code attempts to access the value that an instance of CheckedValue contains.
The problems with this approach.
We are now throwing "Exception" effectively hiding the specific type originally thrown.
We are unaware that an exception occurred until CheckedValue#get() is called.
Consumer et al
Some functional interfaces (Consumer for example) must be handled in a different manner as they don't provide a return value.
Function in lieu of Consumer
One approach is to use a function instead of a consumer, which applies when handling streams.
List<String> lst = Lists.newArrayList();
// won't compile
lst.stream().forEach(e -> throwyMethod(e));
// compiles
lst.stream()
.map(e -> CheckedValueSupplier.from(
() -> {throwyMethod(e); return e;}))
.filter(v -> v.getException().isPresent()); //this example may not actually run due to lazy stream behavior
Escalate
Alternatively, you can always escalate to a RuntimeException. There are other answers that cover escalation of a checked exception from within a Consumer.
Don't consume.
Just avoid functional interfaces all together and use a good-ole-fashioned for loop.
Another solution using a Function wrapper would be to return either an instance of a wrapper of your result, say Success, if everything went well, either an instance of, say Failure.
Some code to clarify things :
public interface ThrowableFunction<A, B> {
B apply(A a) throws Exception;
}
public abstract class Try<A> {
public static boolean isSuccess(Try tryy) {
return tryy instanceof Success;
}
public static <A, B> Function<A, Try<B>> tryOf(ThrowableFunction<A, B> function) {
return a -> {
try {
B result = function.apply(a);
return new Success<B>(result);
} catch (Exception e) {
return new Failure<>(e);
}
};
}
public abstract boolean isSuccess();
public boolean isError() {
return !isSuccess();
}
public abstract A getResult();
public abstract Exception getError();
}
public class Success<A> extends Try<A> {
private final A result;
public Success(A result) {
this.result = result;
}
#Override
public boolean isSuccess() {
return true;
}
#Override
public A getResult() {
return result;
}
#Override
public Exception getError() {
return new UnsupportedOperationException();
}
#Override
public boolean equals(Object that) {
if(!(that instanceof Success)) {
return false;
}
return Objects.equal(result, ((Success) that).getResult());
}
}
public class Failure<A> extends Try<A> {
private final Exception exception;
public Failure(Exception exception) {
this.exception = exception;
}
#Override
public boolean isSuccess() {
return false;
}
#Override
public A getResult() {
throw new UnsupportedOperationException();
}
#Override
public Exception getError() {
return exception;
}
}
A simple use case :
List<Try<Integer>> result = Lists.newArrayList(1, 2, 3).stream().
map(Try.<Integer, Integer>tryOf(i -> someMethodThrowingAnException(i))).
collect(Collectors.toList());
This problem has been bothering me as well; this is why I have created this project.
With it you can do:
final ThrowingFunction<String, Integer> f = yourMethodReferenceHere;
There are a totla of 39 interfaces defined by the JDK which have such a Throwing equivalent; those are all #FunctionalInterfaces used in streams (the base Stream but also IntStream, LongStream and DoubleStream).
And as each of them extend their non throwing counterpart, you can directly use them in lambdas as well:
myStringStream.map(f) // <-- works
The default behavior is that when your throwing lambda throws a checked exception, a ThrownByLambdaException is thrown with the checked exception as the cause. You can therefore capture that and get the cause.
Other features are available as well.
There are a lot of great responses already posted here. Just attempting to solve the problem with a different perspective. Its just my 2 cents, please correct me if I am wrong somewhere.
Throws clause in FunctionalInterface is not a good idea
I think this is probably not a good idea to enforce throws IOException because of following reasons
This looks to me like an anti-pattern to Stream/Lambda. The whole idea is that the caller will decide what code to provide and how to handle the exception. In many scenarios, the IOException might not be applicable for the client. For example, if the client is getting value from cache/memory instead of performing actual I/O.
Also, the exceptions handling in streams becomes really hideous. For example, here is my code will look like if I use your API
acceptMyMethod(s -> {
try {
Integer i = doSomeOperation(s);
return i;
} catch (IOException e) {
// try catch block because of throws clause
// in functional method, even though doSomeOperation
// might not be throwing any exception at all.
e.printStackTrace();
}
return null;
});
Ugly isn't it? Moreover, as I mentioned in my first point, that the doSomeOperation method may or may not be throwing IOException (depending on the implementation of the client/caller), but because of the throws clause in your FunctionalInterface method, I always have to write the try-catch.
What do I do if I really know this API throws IOException
Then probably we are confusing FunctionalInterface with typical Interfaces. If you know this API will throw IOException, then most probably you also know some default/abstract behavior as well. I think you should define an interface and deploy your library (with default/abstract implementation) as follows
public interface MyAmazingAPI {
Integer myMethod(String s) throws IOException;
}
But, the try-catch problem still exists for the client. If I use your API in stream, I still need to handle IOException in hideous try-catch block.
Provide a default stream-friendly API as follows
public interface MyAmazingAPI {
Integer myMethod(String s) throws IOException;
default Optional<Integer> myMethod(String s, Consumer<? super Exception> exceptionConsumer) {
try {
return Optional.ofNullable(this.myMethod(s));
} catch (Exception e) {
if (exceptionConsumer != null) {
exceptionConsumer.accept(e);
} else {
e.printStackTrace();
}
}
return Optional.empty();
}
}
The default method takes the consumer object as argument, which will be responsible to handle the exception. Now, from client's point of view, the code will look like this
strStream.map(str -> amazingAPIs.myMethod(str, Exception::printStackTrace))
.filter(Optional::isPresent)
.map(Optional::get).collect(toList());
Nice right? Of course, logger or other handling logic could be used instead of Exception::printStackTrace.
You can also expose a method similar to https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#exceptionally-java.util.function.Function- . Meaning that you can expose another method, which will contain the exception from previous method call. The disadvantage is that you are now making your APIs stateful, which means that you need to handle thread-safety and which will be eventually become a performance hit. Just an option to consider though.
By default, Java 8 Function does not allow to throw exception and as suggested in multiple answers there are many ways to achieve it, one way is:
#FunctionalInterface
public interface FunctionWithException<T, R, E extends Exception> {
R apply(T t) throws E;
}
Define as:
private FunctionWithException<String, Integer, IOException> myMethod = (str) -> {
if ("abc".equals(str)) {
throw new IOException();
}
return 1;
};
And add throws or try/catch the same exception in caller method.
I use an overloaded utility function called unchecked() which handles multiple use-cases.
SOME EAMPLE USAGES
unchecked(() -> new File("hello.txt").createNewFile());
boolean fileWasCreated = unchecked(() -> new File("hello.txt").createNewFile());
myFiles.forEach(unchecked(file -> new File(file.path).createNewFile()));
SUPPORTING UTILITIES
public class UncheckedUtils {
#FunctionalInterface
public interface ThrowingConsumer<T> {
void accept(T t) throws Exception;
}
#FunctionalInterface
public interface ThrowingSupplier<T> {
T get() throws Exception;
}
#FunctionalInterface
public interface ThrowingRunnable {
void run() throws Exception;
}
public static <T> Consumer<T> unchecked(
ThrowingConsumer<T> throwingConsumer
) {
return i -> {
try {
throwingConsumer.accept(i);
} catch (Exception ex) {
throw new RuntimeException(ex);
}
};
}
public static <T> T unchecked(
ThrowingSupplier<T> throwingSupplier
) {
try {
return throwingSupplier.get();
} catch (Exception ex) {
throw new RuntimeException(ex);
}
}
public static void unchecked(
ThrowingRunnable throwing
) {
try {
throwing.run();
} catch (Exception ex) {
throw new RuntimeException(ex);
}
}
}
You can use ET for this. ET is a small Java 8 library for exception conversion/translation.
With ET it looks like this:
// Do this once
ExceptionTranslator et = ET.newConfiguration().done();
...
// if your method returns something
Function<String, Integer> f = (t) -> et.withReturningTranslation(() -> myMethod(t));
// if your method returns nothing
Consumer<String> c = (t) -> et.withTranslation(() -> myMethod(t));
ExceptionTranslator instances are thread safe an can be shared by multiple components. You can configure more specific exception conversion rules (e.g. FooCheckedException -> BarRuntimeException) if you like.
If no other rules are available, checked exceptions are automatically converted to RuntimeException.
(Disclaimer: I am the author of ET)
If you don't mind using a third party library, with cyclops-react, a library I contribute to, you can use the FluentFunctions API to write
Function<String, Integer> standardFn = FluentFunctions.ofChecked(this::myMethod);
ofChecked takes a jOOλ CheckedFunction and returns the reference softened back to a standard (unchecked) JDK java.util.function.Function.
Alternatively you can keep working with the captured function via the FluentFunctions api!
For example to execute your method, retrying it up to 5 times and logging it's status you can write
FluentFunctions.ofChecked(this::myMethod)
.log(s->log.debug(s),e->log.error(e,e.getMessage())
.try(5,1000)
.apply("my param");
What I'm doing is to allow the user to give the value he actually want in case of exception .
So I've something looking like this
public static <T, R> Function<? super T, ? extends R> defaultIfThrows(FunctionThatThrows<? super T, ? extends R> delegate, R defaultValue) {
return x -> {
try {
return delegate.apply(x);
} catch (Throwable throwable) {
return defaultValue;
}
};
}
#FunctionalInterface
public interface FunctionThatThrows<T, R> {
R apply(T t) throws Throwable;
}
And this can then be call like :
defaultIfThrows(child -> child.getID(), null)
Use Jool Library or say jOOλ library from JOOQ. It not only provides unchecked exception handled interfaces but also provides Seq class with lots of useful methods.
Also, it contains Functional Interfaces with up to 16 parameters. Also, it provides Tuple class which is used in different scenarios.
Jool Git Link
Specifically in library lookup for org.jooq.lambda.fi.util.function package. It contains all the Interfaces from Java-8 with Checked prepended. See below for reference:-
If you have lombok, you can annotate your method with #SneakyThrows
SneakyThrow does not silently swallow, wrap into RuntimeException, or otherwise modify any exceptions of the listed checked exception types. The JVM does not check for the consistency of the checked exception system; javac does, and this annotation lets you opt out of its mechanism.
https://projectlombok.org/features/SneakyThrows
Several of the offered solutions use a generic argument of E to pass in the type of the exception which gets thrown.
Take that one step further, and rather than passing in the type of the exception, pass in a Consumer of the type of exception, as in...
Consumer<E extends Exception>
You might create several re-usable variations of Consumer<Exception> which would cover the common exception handling needs of your application.
I will do something generic:
public interface Lambda {
#FunctionalInterface
public interface CheckedFunction<T> {
T get() throws Exception;
}
public static <T> T handle(CheckedFunction<T> supplier) {
try {
return supplier.get();
} catch (Exception exception) {
throw new RuntimeException(exception);
}
}
}
usage:
Lambda.handle(() -> method());
I'm the author of a tiny lib with some generic magic to throw any Java Exception anywhere without the need of catching them nor wrapping them into RuntimeException.
Usage:
unchecked(() -> methodThrowingCheckedException())
public class UncheckedExceptions {
/**
* throws {#code exception} as unchecked exception, without wrapping exception.
*
* #return will never return anything, return type is set to {#code exception} only to be able to write <code>throw unchecked(exception)</code>
* #throws T {#code exception} as unchecked exception
*/
#SuppressWarnings("unchecked")
public static <T extends Throwable> T unchecked(Exception exception) throws T {
throw (T) exception;
}
#FunctionalInterface
public interface UncheckedFunction<R> {
R call() throws Exception;
}
/**
* Executes given function,
* catches and rethrows checked exceptions as unchecked exceptions, without wrapping exception.
*
* #return result of function
* #see #unchecked(Exception)
*/
public static <R> R unchecked(UncheckedFunction<R> function) {
try {
return function.call();
} catch (Exception e) {
throw unchecked(e);
}
}
#FunctionalInterface
public interface UncheckedMethod {
void call() throws Exception;
}
/**
* Executes given method,
* catches and rethrows checked exceptions as unchecked exceptions, without wrapping exception.
*
* #see #unchecked(Exception)
*/
public static void unchecked(UncheckedMethod method) {
try {
method.call();
} catch (Exception e) {
throw unchecked(e);
}
}
}
source: https://github.com/qoomon/unchecked-exceptions-java
For me the preferred solution is to use Lombok. It is a nice library anyway.
Instead of:
Integer myMethod(String s) throws IOException
you will have
import lombok.SneakyThrows;
#SneakyThrows
Integer myMethod(String s)
The exception is still thrown but you do not need to declare it with throws.
public void frankTest() {
int pageId= -1;
List<Book> users= null;
try {
//Does Not Compile: Object page=DatabaseConnection.getSpringConnection().queryForObject("SELECT * FROM bookmark_page", (rw, n) -> new Portal(rw.getInt("id"), "", users.parallelStream().filter(uu -> uu.getVbid() == rw.getString("user_id")).findFirst().get(), rw.getString("name")));
//Compiles:
Object page= DatabaseConnection.getSpringConnection().queryForObject("SELECT * FROM bookmark_page", (rw, n) -> {
try {
final Book bk= users.stream().filter(bp -> {
String name= null;
try {
name = rw.getString("name");
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return bp.getTitle().equals(name);
}).limit(1).collect(Collectors.toList()).get(0);
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return new Portal(rw.getInt("id"), "", users.get(0), rw.getString("name"));
} );
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}

how to read the request in chunks in httpcomponents NIO server?

I am using httpcomponenets nio server to handle post request.
Below is the sample code. It gets the complete data in byte array using EntityUtils.toByteArray(). This fails if the requester sends a large file.
I couldnt figure out how to read the data in the request in chunks.
HttpEntity.getContent().read() always returns null
public static void main(String[] args) throws Exception {
int port = 8280;
// Create HTTP protocol processing chain
HttpProcessor httpproc = HttpProcessorBuilder.create()
.add(new ResponseDate())
.add(new ResponseServer("Test/1.1"))
.add(new ResponseContent())
.add(new ResponseConnControl()).build();
// Create request handler registry
UriHttpAsyncRequestHandlerMapper reqistry = new UriHttpAsyncRequestHandlerMapper();
// Register the default handler for all URIs
reqistry.register("/test*", new RequestHandler());
// Create server-side HTTP protocol handler
HttpAsyncService protocolHandler = new HttpAsyncService(httpproc, reqistry) {
#Override
public void connected(final NHttpServerConnection conn) {
System.out.println(conn + ": connection open");
super.connected(conn);
}
#Override
public void closed(final NHttpServerConnection conn) {
System.out.println(conn + ": connection closed");
super.closed(conn);
}
};
// Create HTTP connection factory
NHttpConnectionFactory<DefaultNHttpServerConnection> connFactory;
connFactory = new DefaultNHttpServerConnectionFactory(
ConnectionConfig.DEFAULT);
// Create server-side I/O event dispatch
IOEventDispatch ioEventDispatch = new DefaultHttpServerIODispatch(protocolHandler, connFactory);
// Set I/O reactor defaults
IOReactorConfig config = IOReactorConfig.custom()
.setIoThreadCount(1)
.setSoTimeout(3000)
.setConnectTimeout(3000)
.build();
// Create server-side I/O reactor
ListeningIOReactor ioReactor = new DefaultListeningIOReactor(config);
try {
// Listen of the given port
ioReactor.listen(new InetSocketAddress(port));
// Ready to go!
ioReactor.execute(ioEventDispatch);
} catch (InterruptedIOException ex) {
System.err.println("Interrupted");
} catch (IOException e) {
System.err.println("I/O error: " + e.getMessage());
}
System.out.println("Shutdown");
}
public static class RequestHandler implements HttpAsyncRequestHandler<HttpRequest> {
public void handleInternal(HttpRequest httpRequest, HttpResponse httpResponse, HttpContext httpContext) throws HttpException, IOException {
HttpEntity entity = null;
if (httpRequest instanceof HttpEntityEnclosingRequest)
entity = ((HttpEntityEnclosingRequest)httpRequest).getEntity();
byte[] data;
if (entity == null) {
data = new byte [0];
} else {
data = EntityUtils.toByteArray(entity);
}
System.out.println(new String(data));
httpResponse.setEntity(new StringEntity("success response"));
}
#Override public HttpAsyncRequestConsumer<HttpRequest> processRequest(HttpRequest request, HttpContext context) throws HttpException, IOException {
return new BasicAsyncRequestConsumer();
}
#Override
public void handle(HttpRequest request, HttpAsyncExchange httpExchange, HttpContext context) throws HttpException, IOException {
HttpResponse response = httpExchange.getResponse();
handleInternal(request, response, context);
httpExchange.submitResponse(new BasicAsyncResponseProducer(response));
}
}
Please consider implementing a custom AbstractAsyncRequestConsumer instead of BasicAsyncRequestConsumer if you want to have full control over request processing.
You might use these classes as a starting point [1][2]. Please note these are response consumers though one can use the same approach to create custom request consumers:
[1] http://hc.apache.org/httpcomponents-asyncclient-4.1.x/httpasyncclient/xref/org/apache/http/nio/client/methods/AsyncCharConsumer.html
[2] http://hc.apache.org/httpcomponents-asyncclient-4.1.x/httpasyncclient/xref/org/apache/http/nio/client/methods/AsyncByteConsumer.html

Resources