Using Springboot 1.5.x, Spring Cloud, and JAX-RS:
I could use a second pair of eyes since it is not clear to me whether the Spring configured, Javanica HystrixCommand works for all use cases or whether I may have an error in my code. Below is an approximation of what I'm doing, the code below will not actually compile.
From below WebService lives in a library with separate package path to the main application(s). Meanwhile MyWebService lives in the application that is in the same context path as the Springboot application. Also MyWebService is functional, no issues there. This just has to do with the visibility of HystrixCommand annotation in regards to Springboot based configuration.
At runtime, what I notice is that when a code like the one below runs, I do see "commandKey=A" in my response. This one I did not quite expect since it's still running while the data is obtained. And since we log the HystrixRequestLog, I also see this command key in my logs.
But all the other Command keys are not visible at all, regardless of where I place them in the file. If I remove CommandKey-A then no commands are visible whatsoever.
Thoughts?
// Example WebService that we use as a shared component for performing a backend call that is the same across different resources
#RequiredArgsConstructor
#Accessors(fluent = true)
#Setter
public abstract class WebService {
private final #Nonnull Supplier<X> backendFactory;
#Setter(AccessLevel.PACKAGE)
private #Nonnull Supplier<BackendComponent> backendComponentSupplier = () -> new BackendComponent();
#GET
#Produces("application/json")
#HystrixCommand(commandKey="A")
public Response mainCall() {
Object obj = new Object();
try {
otherCommandMethod();
} catch (Exception commandException) {
// do nothing (for this example)
}
// get the hystrix request information so that we can determine what was executed
Optional<Collection<HystrixInvokableInfo<?>>> executedCommands = hystrixExecutedCommands();
// set the hystrix data, viewable in the response
obj.setData("hystrix", executedCommands.orElse(Collections.emptyList()));
if(hasError(obj)) {
return Response.serverError()
.entity(obj)
.build();
}
return Response.ok()
.entity(healthObject)
.build();
}
#HystrixCommand(commandKey="B")
private void otherCommandMethod() {
backendComponentSupplier
.get()
.observe()
.toBlocking()
.subscribe();
}
Optional<Collection<HystrixInvokableInfo<?>>> hystrixExecutedCommands() {
Optional<HystrixRequestLog> hystrixRequest = Optional
.ofNullable(HystrixRequestLog.getCurrentRequest());
// get the hystrix executed commands
Optional<Collection<HystrixInvokableInfo<?>>> executedCommands = Optional.empty();
if (hystrixRequest.isPresent()) {
executedCommands = Optional.of(hystrixRequest.get()
.getAllExecutedCommands());
}
return executedCommands;
}
#Setter
#RequiredArgsConstructor
public class BackendComponent implements ObservableCommand<Void> {
#Override
#HystrixCommand(commandKey="Y")
public Observable<Void> observe() {
// make some backend call
return backendFactory.get()
.observe();
}
}
}
// then later this component gets configured in the specific applications with sample configuraiton that looks like this:
#SuppressWarnings({ "unchecked", "rawtypes" })
#Path("resource/somepath")
#Component
public class MyWebService extends WebService {
#Inject
public MyWebService(Supplier<X> backendSupplier) {
super((Supplier)backendSupplier);
}
}
There is an issue with mainCall() calling otherCommandMethod(). Methods with #HystrixCommand can not be called from within the same class.
As discussed in the answers to this question this is a limitation of Spring's AOP.
Related
I work with spring boot and axon example, i implement the snapshot feature, with the below code is working fine, after 3 events i found the data in the table snapshot_event_entry in the database
#Configuration
#AutoConfigureAfter(value = { AxonAutoConfiguration.class })
public class AxonConfig {
#Bean
public SnapshotTriggerDefinition catalogSnapshotTrigger(Snapshotter snapshotter) {
return new EventCountSnapshotTriggerDefinition(snapshotter, 3);
}
}
#Aggregate(snapshotTriggerDefinition = "catalogSnapshotTrigger")
public class CatalogAggregate { }
My question, is there a method to do a snapshot in demand? That means i want to implement an api to do the snapshot, not automatically after 3 events
there is nothing already in place.
One way to implement what you need is to create a dedicated Command, eg PerformShapshotCmd, that will carry the aggregateId information, and a #CommandHandler into your Aggregate. You could then let Spring autowire the Snapshotter instance bean, and call for the scheduleSnapshot(Class<?> aggregateType, String aggregateIdentifier) method.
Below some code snippet that could guide you.
data class PerformShapshotCmd(#TargetAggregateIdentifier val id: String)
#CommandHandler
public void handle(PerformShapshotCmd cmd, Snapshotter snapshotter) {
logger.debug("handling {}", cmd);
snapshotter.scheduleSnapshot(this.getClass(), cmd.getId());
}
You should also define one Bean of type Snapshotter into your config
#Bean
public SpringAggregateSnapshotterFactoryBean snapshotter() {
SpringAggregateSnapshotterFactoryBean springAggregateSnapshotterFactoryBean = new SpringAggregateSnapshotterFactoryBean();
//Setting async executors
springAggregateSnapshotterFactoryBean.setExecutor(Executors.newSingleThreadExecutor());
return springAggregateSnapshotterFactoryBean;
}
Please note that the first argument of your commandHandler needs to be the command, otherwise the framework will complain with an exception at startup time.
I have a SOAP service implemented using Spring. The service accepts the username/password via the <UsernameToken> element in the SOAP header. That all works fine.
However, the client consuming this SOAP service requests that I include in the WSDL file that the username/password is required via a <Policy> in the <wsdl:binding> element.
I have a method in my code like:
#EnableWs
#Configuration
public class SoapWebServiceConfig extends WsConfigurerAdapter {
#Bean
public DefaultWsdl11Definition defaultWsdl11Definition(XsdSchema s) {
var wsdl11Definition = new DefaultWsdl11Definition();
wsdl11Definition.setPortTypeName("MyPort");
wsdl11Definition.setLocationUri("/soap");
wsdl11Definition.setTargetNamespace("http://myservice.com/");
wsdl11Definition.setSchema(s);
return wsdl11Definition;
}
...
}
This produces a WSDL file, but without information in it that the <UsernameToken> is required.
How can I persuade Spring to include the necessary <Policy> information in the WSDL file?
I have looked at the Spring documentation but was unable to determine the incantation necessary. I have looked through the Spring source code but was also not able to see an obvious hook to add the extra information. What I am looking for is something similar to https://stackoverflow.com/a/19726325/220627 but for Spring.
What I do now is a bit of a hack, but I'll put it here in case it's helpful for someone. I'll leave the question open in the hope that someone comes along with a better solution!
private Source addWsdlUsernameTokenPolicy(#Source xml) {
try {
var originalDocumentDOMResult = new DOMResult();
var transformer = TransformerFactory.newInstance().newTransformer();
transformer.transform(xml, originalDocumentDOMResult);
var doc = (Document) originalDocumentDOMResult.getNode();
// ... alter the document using DOM methods as necessary ...
return new DOMSource(doc);
}
catch (TransformerConfigurationException | TransformerException e) {
throw new RuntimeException(e);
}
}
#Bean
public DefaultWsdl11Definition defaultWsdl11Definition(XsdSchema s) {
var wsdl11Definition = new DefaultWsdl11Definition() {
// This is the trick. The getSource() returns the XML of the WSDL.
// You can convert this "Source" (of XML) into a DOM XML structure
// then alter the XML as you like with normal DOM operations.
#Override public Source getSource() {
return addWsdlUsernameTokenPolicy(super.getSource());
}
};
...
return wsdl11Definition;
}
I have a service that requires a configuration
#Component(service=InstrumenterService.class ,configurationPid = "InstrumenterService", configurationPolicy = ConfigurationPolicy.REQUIRE, scope = ServiceScope.PROTOTYPE)
public class InstrumenterService
This service is referenced inside another service :
#Component(service = SampleService.class, scope = ServiceScope.PROTOTYPE)
public class SampleService {
#Reference(cardinality = ReferenceCardinality.OPTIONAL, scope = ReferenceScope.PROTOTYPE_REQUIRED, policyOption = ReferencePolicyOption.GREEDY)
InstrumenterService coverageInstrumenter;
public boolean hasInstrumenter() {
if(coverageInstrumenter == null)
return false;
return true;
}
}
This SampleService is used inside a Main class hooked to the main osgi thread.
I'm using ComponentServiceObjects as I want to create on demand SampleServices.
#Component(immediate = true, property = "main.thread=true")
public class Main implements Runnable {
#Reference
ConfigurationAdmin cfgAdm;
#Reference(scope = ReferenceScope.PROTOTYPE_REQUIRED)
private ComponentServiceObjects<SampleService> sampleServices;
public void run() {
if (cfgAdm != null) {
Configuration configuration;
try {
configuration = cfgAdm.getConfiguration("InstrumenterService", "?");
Hashtable<String, Object> props = new Hashtable<>();
props.put("some_prop", "some_value");
configuration.update(props);
} catch (IOException e1) {
e1.printStackTrace();
}
}
SampleService servicess = sampleServices.getService();
System.out.println(servicess.hasInstrumenter());
}
}
The problem I have is that the configuration set by the ConfigurationAdmin is not visible in the InstrumenterService unless I put a Thread.sleep(500); command after calling the configuration.update.
I'm not really confortable using a Thread.sleep command to ensure the configuration update is visible.
Is there an API to check that the configuration has been updated and is available to use ?
Thanks to Neil I was able to find a workable solution.
I used a ServiceTracker after the configuration was set to wait for the service:
BundleContext bundleContext = FrameworkUtil.getBundle(getClass()).getBundleContext();
ServiceTracker serviceTracker = new ServiceTracker(bundleContext, InstrumenterService.class.getName(), null);
serviceTracker.open();
try {
serviceTracker.waitForService(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
serviceTracker.close();
The reason I needed ConfigurationAdmin in the first place is because there is an interface IInstrumenter which can be implemented by many different classes.
The name of this instrumenter is set in the ConfigurationAdmin and then further on in other services the required instrumeter service is fetch "automagically".
This way any number of instrumenter could be added to the application and only the name of the instrumeter needs to be known in order for it to be used.
I want to mention also that with OSGI we managed to split our monolith legacy application in more modules (~15) and they do not depend directly on each other but use an API layer.
Thanks again for the good job you are doing with OSGI.
As clarified in the comments, this code is not exactly realistic. In production code there is not normally a requirement to update a configuration record and then immediately obtain a service published by a component. This is because any such code makes too many assumptions about the effect of the configuration update.
A call to getServiceReference and getService returns only a snapshot of the service registry state at a particular instant. It is inherently unreliable to call getService expecting it to return a value.
In reality, we always use a pattern where we react to being notified of the existence of the service. This can be done in various ways, including ServiceListener and ServiceTracker, but the simplest is to write a component with a reference, e.g.:
#Component
public class MyComponent {
#Reference
SampleService service;
public void doSomething() {
println(service.hasInstrumenter());
}
}
This component has a mandatory reference to SampleService and will only be activated only when an instance of SampleService is available.
I am trying to set up a service that has both a REST (POST) endpoint and a Kafka endpoint, both of which should take a JSON representation of the request object (let's call it Foo). I would want to make sure that the Foo object is valid (via JSR-303 or whatever). So Foo might look like:
public class Foo {
#Max(10)
private int bar;
// Getter and setter boilerplate
}
Setting up the REST endpoint is easy:
#PostMapping(value = "/", produces = MediaType.APPLICATION_JSON_VALUE)
public ResponseEntity<String> restEndpoint(#Valid #RequestBody Foo foo) {
// Do stuff here
}
and if I POST, { "bar": 9 } it processes the request, but if I post: { "bar": 99 } I get a BAD REQUEST. All good so far!
The Kafka endpoint is easy to create (along with adding a StringJsonMessageConverter() to my KafkaListenerContainerFactory so that I get JSON->Object conversion:
#KafkaListener(topics = "fooTopic")
public void kafkaEndpoint(#Valid #Payload Foo foo) {
// I shouldn't get here with an invalid object!!!
logger.debug("Successfully processed the object" + foo);
// But just to make sure, let's see if hand-validating it works
Validator validator = localValidatorFactoryBean.getValidator();
Set<ConstraintViolation<SlackMessage>> errors = validator.validate(foo);
if (errors.size() > 0) {
logger.debug("But there were validation errors!" + errors);
}
}
But no matter what I try, I can still pass invalid requests in and they process without error.
I've tried both #Valid and #Validated. I've tried adding a MethodValidationPostProcessor bean. I've tried adding a Validator to the KafkaListenerEndpointRegistrar (a la the EnableKafka javadoc):
#Configuration
public class MiscellaneousConfiguration implements KafkaListenerConfigurer {
private Logger logger = LoggerFactory.getLogger(this.getClass());
#Autowired
LocalValidatorFactoryBean validatorFactory;
#Override
public void configureKafkaListeners(KafkaListenerEndpointRegistrar registrar) {
logger.debug("Configuring " + registrar);
registrar.setMessageHandlerMethodFactory(kafkaHandlerMethodFactory());
}
#Bean
public MessageHandlerMethodFactory kafkaHandlerMethodFactory() {
DefaultMessageHandlerMethodFactory factory = new DefaultMessageHandlerMethodFactory();
factory.setValidator(validatorFactory);
return factory;
}
}
I've now spent a few days on this, and I'm running out of other ideas. Is this even possible (without writing validation into every one of my kakfa endpoints)?
Sorry for the delay; we are at SpringOne Platform this week.
The infrastructure currently does not pass a Validator into the payload argument resolver. Please open an issue on GitHub.
Spring kafka listener by default do not scan for #Valid for non Rest controller classes. For more details please refer this answer
https://stackoverflow.com/a/71859991/13898185
In the olden days, we had ThreadLocal for programs to carry data along with the request path since all request processing was done on that thread and stuff like Logback used this with MDC.put("requestId", getNewRequestId());
Then Scala and functional programming came along and Futures came along and with them came Local.scala (at least I know the twitter Futures have this class). Future.scala knows about Local.scala and transfers the context through all the map/flatMap, etc. etc. functionality such that I can still do Local.set("requestId", getNewRequestId()); and then downstream after it has travelled over many threads, I can still access it with Local.get(...)
Soooo, my question is in Java, can I do the same thing with the new CompletableFuture somewhere with LocalContext or some object (not sure of the name) and in this way, I can modify Logback MDC context to store it in that context instead of a ThreadLocal such that I don't lose the request id and all my logs across the thenApply, thenAccept, etc. etc. still work just fine with logging and the -XrequestId flag in Logback configuration.
EDIT:
As an example. If you have a request come in and you are using Log4j or Logback, in a filter, you will set MDC.put("requestId", requestId) and then in your app, you will log many log statements line this:
log.info("request came in for url="+url);
log.info("request is complete");
Now, in the log output it will show this:
INFO {time}: requestId425 request came in for url=/mypath
INFO {time}: requestId425 request is complete
This is using a trick of ThreadLocal to achieve this. At Twitter, we use Scala and Twitter Futures in Scala along with a Local.scala class. Local.scala and Future.scala are tied together in that we can achieve the above scenario still which is very nice and all our log statements can log the request id so the developer never has to remember to log the request id and you can trace through a single customers request response cycle with that id.
I don't see this in Java :( which is very unfortunate as there are many use cases for that. Perhaps there is something I am not seeing though?
If you come across this, just poke the thread here
http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-May/047867.html
to implement something like twitter Futures which transfer Locals (Much like ThreadLocal but transfers state).
See the def respond() method in here and how it calls Locals.save() and Locals.restort()
https://github.com/simonratner/twitter-util/blob/master/util-core/src/main/scala/com/twitter/util/Future.scala
If Java Authors would fix this, then the MDC in logback would work across all 3rd party libraries. Until then, IT WILL NOT WORK unless you can change the 3rd party library(doubtful you can do that).
My solution theme would be to (It would work with JDK 9+ as a couple of overridable methods are exposed since that version)
Make the complete ecosystem aware of MDC
And for that, we need to address the following scenarios:
When all do we get new instances of CompletableFuture from within this class? → We need to return a MDC aware version of the same rather.
When all do we get new instances of CompletableFuture from outside this class? → We need to return a MDC aware version of the same rather.
Which executor is used when in CompletableFuture class? → In all circumstances, we need to make sure that all executors are MDC aware
For that, let's create a MDC aware version class of CompletableFuture by extending it. My version of that would look like below
import org.slf4j.MDC;
import java.util.Map;
import java.util.concurrent.*;
import java.util.function.Function;
import java.util.function.Supplier;
public class MDCAwareCompletableFuture<T> extends CompletableFuture<T> {
public static final ExecutorService MDC_AWARE_ASYNC_POOL = new MDCAwareForkJoinPool();
#Override
public CompletableFuture newIncompleteFuture() {
return new MDCAwareCompletableFuture();
}
#Override
public Executor defaultExecutor() {
return MDC_AWARE_ASYNC_POOL;
}
public static <T> CompletionStage<T> getMDCAwareCompletionStage(CompletableFuture<T> future) {
return new MDCAwareCompletableFuture<>()
.completeAsync(() -> null)
.thenCombineAsync(future, (aVoid, value) -> value);
}
public static <T> CompletionStage<T> getMDCHandledCompletionStage(CompletableFuture<T> future,
Function<Throwable, T> throwableFunction) {
Map<String, String> contextMap = MDC.getCopyOfContextMap();
return getMDCAwareCompletionStage(future)
.handle((value, throwable) -> {
setMDCContext(contextMap);
if (throwable != null) {
return throwableFunction.apply(throwable);
}
return value;
});
}
}
The MDCAwareForkJoinPool class would look like (have skipped the methods with ForkJoinTask parameters for simplicity)
public class MDCAwareForkJoinPool extends ForkJoinPool {
//Override constructors which you need
#Override
public <T> ForkJoinTask<T> submit(Callable<T> task) {
return super.submit(MDCUtility.wrapWithMdcContext(task));
}
#Override
public <T> ForkJoinTask<T> submit(Runnable task, T result) {
return super.submit(wrapWithMdcContext(task), result);
}
#Override
public ForkJoinTask<?> submit(Runnable task) {
return super.submit(wrapWithMdcContext(task));
}
#Override
public void execute(Runnable task) {
super.execute(wrapWithMdcContext(task));
}
}
The utility methods to wrap would be such as
public static <T> Callable<T> wrapWithMdcContext(Callable<T> task) {
//save the current MDC context
Map<String, String> contextMap = MDC.getCopyOfContextMap();
return () -> {
setMDCContext(contextMap);
try {
return task.call();
} finally {
// once the task is complete, clear MDC
MDC.clear();
}
};
}
public static Runnable wrapWithMdcContext(Runnable task) {
//save the current MDC context
Map<String, String> contextMap = MDC.getCopyOfContextMap();
return () -> {
setMDCContext(contextMap);
try {
return task.run();
} finally {
// once the task is complete, clear MDC
MDC.clear();
}
};
}
public static void setMDCContext(Map<String, String> contextMap) {
MDC.clear();
if (contextMap != null) {
MDC.setContextMap(contextMap);
}
}
Below are some guidelines for usage:
Use the class MDCAwareCompletableFuture rather than the class CompletableFuture.
A couple of methods in the class CompletableFuture instantiates the self version such as new CompletableFuture.... For such methods (most of the public static methods), use an alternative method to get an instance of MDCAwareCompletableFuture. An example of using an alternative could be rather than using CompletableFuture.supplyAsync(...), you can choose new MDCAwareCompletableFuture<>().completeAsync(...)
Convert the instance of CompletableFuture to MDCAwareCompletableFuture by using the method getMDCAwareCompletionStage when you get stuck with one because of say some external library which returns you an instance of CompletableFuture. Obviously, you can't retain the context within that library but this method would still retain the context after your code hits the application code.
While supplying an executor as a parameter, make sure that it is MDC Aware such as MDCAwareForkJoinPool. You could create MDCAwareThreadPoolExecutor by overriding execute method as well to serve your use case. You get the idea!
You can find a detailed explanation of all of the above here in a post about the same.