I am struggling with configuring a "custom" ObjectMapper to be used by the Spring Integration DSL transformers.
I receive an java.time.Instant json representations that I would like to parse to object properties. i.e:
{"type": "TEST", "source":"TEST", "timestamp":{"epochSecond": 1454503381, "nano": 335000000}}
The message is a kafka message which raises a question: Should I write a custom serializer implementing Kafka encoders/decoders in order to be able to transform the kafka message to the right object or spring-integration have to do this automatically?
fw/dependencies and version:
Spring Boot - 1.3.2.RELEASE
Spring Integration Java Dsl - 1.1.1.RELEASE
FasterXml Jackson - 2.6.5
I've added this Java Configuration to the project following the Jackson documentation:
https://github.com/FasterXML/jackson-datatype-jsr310
#Configuration
public class IntegrationConfiguration {
#Bean
public JsonObjectMapper<JsonNode, JsonParser> jsonObjectMapper() {
ObjectMapper mapper = new ObjectMapper();
mapper.registerModule(new JavaTimeModule());
return new Jackson2JsonObjectMapper(mapper);
}
}
and the following Jackson JSR-310 artefact as well:
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jsr310</artifactId>
<version>2.6.5</version>
</dependency>
Based on this post on the Spring blog I don't even have to register the new Java8 time module.
https://spring.io/blog/2014/12/02/latest-jackson-integration-improvements-in-spring#jackson-modules
This is the exception I got:
Caused by: com.fasterxml.jackson.databind.JsonMappingException: No suitable constructor found for type [simple type, class java.time.Instant]: can not instantiate from JSON object (missing default constructor or creator, or perhaps need to add/enable type information?)
at [Source: {"type":"TEST","source":"TEST","timestamp":{"epochSecond":1454503381,"nano":335000000}}; line: 1, column: 71] (through reference chain: my.app.MyDto["timestamp"])
at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:148)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1106)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:296)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:133)
at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:520)
at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:95)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:258)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:125)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3736)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2764)
at org.springframework.integration.support.json.Jackson2JsonObjectMapper.fromJson(Jackson2JsonObjectMapper.java:75)
at org.springframework.integration.support.json.Jackson2JsonObjectMapper.fromJson(Jackson2JsonObjectMapper.java:44)
at org.springframework.integration.support.json.AbstractJacksonJsonObjectMapper.fromJson(AbstractJacksonJsonObjectMapper.java:56)
at org.springframework.integration.json.JsonToObjectTransformer.doTransform(JsonToObjectTransformer.java:78)
at org.springframework.integration.transformer.AbstractTransformer.transform(AbstractTransformer.java:33)
... 74 more
RESOLUTION:
The problem was that I expected that Spring will detect the jackson-datatype-jsr310 archetype and register the JavaTimeModule, but it doesn't which is totally fine.
There are two way we can fix this:
1. The accepted answer if we use Spring Boot with Spring Integration as is.
2. If using the Spring Integration Dsl, just keep the IntegrationConfiguration class with the jsonObjectMapper() bean and use it like that:
#Autowired
private JsonObjectMapper jsonObjectMapper;
return IntegrationFlows
.from(inboundChannel())
.transform(Transformers.fromJson(myDto.class, jsonObjectMapper))
...
There is nothing to do with the Spring Boot on the matter to force Spring Integration to use that.
You just need to configure JsonToObjectTransformer with that your jsonObjetMapper():
#Bean
#Transformer(inputChannel="input", outputChannel="output")
JsonToObjectTransformer jsonToObjectTransformer() {
return new JsonToObjectTransformer(jsonObjectMapper());
}
Although there is no reason to register JsonObjectMapper as a bean.
Have you defined an encoder to your channel adapter?
You should always use an encoder for whichever adapter you're using, inbound channel adapter or message drive channel adapter.
In your case StringEncoder should solve the problem.
<bean id="myEncoder" class="org.springframework.integration.kafka.serializer.common.StringEncoder"/>
Related
We are attempting to move away from spring cloud aws to the new io.awspring.cloud project. The documentation states:
The AmazonSQSAsync client is automatically created and passed to the template’s constructor based on the provided credentials.
I am attempting to run this locally. In my pom.xml I have added the dependency:
<dependency>
<groupId>io.awspring.cloud</groupId>
<artifactId>spring-cloud-aws-messaging</artifactId>
<version>${spring.cloud.aws}</version>
</dependency>
And I removed the code to create an AmazonSQSAsync client. As per the documentation, I still define a QueueMessagingTemplate
import io.awspring.cloud.messaging.core.QueueMessagingTemplate;
import com.amazonaws.services.sqs.AmazonSQSAsync;
#Bean
#Qualifier
public QueueMessagingTemplate queueMessagingTemplateFifoSupport(AmazonSQSAsync amazonSQSAsync,
ObjectMapper objectMapper) {
MappingJackson2MessageConverter converter = new MappingJackson2MessageConverter();
converter.setSerializedPayloadClass(String.class);
converter.setObjectMapper(objectMapper);
QueueMessagingTemplate messagingTemplate = new QueueMessagingTemplate(amazonSQSAsync);
messagingTemplate.setMessageConverter(converter);
return messagingTemplate;
}
The package namespace was updated to use io.awspring. When running the app I get the error:
No qualifying bean of type 'com.amazonaws.services.sqs.AmazonSQSAsync' available
My belief is that Spring should supply this client automatically. Am I missing something?
In my application properties I have defined:
cloud.aws.credentials.access-key
cloud.aws.credentials.secret-key
cloud.aws.region.static
I was missing a dependency:
<dependency>
<groupId>io.awspring.cloud</groupId>
<artifactId>spring-cloud-aws-autoconfigure</artifactId>
<version>${spring.cloud.aws}</version>
</dependency>
which is necessary for automatic creation of default clients.
On a side note- best to avoid using this for now as you will be locked in to v1 of the Java AWS SDK until you can upgrade io.awspring to a version compatible with both a newer version of spring boot and aws
The OpenApi documentation says that it supports micrometer. How does the integration works? I could not find anything except this little documentation.
I have a FeignClient in a spring boot application
#FeignClient(name = "SomeService", url = "xxx", configuration = FeignConfiguration.class)
public interface SomeService {
#GET
#Path("/something")
Something getSomething();
}
with the configuration
public class FeignConfiguration {
#Bean
public Capability capability() {
return new MicrometerCapability();
}
}
and the micrometer integration as a dependency
<dependency>
<groupId>io.github.openfeign</groupId>
<artifactId>feign-micrometer</artifactId>
<version>10.12</version>
</dependency>
The code makes a call but I could not find any new metrics via the actuator overview, expecting some general information about my HTTP requests. What part is missing?
Update
I added the support for this to spring-cloud-openfeign. After the next release (2020.0.2), if micrometer is set-up, the only thing you need to do is putting feign-micrometer onto your classpath.
Old answer
I'm not sure if you do but I recommend to use spring-cloud-openfeign which autoconfigures Feign components for you. Unfortunately, it seems it does not autoconfigure Capability (that's one reason why your solution does not work) so you need to do it manually, please see the docs how to do it.
I was able to make this work combining the examples in the OpenFeign and Spring Cloud OpenFeign docs:
#Import(FeignClientsConfiguration.class)
class FooController {
private final FooClient fooClient;
public FooController(Decoder decoder, Encoder encoder, Contract contract, MeterRegistry meterRegistry) {
this.fooClient = Feign.builder()
.encoder(encoder)
.decoder(decoder)
.contract(contract)
.addCapability(new MicrometerCapability(meterRegistry))
.target(FooClient.class, "https://PROD-SVC");
}
}
What I did:
Used spring-cloud-openfeign
Added feign-micrometer (see feign-bom)
Created the client in the way you can see above
Importing FeignClientsConfiguration and passing MeterRegistry to MicrometerCapability are vital
After these, and calling the client, I had new metrics:
feign.Client
feign.Feign
feign.codec.Decoder
feign.codec.Decoder.response_size
Metrics http_server_requests_seconds_count in Spring Boot application with version 2.0.8.Release exposed using spring actuator contains URI as:
"UNKNOWN".
Spring Boot application is using cxf-spring-boot-starter-jaxrs for exposing rest endpoints.
I have added micrometer-registry-prometheus dependency in my project.
http_server_requests_seconds_count{exception="None",method="POST",status="200",uri="UNKNOWN",} 2.0
I have tried adding micrometer-jersey2 dependency in my project.
Actual
http_server_requests_seconds_count{exception="None",method="POST",status="200",uri="UNKNOWN",} 2.0
Expected:
http_server_requests_seconds_count{exception="None",method="GET",status="200",uri="/sayHello",} 2.0
After the clarification in OP comments (CXF being another JAX-RS implementation): There's currently no support in Micrometer to handle CXF requests. It (Spring WebMvc) can't extract the optionally parameterized request url and in that case falls back to UNKNOWN. (Otherwise this could lead to a metrics explosion if your CXF endpoints provide some highly parameterizable URLs which get a lot of traffic.)
So you could have a look at the micrometer-jersey2 implementation and derive a micrometer-cxf implementation ;) (Or if not already the case (use the search) - open up an issue with the Micrometer or CXF project. I am mentioning the latter, because they might be interessted in taking care of that implementation.)
If you need cxf statistics for micrometer report, you can try
<dependency>
<groupId>io.github.kdprog</groupId>
<artifactId>cxf-micrometer-metrics</artifactId>
<version>1.0.0</version>
</dependency>
The following statistics will be reported
cxf_requests_processed_total - total number of cxf requests ,
cxf_requests_seconds_sum - total execution time of cxf requests,
cxf_requests_seconds_max - maximum execution time of cxf request,
cxf_requests_success_total - total number of successfully processed cxf requests,
cxf_requests_failed_total - total number of failed cxf requests
for each web service method of every client or server cxf endpoint.
For spring applications add the following bean to your application configuration.
#Bean
public FactoryBeanListener cxfMicrometerBean(final MeterRegistry registry) {
return new MicrometerFactoryBeanListener(registry);
}
https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#actuator.metrics.supported
You can make custom tag provider to override it:
#Bean
WebMvcTagsProvider webMvcTagsProvider() {
return new DefaultWebMvcTagsProvider() {
#Override
public Iterable<Tag> getTags(HttpServletRequest request, HttpServletResponse response,
Object handler, Throwable exception) {
return Tags.concat(
super.getTags(request, response, handler, exception),
Tags.of(Tag.of("uri",request.getRequestURI()))
);
}
};
}
More examples.
You can also collect cxf metrics for prometheus using io.github.ddk-prog:cxf-prometheus-metrics.
Add dependency to your pom.xml
<dependency>
<groupId>io.github.ddk-prog</groupId>
<artifactId>cxf-prometheus-metrics</artifactId>
<version>1.0.0</version>
</dependency>
and the following bean to your application configuration.
#Bean
public FactoryBeanListener cxfPrometheusFeatureBean(final CollectorRegistry registry) {
return new PrometheusFactoryBeanListener(registry);
}
You will get cxf_requests_total, cxf_requests_success, cxf_requests_failed, cxf_requests_seconds for each endpoint and operation in your spring boot actuator prometheus report.
For example,
cxf_requests_seconds{endpoint="server1",operation="server1Method",} 0.0157349
If you are using WebFlux on your project, you can make your custom tag provider by overriding:
#Bean
WebFluxTagsProvider webFluxTagsProvider() {
return new DefaultWebFluxTagsProvider() {
#Override
public Iterable<Tag> httpRequestTags(ServerWebExchange exchange, Throwable exception) {
return Tags.concat(super.httpRequestTags(exchange, exception), Tags.of(Tag.of("uri", exchange.getRequest().getPath().value())));
}
};
}
It works for me.
I'm developing a project using ElasticSearch and I'm having some problems with serialization/deserialization with Jackson. My project was created using JHipster, so, I'm using spring to store my entity to the database and to index in ElasticSearch. All entities and other objects can be (de)serialized with Jackson, except when I try to add it to ES.
This is my global configuration for Jackson:
#Configuration
public class JacksonConfiguration {
#Bean
Jackson2ObjectMapperBuilder jackson2ObjectMapperBuilder() {
SimpleModule timeModule = new JavaTimeModule();
timeModule.addSerializer(OffsetDateTime.class, JSR310DateTimeSerializer.INSTANCE);
timeModule.addSerializer(ZonedDateTime.class, JSR310DateTimeSerializer.INSTANCE);
timeModule.addSerializer(LocalDateTime.class, JSR310DateTimeSerializer.INSTANCE);
timeModule.addSerializer(Instant.class, JSR310DateTimeSerializer.INSTANCE);
timeModule.addDeserializer(LocalDate.class, JSR310LocalDateDeserializer.INSTANCE);
SimpleModule geoModule=new GeoModule();
geoModule.addSerializer(Point.class, PointSerializer.INSTANCE);
geoModule.addDeserializer(Point.class, PointDeserializer.INSTANCE);
return new Jackson2ObjectMapperBuilder()
.featuresToDisable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS)
.findModulesViaServiceLoader(true)
.modulesToInstall(timeModule,geoModule);
}
}
This configuration works fine, except when I try add an entity to ES, for example, PointSerializer is never called. The only way I can see this serializer running (and consequently indexing correctly) for ES is adding #JsonSerialize(using = PointSerializer.class) to the field. Why is it happening and how can I configure it globally?
It seems that Spring Data elasticsearch doesn't utilize the default spring Jackson2ObjectMapperBuilder for this. Per default this configuration is used:
https://github.com/spring-projects/spring-data-elasticsearch/blob/master/src/main/java/org/springframework/data/elasticsearch/core/DefaultEntityMapper.java
... which you can overwrite by providing some custom object mapper as described here:
https://github.com/spring-projects/spring-data-elasticsearch/wiki/Custom-ObjectMapper
Here you can of course directly use your Jackson ObjectMappers. For more details, have a look at this issue at the jhipster github repo:
https://github.com/jhipster/generator-jhipster/issues/2241#issuecomment-151933768
I am upgrading my project from spring-boot 1.5.12.release to 2.1.9.release. I am unable to find LoggersMvcEndpoint (https://docs.spring.io/spring-boot/docs/1.5.12.RELEASE/api/org/springframework/boot/actuate/endpoint/mvc/LoggersMvcEndpoint.html) in latest version.
In one of my controller I had this. Can some one help me to fix this.
public class LoggerController extends CloudRestTemplate {
#Autowired
LoggersMvcEndpoint loggerAPI;
#Override
public Object getFromInternalApi(final String param) {
return StringUtils.isEmpty(param) ? loggerAPI.invoke() : loggerAPI.get(param);
}
#Override
public Object postToInternalApi(final String param, final Object request) {
return loggerAPI.set(param, (Map<String, String>) request);
}
}
As per Spring docs here
Endpoint infrastructure
Spring Boot 2 brings a brand new endpoint
infrastructure that allows you to define one or several operations in
a technology independent fashion with support for Spring MVC, Spring
WebFlux and Jersey! Spring Boot 2 will have native support for Jersey
and writing an adapter for another JAX-RS implementation should be
easy as long as there is a way to programmatically register resources.
The new #Endpoint annotation declares this type to be an endpoint with
a mandatory, unique id. As we will see later, a bunch of properties
will be automatically inferred from that. No additional code is
required to expose this endpoint at /applications/loggers or as a
org.springframework.boot:type=Endpoint,name=Loggers JMX MBean.
Refer to documentation, it will help you further
and for your info LoggersMvcEndpoint was there until 2.0.0.M3 https://docs.spring.io/spring-boot/docs/2.0.0.M3/api/org/springframework/boot/actuate/endpoint/mvc/LoggersMvcEndpoint.html however there is no reference of deprecation in subsequent version's release notes of 2.0.0.M4
https://docs.spring.io/spring-boot/docs/2.0.0.M4/api/deprecated-list.html#class