I have test that works properly with Spring 2.4.0-M2 but after upgrading to 2.4.0-M3 it breaks - returns 404 for a route that is registered.
My app:
#SpringBootApplication(proxyBeanMethods = false)
class ExampleApp
fun main(args: Array<String>) {
runApplication<ExampleApp>(
init = {
addInitializers(BeansInitializer())
},
args = args
)
}
beans:
class BeansInitializer : ApplicationContextInitializer<GenericApplicationContext> {
#Suppress("LongMethod")
override fun initialize(applicationContext: GenericApplicationContext) {
beans {
bean {
router {
"/routes".nest {
GET("/{id}") { ServerResponse.ok().bodyValue(Foo("ok")) }
POST("/") { ServerResponse.ok().bodyValue(Foo("ok")) }
}
}
}
}
.initialize(applicationContext)
}
}
data class Foo(val status: String)
My test:
#SpringBootTest(
webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT,
classes = [
ExampleApp::class
]
)
class FailingTest #Autowired constructor(
context: ApplicationContext,
) {
val webTestClient: WebTestClient = WebTestClient.bindToApplicationContext(context)
.configureClient()
.build()
#Test
fun `should interact with routes`() {
webTestClient
.post()
.uri("/routes")
.bodyValue(SampleBody("123"))
.exchange()
.expectStatus()
.isOk // returns 404 on 2.4.0-M3 / passes on 2.4.0-M2
}
data class SampleBody(val id: String)
}
test application.yml
context:
initializer:
classes: com.example.BeansInitializer
On 2.4.0-M3 tests fail with following message:
java.lang.AssertionError: Status expected:<200 OK> but was:<404 NOT_FOUND>
On 2.4.0-M2 they pass.
Is there something that changed through the versions? Or this is a bug?
The change in behaviour that you are seeing is due to an improvement in Spring Framework during the development of 5.3.
By default, Spring Framework will match an optional trailing path separator (/). This optional / should be in addition to the path specified in your routes.
You have two routes:
GET /routes/{id}
POST /routes/
The support for an optional trailing path separator means that you could make a get request to /routes/56/ (an additional trailing /), but it should not mean that you can make a request to POST /routes (removal of a trailing /).
If you want to be able to make POST requests to both /routes and /routes/, you should define the route as /routes:
beans {
bean {
router {
"/routes".nest {
GET("/{id}") { ServerResponse.ok().bodyValue(Foo("ok")) }
POST("") { ServerResponse.ok().bodyValue(Foo("ok")) }
}
}
}
}
I have some issues with the new routing feature in spring cloud stream
I tried to implement a simple scenario, I want to send a message with a header spring.cloud.function.definition = consume1 or consume2
I expect that consume1 or consume2 should be called based on what is sent on the header but the methods are called randomly.
I send the message to the exchange consumer using the rabbit admin console
I'm having the following logs:
2020-02-27 14:48:25.896 INFO 22132 --- [ consumer.app-1] com.example.demo.TestConsumer : ==============>consume1 messge [[payload=ok, headers={amqp_receivedDeliveryMode=NON_PERSISTENT, amqp_receivedRoutingKey=#, amqp_receivedExchange=consumer, amqp_deliveryTag=1, deliveryAttempt=1, amqp_consumerQueue=consumer.app, amqp_redelivered=false, id=9a4dff25-88ef-4d76-93e2-c8719cda122d, spring.cloud.function.definition=consume1, amqp_consumerTag=amq.ctag-gGChFNCKIVd25yyR9H6-fQ, sourceData=(Body:'[B#3a92faa7(byte[2])' MessageProperties [headers={spring.cloud.function.definition=consume1}, contentLength=0, receivedDeliveryMode=NON_PERSISTENT, redelivered=false, receivedExchange=consumer, receivedRoutingKey=#, deliveryTag=1, consumerTag=amq.ctag-gGChFNCKIVd25yyR9H6-fQ, consumerQueue=consumer.app]), timestamp=1582811303347}]]
2020-02-27 14:48:25.984 INFO 22132 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet 'dispatcherServlet'
2020-02-27 14:48:25.984 INFO 22132 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet'
2020-02-27 14:48:25.991 INFO 22132 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : Completed initialization in 7 ms
2020-02-27 14:48:26.037 INFO 22132 --- [oundedElastic-1] o.s.i.monitor.IntegrationMBeanExporter : Registering MessageChannel customer-1
2020-02-27 14:48:26.111 INFO 22132 --- [oundedElastic-1] o.s.c.s.m.DirectWithAttributesChannel : Channel 'application.customer-1' has 1 subscriber(s).
2020-02-27 14:48:26.116 INFO 22132 --- [oundedElastic-1] o.s.a.r.c.CachingConnectionFactory : Attempting to connect to: [localhost:5672]
2020-02-27 14:48:26.123 INFO 22132 --- [oundedElastic-1] o.s.a.r.c.CachingConnectionFactory : Created new connection: rabbitConnectionFactory.publisher#32438e24:0/SimpleConnection#3e58666d [delegate=amqp://guest#127.0.0.1:5672/, localPort= 62514]
2020-02-27 14:48:26.139 INFO 22132 --- [-1.customer-1-1] o.s.i.h.s.MessagingMethodInvokerHelper : Overriding default instance of MessageHandlerMethodFactory with provided one.
2020-02-27 14:48:26.140 INFO 22132 --- [-1.customer-1-1] com.example.demo.TestSink : Data received customer-1...body
2020-02-27 14:49:14.185 INFO 22132 --- [ consumer.app-1] o.s.i.h.s.MessagingMethodInvokerHelper : Overriding default instance of MessageHandlerMethodFactory with provided one.
2020-02-27 14:49:14.194 INFO 22132 --- [ consumer.app-1] com.example.demo.TestConsumer : ==============>consume2 messge [[payload=ok, headers={amqp_receivedDeliveryMode=NON_PERSISTENT, amqp_receivedRoutingKey=#, amqp_receivedExchange=consumer, amqp_deliveryTag=1, deliveryAttempt=1, amqp_consumerQueue=consumer.app, amqp_redelivered=false, id=33581edb-2832-1c92-b765-a05794512b34, spring.cloud.function.definition=consume1, amqp_consumerTag=amq.ctag-RIp2nZdcG2a0hNQeImwtBw, sourceData=(Body:'[B#8159793(byte[2])' MessageProperties [headers={spring.cloud.function.definition=consume1}, contentLength=0, receivedDeliveryMode=NON_PERSISTENT, redelivered=false, receivedExchange=consumer, receivedRoutingKey=#, deliveryTag=1, consumerTag=amq.ctag-RIp2nZdcG2a0hNQeImwtBw, consumerQueue=consumer.app]), timestamp=1582811354186}]]
2020-02-27 14:49:14.203 INFO 22132 --- [oundedElastic-1] o.s.i.monitor.IntegrationMBeanExporter : Registering MessageChannel customer-2
2020-02-27 14:49:14.213 INFO 22132 --- [oundedElastic-1] o.s.c.s.m.DirectWithAttributesChannel : Channel 'application.customer-2' has 1 subscriber(s).
2020-02-27 14:49:14.216 INFO 22132 --- [-2.customer-2-1] o.s.i.h.s.MessagingMethodInvokerHelper : Overriding default instance of MessageHandlerMethodFactory with provided one.
2020-02-27 14:49:14.216 INFO 22132 --- [-2.customer-2-1] com.example.demo.TestSink : Data received customer-2...body
application.yml
spring:
main:
allow-bean-definition-overriding: true
spring.cloud.stream:
function.definition: supplier;receive1;receive2;consume1;consume2
function.routing:
enabled: true
bindings:
consume1-in-0.destination: consumer
consume1-in-0.group: app
consume2-in-0.destination: consumer
consume2-in-0.group: app
receive1-in-0.destination: customer-1
receive1-in-0.group: customer-1
receive2-in-0.destination: customer-2
receive2-in-0.group: customer-2
DemoApplication.java
import com.fasterxml.jackson.databind.ObjectMapper
import org.apache.commons.logging.Log
import org.apache.commons.logging.LogFactory
import org.springframework.boot.autoconfigure.SpringBootApplication
import org.springframework.boot.runApplication
import org.springframework.context.annotation.Bean
import org.springframework.http.HttpStatus
import org.springframework.messaging.Message
import org.springframework.messaging.support.MessageBuilder
import org.springframework.stereotype.Component
import org.springframework.web.bind.annotation.PathVariable
import org.springframework.web.bind.annotation.RequestMapping
import org.springframework.web.bind.annotation.RequestMethod.GET
import org.springframework.web.bind.annotation.ResponseStatus
import org.springframework.web.bind.annotation.RestController
import org.springframework.web.client.RestTemplate
import reactor.core.publisher.EmitterProcessor
import reactor.core.publisher.Flux
import java.util.function.Consumer
import java.util.function.Supplier
#SpringBootApplication
class DemoApplication
fun main(args: Array<String>) {
runApplication<DemoApplication>(*args)
}
#RestController
class DynamicDestinationController(private val jsonMapper: ObjectMapper) {
private val processor: EmitterProcessor<Message<String>> = EmitterProcessor.create<Message<String>>()
#RequestMapping(path = ["/api/dest/{destName}"], method = [GET], consumes = ["*/*"])
#ResponseStatus(HttpStatus.ACCEPTED)
fun handleRequest(#PathVariable destName:String) {
val message: Message<String> = MessageBuilder.withPayload("body")
.setHeader("spring.cloud.stream.sendto.destination", destName).build()
processor.onNext(message)
}
#Bean
fun supplier(): Supplier<Flux<Message<String>>> {
return Supplier { processor }
}
}
const val destResourceUrl = "http://localhost:8080/api/dest"
#Component
class TestConsumer() {
private val restTemplate: RestTemplate = RestTemplate()
private val logger: Log = LogFactory.getLog(javaClass)
#Bean
fun consume1(): Consumer<Message<String>> = Consumer {
logger.info("==============>consume1 messge [[payload=${it.payload}, headers=${it.headers}]]")
restTemplate.getForEntity("$destResourceUrl/customer-1", String::class.java)
}
#Bean
fun consume2(): Consumer<Message<String>> = Consumer {
logger.info("==============>consume2 messge [[payload=${it.payload}, headers=${it.headers}]]")
restTemplate.getForEntity("$destResourceUrl/customer-2", String::class.java)
}
}
#Component
class TestSink {
private val logger: Log = LogFactory.getLog(javaClass)
#Bean
fun receive1(): Consumer<String> = Consumer {
logger.info("Data received customer-1..." + it);
}
#Bean
fun receive2(): Consumer<String> = Consumer {
logger.info("Data received customer-2..." + it);
}
}
Any idea how to fix the route to consumer?
thanks in advance.
demo-repo
Actually I am a bit confused, so let's do one step at the time. Here is the functioning (modelled after yours) app which uses sendto feature allowing you to send messages to the specific (existing and/or dynamically resolved) destinations.
(in java but you can rework it to Kotlin)
#Controller
public class WebSourceApplication {
public static void main(String[] args) {
SpringApplication.run(WebSourceApplication.class,
"--spring.cloud.function.definition=supplier;consA;consB",
"--spring.cloud.stream.bindings.consA-in-0.destination=consumerA",
"--spring.cloud.stream.bindings.consA-in-0.group=consumerA-grp",
"--spring.cloud.stream.bindings.consB-in-0.destination=consumerB",
"--spring.cloud.stream.bindings.consB-in-0.group=consumerB-grp"
);
}
EmitterProcessor<Message<String>> processor = EmitterProcessor.create();
#RequestMapping(path = "/api/dest/{destName}", consumes = "*/*")
#ResponseStatus(HttpStatus.ACCEPTED)
public void delegateToSupplier(#RequestBody String body, #PathVariable String destName) {
Message<String> message = MessageBuilder.withPayload(body)
.setHeader("spring.cloud.stream.sendto.destination", destName)
.build();
processor.onNext(message);
}
#Bean
public Supplier<Flux<Message<String>>> supplier() {
return () -> processor;
}
#Bean
public Consumer<String> consA() {
return v -> {
System.out.println("Consuming from consA: " + v);
};
}
#Bean
public Consumer<String> consB() {
return v -> {
System.out.println("Consuming from consB: " + v);
};
}
}
And when i curl it i get consistent invocation pr the appropriate consumer:
curl -H "Content-Type: application/json" -X POST -d "Hello Spring Cloud Stream" http://localhost:8080/api/dest/consumerA
log: Consuming from consA: Hello Spring Cloud Stream
. . .
curl -H "Content-Type: application/json" -X POST -d "Hello Spring Cloud Stream" http://localhost:8080/api/dest/consumerB
log: Consuming from consB: Hello Spring Cloud Stream
Notice: There is no enable routing property. That feature is mainly aimed to always call one function functionRouter and have it call other functions on your behalf. It is a feature of spring-cloud-function which means it works outside of spring-cloud-srteam and channels/destinations etc.
Isn't that what you are trying to accomplish? Send message to a different destination based on some oath variable in your HTTP request?
Here is an example of a different microservice which receives on routing function which hen routes to different functions
public class FunctionRoutingApplication {
public static void main(String[] args) {
SpringApplication.run(FunctionRoutingApplication.class,
"--spring.cloud.stream.function.routing.enabled=true"
);
}
#Bean
public Consumer<String> consA() {
return v -> {
System.out.println("Consuming from consA: " + v);
};
}
#Bean
public Consumer<String> consB() {
return v -> {
System.out.println("Consuming from consB: " + v);
};
}
}
And that's pretty much it. Go to your broker and send data to functionRouter-in-0 exchange while providing spring.cloud.function.definition=consA/consB headers and you will see consistent invocations.
Am I still missing something?
I have this specific problem with my yml configuration file.
I have a multi-module maven project as follows:
app
|-- core
|-- web
|-- app
I have this configuration file in core project
#Configuration
#PropertySource("core-properties.yml")
public class CoreConfig {
}
And this mapping:
#Component
#ConfigurationProperties(prefix = "some.key.providers.by")
#Getter
#Setter
public class ProvidersByMarket {
private Map<String, List<String>> market;
}
Here are my core-properties.yml
some.key.providers:
p1: 'NAME1'
p2: 'NAME2'
some.key.providers.by.market:
de:
- ${some.key.providers.p1}
- ${some.key.providers.p2}
gb:
- ${some.key.providers.p1}
When I load the file via profile activation, for example, rename the file to application-core-properties.yml and then -Dspring.profiles.active=core-propertiesit does work however if when I try to load the file via #PropertySource("core-properties.yml") it does not and I get the following error:
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2019-03-27 10:07:36.397 -ERROR 13474|| --- [ restartedMain] o.s.b.d.LoggingFailureAnalysisReporter :
***************************
APPLICATION FAILED TO START
***************************
Description:
Failed to bind properties under 'some.key.providers.by.market' to java.util.Map<java.lang.String, java.util.List<java.lang.String>>:
Reason: No converter found capable of converting from type [java.lang.String] to type [java.util.Map<java.lang.String, java.util.List<java.lang.String>>]
Action:
Update your application's configuration
Process finished with exit code 1
Bacouse you don't have equivalent properties stracture,
example
spring:
profiles: test
name: test-YAML
environment: test
servers:
- www.abc.test.com
- www.xyz.test.com
---
spring:
profiles: prod
name: prod-YAML
environment: production
servers:
- www.abc.com
- www.xyz.com
And config class should be
#Configuration
#EnableConfigurationProperties
#ConfigurationProperties
public class YAMLConfig {
private String name;
private String environment;
private List<String> servers = new ArrayList<>();
// standard getters and setters
I have resolved the issue implementing the following PropertySourceFactory detailed described in here
import java.io.FileNotFoundException;
import java.io.IOException;
import java.util.Properties;
import org.springframework.beans.factory.config.YamlPropertiesFactoryBean;
import org.springframework.core.env.PropertiesPropertySource;
import org.springframework.core.env.PropertySource;
import org.springframework.core.io.support.EncodedResource;
import org.springframework.core.io.support.PropertySourceFactory;
import org.springframework.lang.Nullable;
public class YamlPropertySourceFactory implements PropertySourceFactory {
#Override
public PropertySource<?> createPropertySource(#Nullable String name, EncodedResource resource) throws IOException {
Properties propertiesFromYaml = loadYamlIntoProperties(resource);
String sourceName = name != null ? name : resource.getResource().getFilename();
return new PropertiesPropertySource(sourceName, propertiesFromYaml);
}
private Properties loadYamlIntoProperties(EncodedResource resource) throws FileNotFoundException {
try {
YamlPropertiesFactoryBean factory = new YamlPropertiesFactoryBean();
factory.setResources(resource.getResource());
factory.afterPropertiesSet();
return factory.getObject();
} catch (IllegalStateException e) {
// for ignoreResourceNotFound
Throwable cause = e.getCause();
if (cause instanceof FileNotFoundException)
throw (FileNotFoundException) e.getCause();
throw e;
}
}
}
I had a similar problem and found a workaround like this:
diacritic:
isEnabled: true
chars: -> I wanted this to be parsed to map but it didn't work
ą: a
ł: l
ę: e
And my solution so far:
diacritic:
isEnabled: true
chars[ą]: a -> these ones could be parsed to Map<String, String>
chars[ł]: l
chars[ę]: e
I am trying to remove file from remote by implementing streaming inbound but connection is closing before adviceChain implementing.
CODE:
#Bean
public SessionFactory<LsEntry> sftpSessionFactory() {
DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(true);
factory.setHost(sftpHost);
factory.setPort(sftpPort);
factory.setUser(sftpUser);
factory.setPassword(sftpPwd);
factory.setAllowUnknownKeys(true);
return new CachingSessionFactory<LsEntry>(factory);
}
#Bean
#InboundChannelAdapter(channel = "stream", poller = #Poller(cron = "2 * * * * ?"))
public MessageSource<InputStream> sftpMessageSource() {
SftpStreamingMessageSource messageSource = new SftpStreamingMessageSource(template());
messageSource.setRemoteDirectory(remoteDirecotry);
messageSource.setFilter(new AcceptAllFileListFilter<>());
return messageSource;
}
#Bean
public SftpRemoteFileTemplate template() {
return new SftpRemoteFileTemplate(sftpSessionFactory());
}
#Bean
#Transformer(inputChannel = "stream", outputChannel = "data")
public org.springframework.integration.transformer.Transformer transformer() {
return new StreamTransformer("UTF-8");
}
#ServiceActivator(inputChannel = "data" ,adviceChain = "afterChain")
#Bean
public MessageHandler handler() {
return new MessageHandler() {
#Override
public void handleMessage(Message<?> message) throws MessagingException {
String fileName = message.getHeaders().get("file_remoteFile").toString();
if (!StringUtils.isEmpty(message.toString())) {
else{
log.info("No file found in the Remote location");
}
}
};
}
#Bean
public ExpressionEvaluatingRequestHandlerAdvice afterChain() {
ExpressionEvaluatingRequestHandlerAdvice advice = new ExpressionEvaluatingRequestHandlerAdvice();
advice.setOnSuccessExpression(
"#template.remove(headers['file_remoteDirectory'] + headers['file_remoteFile'])");
//advice.setOnSuccessExpressionString("#template.remove(headers['file_remoteFile'])");
advice.setPropagateEvaluationFailures(true);
return advice;
}
wherever i search every one is suggesting to implement ExpressionEvaluatingRequestHandlerAdvice but it is throwing me below error.
2018-03-27 12:32:02.618 INFO 23216 --- [ask-scheduler-1] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=starsBatchJob]] completed with the following parameters: [{JobID=1522168322277}] and the following status: [COMPLETED]
2018-03-27 12:32:02.618 INFO 23216 --- [ask-scheduler-1] c.f.u.config.ParentBatchConfiguration : Job Status Completed
2018-03-27 12:32:02.618 INFO 23216 --- [ask-scheduler-1] c.f.u.config.ParentBatchConfiguration : Total time tokk for Stars Batch execution: 0 seconds.
2018-03-27 12:32:02.618 INFO 23216 --- [ask-scheduler-1] c.f.u.config.ParentBatchConfiguration : Batch Job lock is released
2018-03-27 12:32:02.633 INFO 23216 --- [ask-scheduler-1] com.jcraft.jsch : Disconnecting from hpchd1e.hpc.ford.com port 22
2018-03-27 12:32:02.633 ERROR 23216 --- [ask-scheduler-1] o.s.integration.handler.LoggingHandler : org.springframework.messaging.MessagingException: Dispatcher failed to deliver Message; nested exception is org.springframework.messaging.MessagingException: Failed to execute on session; nested exception is org.springframework.core.NestedIOException: Failed to remove file: 2: No such file; nested exception is 2
I had this problem. My path to the remote file was incorrect. I needed a trailing /. It is a little difficult to see since the path is being created inside a Spel Expression. You can see the path using the following in the handleMessage() method.
String remoteDirectory = (String) message.getHeaders().get("file_remoteDirectory");
String remoteFile = (String) message.getHeaders().get("file_remoteFile");
I did have to use the advice.setOnSuccessExpressionString("#template.remove(headers['file_remoteFile'])"); that is commented out above instead of advice.setOnSuccessExpression"#template.remove(headers['file_remoteDirectory'] + headers['file_remoteFile'])");
It is incorrect in the documentation https://docs.spring.io/spring-integration/reference/html/sftp.html#sftp-streaming which is why I believe people who struggle with this lose faith in the doc. But this seems to be the only error.
How to dump actuator metrics to log file.
Tried using metricsEndpointMetricReader but could not get any results.
Any help is appreciated.
You can autowire the all the PublicMetrics , and extract the result to a map and log it.
public class MetricExtractor {
#Autowired
List<PublicMetrics> publicMetrics;
public void extract() {
Map<String,Object>map = new LinkedHashMap<>();
for(PublicMetrics publicMetrics : publicMetrics) {
for(Metric<?> metric : publicMetrics.metrics()) {
map.put(metric.getName(), metric.getValue());
}
}
log.info("metrics :: {} ", map );
}
}