I have a spring-boot application where using webflux and r2dbc-postgres. I have discovered a strange issue when trying to do some db operations in a flatMap().
Code example:
#Transactional
public Mono<Void> insertDummyFooBars() {
return Flux.fromIterable(IntStream.rangeClosed(1, 260).boxed().collect(Collectors.toList()))
.log()
.flatMap(i -> this.repository.save(FooBar.builder().foo("test-" + i).build()))
.log()
.concatMap(i -> this.repository.findAll())
.then();
}
It seems like flatMap can process max 256 elements in batches. (Queues.SMALL_BUFFER_SIZE default value is 256). So when I tried to run the code above (with 260 elements) I've got an exception - TransientDataAccessResourceException and the following message:
Cannot exchange messages because the request queue limit is exceeded; nested exception is io.r2dbc.postgresql.client.ReactorNettyClient$RequestQueueException
There is no Releasing R2DBC Connection after this exception. The pgdb connection/session remains in idle in transaction state and the app is not able to run properly when pool max size is reached and all of the connections are in idle in transaction state. I think the connection should be released even if an exception happened or not.
If I use concatMap instead of flatMap it works as expected - no exception, connection released! It's also ok with flatMap when the elements are less than or equal to 256.
Is it possible to force pgdb connection closure? What should I do If I have lot of db operations in flatMap like this? Should I replace all of them with concatMap? Is there a global solution for this?
Versions:
Postgres: 12.6, Spring-boot: 2.7.6
Demo project
LOG:
2022-12-08 16:32:13.092 INFO 17932 --- [actor-tcp-nio-1] reactor.Flux.Iterable.1 : | onNext(256)
2022-12-08 16:32:13.092 DEBUG 17932 --- [actor-tcp-nio-1] o.s.r2dbc.core.DefaultDatabaseClient : Executing SQL statement [INSERT INTO foo_bar (foo) VALUES ($1)]
2022-12-08 16:32:13.114 INFO 17932 --- [actor-tcp-nio-1] reactor.Flux.FlatMap.2 : onNext(FooBar(id=258, foo=test-1))
2022-12-08 16:32:13.143 DEBUG 17932 --- [actor-tcp-nio-1] o.s.r2dbc.core.DefaultDatabaseClient : Executing SQL statement [SELECT foo_bar.* FROM foo_bar]
2022-12-08 16:32:13.143 INFO 17932 --- [actor-tcp-nio-1] reactor.Flux.Iterable.1 : | request(1)
2022-12-08 16:32:13.143 INFO 17932 --- [actor-tcp-nio-1] reactor.Flux.Iterable.1 : | onNext(257)
2022-12-08 16:32:13.144 DEBUG 17932 --- [actor-tcp-nio-1] o.s.r2dbc.core.DefaultDatabaseClient : Executing SQL statement [INSERT INTO foo_bar (foo) VALUES ($1)]
2022-12-08 16:32:13.149 INFO 17932 --- [actor-tcp-nio-1] reactor.Flux.Iterable.1 : | onComplete()
2022-12-08 16:32:13.149 INFO 17932 --- [actor-tcp-nio-1] reactor.Flux.Iterable.1 : | cancel()
2022-12-08 16:32:13.160 ERROR 17932 --- [actor-tcp-nio-1] reactor.Flux.FlatMap.2 : onError(org.springframework.dao.TransientDataAccessResourceException: executeMany; SQL [INSERT INTO foo_bar (foo) VALUES ($1)]; Cannot exchange messages because the request queue limit is exceeded; nested exception is io.r2dbc.postgresql.client.ReactorNettyClient$RequestQueueException: [08006] Cannot exchange messages because the request queue limit is exceeded)
2022-12-08 16:32:13.167 ERROR 17932 --- [actor-tcp-nio-1] reactor.Flux.FlatMap.2 :
org.springframework.dao.TransientDataAccessResourceException: executeMany; SQL [INSERT INTO foo_bar (foo) VALUES ($1)]; Cannot exchange messages because the request queue limit is exceeded; nested exception is io.r2dbc.postgresql.client.ReactorNettyClient$RequestQueueException: [08006] Cannot exchange messages because the request queue limit is exceeded
at org.springframework.r2dbc.connection.ConnectionFactoryUtils.convertR2dbcException(ConnectionFactoryUtils.java:215) ~[spring-r2dbc-5.3.24.jar:5.3.24]
at org.springframework.r2dbc.core.DefaultDatabaseClient.lambda$inConnectionMany$8(DefaultDatabaseClient.java:147) ~[spring-r2dbc-5.3.24.jar:5.3.24]
at reactor.core.publisher.Flux.lambda$onErrorMap$29(Flux.java:7105) ~[reactor-core-3.4.25.jar:3.4.25]
at reactor.core.publisher.Flux.lambda$onErrorResume$30(Flux.java:7158) ~[reactor-core-3.4.25.jar:3.4.25]
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:94) ~[reactor-core-3.4.25.jar:3.4.25]
I have tried to change the Queues.SMALL_BUFFER_SIZE, and also tried to add a concurrency value to the flatmap. It works when I reduced the value to 255 but I think it is not a good solution.
Given the setup https://gist.github.com/gel-hidden/0a8627cf93f5396d6b73c2a6e71aad3e, I would expect when I send a message that the ServiceActivator would be called with a delay of 10 000 between messages.
The first channel takes in a list, then split the messages and then call another QueueChannel. But for some reason each pull polls all the split messages. I know I am missing something stupid, or I'm just too stupid to understand whats happening.
Related test case: https://gist.github.com/gel-hidden/de7975fffd0853ec8ce49f9d6fa6531d
Output:
2022-10-26 15:22:02.708 INFO 78647 --- [ scheduling-1] com.example.demo.DemoApplicationTests : Received message Hello
2022-10-26 15:22:02.708 INFO 78647 --- [ scheduling-1] com.example.demo.UpdateLocationFlow : Doing some work for model with id 2
2022-10-26 15:22:03.009 INFO 78647 --- [ scheduling-1] com.example.demo.UpdateLocationFlow : Completed some work for model with id 2
2022-10-26 15:22:03.017 INFO 78647 --- [ scheduling-1] com.example.demo.DemoApplicationTests : Received message World
2022-10-26 15:22:03.018 INFO 78647 --- [ scheduling-1] com.example.demo.UpdateLocationFlow : Doing some work for model with id 3
2022-10-26 15:22:03.319 INFO 78647 --- [ scheduling-1] com.example.demo.UpdateLocationFlow : Completed some work for model with id 3
2022-10-26 15:22:04.322 INFO 78647 --- [ scheduling-1] o.s.i.a.AggregatingMessageHandler : Expiring MessageGroup with correlationKey[1]
My thoughts is that the messages should be something like:
00:01 Doing some work for model with id 2
00:02 Completed some work for model with id 2
00:12 Doing some work for model with id 3
00:13 Completed some work for model it id 3
So, it is a bug in the Spring Integration around lifecycle management for the IntegrationFlowAdapter management. It just starts twice.
As a workaround I suggest to pull your #ServiceActivator handle() into an individual component with its own #Poller configuration and an inputChannel and outputChannel. In other words int must go outside of your UpdateLocationFlow. This way the IntegrationFlowAdapter won't have a control for its lifecycle and won't start it twice.
Meanwhile I'm looking how to fix it.
Thank you for reporting this!
I hope someone of you can help me.
I'm using spring boot 2.3.4 with spring kafka 2.5.6. I recently had to reset an offset and saw some strange behavior. We consumed the messages, but after every X (variating) messages we had a timeout of 10 seconds before the consumption continued.
This is my configuration:
spring:
kafka:
bootstrap-servers: localhost:9092
consumer:
enable-auto-commit: false
auto-offset-reset: earliest
heartbeat-interval: 1000
max-poll-records: 50
group-id: kafka-fetch-demo
fetch-max-wait: 10000
listener:
type: single
concurrency: 1
poll-timeout: 1000
no-poll-threshold: 2
monitor-interval: 10
ack-mode: manual
producer:
acks: all
batch-size: 0
retries: 0
This is an examle listener code:
#KafkaListener(id = LISTENER_ID, idIsGroup = false, topicPattern = "#{demoProperties.getTopicPattern()}")
public void onEvent(Acknowledgment acknowledgment, ConsumerRecord<byte[], String> record) {
log.info("Received record on topic {}, partition {} and offset {}",
record.topic(),
record.partition(),
record.offset());
acknowledgment.acknowledge();
}
Analysis
I figured out that the 10 second timeout came from the fetch.max.wait.ms property. However I'm not able to figure out why this property applies.
As far as I understand the fetch-max-wait property only determines the maximum time the broker waits before providing the consumer with new records even if the fetch.min.bytes is not exceeded. (Which in my case is set to the default 1 and should always be fullfilled)
Furthermore I analyzed that this problem only applies when using topic patterns and "larger" messages.
Reproduction
I uploaded an demo application on Github to reproduce the issue: https://github.com/kraennix/kafka-fetch-demo.
How I did reproduce it:
I put a thousand messages with 17,1 KB per message on a kafka topic.
I start my consuming application that listens per topic pattern to this topic. Then you can see this stopping behaviour.
Note: If I do the same with "small" messages (89 Bytes) it works as expected.
Logs
In the logs you can see the successful commit, but then the it says Skipping fetch
2021-01-16 15:04:40.773 DEBUG 19244 --- [_LISTENER-0-C-1] essageListenerContainer$ListenerConsumer : Commit list: {publish.LargeTopic.2.test-0=OffsetAndMetadata{offset=488, leaderEpoch=null, metadata=''}}
2021-01-16 15:04:40.773 DEBUG 19244 --- [_LISTENER-0-C-1] essageListenerContainer$ListenerConsumer : Committing: {publish.LargeTopic.2.test-0=OffsetAndMetadata{offset=488, leaderEpoch=null, metadata=''}}
2021-01-16 15:04:40.773 TRACE 19244 --- [_LISTENER-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator : [Consumer clientId=consumer-kafka-fetch-demo-1, groupId=kafka-fetch-demo] Sending OffsetCommit request with {publish.LargeTopic.2.test-0=OffsetAndMetadata{offset=488, leaderEpoch=null, metadata=''}} to coordinator localhost:9092 (id: 2147483647 rack: null)
2021-01-16 15:04:40.773 DEBUG 19244 --- [_LISTENER-0-C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-kafka-fetch-demo-1, groupId=kafka-fetch-demo] Using older server API v7 to send OFFSET_COMMIT {group_id=kafka-fetch-demo,generation_id=4,member_id=consumer-kafka-fetch-demo-1-cf8e747f-531d-457a-aca8-18960c518ef9,group_instance_id=null,topics=[{name=publish.LargeTopic.2.test,partitions=[{partition_index=0,committed_offset=488,committed_leader_epoch=-1,committed_metadata=}]}]} with correlation id 62 to node 2147483647
2021-01-16 15:04:40.778 TRACE 19244 --- [_LISTENER-0-C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-kafka-fetch-demo-1, groupId=kafka-fetch-demo] Completed receive from node 2147483647 for OFFSET_COMMIT with correlation id 62, received {throttle_time_ms=0,topics=[{name=publish.LargeTopic.2.test,partitions=[{partition_index=0,error_code=0}]}]}
2021-01-16 15:04:40.779 DEBUG 19244 --- [_LISTENER-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator : [Consumer clientId=consumer-kafka-fetch-demo-1, groupId=kafka-fetch-demo] Committed offset 488 for partition publish.LargeTopic.2.test-0
2021-01-16 15:04:40.779 TRACE 19244 --- [_LISTENER-0-C-1] o.a.k.c.consumer.internals.Fetcher : [Consumer clientId=consumer-kafka-fetch-demo-1, groupId=kafka-fetch-demo] Skipping fetch for partition publish.LargeTopic.1.test-0 because previous request to localhost:9092 (id: 0 rack: null) has not been processed
2021-01-16 15:04:40.779 TRACE 19244 --- [_LISTENER-0-C-1] o.a.k.c.consumer.internals.Fetcher : [Consumer clientId=consumer-kafka-fetch-demo-1, groupId=kafka-fetch-demo] Skipping fetch for partition publish.LargeTopic.2.test-0 because previous request to localhost:9092 (id: 0 rack: null) has not been processed
2021-01-16 15:04:40.779 TRACE 19244 --- [_LISTENER-0-C-1] o.a.k.c.consumer.internals.Fetcher : [Consumer clientId=consumer-kafka-fetch-demo-1, groupId=kafka-fetch-demo] Skipping fetch for partition publish.LargeTopic.1.test-0 because previous request to localhost:9092 (id: 0 rack: null) has not been processed
2021-01-16 15:04:40.779 TRACE 19244 --- [_LISTENER-0-C-1] o.a.k.c.consumer.internals.Fetcher : [Consumer clientId=consumer-kafka-fetch-demo-1, groupId=kafka-fetch-demo] Skipping fetch for partition publish.LargeTopic.2.test-0 because previous request to localhost:9092 (id: 0 rack: null) has not been processed
When there is a change in the Size of the message, you might need to change below 2 Props
heartbeat-interval: 1000
max-poll-records: 50
Your heart beat interval is 1sec and Max poll wait is 10secs. If the size of the message is high and you are processing the consumed messages in the same thread, then Heartbeat check will fail by the time the next Pull triggered. Make sure to process messages by an Executor using Callable.
Increase the Heart Beat Interval to 5 to 10 secs and Reduce Max Poll records to 15 when the messages size is high. Hope, this can help
I have spent days on this simple issue , I am giving up and finally posting this issue which I am facing locally. I am trying to set up a microservices flow in my local for my hand itching learning purpose. This is no brainer. I have Eureka , Zuul Gateway , Simple Microservice. When I try to reach to the underlying service with the "url route" its working. But when I try to do serviceId look up its not working. Guys help me fixing it.
Git hub link is Git hub source code link
I have also raised an issue Git hut Issue link
Eureka Screenshot
Zuul Gateway logs
2019-10-06 11:11:24.611 INFO 26980 --- [nio-2020-exec-4] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet 'dispatcherServlet'
2019-10-06 11:11:24.611 INFO 26980 --- [nio-2020-exec-4] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet'
2019-10-06 11:11:24.633 INFO 26980 --- [nio-2020-exec-4] o.s.web.servlet.DispatcherServlet : Completed initialization in 22 ms
2019-10-06 11:11:25.103 INFO 26980 --- [nio-2020-exec-4] c.netflix.config.ChainedDynamicProperty : Flipping property: CHECKOUT-SERVICE.ribbon.ActiveConnectionsLimit to use NEXT property: niws.loadbalancer.availabilityFilteringRule.activeConnectionsLimit = 2147483647
2019-10-06 11:11:25.157 INFO 26980 --- [nio-2020-exec-4] c.n.u.concurrent.ShutdownEnabledTimer : Shutdown hook installed for: NFLoadBalancer-PingTimer-CHECKOUT-SERVICE
2019-10-06 11:11:25.157 INFO 26980 --- [nio-2020-exec-4] c.netflix.loadbalancer.BaseLoadBalancer : Client: CHECKOUT-SERVICE instantiated a LoadBalancer: DynamicServerListLoadBalancer:{NFLoadBalancer:name=CHECKOUT-SERVICE,current list of Servers=[],Load balancer stats=Zone stats: {},Server stats: []}ServerList:null
2019-10-06 11:11:25.167 INFO 26980 --- [nio-2020-exec-4] c.n.l.DynamicServerListLoadBalancer : Using serverListUpdater PollingServerListUpdater
2019-10-06 11:11:25.215 INFO 26980 --- [nio-2020-exec-4] c.netflix.config.ChainedDynamicProperty : Flipping property: CHECKOUT-SERVICE.ribbon.ActiveConnectionsLimit to use NEXT property: niws.loadbalancer.availabilityFilteringRule.activeConnectionsLimit = 2147483647
2019-10-06 11:11:25.218 INFO 26980 --- [nio-2020-exec-4] c.n.l.DynamicServerListLoadBalancer : DynamicServerListLoadBalancer for client CHECKOUT-SERVICE initialized: DynamicServerListLoadBalancer:{NFLoadBalancer:name=CHECKOUT-SERVICE,current list of Servers=[192.168.0.6:8098],Load balancer stats=Zone stats: {defaultzone=[Zone:defaultzone; Instance count:1; Active connections count: 0; Circuit breaker tripped count: 0; Active connections per server: 0.0;]
},Server stats: [[Server:192.168.0.6:8098; Zone:defaultZone; Total Requests:0; Successive connection failure:0; Total blackout seconds:0; Last connection made:Wed Dec 31 19:00:00 EST 1969; First connection made: Wed Dec 31 19:00:00 EST 1969; Active Connections:0; total failure count in last (1000) msecs:0; average resp time:0.0; 90 percentile resp time:0.0; 95 percentile resp time:0.0; min resp time:0.0; max resp time:0.0; stddev resp time:0.0]
]}ServerList:org.springframework.cloud.netflix.ribbon.eureka.DomainExtractingServerList#6f7f7ca0
2019-10-06 11:11:26.177 INFO 26980 --- [erListUpdater-0] c.netflix.config.ChainedDynamicProperty : Flipping property: CHECKOUT-SERVICE.ribbon.ActiveConnectionsLimit to use NEXT property: niws.loadbalancer.availabilityFilteringRule.activeConnectionsLimit = 2147483647
Never mind guys it was a mistake from my side in resolving the API path
I've got Component Integration Test suite setup with Citrus Framework. In order to create a mock service for different components. I've been using simple mapping strategy with the help of XPathPayloadMappingKeyExtractor.
The challenge I've got is to create a multiple responses for different soap requests.
I've been struggling to understand the way it's responding. Any experienced person who can help me out on Citrus and Spring please ?
My Context file:
<citrus-ws:client id="ClientEndpoint"
request-url="http://localhost:8085/"/>
<citrus-ws:server id="BS_Customer_Information"
port="8085"
auto-start="true"/>
My Test Method:
#CitrusTest
public void testingServer() {
soap().client(ClientEndpoint)
.send()
.soapAction("urn:RetrieveAddressBookOP_01")
.payload(new ClassPathResource("Requests/Sample1.xml"));
soap().server(BS_Customer_Information)
.receive()
.soapAction("urn:RetrieveAddressBookOP_01");
soap().server(BS_Customer_Information)
.send()
.payload(new ClassPathResource("AtomicResponses/Response1.xml"));
soap().client(ClientEndpoint)
.receive()
.messageType(MessageType.XML);
}
Issue 1: My Soap Client times out and could not get the response
Question 1: Should the server response be parsed into Soap Env and Body ?
Question 2: Why Would it still call go to Channel Endpoint.inbound when I've already defined my response?
Console Log:
2521 DEBUG gEndpointInterceptor| Received SOAP request:
<?xml version="1.0" encoding="UTF-8"?><SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header/>
<SOAP-ENV:Body>
<RetrieveAddressBookOP_01"/>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
2527 DEBUG r.WebServiceEndpoint| Received SOAP request:
SOAPMESSAGE [payload: <?xml version="1.0" encoding="UTF-8"?><au:RetrieveAddressBookOP_01 xmlns:au="au.com.teysau.dataservice"/>][headers: {citrus_message_id=e1311140-6ee3-415a-815e-0d162cbc03c1, citrus_message_timestamp=1490649843807, citrus_soap_action=urn:RetrieveAddressBookOP_01, citrus_http_request_uri=/, citrus_http_context_path=, citrus_http_query_params=, citrus_http_method=POST}][header-data: [<?xml version="1.0" encoding="UTF-8"?><SOAP-ENV:Header xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"/>]][attachments: []]
2527 DEBUG annelEndpointAdapter| Forwarding request to message channel ...
2527 DEBUG t.TestContextFactory| Created new test context - using global variables: '{}'
2527 DEBUG ltCorrelationManager| Saving correlation key for 'citrus_message_correlator_ChannelEndpointAdapter:producer'
2527 DEBUG context.TestContext| Setting variable: citrus_message_correlator_ChannelEndpointAdapter:producer with value: 'citrus_message_id = 'e1311140-6ee3-415a-815e-0d162cbc03c1''
2527 DEBUG .ChannelSyncProducer| Sending message to channel: 'BS_Customer_Information.inbound'
2527 DEBUG .ChannelSyncProducer| Message to send is:
SOAPMESSAGE [payload: <?xml version="1.0" encoding="UTF-8"?><RetrieveAddressBookOP_01"/>][headers: {citrus_message_id=e1311140-6ee3-415a-815e-0d162cbc03c1, citrus_message_timestamp=1490649843807, citrus_soap_action=urn:RetrieveAddressBookOP_01, citrus_http_request_uri=/, citrus_http_context_path=, citrus_http_query_params=, citrus_http_method=POST}][header-data: [<?xml version="1.0" encoding="UTF-8"?><SOAP-ENV:Header xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"/>]][attachments: []]
2527 INFO .ChannelSyncProducer| Message was sent to channel: 'BS_Customer_Information.inbound'
3539 WARN annelEndpointAdapter| Reply timed out after 1000ms. Did not receive reply message on reply channel
3540 DEBUG annelEndpointAdapter| Did not receive reply message - no response is simulated
3540 DEBUG r.WebServiceEndpoint| No reply message from endpoint adapter 'com.consol.citrus.channel.ChannelEndpointAdapter#6dbd0a41'
3540 WARN r.WebServiceEndpoint| No SOAP response for calling client
3542 DEBUG ageDispatcherServlet| Successfully completed request
3546 DEBUG server.Server| RESPONSE / 202 handled=true
3546 DEBUG ver.HttpChannelState| HttpChannelState#244a4722{s=DISPATCHED i=true a=null} unhandle DISPATCHED
3553 DEBUG erver.HttpConnection| org.eclipse.jetty.server.HttpConnection$SendCallback#27d2cb2e[PROCESSING][i=ResponseInfo{HTTP/1.1 202 null,-1,false},cb=org.eclipse.jetty.server.HttpChannel$CommitCallback#2f1ea019] generate: NEED_HEADER (null,[p=0,l=0,c=0,r=0],true)#START
3556 DEBUG erver.HttpConnection| org.eclipse.jetty.server.HttpConnection$SendCallback#27d2cb2e[PROCESSING][i=ResponseInfo{HTTP/1.1 202 null,-1,false},cb=org.eclipse.jetty.server.HttpChannel$CommitCallback#2f1ea019] generate: FLUSH ([p=0,l=114,c=8192,r=114],[p=0,l=0,c=0,r=0],true)#COMPLETING
3556 DEBUG io.WriteFlusher| write: WriteFlusher#76f08437{IDLE} [HeapByteBuffer#2c34f7c0[p=0,l=114,c=8192,r=114]={<<<HTTP/1.1 202 Acce....v20160210)\r\n\r\n>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}]
3557 DEBUG io.WriteFlusher| update WriteFlusher#76f08437{WRITING}:IDLE-->WRITING
3562 INFO ent.WebServiceClient| SOAP message was sent to endpoint: 'http://localhost:8085/'
3562 INFO ent.WebServiceClient| Received no SOAP response from endpoint: 'http://localhost:8085/'
3562 INFO citrus.Citrus|
3562 DEBUG citrus.Citrus| TEST STEP 1/4 SUCCESS
3562 INFO citrus.Citrus|
3562 DEBUG citrus.Citrus| TEST STEP 2/4: receive
3563 DEBUG io.ChannelEndPoint| flushed 114 SelectChannelEndPoint#1c2a40b4{/127.0.0.1:54924<->8085,Open,in,out,-,W,1092/30000,HttpConnection}{io=0,kio=0,kro=1}
3566 DEBUG nnel.ChannelConsumer| Receiving message from: BS_Customer_Information.inbound
3567 DEBUG nnel.ChannelConsumer| Received message from: BS_Customer_Information.inbound
3567 DEBUG ltCorrelationManager| Saving correlation key for 'citrus_message_correlator_BS_Customer_Information:consumer'
3567 DEBUG context.TestContext| Setting variable: citrus_message_correlator_BS_Customer_Information:consumer with value: 'citrus_message_id = 'e1311140-6ee3-415a-815e-0d162cbc03c1''
3567 DEBUG ltCorrelationManager| Saving correlated object for 'citrus_message_id = 'e1311140-6ee3-415a-815e-0d162cbc03c1''
3567 DEBUG ageValidatorRegistry| Found 3 message validators for message type: XML
3567 DEBUG mXmlMessageValidator| Start message validation ...
3567 DEBUG mXmlMessageValidator| Start XML message validation
3569 DEBUG mXmlMessageValidator| Starting XML schema validation ...
3569 WARN mXmlMessageValidator| Neither schema instance nor schema repository defined - skipping XML schema validation
3569 INFO mXmlMessageValidator| XML message validation successful: All values OK
3569 DEBUG mXmlMessageValidator| Start message header validation ...
3570 DEBUG io.WriteFlusher| update WriteFlusher#76f08437{IDLE}:WRITING-->IDLE
3570 DEBUG erver.HttpConnection| org.eclipse.jetty.server.HttpConnection$SendCallback#27d2cb2e[PROCESSING][i=ResponseInfo{HTTP/1.1 202 null,-1,false},cb=org.eclipse.jetty.server.HttpChannel$CommitCallback#2f1ea019] generate: DONE ([p=114,l=114,c=8192,r=0],[p=0,l=0,c=0,r=0],true)#END
3570 DEBUG mXmlMessageValidator| Validating header element: citrus_soap_action='urn:RetrieveAddressBookOP_01': OK.
3571 INFO mXmlMessageValidator| Message header validation successful: All properties OK
3571 INFO mXmlMessageValidator| Message validation successful: All values OK
3571 INFO citrus.Citrus|
3571 DEBUG citrus.Citrus| TEST STEP 2/4 SUCCESS
3571 INFO citrus.Citrus|
3571 DEBUG citrus.Citrus| TEST STEP 3/4: send
3571 DEBUG ngCorrelationManager| Get correlation key for 'citrus_message_correlator_BS_Customer_Information:consumer'
3571 DEBUG ltCorrelationManager| Finding correlated object for 'citrus_message_id = 'e1311140-6ee3-415a-815e-0d162cbc03c1''
3571 DEBUG .ChannelSyncConsumer| Sending message to reply channel: 'org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#66f66866'
3571 DEBUG .ChannelSyncConsumer| Message to send is:
SOAPMESSAGE [payload: <AddressBook>
<Address>
<Address_Number>14</Address_Number>
<Long_Address>Willis Street</Long_Address>
<Tax_ID>7987398</Tax_ID>
<Alpha_Name>what is alpha</Alpha_Name>
<Code>UN_</Code>
<Mailing_Name>James Bond</Mailing_Name>
<Address_Line1>take the first part of the long address</Address_Line1>
<Address_Line2>You may take the second part not</Address_Line2>
<Postal_Code>8900</Postal_Code>
<City>WLN</City>
<Country>NZ</Country>
<Prefix>233</Prefix>
<Phone_Number>025364</Phone_Number>
<Prefix_2 xsi:nil="true" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"/>
<Phone_Number2 xsi:nil="true" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"/>
<userID>s3b9d</userID>
<BatchNumber>jkdhj59</BatchNumber>
<Tansaction_Number>jkd3j29</Tansaction_Number>
<Line_Number>291597</Line_Number>
</Address>
</AddressBook>][headers: {citrus_message_id=fd946a46-f67a-405d-b118-b1b6b3d562c1, citrus_message_timestamp=1490649843443}][attachments: []]
3572 WARN emporaryReplyChannel| Reply message received but the receiving thread has exited due to a timeout:GenericMessage [payload=SOAPMESSAGE [payload: <AddressBook>
<Address>
<Address_Number>14</Address_Number>
<Long_Address>Willis Street</Long_Address>
<Tax_ID>7987398</Tax_ID>
<Alpha_Name>what is alpha</Alpha_Name>
<Code>UN_</Code>
<Mailing_Name>James Bond</Mailing_Name>
<Address_Line1>take the first part of the long address</Address_Line1>
<Address_Line2>You may take the second part not</Address_Line2>
<Postal_Code>8900</Postal_Code>
<City>WLN</City>
<Country>NZ</Country>
<Prefix>233</Prefix>
<Phone_Number>025364</Phone_Number>
<Prefix_2 xsi:nil="true" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"/>
<Phone_Number2 xsi:nil="true" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"/>
<userID>s3b9d</userID>
<BatchNumber>jkdhj59</BatchNumber>
<Tansaction_Number>jkd3j29</Tansaction_Number>
<Line_Number>291597</Line_Number>
</Address>
</AddressBook>][headers: {citrus_message_id=fd946a46-f67a-405d-b118-b1b6b3d562c1, citrus_message_timestamp=1490649843443}][attachments: []], headers={id=cfc4eeaa-eac8-8f00-2f45-2e3fcb16fdb6, timestamp=1490649844854}]
3572 INFO .ChannelSyncConsumer| Message was sent to reply channel: 'org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#66f66866'
3572 INFO citrus.Citrus|
3572 DEBUG citrus.Citrus| TEST STEP 3/4 SUCCESS
3572 INFO citrus.Citrus|
3572 DEBUG citrus.Citrus| TEST STEP 4/4: receive
3572 DEBUG ngCorrelationManager| Get correlation key for 'citrus_message_correlator_ClientEndpoint'
3572 DEBUG ltCorrelationManager| Finding correlated object for 'citrus_message_id = '69b4056a-cae6-4004-a837-41de069865b8''
3572 DEBUG citrus.RetryLogger| Reply message did not arrive yet - retrying in 500ms
3575 DEBUG http.HttpParser| reset HttpParser{s=END,214 of 214}
3576 DEBUG http.HttpParser| END --> START
3576 DEBUG server.HttpChannel| HttpChannelOverHttp#4820529f{r=1,c=false,a=IDLE,uri=} handle exit, result COMPLETE
3576 DEBUG io.ChannelEndPoint| filled 0 SelectChannelEndPoint#1c2a40b4{/127.0.0.1:54924<->8085,Open,in,out,-,-,13/30000,HttpConnection}{io=0,kio=0,kro=1}
3576 DEBUG io.ChannelEndPoint| filled 0 SelectChannelEndPoint#1c2a40b4{/127.0.0.1:54924<->8085,Open,in,out,-,-,13/30000,HttpConnection}{io=0,kio=0,kro=1}
3576 DEBUG http.HttpParser| parseNext s=START HeapByteBuffer#6b681750[p=0,l=0,c=16384,r=0]={<<<>>>POST / HTTP/1.1\r\n...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}
3576 DEBUG o.AbstractConnection| fillInterested HttpConnection#2c01adea[FILLING,SelectChannelEndPoint#1c2a40b4{/127.0.0.1:54924<->8085,Open,in,out,-,-,13/30000,HttpConnection}{io=0,kio=0,kro=1}][p=HttpParser{s=START,0 of -1},g=HttpGenerator{s=START},c=HttpChannelOverHttp#4820529f{r=1,c=false,a=IDLE,uri=}]
3576 DEBUG o.AbstractConnection| FILLING-->FILLING_FILL_INTERESTED HttpConnection#2c01adea[FILLING_FILL_INTERESTED,SelectChannelEndPoint#1c2a40b4{/127.0.0.1:54924<->8085,Open,in,out,-,-,13/30000,HttpConnection}{io=0,kio=0,kro=1}][p=HttpParser{s=START,0 of -1},g=HttpGenerator{s=START},c=HttpChannelOverHttp#4820529f{r=1,c=false,a=IDLE,uri=}]
3577 DEBUG o.AbstractConnection| FILLING_FILL_INTERESTED-->FILL_INTERESTED HttpConnection#2c01adea[FILL_INTERESTED,SelectChannelEndPoint#1c2a40b4{/127.0.0.1:54924<->8085,Open,in,out,-,-,13/30000,HttpConnection}{io=0,kio=0,kro=1}][p=HttpParser{s=START,0 of -1},g=HttpGenerator{s=START},c=HttpChannelOverHttp#4820529f{r=1,c=false,a=IDLE,uri=}]
3577 DEBUG electChannelEndPoint| Local interests updating 0 -> 1 for SelectChannelEndPoint#1c2a40b4{/127.0.0.1:54924<->8085,Open,in,out,R,-,0/30000,HttpConnection}{io=1,kio=0,kro=1}
3577 DEBUG io.SelectorManager| Queued change org.eclipse.jetty.io.SelectChannelEndPoint$1#67e66c5c
3578 DEBUG io.SelectorManager| Selector loop woken up from select, 0/1 selected
3578 DEBUG io.SelectorManager| Running change org.eclipse.jetty.io.SelectChannelEndPoint$1#67e66c5c
3578 DEBUG electChannelEndPoint| Key interests updated 0 -> 1 on SelectChannelEndPoint#1c2a40b4{/127.0.0.1:54924<->8085,Open,in,out,R,-,1/30000,HttpConnection}{io=1,kio=1,kro=1}
3578 DEBUG io.SelectorManager| Selector loop waiting on select
4072 DEBUG ltCorrelationManager| Finding correlated object for 'citrus_message_id = '69b4056a-cae6-4004-a837-41de069865b8''
4072 DEBUG citrus.RetryLogger| Reply message did not arrive yet - retrying in 500ms
4573 DEBUG ltCorrelationManager| Finding correlated object for 'citrus_message_id = '69b4056a-cae6-4004-a837-41de069865b8''
4573 DEBUG citrus.RetryLogger| Reply message did not arrive yet - retrying in 500ms
5073 DEBUG ltCorrelationManager| Finding correlated object for 'citrus_message_id = '69b4056a-cae6-4004-a837-41de069865b8''
5073 DEBUG citrus.RetryLogger| Reply message did not arrive yet - retrying in 500ms
5574 DEBUG ltCorrelationManager| Finding correlated object for 'citrus_message_id = '69b4056a-cae6-4004-a837-41de069865b8''
5574 DEBUG citrus.RetryLogger| Reply message did not arrive yet - retrying in 500ms
6075 DEBUG ltCorrelationManager| Finding correlated object for 'citrus_message_id = '69b4056a-cae6-4004-a837-41de069865b8''
6075 DEBUG citrus.RetryLogger| Reply message did not arrive yet - retrying in 500ms
6575 DEBUG ltCorrelationManager| Finding correlated object for 'citrus_message_id = '69b4056a-cae6-4004-a837-41de069865b8''
6575 DEBUG citrus.RetryLogger| Reply message did not arrive yet - retrying in 500ms
7075 DEBUG ltCorrelationManager| Finding correlated object for 'citrus_message_id = '69b4056a-cae6-4004-a837-41de069865b8''
7075 DEBUG citrus.RetryLogger| Reply message did not arrive yet - retrying in 500ms
7576 DEBUG ltCorrelationManager| Finding correlated object for 'citrus_message_id = '69b4056a-cae6-4004-a837-41de069865b8''
7576 DEBUG citrus.RetryLogger| Reply message did not arrive yet - retrying in 500ms
8076 DEBUG ltCorrelationManager| Finding correlated object for 'citrus_message_id = '69b4056a-cae6-4004-a837-41de069865b8''
8076 DEBUG citrus.RetryLogger| Reply message did not arrive yet - retrying in 500ms
8576 DEBUG ltCorrelationManager| Finding correlated object for 'citrus_message_id = '69b4056a-cae6-4004-a837-41de069865b8''
8580 INFO report.JIRAConsumer| Invoking JIRA API...
works.integration.jira.exceptions.ServiceBindingException: Missing Required Properties - SUMMARY, EXMESSAGE, DETAILEDEXCEPTION, PROJECT
You have to add a fork(true) option to the first `soap().client().send()' action because the Http SOAP protocol is synchronous by nature. The first action in your test waits for a synchronous response and blocks the rest of the test case execution.
Obviously your test needs to receive some other messages before that client response with soap().server(). That is why you need to fork the client send action in the first place so the server actions can perform before the client response has arrived.
In general the SOAP components in Citrus automatically handle SOAP Envelope and SOAP body. So you just need to define the pure body content as payload. SOAP Envelope is added automatically.
Hope this is more clear now.