I have a spring-boot application where using webflux and r2dbc-postgres. I have discovered a strange issue when trying to do some db operations in a flatMap().
Code example:
#Transactional
public Mono<Void> insertDummyFooBars() {
return Flux.fromIterable(IntStream.rangeClosed(1, 260).boxed().collect(Collectors.toList()))
.log()
.flatMap(i -> this.repository.save(FooBar.builder().foo("test-" + i).build()))
.log()
.concatMap(i -> this.repository.findAll())
.then();
}
It seems like flatMap can process max 256 elements in batches. (Queues.SMALL_BUFFER_SIZE default value is 256). So when I tried to run the code above (with 260 elements) I've got an exception - TransientDataAccessResourceException and the following message:
Cannot exchange messages because the request queue limit is exceeded; nested exception is io.r2dbc.postgresql.client.ReactorNettyClient$RequestQueueException
There is no Releasing R2DBC Connection after this exception. The pgdb connection/session remains in idle in transaction state and the app is not able to run properly when pool max size is reached and all of the connections are in idle in transaction state. I think the connection should be released even if an exception happened or not.
If I use concatMap instead of flatMap it works as expected - no exception, connection released! It's also ok with flatMap when the elements are less than or equal to 256.
Is it possible to force pgdb connection closure? What should I do If I have lot of db operations in flatMap like this? Should I replace all of them with concatMap? Is there a global solution for this?
Versions:
Postgres: 12.6, Spring-boot: 2.7.6
Demo project
LOG:
2022-12-08 16:32:13.092 INFO 17932 --- [actor-tcp-nio-1] reactor.Flux.Iterable.1 : | onNext(256)
2022-12-08 16:32:13.092 DEBUG 17932 --- [actor-tcp-nio-1] o.s.r2dbc.core.DefaultDatabaseClient : Executing SQL statement [INSERT INTO foo_bar (foo) VALUES ($1)]
2022-12-08 16:32:13.114 INFO 17932 --- [actor-tcp-nio-1] reactor.Flux.FlatMap.2 : onNext(FooBar(id=258, foo=test-1))
2022-12-08 16:32:13.143 DEBUG 17932 --- [actor-tcp-nio-1] o.s.r2dbc.core.DefaultDatabaseClient : Executing SQL statement [SELECT foo_bar.* FROM foo_bar]
2022-12-08 16:32:13.143 INFO 17932 --- [actor-tcp-nio-1] reactor.Flux.Iterable.1 : | request(1)
2022-12-08 16:32:13.143 INFO 17932 --- [actor-tcp-nio-1] reactor.Flux.Iterable.1 : | onNext(257)
2022-12-08 16:32:13.144 DEBUG 17932 --- [actor-tcp-nio-1] o.s.r2dbc.core.DefaultDatabaseClient : Executing SQL statement [INSERT INTO foo_bar (foo) VALUES ($1)]
2022-12-08 16:32:13.149 INFO 17932 --- [actor-tcp-nio-1] reactor.Flux.Iterable.1 : | onComplete()
2022-12-08 16:32:13.149 INFO 17932 --- [actor-tcp-nio-1] reactor.Flux.Iterable.1 : | cancel()
2022-12-08 16:32:13.160 ERROR 17932 --- [actor-tcp-nio-1] reactor.Flux.FlatMap.2 : onError(org.springframework.dao.TransientDataAccessResourceException: executeMany; SQL [INSERT INTO foo_bar (foo) VALUES ($1)]; Cannot exchange messages because the request queue limit is exceeded; nested exception is io.r2dbc.postgresql.client.ReactorNettyClient$RequestQueueException: [08006] Cannot exchange messages because the request queue limit is exceeded)
2022-12-08 16:32:13.167 ERROR 17932 --- [actor-tcp-nio-1] reactor.Flux.FlatMap.2 :
org.springframework.dao.TransientDataAccessResourceException: executeMany; SQL [INSERT INTO foo_bar (foo) VALUES ($1)]; Cannot exchange messages because the request queue limit is exceeded; nested exception is io.r2dbc.postgresql.client.ReactorNettyClient$RequestQueueException: [08006] Cannot exchange messages because the request queue limit is exceeded
at org.springframework.r2dbc.connection.ConnectionFactoryUtils.convertR2dbcException(ConnectionFactoryUtils.java:215) ~[spring-r2dbc-5.3.24.jar:5.3.24]
at org.springframework.r2dbc.core.DefaultDatabaseClient.lambda$inConnectionMany$8(DefaultDatabaseClient.java:147) ~[spring-r2dbc-5.3.24.jar:5.3.24]
at reactor.core.publisher.Flux.lambda$onErrorMap$29(Flux.java:7105) ~[reactor-core-3.4.25.jar:3.4.25]
at reactor.core.publisher.Flux.lambda$onErrorResume$30(Flux.java:7158) ~[reactor-core-3.4.25.jar:3.4.25]
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:94) ~[reactor-core-3.4.25.jar:3.4.25]
I have tried to change the Queues.SMALL_BUFFER_SIZE, and also tried to add a concurrency value to the flatmap. It works when I reduced the value to 255 but I think it is not a good solution.
I have a Spring Boot application protected by Keycloak (15.0.2).
Let's say that I have a browser with two tabs opened for the same user and that user is authenticated.
Now I logout in tab 1 and I get the Keycloak login form with the following Url:
http://localhost:8180/auth/realms/AUTOTEST_PG/protocol/openid-connect/auth?response_type=code&client_id=mcd-client&state=e01b8a88-6945-4c56-9ee1-46fb156d33be&login=true&scope=openid
Now I logout in tab 2 and I get the Keycloak login form with the following Url:
http://localhost:8180/auth/realms/AUTOTEST_PG/protocol/openid-connect/auth?response_type=code&client_id=mcd-client&state=b9607387-7a05-4226-8313-4c5c80a1b145&login=true&scope=openid
Next, I tried to login in tab 1 and I get a 400 Bad Request response.
Looking at the Spring Boot log I get the following:
14554520 2021-12-14 00:02:06,100 [ajp-nio-0.0.0.0-8009-exec-8] DEBUG sample.controller.filters.MyKeycloakAuthenticationProcessingFilter ?:? - - - - - attemptAuthentication
14554520 2021-12-14 00:02:06,100 [ajp-nio-0.0.0.0-8009-exec-8] DEBUG sample.security.RealmProvider ?:? - - - - - getRealmName - requestUri: https://10.161.54.36/sso/login?redirect_url=/sctools/&state=e01b8a88-6945-4c56-9ee1-46fb156d33be&session_state=4fe8d159-5ada-4522-a422-07c5a0ce9d15&code=c5dda8ef-0fd9-4323-9922-fa5c73b28f89.4fe8d159-5ada-4522-a422-07c5a0ce9d15.442a440e-6df0-40c7-b8cb-3a52123430a0
14554520 2021-12-14 00:02:06,100 [ajp-nio-0.0.0.0-8009-exec-8] WARN org.keycloak.adapters.OAuthRequestAuthenticator ?:? - - - - - state parameter invalid
14554520 2021-12-14 00:02:06,100 [ajp-nio-0.0.0.0-8009-exec-8] WARN org.keycloak.adapters.OAuthRequestAuthenticator ?:? - - - - - cookie: b9607387-7a05-4226-8313-4c5c80a1b145
14554520 2021-12-14 00:02:06,100 [ajp-nio-0.0.0.0-8009-exec-8] WARN org.keycloak.adapters.OAuthRequestAuthenticator ?:? - - - - - queryParam: e01b8a88-6945-4c56-9ee1-46fb156d33be
14554521 2021-12-14 00:02:06,101 [ajp-nio-0.0.0.0-8009-exec-8] DEBUG sample.controller.filters.MyKeycloakAuthenticationProcessingFilter ?:? - - - - - unsuccessfulAuthentication - committed: true
14554521 2021-12-14 00:02:06,101 [ajp-nio-0.0.0.0-8009-exec-8] DEBUG sample.security.MyKeycloakAuthenticationFailureHandler ?:? - - - - - onAuthenticationFailure org.keycloak.adapters.springsecurity.KeycloakAuthenticationException: Invalid authorization header, see WWW-Authenticate header for details
14554521 2021-12-14 00:02:06,101 [ajp-nio-0.0.0.0-8009-exec-8] DEBUG sample.security.MyKeycloakAuthenticationFailureHandler ?:? - - - - - onAuthenticationFailure - response isCommitted - Status: 400
It seems that the cookie sent and the state value doesn't match and that is the reason why I get a bad request.
How can I login from any of the tabs without getting the Bad Request ?
Note: If I go to the tab2 and login, then I get a message telling me that the user is already login.
UPDATE
When doing the same operation on the Keycloak console, if the user logout from one tab the second tab is automatically logout and the Url doesn't get an state variable it gets a challenge variable, something like:
http://10.161.54.36:8180/auth/realms/master/protocol/openid-connect/auth?client_id=security-admin-console&redirect_uri=http%3A%2F%2F10.161.54.36%3A8180%2Fauth%2Fadmin%2Fmaster%2Fconsole%2F%23%2Frealms&state=c3015df8-4c50-4454-be8d-0555f25e3bd0&response_mode=fragment&response_type=code&scope=openid&nonce=acb2a07d-c4a3-4cd3-8a7a-e23580a97d14&code_challenge=2W6-09eeD_WEwtWct3a5MojpIQJMe-9brcOH-7fbT6A&code_challenge_method=S256
How is it done ?
I am following the NiFi 1.10 stateless guildeline to create a simple process group of executing a sql in mysql db. I have put necessary parm of db controller service to parameter context.
it works well in nifi canvas. Then i add it to registry and prepare a json parm file: stateless-simpledb.json
{
"registryUrl": "http://localhost:18080",
"bucketId": "cac8f127-e328-45c1-a4cb-0e03dc837ceb",
"flowId": "cc2753f2-78f3-4449-a2fd-343dfeaafe15",
"flowVersion": "3",
"parameters": {
"lastIngestId" : "20000",
"mysql-jdbc-driver-name" : "com.mysql.jdbc.Driver",
"db-user" : "root",
"db-password" : "password",
"db-con-url" : "jdbc:mysql://localhost:3306/mms",
"jdbc-jar-path" : "/program/jdbc/mysql-connector-java.jar"
}
}
and run the one-off command:
/program/nifi/bin/nifi.sh stateless RunFromRegistry Once --file /app/poc/nifi-stateless/conf/stateless-simpledb.json
It raise error:
=== FlowFileRepository Type ===
org.apache.nifi.controller.repository.RocksDBFlowFileRepository
org.apache.nifi:nifi-framework-nar:1.10.0 || /program/nifi-1.10.0/work/stateless-nars/nifi-framework-nar-1.10.0.nar-unpacked
org.apache.nifi.controller.repository.WriteAheadFlowFileRepository
org.apache.nifi:nifi-framework-nar:1.10.0 || /program/nifi-1.10.0/work/stateless-nars/nifi-framework-nar-1.10.0.nar-unpacked
org.apache.nifi.controller.repository.VolatileFlowFileRepository
org.apache.nifi:nifi-framework-nar:1.10.0 || /program/nifi-1.10.0/work/stateless-nars/nifi-framework-nar-1.10.0.nar-unpacked
=== End FlowFileRepository types ===
23:32:32.626 [main] INFO org.apache.nifi.stateless.bootstrap.ExtensionDiscovery - Successfully discovered extensions in 4411 milliseconds
23:32:32.633 [main] DEBUG org.apache.nifi.stateless.core.ComponentFactory - Setting context class loader to org.apache.nifi.nar.InstanceClassLoader#50fa5938 (parent = org.apache.nifi.nar.NarClassLoader[/program/nifi-1.10.0/work/stateless-nars/nifi-dbcp-service-nar-1.10.0.nar-unpacked]) to create org.apache.nifi.dbcp.DBCPConnectionPool
23:32:32.647 [main] DEBUG org.apache.nifi.parameter.ExpressionLanguageAwareParameterParser - For input #{jdbc-jar-path} found 1 Parameter references: [org.apache.nifi.parameter.StandardParameterReference#2d3eecda]
23:32:32.650 [main] DEBUG org.apache.nifi.parameter.ExpressionLanguageAwareParameterParser - For input /program/jdbc/mysql-connector-java.jar found 0 Parameter references: []
23:32:32.651 [main] DEBUG org.apache.nifi.parameter.ExpressionLanguageAwareParameterParser - For input 500 millis found 0 Parameter references: []
23:32:32.651 [main] DEBUG org.apache.nifi.parameter.ExpressionLanguageAwareParameterParser - For input 8 found 0 Parameter references: []
23:32:32.651 [main] DEBUG org.apache.nifi.parameter.ExpressionLanguageAwareParameterParser - For input 0 found 0 Parameter references: []
23:32:32.651 [main] DEBUG org.apache.nifi.parameter.ExpressionLanguageAwareParameterParser - For input 8 found 0 Parameter references: []
23:32:32.651 [main] DEBUG org.apache.nifi.parameter.ExpressionLanguageAwareParameterParser - For input -1 found 0 Parameter references: []
23:32:32.651 [main] DEBUG org.apache.nifi.parameter.ExpressionLanguageAwareParameterParser - For input -1 found 0 Parameter references: []
23:32:32.651 [main] DEBUG org.apache.nifi.parameter.ExpressionLanguageAwareParameterParser - For input 30 mins found 0 Parameter references: []
23:32:32.651 [main] DEBUG org.apache.nifi.parameter.ExpressionLanguageAwareParameterParser - For input -1 found 0 Parameter references: []
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.nifi.bootstrap.RunStatelessNiFi.main(RunStatelessNiFi.java:69)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.nifi.StatelessNiFi.main(StatelessNiFi.java:103)
... 5 more
Caused by: java.lang.RuntimeException: Failed to enable Controller Service {id=691ecc97-ff46-3a5e-8aad-37dc568bc247, name=MYSQL-MMS-stateless-test, type=class org.apache.nifi.dbcp.DBCPConnectionPool} because validation failed: ['Database Connection URL' is invalid because Database Connection URL is required, 'Database Driver Class Name' is invalid because Database Driver Class Name is required]
at org.apache.nifi.stateless.core.StatelessControllerServiceLookup.enableControllerServices(StatelessControllerServiceLookup.java:133)
at org.apache.nifi.stateless.core.StatelessFlow.<init>(StatelessFlow.java:153)
at org.apache.nifi.stateless.core.StatelessFlow.createAndEnqueueFromJSON(StatelessFlow.java:469)
at org.apache.nifi.stateless.runtimes.Program.runLocal(Program.java:133)
at org.apache.nifi.stateless.runtimes.Program.launch(Program.java:67)
... 10 more
Seems the apache nifi stateless function failed to set controller service even it's in "process group" scope.
Would anyone has any advice?
As mentioned in the comments, this appears to be a known problem with the validation of controller services.
This can be avoided by using Nifi 1.12 and above as it got fixed in the following jira: https://issues.apache.org/jira/plugins/servlet/mobile#issue/NIFI-7380
Though I am not entirely sure of this, it may also be possible that this simply indicates that your controller service is not configured correctly. This would be worth double checking.
guys !
I notices that from time to time, my response is generated about approx. 10 sec. which is not normal. This normally happens when i submit a post form, but not always. I'm using Spring Boot 1.5.6 with Thymeleaf 3.0.2.
2017-10-16 09:09:21.737 TRACE 19366 --- [https-jsse-nio-8443-exec-1] o.t.s.e.LinkExpression : [THYMELEAF][https-jsse-nio-8443-exec-1] Evaluating link: "#{'/resources/js/lightbox.js'}"
2017-10-16 09:09:21.737 TRACE 19366 --- [https-jsse-nio-8443-exec-1] o.t.s.e.TextLiteralExpression : [THYMELEAF][https-jsse-nio-8443-exec-1] Evaluating text literal: "'/resources/js/lightbox.js'"
2017-10-16 09:09:21.737 TRACE 19366 --- [https-jsse-nio-8443-exec-1] o.t.TemplateEngine : [THYMELEAF][https-jsse-nio-8443-exec-1] FINISHED PROCESS AND OUTPUT OF TEMPLATE "distributor/createDistributor" WITH LOCALE en_US
2017-10-16 09:09:21.737 TRACE 19366 --- [https-jsse-nio-8443-exec-1] o.t.T.TIMER : [THYMELEAF][https-jsse-nio-8443-exec-1][distributor/createDistributor][en_US][8336337999][8336] TEMPLATE "distributor/createDistributor" WITH LOCALE en_US PROCESSED IN 8336337999 nanoseconds (approx. 8336ms)
2017-10-16 09:09:21.738 TRACE 19366 --- [https-jsse-nio-8443-exec-1] ationConfigEmbeddedWebApplicationContext : Publishing event in org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext#1c6b6478: ServletRequestHandledEvent: url=[/distributor/add]; client=[192........]; method=[GET]; servlet=[dispatcherServlet]; session=[F2E366C8DA7EC3C5BD2026FA388F3437]; user=[admin]; time=[8338ms]; status=[OK]
On line 4 ( TIMER ) is showing that processing time is time=[8338ms]
Can somebody help me how to debug this ?
I'm trying to configure a seqhilo generator for a Hibernate application on Oracle.
<id name="idTest" type="int">
<column name="ID_TEST" precision="6" scale="0" />
<generator class="seqhilo">
<param name="sequence">S_TEST</param>
<param name="max_lo">1000</param>
</generator>
</id>
I created a sequence(S_TEST) in oracle database 10g, Unfortunately its not working: the id is always null.
could you explain to me how to use seqhilo generator within oracle database, may be i'm confused :(
here is the generated sql trace:
08:52:44,441 DEBUG AnnotationTransactionAttributeSource:107 - Adding transactional method [create] with attribute [PROPAGATION_REQUIRED,ISOLATION_DEFAULT]
08:52:44,441 DEBUG HibernateTransactionManager:346 - Using transaction object [org.springframework.orm.hibernate3.HibernateTransactionManager$HibernateTransactionObject#1786a3c]
08:52:44,441 DEBUG HibernateTransactionManager:374 - Creating new transaction with name [com.cylande.utilities.GenericDAO.create]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
08:52:44,472 DEBUG HibernateTransactionManager:496 - Opened new Session [org.hibernate.impl.SessionImpl#1bf68a9] for Hibernate transaction
08:52:44,472 DEBUG HibernateTransactionManager:507 - Preparing JDBC Connection of Hibernate Session [org.hibernate.impl.SessionImpl#1bf68a9]
08:52:44,487 DEBUG DriverManagerDataSource:163 - Creating new JDBC DriverManager Connection to [jdbc:oracle:thin:#localhost:1521:orcl]
08:52:44,519 DEBUG HibernateTransactionManager:572 - Exposing Hibernate transaction as JDBC transaction [oracle.jdbc.driver.T4CConnection#8c7be5]
08:52:44,519 DEBUG TransactionSynchronizationManager:186 - Bound value [org.springframework.jdbc.datasource.ConnectionHolder#11a0d35] for key [org.springframework.jdbc.datasource.DriverManagerDataSource#13b5a3a] to thread [main]
08:52:44,519 DEBUG TransactionSynchronizationManager:186 - Bound value [org.springframework.orm.hibernate3.SessionHolder#12c4c57] for key [org.hibernate.impl.SessionFactoryImpl#1594a88] to thread [main]
08:52:44,519 DEBUG TransactionSynchronizationManager:261 - Initializing transaction synchronization
08:52:44,534 DEBUG TransactionInterceptor:290 - Getting transaction for [com.cylande.utilities.GenericDAO.create]
08:52:44,534 DEBUG TransactionSynchronizationManager:142 - Retrieved value [org.springframework.orm.hibernate3.SessionHolder#12c4c57] for key [org.hibernate.impl.SessionFactoryImpl#1594a88] bound to thread [main]
08:52:44,550 DEBUG SQL:102 - select categorie_.ID_CATEGORIE, categorie_.CATEGORIE as CATEGORIE9_ from AHMED.CATEGORIE categorie_ where categorie_.ID_CATEGORIE=?
08:52:44,550 TRACE IntegerType:128 - binding '1' to parameter: 1
08:52:44,550 TRACE StringType:170 - returning 'Test dintegration' as column: CATEGORIE9_
08:52:44,550 DEBUG TransactionInterceptor:319 - Completing transaction for [com.cylande.utilities.GenericDAO.create]
08:52:44,550 DEBUG HibernateTransactionManager:880 - Triggering beforeCommit synchronization
08:52:44,550 DEBUG HibernateTransactionManager:893 - Triggering beforeCompletion synchronization
08:52:44,550 DEBUG HibernateTransactionManager:707 - Initiating transaction commit
08:52:44,566 DEBUG HibernateTransactionManager:651 - Committing Hibernate transaction on Session [org.hibernate.impl.SessionImpl#1bf68a9]
08:52:44,566 DEBUG SQL:102 - insert into AHMED.TEST (ID_APPLICATION, ID_MODULE, ID_CATEGORIE, ID_PAGE, NOM_TEST, DESCRIPTION_TEST, ID_TEST) values (?, ?, ?, ?, ?, ?, ?)
08:52:44,566 TRACE IntegerType:121 - binding null to parameter: 1
08:52:44,566 TRACE IntegerType:121 - binding null to parameter: 2
08:52:44,566 TRACE IntegerType:128 - binding '1' to parameter: 3
08:52:44,566 TRACE IntegerType:121 - binding null to parameter: 4
08:52:44,566 TRACE StringType:128 - binding 'nomTest2' to parameter: 5
08:52:44,566 TRACE IntegerType:128 - binding '0' to parameter: 7
08:52:44,581 WARN JDBCExceptionReporter:77 - SQL Error: 1, SQLState: 23000
08:52:44,581 ERROR JDBCExceptionReporter:78 - ORA-00001: violation de contrainte unique
the field that must be generated (id_Test) is always equal to zero even if i changed the generator to sequence or native the result remain the same
The declaration looks correct, or at least very close to the sample from the documentation:
5.1.4.2. Hi/lo algorithm
The hilo and seqhilo generators
provide two alternate implementations
of the hi/lo algorithm. The first
implementation requires a "special"
database table to hold the next
available "hi" value. Where supported,
the second uses an Oracle-style
sequence.
<id name="id" type="long" column="cat_id">
<generator class="seqhilo">
<param name="sequence">hi_value</param>
<param name="max_lo">100</param>
</generator>
</id>
But in order to debug the issue, I would try to
activate the logging of the generated SQL to see what is happening exactly (and post the result)
stick to the above example i.e. define the column in the id element and use a long
if this works, start modifying the configuration