In the WSO2 documentation you state that the embedded H2 database is suitable for development. However, for most enterprise testing and production environments we recommend an industry-standard RDBMS such as Oracle, PostgreSQL, MySQL, MS SQL, etc. (https://docs.wso2.com/display/Carbon420/Working+with+Databases)
Can you kindly give us some use cases where H2 Databases are suitable for production environments?
How can we better assess if the H2 Database is suitable or not for our production environment?
We do not recommend H2 Database to be used in production. However H2 is recommended to be used as the "Local" Registry. See "Registry and governance" in Production Deployment Guidelines.
See also Sharing Databases
I answered a similar question before: https://stackoverflow.com/a/23090822/1955702
In my case, it was perfectly suitable for testing project on build time. In-memory database have been used to store some lookup table values. Very simple usage as follows:
pom.xml
...
<dependencies>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<version>1.4.193</version>
<scope>testing</scope>
</dependency>
...
</dependencies>
...
Test class
...
#BeforeClass
public static void setupOnce() throws SQLException {
conn = DriverManager
.getConnection("jdbc:h2:mem:test;MODE=Oracle;INIT=runscript from 'classpath:scripts/test.sql'");
dao = new Dao(conn);
...
}
Related
Im working in a quarkus project, I have to connect to an elasticsearch clusert and in production exists a mysql database with data.
Im thinking about using Hibernate Search but I have some questions.
1-Which version of hibernate search use quarkus? In the pom is not specified. Is 6?
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-hibernate-search-orm-elasticsearch</artifactId>
</dependency>
2-It compatible with elasticsearch 7.11.1?
3-In my project I will connect to the mysql database just once to initialize all the index, then the connection is going to be closed, is this possible? or hibernate search needs to be connected to mysql database always?
4-To initialize the indexs with hibernate search is mandatory to use hibernate annotations (for example #Entity and #Column) in the entitys?
5-As I said, the connection with mysql database is going to be close after first indexing, is there a way to add new records to index if I get a list of objects from other system? (for example something like batch)
Thanks
It's Hibernate Search 6 - in Quarkus 1.13, 6.0.2.Final
Yes, it should be. Our main testing is now against the latest Open Source version of Elasticsearch but we are still testing 7.11.
Hibernate Search handles reads/writes and also hydrate your search data from the database so you should have the MySQL database around. If you are only doing read-only stuff AND only using projections, maybe not having the database around is possible but I don't think it's a supported use case
Yes.
You will have to implement it yourself, there's nothing built-in.
I am working on a micro service oriented multi-tenant application using Spring MVC, Spring Boot and PostgreSQL. In my service domain, I have different 30 number of Spring Boot projects connecting with same database.
Issue
When I am starting more than 11 Spring Boot project parallel, I am not able to retrieve data from database. I am only getting blank JSON response. But If I am starting only less than 11 projects, then at that time, I am able to get the micro service result. Problem getting when I am starting from 12th service.
Troubleshooting and my Investigation
According to reading, I made little bit changes as follows,
I set PostgreSQL config file, max_connections = 100 and shared_buffers = 128MB.
But that not solved my problem. And also added additional lines in my application.property for hikari connection pool like the following,
spring.datasource.hikari.minimumIdle=3
spring.datasource.hikari.maximum-pool-size=3
Even this also not fixing my issue. Still I am only able to start maximum 12 Spring Boot projects at one time.
On pom.xml added the following,
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
<exclusions>
<exclusion>
<groupId>com.zaxxer</groupId>
<artifactId>HikariCP</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.tomcat</groupId>
<artifactId>tomcat-jdbc</artifactId>
<version>9.0.10</version>
</dependency>
application.property updated with the following,
spring.datasource.tomcat.initial-size=15
spring.datasource.tomcat.max-wait=20000
spring.datasource.tomcat.max-active=20
spring.datasource.tomcat.max-idle=20
spring.datasource.tomcat.min-idle=5
spring.datasource.tomcat.default-auto-commit=true
Updates
As I mentioned above, in total I have 100 max connection. When I am starting first microservice and monitoring the DB connection by querying (below query), I am getting 10 connections.
select max_conn,used,res_for_super,max_conn-used-res_for_super res_for_normal
from
(select count(*) used from pg_stat_activity) t1,
(select setting::int res_for_super from pg_settings where
name=$$superuser_reserved_connections$$) t2,
(select setting::int max_conn from pg_settings where
name=$$max_connections$$) t3
Activity screenshot - When I started only 1 microservice I am getting like these,
Activity screenshot 2:
So here, for each micro service starting, 10 Database connections are initiating at the back end database level. So if I am starting more than 30 microservice application, it will go to more than 300 connection.
Confusion
Is this because of any database default settings?
Or is this because of Tomcat problem when connecting to DB engine?
Even I tried to use connection pool tool pgbouncer in my database node. Then also I am getting the same problems.
If this is due to database default settings or configuration, how can I override this?
A micro-service architecture is a kind of architecture in which an application is broken down into multiple independent services which are easily to understand, develop, test and are error-prone.
Now according to above definition, each independent service should have their own separate independent databases. But those services can inter-communicate for data if needed.
Each service should have a selected entities in their particular database.
If you have a common database for all those independent services, then you are violating the rule of micro-service architecture.
In this case, there is a high risk of data redundancy.
May be in your case, it is happening that services are containing duplicate version of the data and potentially using up the whole connection pool.
Better approach is to have separate database (containing only the entities specific to that service only) for separate independent service.
Clarification:
Explain above.
Maintain separate datasource hikari connection pool for all the spring boot modules/projects.
For example : Suppose you have two datasource for two modules/projects -
In application.properties.
## datasource for first service/module ##
datasource1.url=jdbc:postgresql://localhost:3306/service1
datasource1.username=service1
datasource1.password=password1
datasource1.driver-class-name=org.postgresql.Driver
datasource1.initial-size=15
datasource1.max-wait=20000
datasource1.max-active=20
datasource1.max-idle=20
datasource1.min-idle=5
datasource1.default-auto-commit=true
## datasource for second service/module ##
datasource2.url=jdbc:postgresql://localhost:3306/service2
datasource2.username=service2
datasource2.password=password2
datasource2.driver-class-name=org.postgresql.Driver
datasource2.initial-size=15
datasource2.max-wait=20000
datasource2.max-active=20
datasource2.max-idle=20
datasource2.min-idle=5
datasource2.default-auto-commit=true
Follow this links to read how to configure :
https://blogs.ashrithgn.com/multiple-data-source-in-spring-boot-using-spring-boot-data-starter/
https://www.ru-rocker.com/2018/01/28/configure-multiple-data-source-spring-boot/
Or you may need to go for Hibernate Multi-tenancy with Hikari-CP.
Is Spring Data JDBC v1.1.5 recommended for Oracle Database and Enterprise Applications? Lot of samples around the net based on Open Source RDBMS (H2 or PostgreSQL). We are using Spring Data JDBC in a Spring Boot Microservice Application, facing following problems.
Force to write custom converters for oracle.sql.TIMESTAMP, oracle.sql.TIMESTAMPTZ and oracle.sql.DATE and oracle.sql.ROWID etc..
Can't type cast oracle.sql.ROWID to java.lang.Number
Identity must not be null after save.
Spring Data JDBC is absolutely recommended for Enterprise Applications.
Not so much for use with Oracle.
Since the necessary resources (database & JDBC driver) weren't available in a form that could be easily used in integration tests on public platforms, Oracle isn't included in regular builds.
Therefore it is likely that one encounters issues when working with Oracle.
Some are already known, for others issues in Jira or even PRs are highly appreciated.
I am trying to connect my springboot app to my netezza database but no luck. I am trying to use the tutorial at: https://www.callicoder.com/spring-boot-rest-api-tutorial-with-mysql-jpa-hibernate/
I replaced the spring.datasource.url with my netezza database url but no luck. How can I connect to netezza please?
Next to properly configuring spring.datasource.url, you also have to add the Netezza JDBC driver to your classpath. According to this article on the IBM Knowledge Center, you should be able to download the client tools for the right environment:
nz-*client-version.tar.z
Netezza client installation packages for the supported client
operating systems. The IBM Netezza client packages are available for a
common group of operating system environments such as:
nz-linuxclient-version.tar.gz for Linux on Intel/AMD, includes ODBC/JDBC and Netezza command line utilities
...
Additionally to this, you also have to properly configure the dialect. As far as I'm aware, Hibernate (the default JPA provider within Spring boot) has no support for the Netazza dialect. You can find a full list of supported dialects on their Javadoc.
However, since Netazza is based on PostgreSQL 7.x, you might be able to get it to work using one of the PostgreSQL dialects, though there are two remarks:
Hibernate no longer comes with a dialect supporting PostgreSQL 7.x
Netazza didn't maintain backwards compatibility, so it may not work
So to answer your question, it's not officially supported by Hibernate, so it probably won't work. The alternatives are:
Writing your own Hibernate dialect for Netezza. This requires some work, but will allow you to use your dialect for re-use in other projects.
Using the more low-level JdbcTemplate. This means you'll have to work with SQL itself and you may have to do things by yourself.
For the lazy ones like myself,
1) You'll need to install the jar on your machine first, this SO answer explains that.
2) Add the dependencies
<dependency>
<groupId>postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>7.4.1-jdbc3</version>
</dependency>
<dependency>
<groupId>net.sf.squirrel-sql.plugins</groupId>
<artifactId>netezza</artifactId>
<version>3.5.0</version>
</dependency>
<dependency>
<groupId>org.netezza</groupId>
<artifactId>netezza</artifactId>
<version>1.0</version>
</dependency>
3) Netezza custom dialect. Please note, these are the data types that were required for my use case. For any additional data types, you'll have to add them explicitly.
import org.hibernate.dialect.PostgreSQLDialect;
import org.hibernate.type.StandardBasicTypes;
#SuppressWarnings("deprecation")
public class NetezzaDialect extends PostgreSQLDialect {
public NetezzaDialect() {
registerHibernateType(Types.CHAR, StandardBasicTypes.STRING.getName());
registerHibernateType(Types.VARCHAR, StandardBasicTypes.STRING.getName());
registerHibernateType(Types.NVARCHAR, StandardBasicTypes.STRING.getName());
registerHibernateType(Types.BIGINT, StandardBasicTypes.LONG.getName());
registerHibernateType(Types.SMALLINT, StandardBasicTypes.SHORT.getName());
registerHibernateType(Types.DATE, StandardBasicTypes.DATE.getName());
}
}
4) The properties for spring boot
# Netezza properties
spring.datasource.url=jdbc:netezza://{host}:{port}/{db/ catalog}
spring.datasource.username=
spring.datasource.password=
spring.jpa.properties.hibernate.dialect=your.org.config.NetezzaDialect
spring.datasource.driver-class-name=org.netezza.Driver
I uploaded my web application to CloudFoundry, but my native language was broken as below.
enter image description here
I have checked that the DB has broken data.
DB uses ClearDB MySQL Database, which CloudFoundry provides as a service.
I ran the following query.
SHOW VARIABLES LIKE 'c%'
character_set_client utf8
character_set_connection utf8
character_set_database utf8
character_set_filesystem binary
character_set_results utf8
character_set_server latin1
character_set_system utf8
character_sets_dir /usr/share/mysql/charsets/
collation_connection utf8_general_ci
collation_database utf8_general_ci
collation_server latin1_swedish_ci
completion_type NO_CHAIN
concurrent_insert AUTO
connect_timeout 10
I suspect that the part that is 'latin' is guessing, but I do not know how to change that.
Is there someone you can help with this problem?
Thank you.
Depending on the particular ClearDB database server, there will be cases when your database's characters set is latin1 (the default configuration for MySQL older than 8.1).
We have contacted the ClearDB support team, but they were not willing to change this configuration because it could affect other customers hosted on the same server and they offered us a more expensive solution: to have our own dedicated servers.
The good point is that the character set can be configured per connection bases, so we decided to override the default DataSource, autoconfigured by the Java buildpack used by the Cloud Foundry.
In order to override the DataSource, we needed to do several things:
Add Maven dependency for Spring Cloud connectors,
Turn-off the default autoconfiguration of the DataSource bean,
Configure the DataSource bean by using connection parameters provided by the environment.
To add Spring Cloud connectors, add this dependency to your pom.xml:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-cloud-connectors</artifactId>
</dependency>
In order to turn off the auto-configuration, be sure to create Cloud bean in spring-boot cloud profile:
#Configuration
#Profile("cloud")
public class DataSourceConfiguration {
#Bean
public Cloud cloud() {
return new CloudFactory().getCloud();
}
}
Finally, to use database connection parameters hosted within VCAP_SERVICES environment variables, you can reference them in the application-cloud.yml in this way:
spring.datasource.url: ${cloud.services.mysql.connection.jdbcurl}&characterEncoding=utf-8
spring.datasource.username: ${cloud.services.mysql.connection.username}
spring.datasource.password: ${cloud.services.mysql.connection.password}
spring.datasource.testOnBorrow: true
When it comes to customizing DataSource configuration, it is important to pay attention to spring.datasource.url that ends with characterEncoding=utf-8: this is where connections are configured to use the UTF-8 encoding.
Also, mysql is the name of the PCF service instance in our case and can be different in your environment.
Finally, to learn more about binding to data services with Spring Boot, you can read this excellent article: https://spring.io/blog/2015/04/27/binding-to-data-services-with-spring-boot-in-cloud-foundry.