I'm currently writing an integration test with spring boot and a Postgis connection. In my original application a query uses the <-> operator. For my tests I used a h2 in-memory database with the h2gis extension. Unfortunately, the <-> is not recognized and throws a syntax error. Do you have any ideas how to do this with an in-memory database or is there only the chance to run a docker container with a proper postgis database running?
Thank you!
Even if it where possible to run a database similar enough to Postgres, I'd recommend against it.
We now have Testcontainers and can therefore easily start any* database in docker container from our tests. This is preferable, because you are using the actual database you'll also see in production.
any*: Some of the commercial variants are either huge or take a long time to start up, but Postgres works great.
Related
I have written a spring batch solution which currently uses the embedded H2 in-memory database.
The read and write operations uses SOLR Cloud API calls.
Ideally, we dont want to introduce an proper relational database as a job repo database, for the read-write batch operation.
I read that H2 in-memory databases are best used for Dev and Test in spring batch.
Does anyone have experience of using this H2 database in spring batch on a proper live environment dealing with millions of records in the batch processing, but batch job will ran only once a day at most?
If H2 is not stable for prod, I might have to ditch spring batch OMG, or anyother alternatives?
Open to any ideas or references.
Thanks in advance.
H2 is a light-weight Java database, as you mentioned yourself, that it is ideal for dev testing !
When considering production, you might be missing on lot of features which a RDBMS , NoSQL databases provide!
For e.g. Replication, memory and performance optimizations etc.
If frequent reads and writes are concerned and you don't want RDBMS, you may choose MongoDB or Couchbase to manipulate records , they are fast too !
So considering Millions of records, I don't think H2 would be a good choice for production databases
A similar article might throw some light & help you decide !
Are there any reasons why h2 database shouldn't be used in production?
i'm the developper of an open-source application designed for self-hosting. As such, it is used by many people on various systems, OSes, and hardware. It's built using Kotlin and Spring Boot, and uses H2 as the database. The problem i have is that many users are facing database corruption from times to times (it happened to me also a few times).
The corruptions are always of the form:
org.h2.jdbc.JdbcSQLNonTransientException: General error: "java.lang.IllegalStateException: Chunk 20221 not found [1.4.200/9]" [50000-200]
I can't pinpoint any particular reason for those corruptions. The application doesn't use any dangerous H2 options. I couldn't find anything on the web or Stackoverflow that relates to my (imho) mundane use of H2.
I'm using:
Spring Boot 2.2.6.RELEASE
jOOQ 3.13.1 (but i had the same issues before with Hibernate)
H2 1.4.200
The connection string is pretty straightforward:
jdbc:h2:~/.komga/database.h2
The application is doing heavy writes once in a while (during file system scanning), but afterwards it's doing mostly reads.
Would you have suggestions on how to better configure H2 to avoid those issues ?
If your application uses Thread.interrupt() for threads doing calls into embedded database, this is the reason of corruption, don't do this or use the async: filesystem (jdbc:h2:async:…).
If classloader with H2 can be forcibly unloaded (on some application servers, for example) or application is going to be terminated in some abnormal way, you need to close all connections before it or execute the SHUTDOWN command and wait for its completion.
If you try to open the database file with some older version of H2, it can corrupt the file created by a more recent version.
You may run into some bug in H2, so if you can build a standalone test case (Java / JDBC / SQL only, no third-party libraries), you need to fill a new issue on GitHub. You can also try to build H2 from its current sources, there were some changes in the storage backend, but current H2 is very different from 1.4.200 in other aspects and third-partly libraries that you use may be not yet ready to work with it.
You can use the legacy PageStore backend instead of default one by appending ;MV_STORE=FALSE to the JDBC URL when you create your database. This backend uses table-level locks (you need to lock tables in the same order in all your transactions) and doesn't execute commands from different sessions in parallel, but it is more reliable.
You can use a separate H2 Server process; such configuration is usually more reliable than embedded databases, but it works slower.
In any case, with any DBMS, you should create backup copies on regular basis. H2 has BACKUP command for this purpose.
I am learning Spring Boot using open-source projects and have stumbled upon their demo project — PetClinic. It has two possible databases configured: MySQL & HSQLDB, it uses the latter by default.
So I was able to launch the project look at it on localhost and can see that the DB (HSQLDB) is being populated but I was unable to set up a connection through the Intellij IDEA because the project does not specify the path that allows to see the contents of this in-memory DB.
Can anyone please tell me, what am I missing in the process of establishing the connection to HSQLDB here?
Thanks!
You can't connect to an in-memory instance of HSQLDB from another process.
The main drawback [of in-memory mode] is that it is not possible by default to connect to
the database from outside your application. As a result you cannot
check the contents of the database with external tools such as
Database Manager while your application is running.
If you want to do this, you need to run HSQL as a server. More details about how to run it in server mode can be found here.
I have an application pointing to a mysql database.
I have been trying to use DBUnit to set up my tests environments, which works fine.
The problem is that when configuring DBUnit I pointed it to the SAME mysql database. So when DBUnit is executed, it takes the specified dataset.xml and overrides the information from my original database. which makes sense because there is where I am pointing it to.
The question is, am I supposed to create a new database only for tests so my DBUnit can point to it? If so, how would I manage the structure synchronization between my original database and the one for tests?
Thanks in advance.
am I supposed to create a new database only for tests so my DBUnit can point to it?
It is a better approach to do so as it eliminates multiple problems.
how would I manage the structure synchronization between my original database and the one for tests?
You don't mention tech in your persistence stack, such as Hibernate, Spring/Spring Boot/Spring Data/Flyway/LiquiBase/etc. to suggest more of how to implement this. In general, run DDL in the schema at tests run startup (either from managed DDL from something like Flyway or auto-generation from Hibernate).
Additionally, my preferred and typical testing approach is with:
An in-memory/embedded database for its speed, such as Apache Derby, automatically started just before launching tests.
Create tables in schema using Hibernate DDL gen from annotated entities.
No existing rows in any tables; happens automatically with an embedded database and a clean build when storing any of its files in a subdirectory of the build output dir.
dbUnit configured with DatabaseOperation.CLEAN_INSERT [0].
Minimal dbUnit data for each test.
In my previous spring projects, I always use hibernate+postgresql to store the data. I rencently start to use spring-boot, and I am looking for a database system which allow me embed it in my project, without be required the installation of a external DBMS.
I try use SQLite, but in my searches I found some afirmations Hibernate isn't compatible with SQLite.
Anyone knows if this is possible and could point me a solution?
We've successfuly used HSQLDB with Hibernate for ages.
This is actually super cool for sales, you can demonstrate a working application on (potential) customers machine with the embedded HSQLDB database. And still be able to switch to "the real thing" later on.
See also this:
Does Hibernate Fully Support SQLite
and this:
https://code.google.com/p/hibernate-sqlite/