I want to have a file based SQL Database for my Spring Boot application that does not need a dedicated database server and works out of the box, something like HSQLDB or H2.
Additionally I want that database to primarily work on the file system and limit its memory usage. Yet whether I use HSQLDB or H2, changes are only saved to disk on shutdown and memory usage explodes.
I'd like to immediately save all changes to my database to disk and not keep all of it in memory all the time.
My Spring Boot datasource url is currently: jdbc:hsqldb:file:data/xxx.
Am I misunderstanding the capabilities of HSQLDB and H2 or am I missing a configuration setting?
This is configurable
jdbc:hsqldb:file:filename;hsql.default_table_type=cached;hsqldb.cache_size=500;hsqldb.write_delay=false
The settings indicate disk based tables, with 500KB used for memory cache (the actual memory usage is perhaps twice the amount) and immediate writing of changes to disk.
Related
I want to start by saying that I have never used Apache Ignite and only briefly played around with H2. I have a client who uses Ignite and in their own code they interact with the H2 database in Ignite. The code is very complex so it's hard for me to explain exactly what their code uses the H2 database for but I believe it to be irrelevant to the question.
Through an SQL injection it was possible to execute H2 functions such as "FILE_WRITE" and "FILE_READ". The H2 database appears to be running the SA account and is also running as root which made it possible to overwrite sensitive files on the system.
If we ignore the fact that there is an SQL injection that obviously needs fixing, can the SA account for H2 in Ignite be changed to a low privileged user to stop the usage of the FILE_READ and FILE_WRITE functions?
If Ignite is run as root, then I assume H2 is run as root as well in the system? Is it correct to assume that it is the Ignite process that needs to be downgraded to a low privileged account for H2 to not be run as root as well?
I currently do not have access to the system in question but it has been requested, but I know for a fact that FILE_WRITE could be used, and that a sensitive file could be overwritten (that was only writable by root).
Not a direct answer to your question, but this is fixed in Apache Ignite 2.8.1. See the announcement.
And yes, don't run Ignite as "root" for exactly the reason you suggest.
I have a Java Spring project which does a lot of database reading. The database I have available is a shared postgresql database on a remote machine and it's quite slow to get data from it, especially when I'm doing a lot of back-and-forth.
When I'm doing local testing and development, I use the embedded H2 in-memory database. Things easily go 100 times faster.
This made me wonder. Is there a way to use the embedded H2 in-memory database so that:
Data manipulation (INSERT, UPDATE, DELETE) is ("eventually") replicated to the PostgreSQL database
Upon start/restart of the Spring project, the H2 database is automatically filled with the data of the PostgreSQL server
This would allow me to use the fast H2 to provide a seamless user experience while at the same time also storing the data in a longer-term data storage. In a way, I'd be using the H2 as a fully cached version of the PostgreSQL database.
Is there a known / standardized way to approach this? Google for H2 and PostgreSQL and replication gives me results on transitioning from one to the other, but I'm not finding much as to using one as a sort of cache for the other.
I remain on the lookout for a Spring / JPA / Hibernate focused answer, but if none comes: I may have found an alternative domain to investigate. Dedicated database replication software might be able to manage this. Specifically, I've discovered SymmetricDS, which seems (I've only given the documentation a cursory glance) like it might be able to be embedded into my Spring application, do an initial load of my embedded in-memory H2 database on startup and then trickle feed data changes to the remote database.
(I'm not in any way affiliated with SymmetricDS, it just looks like it might be a solution.)
We have multiple oracle schema which we want to import in to somekind of inmemory db so that when we run our integration test we can use that db and run our tests faster.
Is there anyway we this can be achieved using something like HSQL db. We are using spring framework and it does support inmemory db.
Any link to some resource would be highly appreciated.
Try force full database caching mode, if you're using 12.1.0.2. It's not exactly the same as a full in-memory database, but it should be closer.
alter database force full database caching;
In-memory database performance is over-rated anyway. Oracle's "old-fashioned" asynchronous IO and caching often work just fine. For example, in this question, accessing a temporary table (which is stored on disk) runs faster than an equivalent solution using in-memory data structures. And I've seen a small Oracle database handle petabytes of IO with the "boring" old buffer cache.
Or when you say "run our tests faster", are you referring to a more agile database; one that can be controlled by an individual, instead of the typical monolithic Oracle database installed on a server? I see that issue a lot, and there's no technical reason why Oracle can't be installed on your desktop. But that can be a tough cultural battle.
Yes, you can use HSQLDB for the purpose of unit testing - see this post for more information on how to integrate with Spring.
Also, see this list as a good starting point for different usages of HSQLDB.
H2 Database is not very stable (But very Fast wich is very good for DEV), especialy during the developpement process, i hope that the number of corruption is du to the immediat shutdown of the Server (during debuging).
How to ensure that a H2 DataBase is not corrupted?
In order garant that a backup is good.
Probably the best way to check if everything is OK is to create a SQL script from the database, using the SCRIPT statement. If that works, then the data is fully readable. The index data might still be corrupt, but indexes can be re-created.
Another option is to always backup the data in the form of a SQL script. This will make a separate check unnecessary; but backup is a bit slower and can't be done online (while updates are happening).
By the way: if a database file gets corrupt, it's due to misconfiguration or wrong usage (H2 supports disabling the transaction log), due to hardware failure, or due to a bug in the database engine itself.
I have the following issue:
Two instances of an application on two different systems should share a small database.
The main problem is that both systems can only exchange data through a network-folder.
I don't have the possibilty to setup a database-server somewhere.
Is it possible to place a H2 database on the network-folder and let both instances connect to the database (also concurrently)?
I could connect with both instances to the db using the embedded mode if I disable the file-locking, right?
The instances can perfom either READ or INSERT operations on the db. Do I risk data corruptions using multiple concurrent embedded connections?
As the documentation says; ( http://h2database.com/html/features.html#auto_mixed_mode
)
Multiple processes can access the same database without having to start the server manually. To do that, append ;AUTO_SERVER=TRUE to the database URL. You can use the same database URL independent of whether the database is already open or not. This feature doesn't work with in-memory databases.
// Application 1:
DriverManager.getConnection("jdbc:h2:/data/test;AUTO_SERVER=TRUE");
// Application 2:
DriverManager.getConnection("jdbc:h2:/data/test;AUTO_SERVER=TRUE");
From H2 documentation:
It is also possible to open the database without file locking; in this
case it is up to the application to protect the database files.
Failing to do so will result in a corrupted database.
I think that if your application use always the same configuration (shared file database on network folder), you need to create an application layer that manages concurrency