I'm using Testcontainers:
protected static JdbcDatabaseContainer<?> platformPostgresContainer = new PostgreSQLContainer<>("postgres:13")
// .withReuse(true)
.withInitScript("create-databases.sql");
Why without the .withReuse(true) the postgres:13 image sometimes disappears from Docker at the end of the tests?
The image is deleted by Ryuk, if the environment variable TESTCONTAINERS_RYUK_DISABLED=true is not specified, then Testcontainers will start the Moby Ryuk container which in turn when will receive SIGTERM will remove all containers with label org.testcontainers=true, then will remove all images without containers (see the argument of cli.ImagesPrune from function prune in moby-ryuk/main.go). Thus, if the only postgres:13 containers where those created by Testcontainers, then after their remove, the postgres:13 image will have no associated containers and will be removed too by Ryuk.
This is by design, the lifecycle of containers managed by Testcontainers can be hooked into the test framework, but will ultimately not extend the life of the JVM process (unless the alpha feature resusable mode is used).
Related
Is there a way of persisting Quarkus devservices databases? Maybe a way of using volumes, but I can not find any reference. I am thinking on something like a property (non existing) quarkus.datasource.devservices.volume=some_volume that will reuse some_volume existing volume with the spin Docker container.
Maybe what you can do for now is, disable database startup from dev-services (see link 3 below) and add a QuarkusTestResource on your test class and startup your own docker image with a volume mount to your disk.
And the next time you startup your test, the data should be available as long as it points to the same volume mount. Also make sure that you don't use TestTransaction, otherwise the transaction will be rollbacked at the end of the test.
Maybe these links can help you:
cheat sheat: continious testing
cheat sheat: dev-services
dev-service guide
I have a spring boot app which loads a yaml file at startup containing an encryption key that it needs to decrypt properties it receives from spring config.
Said yaml file is mounted as a k8s secret file at etc/config/springconfig.yaml
If my springboot is running I can still sh and view the yaml file with "docker exec -it 123456 sh" How can I prevent anyone from being able to view the encryption key?
You need to restrict access to the Docker daemon. If you are running a Kubernetes cluster the access to the nodes where one could execute docker exec ... should be heavily restricted.
You can delete that file, once your process fully gets started. Given your app doesnt need to read from that again.
OR,
You can set those properties via --env-file, and your app should read from environment then. But, still if you have possibility of someone logging-in to that container, he can read environment variables too.
OR,
Set those properties into JVM rather than system environment, by using -D. Spring can read properties from JVM environment too.
In general, the problem is even worse than just simple access to Docker daemon. Even if you prohibit SSH to worker nodes and no one can use Docker daemon directly - there is still possibility to read secret.
If anyone in namespace has access to create pods (which means ability to create deployments/statefulsets/daemonsets/jobs/cronjobs and so on) - it can easily create pod and mount secret inside it and simply read it. Even if someone have only ability to patch pods/deployments and so on - he potentially can read all secrets in namespace. There is no way how you can escape that.
For me that's the biggest security flaw in Kubernetes. And that's why you must very carefully give access to create and patch pods/deployments and so on. Always limit access to namespace, always exclude secrets from RBAC rules and always try to avoid giving pod creation capability.
A possibility is to use sysdig falco (https://sysdig.com/opensource/falco/). This tool will look at pod event, and can take action when a shell is started in your container. Typical action would be to immediately kill the container, so reading secret cannot occurs. And kubernetes will restart the container to avoid service interruption.
Note that you must forbids access to the node itself to avoid docker daemon access.
You can try mounting the secret as an environment variable. Once your application grabs the secret on startup, the application can then unset that variable rendering the secret inaccessible thereon.
We need to deploy our 4 applications (3 spring boot apps and 1 zookeper) with docker stack. As our DevOps guy told us, there is no way how to define in docker stack which application will be depending on another like in docker compose, so we as developers need to solve it in code.
Can you tell me how to do that or what is the best way? One of our applications have to be started as first because that app manage database (migration and so on). Next can start other applications when database is prepared. Any ideas? Thanks.
if you want to run all the 4 applications in one docker container, you can refer to this postRun multiple services in a container
if you want to docker compose the 4 applications, you can refer to this post startup order, it use depends_on your other app images
no matter what the way is, you must write a script to check if your first app has already finish to manage the database, you can refer wait-for-postgres.sh to learn how to use sleep in shell to repeatedly check your first app status
the more precisely way i can suggest one is for example:
put a shared static variable to false
public static boolean is_app_start = false;
when you finish to manage your database, change this value to true;
write a #RequestMapping("/is_app_start") in your controller to return this value
use curl in your shell script to check the value
i never really understood how to start a docker and how to maintain it alive.
I have a question, so when you start a docker in the terminal you must provide a command for the docker so it maintains alive, and when you dont provide a service it restarts everytime, you can provide the /bin/bash so it maintains open. (Could you show me how to do it the right way, maintain it open with bash ?)
When it comes to rancher, when you create a new docker you can provide the command too, but if you dont the docker won't restart it maintains alive, so what does this means, that it have default command ? (/bin/bash)? What command does exactly executes rancher to start the docker?
thank you all
It is probably best if you read some about docker, to get the various concepts clear. From your use of "a docker", it seems that you don't really have all the pieces yet for an easy understanding.
A quick layout would be that you have
Image. I have seen this compared to a 'class' in programming
Container. In the same comparison, this would be an object: an instance of a class.
If you want to run something with docker, you start a container from an image. Just like if you want to create an object, you create one from a class. (lets not take this comparison/simili too far)
Now a containers purpose is to run something, rather, to run a single something. So "keeping a docker open" is not something you 'should want' What you want is to run, for instance, a server. Or a script.
Every container runs a single process (or should run one). As the 'official' usecase is not 'create a virtual server you can play around', it might behave strange or complicated if you want to have place to ssh to and not run a specific thing.
This also means you don't want to run any services as a background: if you run apache, you want to run it not as a daemon, but just run it: that's what the docker container is for. If you need to run something else (for instance, a database server) you would start a second container.
There might be exceptions for this, but to get your head around the why stuff works as it does, you should probably start somewhat religiously with these 'rules', and from that point go on.
I want to run an arbitrary application inside a docker container safely, like within a vm. To do so I save the application (that I donwloaded from the web and that I don't trust) inside a directory of the host system and I create a volume that maps this directory with the home directory of the container and then I run the application inside the container. Are there any security issues with this approach? Are there better solutions to accomplish the same task?
Moreover, to install all the necessary dependencies, I let to execute an arbitrary script inside a bash terminal running inside the container: could this be dangerous?
To add to #Dimitris answer. There are other things you need to consider.
There are certain things container do not contain. Docker uses namespaces to alter process view of the system.i.e N/W Shared memory etc. But you have to keep in mind it is not like KVM. Docker do talk to kernel directly unlike KVM(Vms) like /proc/sys.
So if the arbitrary application tries to access kernel subsystems like Cgroups , /proc/sys , /proc/bus etc. you could be in trouble. I would say its fine unless its a multi-tenant system.
As long as you do not give the application sudo access you should be good to try it out.
Dependencies are better off defined in the Dockerfile in a clear way for other to see. Opting to run a script instead will also do the job but it's more inconvenient.