I'd like to package spring dataflow server into a container which will contain one local jar application. Publish this into local repo, expectation is that end result is same as the normal dataflow server:
https://hub.docker.com/r/springcloud/spring-cloud-dataflow-server
just with the local jar added.
Creating the Dockerfile to include the jar is straightforward, but I'm strugling a bit with how to register the jar into dataflow server.
I know one option is to use the RESTapi, but it feels quite complicated to start the dataflow server during the docker creation. I found documentation that application.yml might be a way to do this as well, but couldn't figure out how exactly.
https://github.com/spring-cloud/spring-cloud-dataflow/blob/main/spring-cloud-dataflow-server/README.adoc
https://docs.spring.io/spring-boot/docs/1.5.13.RELEASE/reference/html/boot-features-external-config.html
So is there a straightforward way to package a jar into dataflow server docker container?
The API is the only practical way to do it. Take a look at how we register apps with the docker-compose installation.
Technically, you could also pre-populate the associated DB table(s), but I don’t recommend this.
Related
Currently my team maintains many spring boot microservices. When running them locally, our workflow is to open a new IntelliJ IDEA window and pressing the "run" button for each microservice. This does the same thing as typing gradle bootRun. At a minimum each service depends on a config server (from which they get their config settings) and a eureka server. Their dependencies are specified in a bootstrap.yml file. I was wondering if there is a way to just launch one microservice (or some script or run configuration), and it would programatically know which dependencies to start along with the service I am testing? It seems cumbersome to start them the way we do now.
If you're using docker then you could use docker compose to launch services in a specific order using the depends_on option. Take a look here and see if that will solve your problem.
https://docs.docker.com/compose/startup-order/
I have a spring boot project with 4 microservices (Eureka service registry, Config server, a Zuul gateway and a userservice) in one repository with a parent project where I have a docker-compose.yml which reads the Dockerfiles in the microservices project and uses the "application-docker.yml" and "bootstrap-docker.yml"
What I'd like to do is to trigger a jenkins pipeline after a commit in git so that it will compile and deploy the microservices in Docker. Eventually I'd like to have a production configuration that deploys the images in Kubernetes maybe AWS.
Now, in order to work, the microservices need to start in order:
configserver
eureka service registry
gateway , etc..
What is the best practise?
If I have separate repositories per microservice, I think I can figure it out. It should be easy to deploy a single microservice assuming that configserver and eureka service registry are already up and running, in reality they should never change.
If I have a single repository, and I keep developing new microservices, do I need to have separate jenkins file per microservices or can I have a jenkinsfile in the parent project and use docker-compose?
How does it work? Any articles online that can help (couldn't find any). Does it make sense?
Or do I need to look at Jenkins X ?
Thanks!
I would recommend using separate repositories for each microservice. You use microservices to prevent monoliths and have small well-defined services; it only seems appropriate to also separate them by space i.e. store them in separate repositories (making it for example easier to reuse one).
You would then have to provide a Jenkinsfile in each repo. These would be mostly identical.
If you want fast release cycles you could automatically deploy a single service upon release.
Alternatively you could use an additional release train module that handles the full deployment.
In both cases I would use a docker-compose file that handles the interconnection between the services.
You can enforce the right order by using 'depends_on, links, volumes_from, and network_mode: "service:..."'. For a full reference see the docker documentation.
If you want to keep your single repository your Jenkinsfile(s) would have to be quite hacky, I suppose... After each commit you would either
build all modules --> monolithic behaviour
somehow determine which modules have changed (e.g. looking at the git log) --> same behaviour as with multiple modules but very hackily
The Docker-Compose File
If you want to release all modules at a specific point of time you could use a Release Train module where the docker-compose.yml resides next to a Jenkinsfile. Then when you want to ship your application you can start this Jenkins-job.
If you want to ship each service as soon as it is released, independently from the others, you would need to access the docker-compose.yml from each module. You could do this manually (since the files won't change too often) or create a docker module that you use as a git-submodule in all your services.
We use a generic docker-compose.yml for this, where every version is replaced by a variable:
example-service:
image: example.service:${EXAMPLE_SERVICE_VERSION}
Then to start that specific service in jenkins we use the command
export EXAMPLE_SERVICE_VERSION=1.1.1
docker-compose -p example-project -f docker-compose.yml up -d example-service
I have a multi-modules vertx application deployed on OpenShift. For integration testing purposes, I would like to deploy a database container with pre-defined data, and destroy it when the test is finished.
How can I achieve this ?
My application uses junit and maven fabric8 plugin to deploy containers in Openshift.
This is something that could be done relatively easy using arquillian-cube, which does support Kubernetes and Openshift.
What arquillian-cube can do for you, is to (optionally) create an ephemeral project, deploy everything you need for your test and once everything is up and running, then start your tests. In the end it can also do the cleaning up for you.
It is quite flexible so according to your needs and requirements it can work with either ephemeral or fixed projects. And also there are pletny of configuration options when it comes to cleaning up.
Last but not least, it does play quite nicely with the fabric8 maven plugin.
https://github.com/arquillian/arquillian-cube/blob/master/docs/kubernetes.adoc
This question is about running a Kubernetes deployment on Azure and we are completely new to Kubernetes. We have a new microservice called xde-deployer that we use to deploy projects onto other microservices. The xde-deployer builds projects using Maven and therefore needs a working Maven configuration. Normally this is provided in the user's settings.xml. In this case we are running it in a docker container, so xde-deployer will look for it in /root/.m2/settings.xml.
Normally when we deploy the docker container, we use a volume to pass the settings.xml which is located on the host. On Kubernetes of course this is not so straightforward. As the answers to this question state one could add the file later, or use a configMap. Both answers are a bit too vague for our purposes though. Is there really no way to do this from the deployment? I cannot imagine we are the only ones who need to pass Maven settings to jobs running on Kubernetes. How are others solving this problem? My main problem after reading the documentation is still: how do I get a file into the Kubernetes cluster at all when I am on Azure? Is there a kind of persistent volume or parameters store that can be easily shared by the pods?
I have a spring boot application that currently runs in embedded Tomcat. I have a file, states.csv, that I want to parse on startup and seed my states database table (I tried via liquibase but that refuses to work).
I put the file in resources/main/ and that appears to work fine. My question is, if I decided against embedded Tomcat in the future (say moving to AWS or a regular Tomcat), is this still the best location to keep files for use?
I don't want to code myself into a corner if there is a better way to do this.
This depends entirely on how you're reading the file. As long as you're grabbing it out of the classpath, you should be fine. (And I've run single-jar applications on both basic AWS VMs and Cloud Foundry on EC2 with no difficulty at all.)