Hello I'm starting with docker and docker compose and I have the following problem:
I'm working in a spring micro services architecture where I have one configuration service, one discovery service, one gateway service and multiple resource services.
To run these services, I build jar files, which I place in separated folder per service with their config files (application.yml and bootstrap.yml):
e.g:
config-service/
config-service.jar
application.yml
discovery-service/
discovery-service.jar
bootstrap.yml
gateway-service/
gateway-service.jar
bootstrap.yml
crm-service/
crm-service.jar
bootstrap.yml
This works so far on my server.
Now I want to deploy my services in different environments as docker images (created with mvn build image and buildpack) using docker compose, where the configuration files vary depending on the environment. How can I deploy a service as a container using an existing image but with a different configuration file?
Thank you in advance!
There are a few possibilities when handling configuration in a containerized environment.
One of the options is that Spring boot allows you to use environment variables for each application property. For example, let's say you have a spring.datasource.url property, in that case you could also define that property by setting a SPRING_DATASOURCE_URL environment variable:
version: '3.8'
services:
my-spring-boot-app:
image: my-image:0.0.1
environment:
- SPRING_DATASOURCE_URL=jdbc:my-database-url
Alternatively, you could use volumes to put an external file on a specific location within a container:
version: '3.8'
services:
my-spring-boot-app:
image: my-image:0.0.1
volumes:
./my-app/bootstrap.yml:/etc/my-app/bootstrap.yml
In this example I'm copyingbootstrap.yml from a relative folder on my host machine, to /etc/my-app within the container. If you put these files within the same folder as your JAR file, you can override the configuration.
Related
I have a Spring Boot app running on port 5000 with the server.servlet.context-path=/api.
When the app starts up, on my machine, I can access the static welcome page (index.html) at http://localhost:5000/api no problem.
I have this app deployed on AWS CodePipeline in a Docker container. The problem is that when I access the endpoint of my app at http://my-api-env.eba-uatdpxjr.us-east-2.elasticbeanstalk.com/api it works fine, but I would like to have it so that the root url of my deployment environment doesn't need to have /api manually tacked to the end of it to access my service.
How can I make it so that the deployed app's URL http://my-api-env.eba-uatdpxjr.us-east-2.elasticbeanstalk.com automatically includes the app's context path /api and renders the static welcome page?
Dockerfile
FROM openjdk:8-jdk-alpine
COPY /target/RestApi-0.0.1-SNAPSHOT.jar RestApi-0.0.1-SNAPSHOT.jar
EXPOSE 5000
ENTRYPOINT ["java", "-jar", "/RestApi-0.0.1-SNAPSHOT.jar"]
docker-compose.yml
version: "1.0"
services:
api_service:
build: .
restart: always
ports:
- 5000:5000
buildspec.yml
version: 0.2
phases:
build:
commands:
- java -version
- mvn clean package
artifacts:
files:
- 'Dockerfile'
- target/RestApi-0.0.1-SNAPSHOT.jar
application.properties
# JPA Settings
spring.jpa.show-sql=true
spring.jpa.hibernate.ddl-auto=create
# Configure Port and Context Path
server.port=5000
server.servlet.context-path=/api
# In the master application.properties file we can control which Spring Profile we want to use
spring.profiles.active=dev
# Set Welcome view to index.html page
spring.mvc.view.suffix = .html
And why exactly removing the context path (it's "/" by default) doesn't work for you?
I want to set variables in my spring project YML configuration from docker environment variable without using Docker Compose.
To use an environment variable in your spring project, first, you need to set in your docker container.
Set ENV in Docker Container:
Docker run -e LOGGING_LEVEL_ROOT="info" -e PROFILE="production" -it your_image
Now, You can use LOGGING_LEVEL_ROOT in your spring project YML configuration.
logging:
level:
root: ${LOGGING_LEVEL_ROOT}
Or
spring:
profiles: ${PROFILE}
Note that, since the application.properties and application.yml files
accept Spring style placeholders (${…}), the Maven filtering is
changed to use #..# placeholders. (You can override that by setting a
Maven property called resource.delimiter.)
oot-features-external-config-placeholders-in-properties
I got two ddev projects that needs to interact with each other. When a ran into some issues, I check the resolved IP for the connection.
I did it by ssh into project1 and ping project2 (ping project2.ddev.local)
The domain resolves to 127.0.0.1
So every request I send to this domain will stay in the current container and is not routet to the other project.
Steps to reproduce:
Start two separate ddev containers and ssh into one of them. Try to ping the the other project by using the ddev domain.
Is there a solution that two (or more) projects can interact with each other?
Edit 2019-01-08: It's actually easy to do this with just the docker name of the container, no extra docker-compose config is required. For a db container that's ddev-<projectname>-db. So you can access the db container of a project named "d8composer" by using the hostname ddev-d8composer-db; for example mysql -udb -pdb -h ddev-d8composer-db db
Here's another technique that actually does have two projects communicating with each other.
Let's say that you have two projects named project1 and project2, and you want project2 to have access to the db container from project1.
Add a .ddev/docker-compose.extradb.yaml to project2's .ddev folder with this content:
version: '3.6'
services:
web:
external_links:
- ddev-project1-db:proj1-db
And now project1's database container is accessible from the web container on project2. For example, you can mysql -h proj1-db from within the project2 web container.
Note that this is often a bad idea, it's best not to have two dev projects depend on each other, it's better to figure out development environments that are as simple as possible. If you just need an extra database, you might want to try How can I create and load a second database in ddev? . If you just need an additional web container as an API server or whatever, the other answer is better.
A simple example of extra_hosts. I needed to use an HTTPS URL in a Drupal module's UI, entity_share, to cURL another ddev site.
On foo I add a .ddev/docker-compose.hosts.yaml
version: '3.6'
services:
web:
extra_hosts:
- bar.ddev.site:172.18.0.6
I tried this and it worked quite nicely; the basic idea is to run a separate ddev-webserver as a service. We usually think of a ddev "service" as something like redis or memcache or solr, but it can really be an API server of any type, and can use the ddev-webserver image (or any other webserver image you want to use).
For example, add this docker-compose.api.yaml to your project's .ddev folder (updated for ddev v1.1.1):
version: '3.6'
services:
myapi:
image: drud/ddev-webserver:v1.1.0
restart: "no"
labels:
com.ddev.site-name: ${DDEV_SITENAME}
com.ddev.approot: $DDEV_APPROOT
com.ddev.app-url: $DDEV_URL
volumes:
- "../myapi_docroot/:/var/www/html:cached"
- ".:/mnt/ddev_config:ro"
web:
links:
- myapi:$DDEV_HOSTNAME
and put a dummy index.html in your project's ./myapi_docroot.
After ddev start you can ddev ssh -s myapi and do whatever you want there (and myapi_docroot is mounted at /var/www/html). If you ddev ssh into the web container you can curl http://myapi and you'll see the contents of your myapi_docroot/index.html. Your myapi container can access the 'db' container, or you can run another db container, or ...
Note that this mounts a subdirectory of the main project as /var/www/html, but it can actually mount anything you want. For example,
volumes:
- "../../fancyapiproject/:/var/www/html:cached"
I have facing an issue with Spring Cloud Config Server and Eureka Server Profiling.
Let's say I have 3 services with their name ("spring.application.name") as :
myapp-svc
myapp-spring-cloud-config-svc
myapp-spring-eureka-svc
I want to deploy each service in 2 regions ( dev and prod ). In Dev region, each service will run on localhost and in prod it will have some different url. 'myapp-spring-cloud-config-svc' in dev region will point to local git repo, while in prod region it will point to remote git repo.I can have 2 configurations:
1) When I start 'myapp-svc' service in local, it should connect to 'myapp-spring-cloud-config-svc' in dev.
I can do this by setting spring.cloud.config.uri = .
But the issue with this set up is that the property needs to be defined in bootstrap.properties.
So, If deploy 'myapp-svc' to prod, I will have to change config uri there to point it to prod config service which in turn would need another build creation.
This doesn't seem like a good solution, what if I have 50 app related services, I can't change this property in each one of them before prod deployment.
I tried setting spring.cloud.config.uri in application-dev.properties of 'myapp-svc' but it doesn't work. As per docs, it must be changed in bootstrap.
So, how do I implement this without having to create new build for prod ?
2) I can first call eureka and then using eureka I can call config service here.
The problem here is also same.
If I use eureka to look up config then "eureka.client.serviceUrl.defaultZone" must be defined in "bootstrap.yml".
See this:https://cloud.spring.io/spring-cloud-config/multi/multi__spring_cloud_config_client.html
So, in this case too, I need to change eureka url before deploying this service to prod.
Please help me on this...!!
Here is how, the properties, yml looks like for each of the above mentioned services:
1) myapp-svc:
1.1)bootstrap.yml
spring:
application:
name: myapp-svc
cloud:
config:
discovery:
enabled: true
serviceId: myapp-spring-cloud-config-svc
eureka:
client:
serviceUrl:
defaultZone: http://localhost:8762/eureka/
server:
port: 8082
2) myapp-spring-cloud-config-svc:
2.1)application-dev.properties:
spring.cloud.config.server.git.uri=file:///C:/config-repo
eureka.client.serviceUrl.defaultZone=http://localhost:8762/eureka
2.2)application-prod.properties:
spring.cloud.config.server.git.uri=https://github.com/<mygit Repo>
2.3)bootstrap.proerties:
spring.application.name=myapp-spring-cloud-config-svc
server.port=8888
3) myapp-spring-eureka-svc
3.1)bootstrap.proerties
spring.application.name=myapp-spring-eureka-svc
server.port=8762
1) You can have profile specific bootstrap-<profile>.properties (like for application-<profile>.properties) for each supported profile to avoid rebuilding your application for each env. Then just pass application profile using to your application during start-up. Spring will load correct bootstrap-<profile>.properties and will connect to proper configuration server (or eureka, etc). Example:
java -jar your-app.jar --spring.profiles.active=dev
2) You can pass your URLs externally as custom properties (same as with profile above) and have smth like this in bootstrap.properties. Example:
spring.cloud.config.uri=${config.server.url}
then pass --config.server.url= ... during start-up.
3) You can pass Spring properties in the same way during start-up. Example:
java -jar your-app.jar --spring.cloud.config.uri= ...
4) You can use system env variables. Example:
spring.cloud.config.uri=${SYSTEM_ENV_CLOUD_CONFIG_URI}
This is a fairly simple question I think, however I haven't seen too many examples or really anything that explains the connection between using docker configs (v 3.3+) and loading that config into Spring Boot for reference.
sample docker-stack.yml
version: '3.3'
services:
test-service:
image: myrepo/test-service:1.0.0
configs:
- service-config
networks:
- test-network
configs:
service-config:
external:true
networks:
test-network:
sample swarm "service-config".
I entered this as a new "configs" entry in Portainer.
services:
test-service:
key1: sample value
key2: sample two
What I'm trying to do it load this config into Spring such that I can reference the values from this config within a component.
Either via #ConfigurationProperties
#ConfigurationProperties("services.test-service")
public MyBeanName myBean() { return new MyBeanName(); }
or via the #Value:
#Value("${services.test-service.key1}")
private String key1;
How can I get this docker "configs" configuration loaded into Spring. This has got to be simple enough.. lol. Thanks!
Sorry for the delay in responding to this question, or at least posting the solution...
It took a bit more digging into how configs work with docker, however as it turns out you need to specify a target for the "config" held within your "configs" entries in your swarm cluster, and then map it into your container, and load as an external config to your spring application. In my case, I didn't want to override the the application.yml in my spring boot app, just wanted to pick up additional configs. So I went with the setting:
--spring.config.additional-location=file:/configs/sample-config.yml
Lets say I create a docker configs entry with the name "sample-config" and having the following data:
Configs Entry => "sample-config"
services:
test-service:
key1: sample value
key2: sample two
Then on my compose/stack file I need to reference the "configs" entry, and provide a target file that corresponds to the file I specified in my "spring.config.additional-location" setting.
version: '3.3'
services:
test-service:
image: myrepo/test-service:1.0.0
configs:
- source: sample-config
target: /configs/sample-config.yml
networks:
- test-network
configs:
sample-config:
external:true
networks:
test-network:
In my Dockerfile then, I'd specify the following to essentially load the "sample-config" configs entry when starting the jar/app:
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar", "--spring.config.additional-location=file:/configs/sample-config.yml"]
This allowed me to access the configuration entries, loaded externally to my spring application, with the #Value and #ConfigurationProperties annotations.