I've just enabled query caching on my symfony application using the following configuration:
Doctine cache config
doctrine_cache:
providers:
cache:
namespace: '%cache_namespace%'
chain:
providers:
- array_cache
- redis_cache
- file_cache
redis_cache:
namespace: '%cache_namespace%'
predis:
host: "%redis_host%"
port: "%redis_port%"
password: "%redis_password%"
timeout: "%redis_timeout%"
array_cache:
namespace: '%cache_namespace%'
array: ~
file_cache:
namespace: '%cache_namespace%'
file_system:
directory: "%kernel.cache_dir%/application"
Doctrine ORM config
orm:
auto_generate_proxy_classes: "%kernel.debug%"
entity_managers:
an_entity_manager:
connection: connection
mappings:
AppBundle: ~
naming_strategy: doctrine.orm.naming_strategy.underscore
metadata_cache_driver:
type: service
id: "doctrine_cache.providers.cache"
query_cache_driver:
type: service
id: "doctrine_cache.providers.cache"
result_cache_driver:
type: service
id: "doctrine_cache.providers.cache"
I'm also having functional tests that populate a local sqlite database instead of the real one. What I'm seeing is the following:
Every time I run my tests, I see the Redis cache creating new keys even for identical records. I'm guessing this must be because the database gets re-created before every test gets executed, and the contents of the newly created rows don't matter as far as caching is concerned, but I can't be sure.
Does anyone know if this expected behaviour?
You should disable Redis (and all unecessary external dependencies) in you test environment. This could be done overriding your configuration in the test env, using the in-memory cache only.
To read more abound environments and configuration, you can look into the Symfony docs: http://symfony.com/doc/current/book/configuration.html#environment-configuration
Related
I have a new MongoDB, that means there's no user, password and/or an authentication database.
Translating this to a .properties file, this should be:
spring.data.mongodb.host=localhost
spring.data.mongodb.port=27017
...and for .yaml/.yml:
spring:
data:
mongodb:
host: localhost
port: 27017
Now let's suppose I want to use environment variables instead, if they're set, like this:
spring:
data:
mongodb:
host: ${MONGODB_HOST:localhost}
port: ${MONGODB_PORT:27017}
Everything until now works as expected.
What I want to achieve is the same for the spring.data.mongodb.user, spring.data.mongodb.password and spring.data.mongodb.authentication-database properties. I've tried doing the same technique for these properties, but in case they're not found in the environment, an exception is thrown, like this:
spring:
data:
mongodb:
host: ${MONGODB_HOST:localhost}
port: ${MONGODB_PORT:27017}
username: ${MONGODB_USERNAME}
password: ${MONGODB_PASSWORD}
authentication-database: ${MONGODB_AUTHENTICATION_DATABASE}
I have even tried setting empty/blank valuesm like ${MONGODB_USERNAME:}, ${MONGODB_USERNAME:''} and ${MONGODB_USERNAME:""}.
How do I get to achieve this? Is it even possible?
I've not tried this, but Spring boot should be able to pickup environment variables and use them as properties without having to put them in the yaml file. For example if you name a OS environment variable SPRING_DATA_MONGODB_USERNAME the value should show up in the spring property spring.data.mongodb.username. This would allow you to specify the username property only when you require it.
The other way to do this is to use externalised config. If you are booting your application from a fat jar, an application.yml outside the jar in the same directory can add in your additional properties:
spring:
data:
mongodb:
username: fred
password: password
without affecting the properties specified in the application.yml provided inside the jar.
There's a long ordered list of where spring-boot looks for properties documented here.
I have application.yaml file which contains database properties fetched from Secrets object in Kubernetes Cluster in separate deployment environment. However, when I try to run that application locally (Spring Boot application), it fails to load for obvious reason that it can't find the datasource due to not having actual values in application.yaml file.
Does anyone have any idea how to start application locally without hardcoding database credentials in yaml file?
url: ${DB_URL}
username: ${DB_USER}
password: ${DB_PASSWORD}
I don't have Kubernetes cluster locally.
I don't have Kubernetes cluster locally.
You will need something to run .yaml files locally probably "minikube". Add secrets to that environment using another file(local-secrets.yaml) or directly using "kubectl".
See here how to add secrets.
The object will look something like this (base64'ed)
apiVersion: v1
kind: Secret
metadata:
name: mysecret
type: Opaque
data:
username: YWRtaW4=
password: MWYyZDFlMmU2N2Rm
without hardcoding database credentials in yaml file
you may use helm charts for that, cause you can provide values with --set parameter when installing the chart.
We have a spring cloud config server with a composite configuration like this:
spring:
profiles:
active: composite
cloud:
config:
server:
composite:
-
type: git
uri: "https://github.comcast.com/config-org/{application}"
username: "mainUsername"
password: "mainPassword"
searchPaths: "*"
-
type: git
uri: "https://github.comcast.com/config-org/shared-repo"
username: "sharedUsername"
password: "sharedPassword"
searchPaths: "*"
health:
enabled: false
Here's the issue we're running into. Given that there is a repository named xsp-reference-service, when the cloud config client makes a request to the config server with a label for a branch that exists in the application (xsp-reference-service) repository, but not in the shared (shared-repo) repository, we get this:
{
"timestamp": "2019-12-04T16:18:43.886+0000",
"status": 404,
"error": "Not Found",
"message": "No such label: schrodingers_branch",
"path": "/xsp-reference-service/dev/schrodingers_branch"
}
Is there a way we can either tell Spring Cloud Config Server to not worry about the missing branch in the shared repository (fail gracefully)? If not, can we force cloud config server to use a default label (branch name) if the provided one does not exist? I'll even take suggestions on maybe a different setup that would serve our use case better. Any comments are welcome.
I don't have the required rep to comment nor do I know how to tell Spring Cloud Config Server to gracefully fail. That being said, I was able to find some information here[1].
Two important points from the above documentation:
Any type of failure when retrieving values from an environment
repository results in a failure for the entire composite environment.
When using a composite environment, it is important that all
repositories contain the same labels. If you have an environment
similar to those in the preceding examples and you request
configuration data with the master label but the Subversion repository
does not contain a branch called master, the entire request fails.
It might be in your best interest to revisit your design with having a shared repo. One alternative could be to use Spring Vault[2]. Then (taken from the same above link) you can define distinct types of repositories and no longer have this composite labeling problem.
spring:
profiles:
active: git, vault
cloud:
config:
server:
git:
uri: file:///path/to/git/repo
order: 2
vault:
host: 127.0.0.1
port: 8200
order: 1
1: https://cloud.spring.io/spring-cloud-config/multi/multi__spring_cloud_config_server.html#composite-environment-repositories
2: https://spring.io/projects/spring-vault
Long story short --->
While passing an ssh-key, which is retrieved from a secret in Openshift to apache-camel SFTP component its not able to connect the server; whereas if I directly pass a path of the actual ssh-key file w/o creating secret to the same component, it works just fine. The exception is, invalid key. I tried to read the key file in java and pass it as ByteArray as a privateKey parameter but no luck. Seems like passing the key as byte is not working as all possible means.
SFTP-COMPONENT Properties->
sftp:
host: my.sftp.server
port: 22
fileDirectory: /to
fileName: /app/home/file.txt
username: sftp-user
privateKeyFilePath: /var/run/secret/secret-volume/ssh-privatekey **(Also tried privateKey param with byte array)**
knownHostsFile: resource:classpath:keys/known_hosts
binary: true
Application Detail:
I am using Openshift 3.11.
Developing Camel-SpringBoot Micro-Integration services configured with fabric8 and spring-cloud-kubernetes plugins for deployment.
I am creating the secret as,
oc secrets new-sshauth sshsecret --ssh-privatekey=$HOME/.ssh/id_rsa
I have tried to refer secret with deployment.yml and bootstrap.yml
Using as env variable with secret-key-ref->
deployment.yml->
- name: SSH_SECRET
valueFrom:
secretKeyRef:
name: sshsecret
key: ssh-privatekey
bootstrap.yml->
spring:
cloud:
kubernetes:
secrets:
enabled: true
enableApi: true
name: sshsecret
Using as mounted volume->
deployment.yml->
volumeMounts:
- mountPath: /var/run/secret/secret-volume
name: secret-volume
volumes:
- name: secret-volume
secret:
secretName: sshsecret
bootstrap.yml->
spring:
cloud:
kubernetes:
secrets:
enabled: true
paths: /var/run/secret/secret-volume
Note: Once the service is deployed I can see the mounted volume is attached with the container and can even bash into the POD and go to the same directory and locate the private key, which completely intact.
Any help will be appreciated. Ask me all questions you need to know to solve this.
It was a very bad mistake from my side. I was using privateKeyUri in camel SFTP component instead of privateKeyFile. I didn't rectify this and always changing those SFTP parameters in config-map directly.
By the way, for those trying to implement similar usecase; use the second option which is, mounting the secret into a volume and then refer the volume path inside Camel. Don't use the secret as ENV variable, so you need not enable secret API inside bootstrap.yml.
Thanks anyway, cheers!
Rito
I'm using spring-cloud-config in my Spring Boot project and I don't understand how to interpolate environment variables.
For example I have MYSQL_PASSWORD variable set in the config server (as environment variable) and I want to get it from other clients, without redefining the variable inside all of them. This is my config:
service.yml (in config-server)
spring:
datasource:
url: jdbc:mysql://mysql:3306/${MYSQL_DATABASE}?autoReconnect=true&useSSL=false
username: ${MYSQL_USER}
password: ${MYSQL_PASSWORD}
bootstrap.yml (into the client)
spring:
application:
name: event-service
cloud:
config:
uri: http://config-service:8888
fail-fast: true
If I start my services with this configuration, placeholder ${MYSQL_***} is not interpolated and I cannot connect to the database obviously.
What I have to do to make it work is to define environment variables like MYSQL_*** also in the client.
This is weird for me as I want config-server to be the central repository for everything. Do you have any advice?
I suggest you try to replace ${MYSQL_USER} and ${MYSQL_PASSWORD} with ${mysql.user} and ${mysql.password} - that way you'll be relying on Spring Boot's default property replacement.
Although I'm not sure that configuration server supports what you're trying to use, I've never tried a similar use case, please write back with a solution if the approach I suggested didn't work :)