I want to use spring-boot, spring-cloud to get all DataSources binded to the cloud foundry application.
Is there a way to get the list?
If I can get service names, I can also use
AbstractCloudConfig.connectionFactory().dataSource(serviceId)
to create the DataSource.
You can do something like this to enumerate over the list of database services and get a DataSource for each one:
Cloud cloud = abstractCloudConfig.cloud();
List<ServiceInfo> serviceInfos = cloud.getServiceInfos(DataSource.class);
List<DataSource> dataSources = new ArrayList<>();
for (ServiceInfo serviceInfo : serviceInfos) {
dataSources.add(cloud.getServiceConnector(serviceInfo.getId, null));
}
DataSources configuration is set in the container environment inside 'VCAP_SERVICES' variable in Cloud Foundry. System.getenv('VCAP_SERVICES') should list all the datasources in your case.
Refer to:
https://docs.run.pivotal.io/devguide/deploy-apps/environment-variable.html#VCAP-SERVICES
Related
I built two different service connector for one service. If I add both service connector to my application Spring will not start saying there is no suitable service connector found. I debugged into spring and found only one of the connectors being added to the internal connector list. Is it possible to add two different spring cloud service connectors for one service and use them within one application?
For a better understanding an example with a rabbitMQ service. Let's say I've build two different Cloud Service Connectors with a CloudFoundryServiceInfoCreator<AMQConnectionInfo> and a CloudFoundryServiceInfoCreator<MQTTConnectionInfo>. I would like to use both connectors in the application (I know I could implement both connection info in one spring cloud connector, but that's not what I would like to do).
edit:
The following exception is raised:
org.springframework.cloud.CloudException: No unique service matching class .... found. Expected 1, found 0
at org.springframework.cloud.Cloud.getSingletonServiceConnector(Cloud.java:149)
I also tried to use cloud.getServiceConnector(id, class, null);.
I also just found that Spring Cloud Connectors just returns the first Connector which is found within this method in org.springframework.cloud.AbstractCloudConnector:
private ServiceInfo getServiceInfo(SD serviceData) {
for (ServiceInfoCreator<? extends ServiceInfo,SD> serviceInfoCreator : serviceInfoCreators) {
if (serviceInfoCreator.accept(serviceData)) {
return serviceInfoCreator.createServiceInfo(serviceData);
}
}
// Fallback with a warning
ServiceInfo fallackServiceInfo = getFallbackServiceInfoCreator().createServiceInfo(serviceData);
logger.warning("No suitable service info creator found for service " + fallackServiceInfo.getId()
+ " Did you forget to add a ServiceInfoCreator?");
return fallackServiceInfo;
}
I think it would be nice if this would return a list of suitable ServiceInfoCreator or searches for the one I requested, wouldn't it?
I want to configure the properties of the Tomcat JDBC Pool with custom parameter values. The pool is bootstrapped by the spring-cloud (Spring Cloud Connector) environment (Cloud Foundry) and connected to a PostgreSQL database. In particular, I want to set the minIdle, maxIdle and initialSize properties for the given pool.
In a "spring-vanilla" environment (non-cloud) the properties can be easily set by using
application.properties / .yaml files with environment properties,
#ConfigurationProperties annotation.
However, this approach doesn't transfer to my Cloud environment, where the URL (and other parameters) are injected from the environment variable VCAP_SERVICES (via the ServiceInfo instances). I don't want to re-implement the logic which Spring Cloud already did with its connectors.
After some searching I also stumbled over some tutorials / guides, which suggest to make use of the PoolConfig object (e.g. http://cloud.spring.io/spring-cloud-connectors/spring-cloud-spring-service-connector.html#_relational_database_db2_mysql_oracle_postgresql_sql_server). However, that way one cannot set the properties I need but merely the following three:
minPoolSize,
maxPoolSize,
maxWaitTime.
Note that I don't want to set connection-related properties (such as charset), but the properties are associated with the pool itself.
In essence, I would like to do the configuration similarly to https://www.baeldung.com/spring-boot-tomcat-connection-pool (using spring.datasource.tomcat.* properties). The problem with that approach is that the properties are not considered if the datasource was created by Spring Cloud. The article https://dzone.com/articles/binding-data-services-spring, section "Using a CloudFactory to create a DataSource", claims that the following code snippet makes it so that the configuration "can be tweaked using application.properties via spring.datasource.* properties":
#Bean
#ConfigurationProperties(DataSourceProperties.PREFIX)
public DataSource dataSource() {
return cloud().getSingletonServiceConnector(DataSource.class, null);
}
However, my own local test (with spring-cloud:Greenwich.RELEASE and spring-boot-starter-parent:2.1.3.RELEASE) showed that those property values are simply ignored.
I found an ugly way to solve my problem but I think it's not appropriate:
Let spring-cloud create the DataSource, which is not the pooled DataSource directly,
check that the reference is a descendant of a DelegatingDataSource,
resolve the delegate, which is then the pool itself,
change the properties programmatically directly at the pool itself.
I do not believe that this is the right way as I am using internal knowledge (on the layering of datasources). Additionally, this approach does not work for the property initialSize, which is only considered when the pool is created.
I have spring boot micro-service with database credentials define in the application properties.
spring.datasource.url=<<url>>
spring.datasource.username=<<username>>
spring.datasource.password=<<password>>
We do not use spring data source to create the connection manually. Only Spring create the database connection with JPA.(org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaAutoConfiguration)
We only provide the application properties, but spring create the connections automatically to use with the database connection pool.
Our requirement to enhance the security without using db properties in clear text. Two possible methods.
Encrypt the database credentials
Use the AWS secret manager. (then get the credential with the application load)
For the option1, jasypt can be used, since we are just providing the properties only and do not want to create the data source manually, how to do to understand by the spring framework is the problem. If better I can get some working sample or methods.
Regarding the option-2,
first we need to define secretName.
use the secertName and get the database credentials from AWS secret manager.
update the application.properties programatically to understand by spring framework. (I need to know this step)
I need to use either option1 and option2. Mentioned the issues with each option.
What you could do is use environment variables for your properties. You can use them like this:
spring.datasource.url=${SECRET_URL}
You could then retrieve these and start your Spring process using a ProcessBuilder. (Or set the variables any other way)
I have found the solution for my problem.
We need to define org.springframework.context.ApplicationListenerin spring.factories file. It should define the required application context listener like below.
org.springframework.context.ApplicationListener=com.sample.PropsLoader
PropsLoader class is like this.
public class PropsLoader implements ApplicationListener<ApplicationEnvironmentPreparedEvent> {
#Override
public void onApplicationEvent(ApplicationEnvironmentPreparedEvent event) {
ConfigurableEnvironment environment = event.getEnvironment();
String appEnv = environment.getProperty("application.env");
//set new properties based on the application environment.
// calling other methods and depends on the enviornment and get the required value set
Properties props = new Properties();
props.put("new_property", "value");
environment.getPropertySources().addFirst(new PropertiesPropertySource("props", props));
}
}
spring.factories file should define under the resources package and META-INF
folder.
This will set the application context with new properties before loading any other beans.
I have a simple Spring Boot app running on PCF. I am wondering if there is a better way to access VCAP environment variables from PCF. Specifically, I am trying to access service credentials for a RabbitMQ service instance running on PCF and bound to my application.
At the moment, I access the credentials using System.getenv:
JSONObject vcapServices = new JSONObject(System.getenv("VCAP_SERVICES"));
JSONArray rabbitmq = (JSONArray) vcapServices.get("p-rabbitmq");
JSONObject serviceInfo = (JSONObject) rabbitmq.get(0);
JSONObject credentials = (JSONObject) serviceInfo.get("credentials");
hostname = credentials.getString("hostname");
virtualHost = credentials.getString("vhost");
username = credentials.getString("username");
password = credentials.getString("password");
I was trying to get it working with the #Value annotation to try to access the VCAP environment variables like this:
#Value("${vcap.services.p-rabbitmq.credentials.hostname}")
private String hostname;
but I haven't been able to grab the values yet.
Is there any way I can access VCAP variables via the #Value annotation? Or is there a better way other than System.getenv to get these credentials from PCF once I deploy my application?
Is there any way I can access VCAP variables via the #Value annotation?
Yes, I think you need to change your format slightly. It's vcap.services.<service-instance-name>.credentials.hostname. You have the name of the service provider itself, not the name of your service instance. I can't see that name in the example above, but I've found the Javadoc to be helpful. It lists the following example.
VCAP_SERVICES: {
"rds-mysql-1.0": [
{
"credentials": {
"host": "mysql-service-public.clqg2e2w3ecf.us-east-1.rds.amazonaws.com",
"hostname": "mysql-service-public.clqg2e2w3ecf.us-east-1.rds.amazonaws.com",
"name": "d04fb13d27d964c62b267bbba1cffb9da",
"password": "pxLsGVpsC9A5S",
"port": 3306,
"user": "urpRuqTf8Cpe6",
"username": "urpRuqTf8Cpe6"
},
"label": "rds-mysql-1.0",
"name": "mysql",
"plan": "10mb"
}
]
}
In this example, the service provider name is "rds-mysql-1.0", which would be the same as "p-rabbitmq" in your example. It then has a service instance name of "mysql" & that is what you'd use in your property placeholder.
Ex: vcap.services.mysql.credentials.host
https://docs.spring.io/spring-boot/docs/current/api/org/springframework/boot/cloud/CloudFoundryVcapEnvironmentPostProcessor.html
Having explained all that, it can still be a fair bit of work to wire up all of your properties. Especially if you're utilizing multiple services or in your case, you're using RabbitMQ which has a lot of info that is passed through VCAP_SERVICES.
UPDATE
Spring Cloud Connectors is still an option, leaving it below for historical context, but java-cfenv is the recommended way to access service properties going forward. It plays nicer with Spring Boot and is every bit as capable.
DEPRECATED
Another option to consider is Spring Cloud Connectors. This is a library that will basically handle all of this for you. You ask for a DataSource or ConnectionFactory and it makes sure one is wired into your application.
Ex:
#Bean
public RabbitConnectionFactory rabbitFactory() {
return connectionFactory().rabbitConnectionFactory();
}
See Getting Started if you're interested in checking this out.
i'm using spring and at the moment running my tests create new objects in my "real" embedded database. I want to create a new one or a temporary db just for testing. I'm new with spring and neo4j so could anyone please help?
thanks a lot
If you are using the embedded driver with SDN/OGM you just need to configure it without providing a path. Then it will create embedded database in /tmp/.. which gets deleted on jvm exit.
E.g. if you are using java configuration
#Bean
public Configuration getConfiguration() {
Configuration config = new Configuration();
config
.driverConfiguration()
.setDriverClassName("org.neo4j.ogm.drivers.embedded.driver.EmbeddedDriver");
return config;
}
See docs for full documentation
http://docs.spring.io/spring-data/data-neo4j/docs/current/reference/html/#_configuring_the_embedded_driver