Spring Boot and how to configure connection details to MongoDB? - spring

Being new to Spring Boot I am wondering on how I can configure connection details for MongoDB.
I have tried the normal examples but none covers the connection details.
I want to specify the database that is going to be used and the url/port of the host that runs MongoDB.
Any hints or tips?

Just to quote Boot Docs:
You can set spring.data.mongodb.uri property to change the url, or alternatively specify a host/port. For example, you might declare the following in your application.properties:
spring.data.mongodb.host=mongoserver
spring.data.mongodb.port=27017
All available options for spring.data.mongodb prefix are fields of MongoProperties:
private String host;
private int port = DBPort.PORT;
private String uri = "mongodb://localhost/test";
private String database;
private String gridFsDatabase;
private String username;
private char[] password;

It's also important to note that MongoDB has the concept of "authentication database", which can be different than the database you are connecting to. For example, if you use the official Docker image for Mongo and specify the environment variables MONGO_INITDB_ROOT_USERNAME and MONGO_INITDB_ROOT_PASSWORD, a user will be created on 'admin' database, which is probably not the database you want to use. In this case, you should specify parameters accordingly on your application.properties file using:
spring.data.mongodb.host=127.0.0.1
spring.data.mongodb.port=27017
spring.data.mongodb.authentication-database=admin
spring.data.mongodb.username=<username specified on MONGO_INITDB_ROOT_USERNAME>
spring.data.mongodb.password=<password specified on MONGO_INITDB_ROOT_PASSWORD>
spring.data.mongodb.database=<the db you want to use>

spring.data.mongodb.host and spring.data.mongodb.port are not supported if you’re using the Mongo 3.0 Java driver. In such cases, spring.data.mongodb.uri should be used to provide all of the configuration, like this:
spring.data.mongodb.uri=mongodb://user:secret#mongo1.example.com:12345

In a maven project create a file src/main/resources/application.yml with the following content:
spring.profiles: integration
# use local or embedded mongodb at localhost:27017
---
spring.profiles: production
spring.data.mongodb.uri: mongodb://<user>:<passwd>#<host>:<port>/<dbname>
Spring Boot will automatically use this file to configure your application. Then you can start your spring boot application either with the integration profile (and use your local MongoDB)
java -jar -Dspring.profiles.active=integration your-app.jar
or with the production profile (and use your production MongoDB)
java -jar -Dspring.profiles.active=production your-app.jar

You can define more details by extending AbstractMongoConfiguration.
#Configuration
#EnableMongoRepositories("demo.mongo.model")
public class SpringMongoConfig extends AbstractMongoConfiguration {
#Value("${spring.profiles.active}")
private String profileActive;
#Value("${spring.application.name}")
private String proAppName;
#Value("${spring.data.mongodb.host}")
private String mongoHost;
#Value("${spring.data.mongodb.port}")
private String mongoPort;
#Value("${spring.data.mongodb.database}")
private String mongoDB;
#Override
public MongoMappingContext mongoMappingContext()
throws ClassNotFoundException {
// TODO Auto-generated method stub
return super.mongoMappingContext();
}
#Override
#Bean
public Mongo mongo() throws Exception {
return new MongoClient(mongoHost + ":" + mongoPort);
}
#Override
protected String getDatabaseName() {
// TODO Auto-generated method stub
return mongoDB;
}
}

In case that somebody is trying to connect to a Atlas MongoDB Cluster in application.properties has to have a config like:
spring.data.mongodb.uri=mongodb+srv://databaseUsername:usernamePassword#cluster0.j4koa.mongodb.net/databaseUsername?retryWrites=true&w=majority

In my case I needed to set up MongoDB for integration tests using Testcontainers. Using properites file was not an option since Mongo port had to be specified during runtime. I wanted to preseve original MongoDB autoconfiguration provided by SpringBoot but override some of the properties. This can be achieved by defining a bean of type MongoClientSettingsBuilderCustomizer which can be used to customize mongo settings :
#Bean
public MongoClientSettingsBuilderCustomizer clientSettingsBuilderCustomizer(final GenericContainer<?> mongoDBContainer) {
String database = environment.getProperty("spring.data.mongodb.database");
ConnectionString connectionString = new ConnectionString(String.format("mongodb://localhost:%s/%s", mongoDBContainer.getFirstMappedPort(), database));
return settings -> settings.applyConnectionString(connectionString);
}

If you simply want to read a MongoDB connection string from an environment variable, one way is to set the following environment variable:
SPRING_DATA_MONGODB_URI=mongodb://localhost:27017/trying-mongo
This doesn't require any changes in the application.properties as the spring data mongo will read the value from the above environment variable by default.

Here is How you can do in Spring Boot 2.0 by creating custom MongoClient adding Providing more control for Connection ,
Please follow github Link for Full Source Code
#Configuration
#EnableMongoRepositories(basePackages = { "com.frugalis.repository" })
#ComponentScan(basePackages = { "com.frugalis.*" })
#PropertySource("classpath:application.properties")
public class MongoJPAConfig extends AbstractMongoConfiguration {
#Value("${com.frugalis.mongo.database}")
private String database;
#Value("${com.frugalis.mongo.server}")
private String host;
#Value("${com.frugalis.mongo.port}")
private String port;
#Value("${com.frugalis.mongo.username}")
private String username;
#Value("${com.frugalis.mongo.password}")
private String password;
#Override
protected String getDatabaseName() {
return database;
}
#Override
protected String getMappingBasePackage() {
return "com.frugalis.entity.mongo";
}
#Bean
public MongoTemplate mongoTemplate() throws Exception {
return new MongoTemplate(mongoClient(), getDatabaseName());
}
#Override
#Bean
public MongoClient mongoClient() {
List<MongoCredential> allCred = new ArrayList<MongoCredential>();
System.out.println("???????????????????"+username+" "+database+" "+password+" "+host+" "+port);
allCred.add(MongoCredential.createCredential(username, database, password.toCharArray()));
MongoClient client = new MongoClient((new ServerAddress(host, Integer.parseInt(port))), allCred);
client.setWriteConcern(WriteConcern.ACKNOWLEDGED);
return client;
}}

Related

Better way to set testcontainer properties from default applicaton.yml in springboottest

I am using posgresql testcontainer in springboottest. As I have multiple tests involving this testcontainer, hence I have used static testcontainer which will be invoked once for all tests of 1 junit class and shutdown after all tests are executed.
This I have implemented using ParameterResolver, BeforeEachCallback.
Problem with this approach is that datasource metadata like jdbc-url, db name , host , port configured in default application.yml is not used directly in testcontainer properties, instead I have hardcoded those values because springboot properties are not available at that time.
is there any better approach where I can use static testcontainers having BeforeEachCallback feature whose values are fetched from default application.yml ?
#SpringBootTest
class SampleTest extends TestContainerBase {
#Test
void test1() {
//some code
}
}
#ExtendWith(ContainerExtension.class)
#ResourceLock(Environment.ID)
public abstract class TestContainerBase {
protected static String jdbcUrl;
protected static String username;
protected static String password;
#BeforeAll
static void prepareContainerEnvironment(Environment env) {
jdbcUrl = env.getJdbcUrl();
username = env.getUsername();
password = env.getPassword();
}
#DynamicPropertySource
static void dynamicPropertySource(DynamicPropertyRegistry registry) {
registry.add("spring.datasource-.jdbc-url", () -> jdbcUrl);
registry.add("spring.datasource-.username", () -> username);
registry.add("spring.datasource-.password", () -> password);
registry.add("spring.datasource-.driver-class-name", () -> "org.postgresql.Driver");
}
}
public class ContainerExtension implements ParameterResolver, BeforeEachCallback {
// overridden supportsParameter and resolveParameter
}
I want that myDB , sa , sa are read from application.yml. How can I get application.yml values here in this class ? As springboot context is not yet loaded so I am unable to think of any alternative to get those values.
public class ContainerResource extends Environment {
#Container
protected static PostgreSQLContainer postgreSQLContainer =
new PostgreSQLContainer("artifactory.devtools.syd.c1.macquarie.com:9996/postgres:11")
.withDatabaseName("myDB")
.withUsername("username")
.withPassword("password");
ContainerEnvironmentResource() {
postgreSQLContainer.start();
this.setJdbcUrl(postgreSQLContainer.getJdbcUrl());
this.setUsername(postgreSQLContainer.getUsername());
this.setPassword(postgreSQLContainer.getPassword());
}
}
It looks like there is now a dedicated project just to integrate Testcontainers and Spring-Boot. As I understand the documentation it should be transparent to the code as everything is done using Spring magic.
https://github.com/Playtika/testcontainers-spring-boot

Properties of custom Spring Cloud Service Connector not published to cloud.services.*

I created a custom Spring Cloud Service Connector by defining the following two classes:
#ServiceLabel("kafka")
public class KafkaServiceInfo extends BaseServiceInfo {
private static Logger logger = Logger.getLogger(KafkaServiceInfo.class.getName());
public static final String BROKERS = "brokers";
public static final String REGISTRY = "schemaregistry";
protected List<String> brokers;
protected String registry;
public KafkaServiceInfo(String id, List<String> brokers, String registry) {
super(id);
this.brokers = brokers;
this.registry = registry;
}
#ServiceProperty
public String getRegistry() {
return registry;
}
#ServiceProperty
public List<String> getBrokers() {
return brokers;
}
}
And this class:
public class KafkaServiceInfoCreator extends CloudFoundryServiceInfoCreator<KafkaServiceInfo> {
private static Logger logger = Logger.getLogger(KafkaServiceInfoCreator.class.getName());
public static final String USER_PROVIDED_SERVICE_NAME = "kafka";
public KafkaServiceInfoCreator() {
super(new Tags(USER_PROVIDED_SERVICE_NAME), null);
}
public KafkaServiceInfo createServiceInfo(Map<String, Object> serviceData) {
String id = getId(serviceData);
Map<String, Object> credentials = getCredentials(serviceData);
List<String> brokers = (List<String>) credentials.get(KafkaServiceInfo.BROKERS);
String registry = (String) credentials.get(KafkaServiceInfo.REGISTRY);
logger.info("KafkaServiceInfo created for Cloud Foundry Service \"" + id + "\"");
logger.info("Kafka Brokers configured for Cloud Foundry Service: " + brokers.toString());
logger.info("Schema Registry configured for Cloud Foundry Service: " + registry);
return new KafkaServiceInfo(id, brokers, registry);
}
#Override
public boolean accept(Map<String, Object> serviceData) {
return getId(serviceData).contains(USER_PROVIDED_SERVICE_NAME);
}
}
On my PCF instance, I created a user-provided service that looks in the VCAPS env variables like this:
"user-provided": [
{
"credentials": {
"brokers": [
"<some-ip-here>:29092"
],
"schemaregistry": "http://<some-ip-here>:8081"
},
"label": "user-provided",
"name": "kafka",
"syslog_drain_url": "",
"tags": [],
"volume_mounts": []
}
]
I also added the service definition file into the META-INF folder.
src/main/resources/META-INF/services/org.springframework.cloud.cloudfoundry.CloudFoundryServiceInfoCreator
with the content:
path.to.my.file.KafkaServiceInfoCreator
I would now expect to see the properties whose getters are annotated with #ServiceProperty in the cloud.services.kafka.* path. However, they do not show up. Instead, I just have the following 2 entries:
"cloud.services.kafka.id": "kafka",
"cloud.services.null.id": "kafka",
I am wondering what is going wrong here and also why I have the second entry with the "null" in between.
Any ideas what could be wrong here? The classes somehow seem to be found since I get the log messages defined in the creator class above.
Regards, Lars
The #ServiceProperty annotation expects to be provided with either a category or name attribute. These values are used to build the key that is placed into the cloud.services map. If neither category or name attribute are provided, the #ServiceProperty annotation does not result in the property appearing in the map.
A typical usage is #ServiceProperty(category="connection"), with the name defaulting to the name of the property using bean naming rules. In your case, adding the category="connection" attribute should result in
"cloud.services.kafka.id": "kafka",
"cloud.services.kafka.connection.registry": "http://<some-ip-here>:8081",
"cloud.services.kafka.connection.brokers": ["<some-ip-here>:29092"],
I'm not sure yet where the "cloud.services.null.id": "kafka" entry is coming from. Let me know if that still appears after adding the attributes to the #ServiceProperty annotation.
After digging quite a bit deeper, I found the "reason" explained here.
The cloud foundry java buildpack includes the auto-reconfiguration library which by itself contains a copy of the org.springframework.cloud namespace. This copy does NOT consider any custom ServiceInfo classes.
An since it is also this auto-reconfiguration that exposes the cloud.services.* properties to the environment, my personal ones are not picked-up and exposed as well. So I will switch this auto-reconfiguration off and configure the required stuff manually.
The spring cloud connector documentation is misleading here as well, since the properties in cloud.service.* are only added by java auto reconfiguration to the environment.

Custom JASPIC on WebSphere error message

Though similar, the specific problem I have is not addressed in Use JASPIC auth module on WebSphere 8.5
I am getting the following error message:
SECJ8027E: The path and name of file where JASPI persistent registrations are stored must be specified using property com.ibm.websphere.jaspi.configuration.
I can set the custom property in the administration to some existing folder but I wanted to make sure that is the right approach or if there is some step I was missing.
Note I am specifically using the "embedded in application" approach rather than a server installed JASPIC module so I have something like this
#WebListener
public class JaspicInitializer implements
ServletContextListener {
#Override
public void contextInitialized(final ServletContextEvent sce) {
final Map<String, String> options = new HashMap<>();
AuthConfigFactory.getFactory()
.registerConfigProvider(AuthModuleConfigProvider.class.getName(), options, "HttpServlet", null, null);
}
}
I had the error on both WebSphere 8.5.5.11 and 9.0.0.3
From #Uux comment, I changed the way I do the registration so it no longer give the error.
#WebListener
public class JaspicInitializer implements
ServletContextListener {
private String registrationID;
#Override
public void contextDestroyed(final ServletContextEvent sce) {
AuthConfigFactory.getFactory().removeRegistration(registrationID);
}
#Override
public void contextInitialized(final ServletContextEvent sce) {
final ServletContext context = sce.getServletContext();
registrationID = AuthConfigFactory.getFactory()
.registerConfigProvider(new AuthModuleConfigProvider(), "HttpServlet",
context.getVirtualServerName() + " " + context.getContextPath(), "JEE Sample");
}
}
Also WebSphere Global Security needs to be configured with
Enable application security
Enable Java Authentication SPI (JASPI)

Connecting to multiple MySQL db instances using jooq in spring boot application

I have a spring boot application which is using gradle as build tool and jooq for dao class generation and db connection. Previously my application was connecting to single mysql instance. Below are the configuration we used for connecting to single db instance:
spring.datasource.username=user
spring.datasource.password=password
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.datasource.name=ds_name
spring.datasource.schema=ds_schema
spring.jooq.sql-dialect=MYSQL
Current project structure is
a) Main application project MainApp having application.properties with above key-value pairs.
b) Separate application project as DBProject which has jooq's generated DAO classes. MainApp include DBProject as a jar.
I am using gradle as build tool for this.
Everything is working fine till here. But now I have to connect to one more instance of MySQL. So, I have created another db project as DBProject2 which also contains dao classes generated by jooq using another mysql schema. I have created DBProject2 exactly as DBProject is created.
Now, my question is if I include both DBProjects in MainApp as jar then both will use same db configuration as in application.properties. How I can make separate db jars to point to their respective db schemas. I googled alot about this but couldn't find helpful solution.
This is what I do to connect to multiple (additional) data sources in my Play app. I am not sure if it is the best approach, but it works great for me. I have changed names below to be generic.
// In my application.conf
// default data source
db.default.driver=com.mysql.jdbc.Driver
db.default.url="jdbc:mysql://localhost:3306/myDb?useSSL=false"
db.default.username=myuser
db.default.password="mypassword"
// additional data source
db.anothersource.driver=com.mysql.jdbc.Driver
db.anothersource.url="jdbc:mysql://localhost:3306/myothersource?useSSL=false"
db.anothersource.username=myuser
db.anothersource.password="mypassword"
// Then in Java, I create a JooqContextProvider class to expose both connections.
public class JooqContextProvider {
#Inject
Database db;
#Inject
play.Configuration config;
public JooqContextProvider(){}
/**
* Creates a default database connection for data access.
* #return DSLConext.
*/
public DSLContext dsl() {
return DSL.using(new JooqConnectionProvider(db), SQLDialect.MYSQL);
}
/**
* Creates an anothersource database connection for data access.
* #return DSLConext for anothersource.
*/
public DSLContext anotherDsl() {
return DSL.using(
new JooqAnotherSourceConnectionProvider(
config.getString("db.anothersource.url"),
config.getString("db.anothersource.username"),
config.getString("db.anothersource.password")),
SQLDialect.MYSQL);
}
}
// Then I needed to implement my JooqAnotherSourceConnectionProvider
public class JooqAnotherSourceConnectionProvider implements ConnectionProvider {
private Connection connection = null;
String url;
String username;
String password;
public JooqAnotherSourceConnectionProvider(String url, String username, String password){
this.url = url;
this.username = username;
this.password = password;
}
#Override
public Connection acquire() throws DataAccessException {
try {
connection = DriverManager.getConnection(url, username, password);
return connection;
}
catch (java.sql.SQLException ex) {
throw new DataAccessException("Error getting connection from data source", ex);
}
}
#Override
public void release(Connection releasedConnection) throws DataAccessException {
if (connection != releasedConnection) {
throw new IllegalArgumentException("Expected " + connection + " but got " + releasedConnection);
}
try {
connection.close();
connection = null;
}
catch (SQLException e) {
throw new DataAccessException("Error closing connection " + connection, e);
}
}
}
// Then in Java code where I need to access one or the other data sources...
jooq.dsl().select().from().where()...
jooq.anotherDsl().select().from().where()...

Test sending email in Spring

I want to test my services in spring which should send emails.
I try to use org.subethamail:subethasmtp.
To acieve my goal I created service MySender where I send email:
#Autowired
private MailSender mailSender;
//...
SimpleMailMessage message = new SimpleMailMessage();
message.setTo("example#example.com");
message.setSubject("Subject");
message.setText("Text");
mailSender.send(message);
// ...
To test this piece of code I created test application.properties (in test scope):
spring.mail.host=127.0.0.1
spring.mail.port=${random.int[4000,6000]}
And test configuration class which should start Wiser SMTP server and make it reusable in tests:
#Configuration
public class TestConfiguration {
#Autowired
private Wiser wiser;
#Value("${spring.mail.host}")
String smtpHost;
#Value("${spring.mail.port}")
int smtpPort;
#Bean
public Wiser provideWiser() {
// provide wiser for verification in tests
Wiser wiser = new Wiser();
return wiser;
}
#PostConstruct
public void initializeMailServer() {
// start server
wiser.setHostname(smtpHost);
wiser.setPort(smtpPort);
wiser.start();
}
#PreDestroy
public void shutdownMailServer() {
// stop server
wiser.stop();
}
}
Expected result is that application sends email using Wiser smtp server and verify number of sended messages.
But when I run service application throws MailSendException(Couldn't connect to host, port: 127.0.0.1, 4688; timeout -1;).
But when I add breakpoint and try connect using telnet smtp server allow to connect and don't throw Connection refused.
Do you have any idea why I can't test sending mails?
Full code preview is available on github:
https://github.com/karolrynio/demo-mail
I faced same problem. If using some constant port number for spring.mail.port in test Spring configuration combined with Maven tests forking, it resulted in tests randomly failing on port conflict when starting Wiser.
As noted here in comments, using random.int doesn't help - it returns different value each time it's referenced, and it's expected behavior (see this issue).
Hence, we need a different way to initialize spring.mail.port with a random value, so it would be constant within the test execution. Here's a way to do it (thanks for advice here):
First, we may not set spring.mail.port in test properties file at all. We'll initialize it in TestPropertySource. We'll need a class like this:
public class RandomPortInitailizer implements ApplicationContextInitializer<ConfigurableApplicationContext> {
#Override
public void initialize(ConfigurableApplicationContext applicationContext) {
int randomPort = SocketUtils.findAvailableTcpPort();
TestPropertySourceUtils.addInlinedPropertiesToEnvironment(applicationContext,
"spring.mail.port=" + randomPort);
}
}
Now we can run our tests this way (not too different from what's found in OP):
#RunWith(SpringRunner.class)
#ContextConfiguration(initializers = RandomPortInitailizer.class)
public class WhenEmailingSomeStuff {
#Value("${spring.mail.host}")
String smtpHost;
#Value("${spring.mail.port}")
int smtpPort;
#Before
public void startEmailServer() {
wiser = new Wiser();
wiser.setPort(smtpPort);
wiser.setHostname(smtpHost);
wiser.start();
}
#After
public void stopEmailServer() {
wiser.stop();
}
#Test
public void testYourJavaMailSenderHere() {
//
}
}
in the application properties can you also add
mail.smtp.auth=false
mail.smtp.starttls.enable=false
The change your code to have these extra two values
#Value("${mail.smtp.auth}")
private boolean auth;
#Value("${mail.smtp.starttls.enable}")
private boolean starttls;
and put these options in your initializeMailServer
Properties mailProperties = new Properties();
mailProperties.put("mail.smtp.auth", auth);
mailProperties.put("mail.smtp.starttls.enable", starttls);
wiser.setJavaMailProperties(mailProperties);
wiser.setHostname(smtpHost);
wiser.setPort(smtpPort);
wiser.start();
let me know if this worked for you

Resources