Better way to set testcontainer properties from default applicaton.yml in springboottest - spring-boot

I am using posgresql testcontainer in springboottest. As I have multiple tests involving this testcontainer, hence I have used static testcontainer which will be invoked once for all tests of 1 junit class and shutdown after all tests are executed.
This I have implemented using ParameterResolver, BeforeEachCallback.
Problem with this approach is that datasource metadata like jdbc-url, db name , host , port configured in default application.yml is not used directly in testcontainer properties, instead I have hardcoded those values because springboot properties are not available at that time.
is there any better approach where I can use static testcontainers having BeforeEachCallback feature whose values are fetched from default application.yml ?
#SpringBootTest
class SampleTest extends TestContainerBase {
#Test
void test1() {
//some code
}
}
#ExtendWith(ContainerExtension.class)
#ResourceLock(Environment.ID)
public abstract class TestContainerBase {
protected static String jdbcUrl;
protected static String username;
protected static String password;
#BeforeAll
static void prepareContainerEnvironment(Environment env) {
jdbcUrl = env.getJdbcUrl();
username = env.getUsername();
password = env.getPassword();
}
#DynamicPropertySource
static void dynamicPropertySource(DynamicPropertyRegistry registry) {
registry.add("spring.datasource-.jdbc-url", () -> jdbcUrl);
registry.add("spring.datasource-.username", () -> username);
registry.add("spring.datasource-.password", () -> password);
registry.add("spring.datasource-.driver-class-name", () -> "org.postgresql.Driver");
}
}
public class ContainerExtension implements ParameterResolver, BeforeEachCallback {
// overridden supportsParameter and resolveParameter
}
I want that myDB , sa , sa are read from application.yml. How can I get application.yml values here in this class ? As springboot context is not yet loaded so I am unable to think of any alternative to get those values.
public class ContainerResource extends Environment {
#Container
protected static PostgreSQLContainer postgreSQLContainer =
new PostgreSQLContainer("artifactory.devtools.syd.c1.macquarie.com:9996/postgres:11")
.withDatabaseName("myDB")
.withUsername("username")
.withPassword("password");
ContainerEnvironmentResource() {
postgreSQLContainer.start();
this.setJdbcUrl(postgreSQLContainer.getJdbcUrl());
this.setUsername(postgreSQLContainer.getUsername());
this.setPassword(postgreSQLContainer.getPassword());
}
}

It looks like there is now a dedicated project just to integrate Testcontainers and Spring-Boot. As I understand the documentation it should be transparent to the code as everything is done using Spring magic.
https://github.com/Playtika/testcontainers-spring-boot

Related

Testcontainers oracle DB

I'm trying to setup a integration test Using Testcontainers and Oracle-xe database.
I'm getting the error below:
application.properties entry:spring.datasource.driver-class-name=org.testcontainers.jdbc.ContainerDatabaseDriver
Driver org.testcontainers.jdbc.ContainerDatabaseDriver claims to not accept jdbcUrl, jdbc:oracle:thin:#localhost:55802/xepdb1
My Test extension:
public class OracleDBContainerExtension implements AfterAllCallback, BeforeAllCallback {
private OracleContainer container;
#Override
public void beforeAll(ExtensionContext context) {
// gvenzl/oracle-xe:18.4.0-slim
container = new OracleContainer();
container.start();
container.waitingFor(
Wait.forListeningPort().withStartupTimeout(Duration.ofSeconds(180L)));
System.setProperty("spring.datasource.url", container.getJdbcUrl());
System.setProperty("spring.datasource.password", container.getPassword());
System.setProperty("spring.datasource.username", container.getUsername());
}
#Override
public void afterAll(ExtensionContext context) {
container.stop();
}
}
test:
#Testcontainers
#SpringBootTest
#ExtendWith(OracleDBContainerExtension.class)
public class HeroRepositoryTest {
#Autowired
private HeroRepository repositoryUnderTest;
#Test
public void shouldReturnHeroesSuccessfully() {
System.out.println("junit version: " + Version.id());
List<Hero> heroes = repositoryUnderTest.allHeros();
assertThat(heroes).hasSize(1);
repositoryUnderTest.addHero(new Hero("bb", "bb"));
Collection<Hero> heroesAfter = repositoryUnderTest.allHeros();
assertThat(heroesAfter).hasSize(2);
}
}
From the docs about Testcontainers JDBC support --
If you're using the JDBC URL support, there is no need to instantiate
an instance of the container - Testcontainers will do it
automagically.
In other words, one should either use the ContainerDatabaseDriver and the JDBC URL with tc: prefix or a container instance with getJdbcUrl() and the original driver (or let the system detect the driver for you).
So if you make this a normal Oracle driver:
spring.datasource.driver-class-name=org.testcontainers.jdbc.ContainerDatabaseDriver, it should work.

Using DelegatingSessionFactory with RemoteFileTemplate.execute(SessionCallback)

I'm trying to declare multiple SFTP sessions, wrap them in a DelegatingSessionFactory, then later use SftpRemoteFileTemplate.execute(...) during a cron job.
On the execute part of things, the code is very simple, it is already used for a single session, but I want to expand it to multiple possible sessions.
Below I extended my single session code. I just copied the methods for reference. At the end I'll show how I think the new methods should look.
public class XSession extends SftpSession {
#Scheduled(cron = "${sftp.scan.x.schedule}")
void scan() {
List<FileHistoryEntity> fileList = template.execute(this::processFiles);
...
}
private List<FileHistoryEntity> processFiles(Session<ChannelSftp.LsEntry> session) {
List.of(session.list(this.remoteDir)).forEach(file -> doWhatever());
...
}
}
But now I have multiple sessions. So I declare the following class:
#Slf4j
#Configuration
#RequiredArgsConstructor
public class DelegateSftpSessionHandler {
private final SessionFactory<ChannelSftp.LsEntry> session1;
private final SessionFactory<ChannelSftp.LsEntry> session2;
private final SessionFactory<ChannelSftp.LsEntry> session3;
private final SessionFactory<ChannelSftp.LsEntry> session4;
private final SessionFactory<ChannelSftp.LsEntry> session5;
#RequiredArgsConstructor
public enum DelegateSessionConfig {
SESSION_1("IN_REALITY_A_RELEVANT_NAME_1");
SESSION_2("IN_REALITY_A_RELEVANT_NAME_2");
SESSION_3("IN_REALITY_A_RELEVANT_NAME_3");
SESSION_4("IN_REALITY_A_RELEVANT_NAME_4");
SESSION_5("IN_REALITY_A_RELEVANT_NAME_5");
public final String threadKey;
}
#Bean
#Primary
public DelegatingSessionFactory<ChannelSftp.LsEntry> delegatingSessionFactory() {
Map<Object, SessionFactory<ChannelSftp.LsEntry>> sessionMap = new HashMap<>();
sessionMap.put(DelegateSessionConfig.SESSION_1.threadKey, session1);
sessionMap.put(DelegateSessionConfig.SESSION_2.threadKey, session2);
sessionMap.put(DelegateSessionConfig.SESSION_3.threadKey, session3);
sessionMap.put(DelegateSessionConfig.SESSION_4.threadKey, session4);
sessionMap.put(DelegateSessionConfig.SESSION_5.threadKey, session5);
DefaultSessionFactoryLocator<ChannelSftp.LsEntry> sessionLocator = new DefaultSessionFactoryLocator<>(sessionMap);
return new DelegatingSessionFactory<>(sessionLocator);
}
#Bean
SftpRemoteFileTemplate ftpRemoteFileTemplate(DelegatingSessionFactory<ChannelSftp.LsEntry> dsf) {
return new SftpRemoteFileTemplate(dsf);
}
}
Ting is, I have no idea how any of this works, and the spring sftp / fpt documentation is by no means clear. The code is virtually undocumented. And I'm just guessing. I think that I have to do the following:
public class XSession extends SftpSession {
#Autowire
DelegatingSessionFactory<ChannelSftp.LsEntry> delegatingSessionFactory;
#Autowired
SftpRemoteFileTemplate template;
#Scheduled(cron = "${sftp.scan.x.schedule}") // x == SESSION_1
#Async // for thread key
void scan() {
delegatingSessionFactory.setThreadKey(DelegateSessionConfig.SESSION_1.threadKey);
// because thread key changes the session globally? So I don't need specify
// which session this template is working with???
List<FileHistoryEntity> fileList = template.execute(this::processFiles);
...
delegatingSessionFactory.clearThreadKey();
}
private List<FileHistoryEntity> processFiles(Session<ChannelSftp.LsEntry> session) {
List.of(session.list(this.remoteDir)).forEach(file -> doWhatever());
...
}
}
I'm basing what I'm saying on the following link, github spring integration test
Honestly, I hardly understand what is happening. But it seems like setting the thread key, changes the session globally.
My only other idea is to just ... create the RemoteFileTemplate on demand
public static SftpRemoteFileTemplate getTemplateFor(DelegatingSessionFactory<ChannelSftp.LsEntry> dsf, DelegateSessionConfig session) {
return new SftpRemoteFileTemplate(dsf.getFactoryLocator().getSessionFactory(session.threadKey));
}
It does not set it globally. That's how a ThreadLocal variable works: you set a value in some thread and only this thread can see it. If you use the same object concurrently, other threads don't see that value because it does not belong to their thread state.
Not sure what is your concern, but pattern to extend an SftpSession for custom logic is not right. You should consider to use an SftpRemoteFileTemplate.execute(SessionCallback<F, T> callback) instead, but thread key must be set into a DelegatingSessionFactory before anyway and in the same thread you going to call that execute().

Test sending email in Spring

I want to test my services in spring which should send emails.
I try to use org.subethamail:subethasmtp.
To acieve my goal I created service MySender where I send email:
#Autowired
private MailSender mailSender;
//...
SimpleMailMessage message = new SimpleMailMessage();
message.setTo("example#example.com");
message.setSubject("Subject");
message.setText("Text");
mailSender.send(message);
// ...
To test this piece of code I created test application.properties (in test scope):
spring.mail.host=127.0.0.1
spring.mail.port=${random.int[4000,6000]}
And test configuration class which should start Wiser SMTP server and make it reusable in tests:
#Configuration
public class TestConfiguration {
#Autowired
private Wiser wiser;
#Value("${spring.mail.host}")
String smtpHost;
#Value("${spring.mail.port}")
int smtpPort;
#Bean
public Wiser provideWiser() {
// provide wiser for verification in tests
Wiser wiser = new Wiser();
return wiser;
}
#PostConstruct
public void initializeMailServer() {
// start server
wiser.setHostname(smtpHost);
wiser.setPort(smtpPort);
wiser.start();
}
#PreDestroy
public void shutdownMailServer() {
// stop server
wiser.stop();
}
}
Expected result is that application sends email using Wiser smtp server and verify number of sended messages.
But when I run service application throws MailSendException(Couldn't connect to host, port: 127.0.0.1, 4688; timeout -1;).
But when I add breakpoint and try connect using telnet smtp server allow to connect and don't throw Connection refused.
Do you have any idea why I can't test sending mails?
Full code preview is available on github:
https://github.com/karolrynio/demo-mail
I faced same problem. If using some constant port number for spring.mail.port in test Spring configuration combined with Maven tests forking, it resulted in tests randomly failing on port conflict when starting Wiser.
As noted here in comments, using random.int doesn't help - it returns different value each time it's referenced, and it's expected behavior (see this issue).
Hence, we need a different way to initialize spring.mail.port with a random value, so it would be constant within the test execution. Here's a way to do it (thanks for advice here):
First, we may not set spring.mail.port in test properties file at all. We'll initialize it in TestPropertySource. We'll need a class like this:
public class RandomPortInitailizer implements ApplicationContextInitializer<ConfigurableApplicationContext> {
#Override
public void initialize(ConfigurableApplicationContext applicationContext) {
int randomPort = SocketUtils.findAvailableTcpPort();
TestPropertySourceUtils.addInlinedPropertiesToEnvironment(applicationContext,
"spring.mail.port=" + randomPort);
}
}
Now we can run our tests this way (not too different from what's found in OP):
#RunWith(SpringRunner.class)
#ContextConfiguration(initializers = RandomPortInitailizer.class)
public class WhenEmailingSomeStuff {
#Value("${spring.mail.host}")
String smtpHost;
#Value("${spring.mail.port}")
int smtpPort;
#Before
public void startEmailServer() {
wiser = new Wiser();
wiser.setPort(smtpPort);
wiser.setHostname(smtpHost);
wiser.start();
}
#After
public void stopEmailServer() {
wiser.stop();
}
#Test
public void testYourJavaMailSenderHere() {
//
}
}
in the application properties can you also add
mail.smtp.auth=false
mail.smtp.starttls.enable=false
The change your code to have these extra two values
#Value("${mail.smtp.auth}")
private boolean auth;
#Value("${mail.smtp.starttls.enable}")
private boolean starttls;
and put these options in your initializeMailServer
Properties mailProperties = new Properties();
mailProperties.put("mail.smtp.auth", auth);
mailProperties.put("mail.smtp.starttls.enable", starttls);
wiser.setJavaMailProperties(mailProperties);
wiser.setHostname(smtpHost);
wiser.setPort(smtpPort);
wiser.start();
let me know if this worked for you

Spring Boot and how to configure connection details to MongoDB?

Being new to Spring Boot I am wondering on how I can configure connection details for MongoDB.
I have tried the normal examples but none covers the connection details.
I want to specify the database that is going to be used and the url/port of the host that runs MongoDB.
Any hints or tips?
Just to quote Boot Docs:
You can set spring.data.mongodb.uri property to change the url, or alternatively specify a host/port. For example, you might declare the following in your application.properties:
spring.data.mongodb.host=mongoserver
spring.data.mongodb.port=27017
All available options for spring.data.mongodb prefix are fields of MongoProperties:
private String host;
private int port = DBPort.PORT;
private String uri = "mongodb://localhost/test";
private String database;
private String gridFsDatabase;
private String username;
private char[] password;
It's also important to note that MongoDB has the concept of "authentication database", which can be different than the database you are connecting to. For example, if you use the official Docker image for Mongo and specify the environment variables MONGO_INITDB_ROOT_USERNAME and MONGO_INITDB_ROOT_PASSWORD, a user will be created on 'admin' database, which is probably not the database you want to use. In this case, you should specify parameters accordingly on your application.properties file using:
spring.data.mongodb.host=127.0.0.1
spring.data.mongodb.port=27017
spring.data.mongodb.authentication-database=admin
spring.data.mongodb.username=<username specified on MONGO_INITDB_ROOT_USERNAME>
spring.data.mongodb.password=<password specified on MONGO_INITDB_ROOT_PASSWORD>
spring.data.mongodb.database=<the db you want to use>
spring.data.mongodb.host and spring.data.mongodb.port are not supported if you’re using the Mongo 3.0 Java driver. In such cases, spring.data.mongodb.uri should be used to provide all of the configuration, like this:
spring.data.mongodb.uri=mongodb://user:secret#mongo1.example.com:12345
In a maven project create a file src/main/resources/application.yml with the following content:
spring.profiles: integration
# use local or embedded mongodb at localhost:27017
---
spring.profiles: production
spring.data.mongodb.uri: mongodb://<user>:<passwd>#<host>:<port>/<dbname>
Spring Boot will automatically use this file to configure your application. Then you can start your spring boot application either with the integration profile (and use your local MongoDB)
java -jar -Dspring.profiles.active=integration your-app.jar
or with the production profile (and use your production MongoDB)
java -jar -Dspring.profiles.active=production your-app.jar
You can define more details by extending AbstractMongoConfiguration.
#Configuration
#EnableMongoRepositories("demo.mongo.model")
public class SpringMongoConfig extends AbstractMongoConfiguration {
#Value("${spring.profiles.active}")
private String profileActive;
#Value("${spring.application.name}")
private String proAppName;
#Value("${spring.data.mongodb.host}")
private String mongoHost;
#Value("${spring.data.mongodb.port}")
private String mongoPort;
#Value("${spring.data.mongodb.database}")
private String mongoDB;
#Override
public MongoMappingContext mongoMappingContext()
throws ClassNotFoundException {
// TODO Auto-generated method stub
return super.mongoMappingContext();
}
#Override
#Bean
public Mongo mongo() throws Exception {
return new MongoClient(mongoHost + ":" + mongoPort);
}
#Override
protected String getDatabaseName() {
// TODO Auto-generated method stub
return mongoDB;
}
}
In case that somebody is trying to connect to a Atlas MongoDB Cluster in application.properties has to have a config like:
spring.data.mongodb.uri=mongodb+srv://databaseUsername:usernamePassword#cluster0.j4koa.mongodb.net/databaseUsername?retryWrites=true&w=majority
In my case I needed to set up MongoDB for integration tests using Testcontainers. Using properites file was not an option since Mongo port had to be specified during runtime. I wanted to preseve original MongoDB autoconfiguration provided by SpringBoot but override some of the properties. This can be achieved by defining a bean of type MongoClientSettingsBuilderCustomizer which can be used to customize mongo settings :
#Bean
public MongoClientSettingsBuilderCustomizer clientSettingsBuilderCustomizer(final GenericContainer<?> mongoDBContainer) {
String database = environment.getProperty("spring.data.mongodb.database");
ConnectionString connectionString = new ConnectionString(String.format("mongodb://localhost:%s/%s", mongoDBContainer.getFirstMappedPort(), database));
return settings -> settings.applyConnectionString(connectionString);
}
If you simply want to read a MongoDB connection string from an environment variable, one way is to set the following environment variable:
SPRING_DATA_MONGODB_URI=mongodb://localhost:27017/trying-mongo
This doesn't require any changes in the application.properties as the spring data mongo will read the value from the above environment variable by default.
Here is How you can do in Spring Boot 2.0 by creating custom MongoClient adding Providing more control for Connection ,
Please follow github Link for Full Source Code
#Configuration
#EnableMongoRepositories(basePackages = { "com.frugalis.repository" })
#ComponentScan(basePackages = { "com.frugalis.*" })
#PropertySource("classpath:application.properties")
public class MongoJPAConfig extends AbstractMongoConfiguration {
#Value("${com.frugalis.mongo.database}")
private String database;
#Value("${com.frugalis.mongo.server}")
private String host;
#Value("${com.frugalis.mongo.port}")
private String port;
#Value("${com.frugalis.mongo.username}")
private String username;
#Value("${com.frugalis.mongo.password}")
private String password;
#Override
protected String getDatabaseName() {
return database;
}
#Override
protected String getMappingBasePackage() {
return "com.frugalis.entity.mongo";
}
#Bean
public MongoTemplate mongoTemplate() throws Exception {
return new MongoTemplate(mongoClient(), getDatabaseName());
}
#Override
#Bean
public MongoClient mongoClient() {
List<MongoCredential> allCred = new ArrayList<MongoCredential>();
System.out.println("???????????????????"+username+" "+database+" "+password+" "+host+" "+port);
allCred.add(MongoCredential.createCredential(username, database, password.toCharArray()));
MongoClient client = new MongoClient((new ServerAddress(host, Integer.parseInt(port))), allCred);
client.setWriteConcern(WriteConcern.ACKNOWLEDGED);
return client;
}}

Spring 3.1 ConfigurableWebApplicationContext properties #Value not resolving

I need to add a bunch of Properties backed in the DB at start up time.
To test the whole thing works, I started with this (the ds.username property below comes from catalina.properties. It's there just to verify I don't break anything):
public class PropertiesInitializer implements ApplicationContextInitializer<ConfigurableWebApplicationContext> {
#Override
public void initialize(ConfigurableWebApplicationContext ctx) {
try {
props.put("hello", "goodbye");
MutablePropertySources propertySources = ctx.getEnvironment().getPropertySources();
propertySources.addFirst(new MapPropertySource("dbProps", props));
}
catch(Exception e) {
e.printStackTrace();
}
}
I have a #Controller and I'm doing this:
#Autowired
Environment env;
#Value( "${hello}" )
public String hello;
#Value( "${ds.username}" )
public String un;
...
So, when I print those, 'hello' and 'un' are empty but the env.getProperties actually return the right values.
Why?
Thanks
Gerardo Blanco
${...} properties are not enabled by default, you need to declare PropertySourcesPlaceholderConfigurer as a bean in order to enable them.
Environment works out of the box because it doesn't require special configuration.

Resources