Integration test in spring boot cant found the path of sql file - spring-boot

I am writing a integration test for rest api entry.The api require to initialize database before the test.However it gives an error which showing it cant find any path to my sql file.
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
#AutoConfigureMockMvc
#TestExecutionListeners(listeners = { SqlScriptsTestExecutionListener.class })
class OrderServiceApplicationTests {
#Autowired
private MockMvc mockMvc;
#Autowired
private ObjectMapper objectMapper;
#Autowired
private JobRepository jobRepository;
//#Sql("INSERT INTO user_info(name) VALUES \"alex\" ")
#Test
#Sql("{/resources/schema.sql}")
void shouldCreatePost() throws Exception {
JobOrder job=jobRepository.save(createjob());
String request=objectMapper.writeValueAsString(job);
mockMvc.perform(MockMvcRequestBuilders.post("/Job/Joborder")
.contentType(MediaType.APPLICATION_JSON)
.content(request))
.andExpect(status().is2xxSuccessful());
}
JobOrder createjob(){
JobOrder job=new JobOrder();
job.setTitle("Hiring software engineer");
job.setDescription("responsible for developing and maintaining mobile app");
job.setRequirement("Need to know basic sql springboot,2 years exp");
job.setSalary(234);
job.setOrder_id(1);
return job;
}
}
schema.sql:
INSERT INTO user_info (name) VALUES ('India');
and i got an error:
org.springframework.jdbc.datasource.init.CannotReadScriptException: Cannot read SQL script from class path resource [com/example/OrderService/{/resources/schema.sql}]
at org.springframework.jdbc.datasource.init.ScriptUtils.executeSqlScript(ScriptUtils.java:239)
at org.springframework.jdbc.datasource.init.ResourceDatabasePopulator.populate(ResourceDatabasePopulator.java:254)
at org.springframework.jdbc.datasource.init.DatabasePopulatorUtils.execute(DatabasePopulatorUtils.java:54)
at org.springframework.jdbc.datasource.init.ResourceDatabasePopulator.execute(ResourceDatabasePopulator.java:269)
at org.springframewo
My properties:
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.url=jdbc:mysql://localhost:3306/test
spring.datasource.username=***
spring.datasource.password=*******
spring.jpa.show-sql=true
spring.jpa.hibernate.ddl-auto=create-drop
spring.jpa.defer-datasource-initialization=true
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQL5Dialect
spring.sql.init.mode=always
My Path:
I dont know what is the problem on my path.And i am wondering if there are problem on my test script or strategy

Several things are wrong with your #Sql("{/resources/schema.sql}").
the { and } do not belong into the path, why did you add it in the first place?
You don't need to include the /resources directory in the path. To find out what the path of your file is, open the target/test-classes and have a look. Everything that resides in the resources directory will be placed in the root directory. If your file was in resources/sql/test.sql it would be available under sql/test.sql.
Solution
#Sql("schema.sql")

Related

Integration test for springboot with initalize query

I am going use an integration test in spring boot with in memory mysql database.But my test component in repository part query like:
#Query(
value = "SELECT order_id,title,description,requirement,salary,user_info.name,user_info.contact,date"+
" FROM job_order " +
" FULL JOIN user_info " +
" ON sender_id=user_info.id" +
" WHERE order_id= ?1 ;",nativeQuery = true
)
Response singlejob(int order_id);
My entrypoint is doing stuff on the job_order table.The query require two tables ,so i am trying to insert the user_info table and then test for the job_order.Then a write test like this:
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
#AutoConfigureMockMvc
#TestExecutionListeners(listeners = { SqlScriptsTestExecutionListener.class })
class OrderServiceApplicationTests {
#Autowired
private MockMvc mockMvc;
#Autowired
private ObjectMapper objectMapper;
#Autowired
private JobRepository jobRepository;
//#Sql("INSERT INTO user_info(name) VALUES \"alex\" ")
#Test
#Sql("{/resources/schema.sql}")
void shouldCreatePost() throws Exception {
JobOrder job=jobRepository.save(createjob());
String request=objectMapper.writeValueAsString(job);
mockMvc.perform(MockMvcRequestBuilders.post("/Job/Joborder")
.contentType(MediaType.APPLICATION_JSON)
.content(request))
.andExpect(status().is2xxSuccessful());
}
JobOrder createjob(){
JobOrder job=new JobOrder();
job.setTitle("Hiring software engineer");
job.setDescription("responsible for developing and maintaining mobile app");
job.setRequirement("Need to know basic sql springboot,2 years exp");
job.setSalary(234);
job.setOrder_id(1);
return job;
}
}
schema.sql:
INSERT INTO user_info (name) VALUES ('India');
and i got an error:
org.springframework.jdbc.datasource.init.CannotReadScriptException: Cannot read SQL script from class path resource [com/example/OrderService/{/resources/schema.sql}]
at org.springframework.jdbc.datasource.init.ScriptUtils.executeSqlScript(ScriptUtils.java:239)
at org.springframework.jdbc.datasource.init.ResourceDatabasePopulator.populate(ResourceDatabasePopulator.java:254)
at org.springframework.jdbc.datasource.init.DatabasePopulatorUtils.execute(DatabasePopulatorUtils.java:54)
at org.springframework.jdbc.datasource.init.ResourceDatabasePopulator.execute(ResourceDatabasePopulator.java:269)
at org.springframewo
My properties:
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.url=jdbc:mysql://localhost:3306/test
spring.datasource.username=***
spring.datasource.password=*******
spring.jpa.show-sql=true
spring.jpa.hibernate.ddl-auto=create-drop
spring.jpa.defer-datasource-initialization=true
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQL5Dialect
spring.sql.init.mode=always
My Path:
I dont know what is the problem on my path.And i am wondering if there are problem on my test script or strategy
Path resource semantics should be followed when #Sql annotation is used. If you want to load the script file from the resources folder you should use classpath: reference:
#Sql(value = {"classpath:schema.sql"})

Injecting #value Resource (json file) while testing method in Junit5

Service class :
#Value("classpath:multipleHearingLocations.json")
public static Resource resource;
public void someMethod(){
ConcurrentHashMap<String,List<String>> multipleHearingLocations = new Objec`enter code here`tMapper().readValue(Files.newInputStream(Paths.get(**resource.getURI()**)), new TypeReference<ConcurrentHashMap<String, List<String>>>(){});
}
someTest.json - resided in my main/resources folder
While testing in Jusint5, test class
#Value("classpath:multipleHearingLocations.json")
public static Resource resource;
But when I test it - the service calss throws a null pointer exception saying resource is null.
Test Method :
#Mock
public static Resource resource;
enter code here
#Test
void getMultipleHearingLocations_shouldReturnCorrespondingMultipleEpimsIdForVenue() throws IOException {
SscsCaseData caseData = SscsCaseData.builder()
.appeal(Appeal.builder()
.hearingOptions(HearingOptions.builder().build())
.build())
.processingVenue(PROCESSING_VENUE_1)
.build();
// = new ClassPathResource("multipleHearingLocationsTest.json");
given(venueService.getEpimsIdForVenue(caseData.getProcessingVenue())).willReturn(Optional.of("443014"));
given(referenceDataServiceHolder.getVenueService()).willReturn(venueService);
given(**resource.getURI()**).willReturn(new ClassPathResource("multipleHearingLocationsTest.json").getURI());
List<HearingLocation> result = HearingsDetailsMapping.getHearingLocations(
caseData,
referenceDataServiceHolder
);
What is it that I am doing wrong?
Or is there a way to inject the Resource in the tests as well, so that it is available as a bean when the service class is called?
Tried running it as below, still no luck
RunWith(SpringRunner.class)
#SpringBootTest
#ContextConfiguration(classes = HearingsDetailsMapping.class)
#TestPropertySource(locations = "classpath:multipleHearingLocationsTest.json")
Any help will be appreciated :)
I've encounter the same problem. This injection method works fine with JUnit 4 but not with Junit5.
#RunsWith is a JUnit 4 annotation the JUnit 5 equivalent is #ExtendWith(SpringExtension.class) see this answer for more details.
To inject a resource in JUnit 5 you could use this working soulution
or
File getFile(String fileLocation) {
return new File(this
.getClass()
.getClassLoader()
.getResource(fileLocation)
.getFile());
}
String filelocation = "multipleHearingLocations.json" //(no "class:" before path)
Path to multipleHearingLocations.json while testing: src/test/resources/multipleHearingLocations.json

Spring Batch is looking for source files during deployment

I am developing Spring batch application on Spring Boot. Batch process is based on fire and forget, which means migration process is triggered by Http GET method and doesn't wait for result. There are 6 (six) csv files which containes migration data and these data must be processed and transferred to DB. Source files path is provided by application.yml file. Batch config (template) looks like as follows:
#Configuration
public class BeginCureBtachConfig {
#Value("$batch-files.begincure-source")
private String dataSourceFilePath;
#Autowired
IteamReader<BeginCure> itemReader;
#Autowired
ItemProcessor<BeginCure, BeginCure> itemProcessor;
#Bean Job beginCureJob(JobBuilderFactory jbFactory, StepBuilderDactory stp, ItemWriter<BeginCure> begnCure {
//// code goes here
}
#Bean
public FlatFileItemReader<BeginCure> beginCureItemReader() {
FlatFileItemReader<BeginCure> ffReader = new FlatFileItemReader<>();
ffReader.setResource(new FileSystemResource(dataSourceFilePath));
ffReader.setLineMapper(lineMapper());
return ffReader;
}
#Bean
public LineMapper<BeginCure> lineMapper () {
//..
}
}
The problem is when project is deployed on openshift, batch looks for files even migratin is not triggered.
In the code, I think this part of code giving some error ffReader.setResource(new FileSystemResource(dataSourceFilePath));
Is there any workaround or solution? Did anybody have this kind of problem?
Thank you in advance^^

Connection Timeout with testcontainers and redis

I do integration tests using Spring Boot, TestContainers, redis and Junit 5.
I am facing a weird behavior, when I all the integration tests, I keep having this log displaying :
Cannot reconnect to [localhost:55133]: Connection refused: localhost/127.0.0.1:55133
and this exception :
org.springframework.dao.QueryTimeoutException: Redis command timed out; nested exception is io.lettuce.core.RedisCommandTimeoutException: Command timed out after 1 minute(s)
at org.springframework.data.redis.connection.lettuce.LettuceExceptionConverter.convert(LettuceExceptionConverter.java:70)
But I run the tests individually, I dont have this behavior.
I use Junit5 and I am using Junit5 extension to start and stop my redis container :
public class RedisTestContainerExtension implements BeforeAllCallback, AfterAllCallback {
private GenericContainer<?> redis;
#Override
public void beforeAll(ExtensionContext extensionContext) throws Exception {
redis = new GenericContainer<>(DockerImageName.parse("redis:5.0.3-alpine"))
.withCommand("redis-server","--requirepass", "password")
.waitingFor(Wait.forListeningPort())
.withStartupTimeout(Duration.ofMinutes(2))
.withExposedPorts(6379);
redis.start();
System.setProperty("APP_REDIS_CONVERSATIONS_HOST",redis.getHost());
System.setProperty("APP_REDIS_CONVERSATIONS_PORT",redis.getFirstMappedPort().toString());
System.setProperty("APP_REDIS_CONVERSATIONS_PASSWORD","password");
System.setProperty("APP_REDIS_CONVERSATIONS_TTL","600m");
}
#Override
public void afterAll(ExtensionContext extensionContext) throws Exception {
if(redis != null){
redis.stop();
}
}
}
And I add this file as an extension to my integration test :
#ExtendWith({SpringExtension.class, RedisTestContainerExtension.class})
#SpringBootTest(classes = ConversationsApplication.class)
class MyIntegrationTest {
...
}
Can anyone help me fix this situation.
We had a similar issue. The issue was occured only when we execute all tests (or at least not only one specific)
We have another test setup - we are using a base class to manage test testcontainers - where the port-mapping of the containers was applied by overriding the properties via DynamicPropertySource
Our fix was to mark the base-test-class with #DirtiesContext that spring does not reuse the application-context over the tests-classes - see documentation of DynamicPropertySource:
NOTE: if you use #DynamicPropertySource in a base class and discover that tests in subclasses fail because the dynamic properties change between subclasses, you may need to annotate your base class with #DirtiesContext to ensure that each subclass gets its own ApplicationContext with the correct dynamic properties.
Example:
#Slf4j
#SpringBootTest
#DirtiesContext
#Testcontainers
public abstract class AbstractContainerTest {
#Container
private static final ElasticsearchContainer elasticsearchContainer = new DealElasticsearchContainer();
#Container
private static final RedisCacheContainer redisCacheContainer = new RedisCacheContainer();
#DynamicPropertySource
static void databaseProperties(DynamicPropertyRegistry registry) {
log.info("Override properties to connect to Testcontainers:");
log.info("* Test-Container 'Elastic': spring.elasticsearch.rest.uris = {}",
elasticsearchContainer.getHttpHostAddress());
log.info("* Test-Container 'Redis': spring.redis.host = {} ; spring.redis.port = {}",
redisCacheContainer.getHost(), redisCacheContainer.getMappedPort(6379));
registry.add("spring.elasticsearch.rest.uris", elasticsearchContainer::getHttpHostAddress);
registry.add("spring.redis.host", redisCacheContainer::getHost);
registry.add("spring.redis.port", () -> redisCacheContainer.getMappedPort(6379));
}
}
So maybe give it a try to use #DirtiesContext or switch to a setup which uses DynamicPropertySource to override the properties. It was especially build for this case:
Method-level annotation for integration tests that need to add properties with dynamic values to the Environment's set of PropertySources.
This annotation and its supporting infrastructure were originally designed to allow properties from Testcontainers based tests to be exposed easily to Spring integration tests. However, this feature may also be used with any form of external resource whose lifecycle is maintained outside the test's ApplicationContext.

Spring Boot Application cannot run test cases involving Two Databases correctly - either get detached entities or no inserts

I am trying to write a Spring Boot app that talks to two databases - primary which is read-write, and secondary which is read only.
This is also using spring-data-jpa for the repositories.
Roughly speaking this giude describes what I am doing: https://www.baeldung.com/spring-data-jpa-multiple-databases
And this documentation from spring:
https://docs.spring.io/spring-boot/docs/current/reference/html/howto-data-access.html#howto-two-datasources
I am having trouble understanding some things - I think about the transaction managers - which is making my program error out either during normal operation, or during unit tests.
I am running into two issues, with two different TransactionManagers that I do not understand well
1)
When I use JPATransactionManager, my secondary entities become detached between function calls. This happens in my application running in full Spring boot Tomcat, and when running the JUnit test as SpringRunner.
2)
When I use DataSourceTransactionManager which was given in some tutorial my application works correctly, but when I try to run a test case with SpringRunner, without running the full server, spring/hibernate will not perform any inserts or updates on the primaryDataSource.
--
Here is a snippet of code for (1) form a service class.
#Transactional
public List<PrimaryGroup> synchronizeAllGroups(){
Iterable<SecondarySite> secondarySiteList = secondarySiteRepo.findAll();
List<PrimaryGroup> allUserGroups = new ArrayList<PrimaryGroup>(0);
for( SecondarySite secondarySite: secondarySiteList){
allUserGroups.addAll(synchronizeSiteGroups( secondarySite.getName(), secondarySite));
}
return allUserGroups;
}
#Transactional
public List<PrimaryGroup> synchronizeSiteGroups(String sitename, SecondarySite secondarySite){
// GET all secondary groups
if( secondarySite == null){
secondarySite = secondarySiteRepo.getSiteByName(sitename);
}
logger.debug("synchronizeGroups started - siteId:{}", secondarySite.getLuid().toString());
List<SecondaryGroup> secondaryGroups = secondarySite.getGroups();// This shows the error because secondarySite passed in is detached
List<PrimaryGroup> primaryUserGroups = primaryGroupRepository.findAllByAppName(sitename);
...
// modify existingUserGroups to have new data from secondary
...
primaryGroupRepository.save(primaryUserGroups );
logger.debug("synchronizeGroups complete");
return existingUserGroups;
}
I am pretty sure I understand what is going on with the detached entities with JPATransactionManager -- When populateAllUsers calls populateSiteUser, it is only carrying over the primaryTransactionManager and the secondary one gets left out, so the entities become detached. I can probably work around that, but I'd like to see if there is any way to have this work, without putting all calls to secondary* into a seperate service layer, that returns non-managed entities.
--
Here is a snippet of code for (2) from a controller class
#GetMapping("synchronize/secondary")
public String synchronizesecondary() throws UnsupportedEncodingException{
synchronizeAllGroups(); // pull all the groups
synchronizeAllUsers(); // pull all the users
synchronizeAllUserGroupMaps(); // add the mapping table
return "true";
}
This references that same synchronizeAllGroups from above. But when I am useing DataSourceTransactionManager I do not get that detached entity error.
What I do get instead, is that the primaryGroupRepository.save(primaryUserGroups ); call does not generate any insert or update statement - when running a JUnit test that calls the controller directly. So when synchronizeAllUserGroupMaps gets called, primaryUserRepository.findAll() returns 0 rows, and same with primaryGroupRepository
That is to say - it works when I run this test case:
#RunWith(SpringRunner.class)
#SpringBootTest(classes=app.DApplication.class, properties={"spring.profiles.active=local,embedded"})
#AutoConfigureMockMvc
public class MockTest {
#Autowired
private MockMvc mockMvc;
#Test
public void shouldSync() throws Exception {
this.mockMvc.perform(get("/admin/synchronize/secondary")).andDo(print()).andExpect(status().isOk());
}
}
But it does not do any inserts or updates when I run this test case:
#RunWith(SpringRunner.class)
#SpringBootTest(classes=app.DApplication.class, properties={"spring.profiles.active=local,embedded"}, webEnvironment=WebEnvironment.MOCK)
#AutoConfigureMockMvc
public class ControllerTest {
#Autowired AdminController adminController;
#Test
public void shouldSync() throws Exception {
String o = adminController.synchronizesecondary();
}
}
Here are the two configuration classes
Primary:
#Configuration
#EnableTransactionManagement
#EntityScan(basePackageClasses = app.primary.dao.BasePackageMarker.class )
#EnableJpaRepositories(
transactionManagerRef = "dataSourceTransactionManager",
entityManagerFactoryRef = "primaryEntityManagerFactory",
basePackageClasses = { app.primary.dao.BasePackageMarker.class }
)
public class PrimaryConfig {
#Bean(name = "primaryDataSourceProperties")
#Primary
#ConfigurationProperties("app.primary.datasource")
public DataSourceProperties primaryDataSourceProperties() {
return new DataSourceProperties();
}
#Bean(name = "primaryDataSource")
#Primary
public DataSource primaryDataSourceEmbedded() {
return primaryDataSourceProperties().initializeDataSourceBuilder().build();
}
#Bean
#Primary
public LocalContainerEntityManagerFactoryBean primaryEntityManagerFactory(
EntityManagerFactoryBuilder builder,
#Qualifier("primaryDataSource") DataSource primaryDataSource) {
return builder
.dataSource(primaryDataSource)
.packages(app.primary.dao.BasePackageMarker.class)
.persistenceUnit("primary")
.build();
}
#Bean
#Primary
public DataSourceTransactionManager dataSourceTransactionManager( #Qualifier("primaryDataSource") DataSource primaryDataSource) {
DataSourceTransactionManager txm = new DataSourceTransactionManager(primaryDataSource);
return txm;
}
}
And Secondary:
#Configuration
#EnableTransactionManagement
#EntityScan(basePackageClasses=app.secondary.dao.BasePackageMarker.class ) /* scan secondary as secondary database */
#EnableJpaRepositories(
transactionManagerRef = "secondaryTransactionManager",
entityManagerFactoryRef = "secondaryEntityManagerFactory",
basePackageClasses={app.secondary.dao.BasePackageMarker.class}
)
public class SecondaryConfig {
private static final Logger log = LoggerFactory.getLogger(SecondaryConfig.class);
#Bean(name = "secondaryDataSourceProperties")
#ConfigurationProperties("app.secondary.datasource")
public DataSourceProperties secondaryDataSourceProperties() {
return new DataSourceProperties();
}
#Bean(name = "secondaryDataSource")
public DataSource secondaryDataSourceEmbedded() {
return secondaryDataSourceProperties().initializeDataSourceBuilder().build();
}
#Bean
public LocalContainerEntityManagerFactoryBean secondaryEntityManagerFactory(
EntityManagerFactoryBuilder builder,
#Qualifier("secondaryDataSource") DataSource secondaryDataSource) {
return builder
.dataSource(secondaryDataSource)
.packages(app.secondary.dao.BasePackageMarker.class)
.persistenceUnit("secondary")
.build();
}
#Bean
public DataSourceTransactionManager secondaryTransactionManager( #Qualifier("secondaryDataSource") DataSource secondaryDataSource) {
DataSourceTransactionManager txm = new DataSourceTransactionManager(secondaryDataSource);
return txm;
}
}
In my real application, the secondary data source - since it is read-only - is used during real run time, and during the unit test I am writing.
I have been having trouble getting spring to initialize both data sources, so I have not attached a complete example.
thanks for any insight people an give me.
Edit: I have read some things that say to use Jta Transaction Manager when using multiple databases, and I have tried that. I get an error when it tries to run the transaction on my second read-only database when I go to commit to the first database
Caused by: org.postgresql.util.PSQLException: ERROR: prepared transactions are disabled
Hint: Set max_prepared_transactions to a nonzero value.
In my case, I cannot set that, because the database is a read-only datbase provided by a vendor, we cannot change anything, and I really sho9udn't be trying to include this database as part of transactions, I just want to be able to call both databases in one service call.

Resources