How to implement spring data mongo multidatabase transactions - spring

I created an application that copies data from one database to another one based on this tutorial.
In fact, I need to make a method that inserts in two different databases transactional.
Is this possible with MongoDB ? and how can I implement it?

To use multi-document transactions in MongoDB across multiple databases in Spring Data MongoDB, you need to configure a MongoTemplate per database, but all of them must use the same MongoDbFactory because it is treated as the transactional resource.
This means that you will need to override a couple of methods of MongoTemplate to make it use the database it should use (and not the one configured inside SimpleMongoClientDbFactory).
Let's assume that your databases are called 'one' and 'two'. Then it goes like this:
public class MongoTemplateWithFixedDatabase extends MongoTemplate {
private final MongoDbFactory mongoDbFactory;
private final String databaseName;
public MongoTemplateWithFixedDatabase(MongoDbFactory mongoDbFactory,
MappingMongoConverter mappingMongoConverter, String databaseName) {
super(mongoDbFactory, mappingMongoConverter);
this.mongoDbFactory = mongoDbFactory;
this.databaseName = databaseName;
}
#Override
protected MongoDatabase doGetDatabase() {
return MongoDatabaseUtils.getDatabase(databaseName, mongoDbFactory, ON_ACTUAL_TRANSACTION);
}
}
and
#Bean
public MongoDbFactory mongoDbFactory() {
// here, some 'default' database name is configured, the following MongoTemplate instances will ignore it
return new SimpleMongoDbFactory(mongoClient(), getDatabaseName());
}
#Bean
public MongoTransactionManager mongoTransactionManager() {
return new MongoTransactionManager(mongoDbFactory());
}
#Bean
public MongoTemplate mongoTemplateOne(MongoDbFactory mongoDbFactory,
MappingMongoConverter mappingMongoConverter) {
MongoTemplate template = new MongoTemplateWithFixedDatabase(mongoDbFactory,
mappingMongoConverter, "one");
return template;
}
#Bean
public MongoTemplate mongoTemplateTwo(MongoDbFactory mongoDbFactory,
MappingMongoConverter mappingMongoConverter) {
MongoTemplate template = new MongoTemplateWithFixedDatabase(mongoDbFactory,
mappingMongoConverter, "two");
return template;
}
Then just inject mongoTemplateOne and mongoTemplateTwo in your service, mark its method with #Transactional and it should work.
Reactive case
In the reactive case, it's very similar. Of course, you need to use reactive versions of the classes like ReactiveMongoTemplate, ReactiveMongoDatabaseFactory, ReactiveMongoTransactionManager.
There is also a couple of caveats. First, you have to override 3 methods, not 2 (as getCollection(String) also needs to be overridden). Also, I had to do it with an abstract class to make it work:
#Bean
public ReactiveMongoOperations reactiveMongoTemplateOne(
#ReactiveMongoDatabaseFactory reactiveMongoDatabaseFactory,
MappingMongoConverter mappingMongoConverter) {
ReactiveMongoTemplate template = new ReactiveMongoTemplate(reactiveMongoDatabaseFactory,
mappingMongoConverter) {
#Override
protected Mono<MongoDatabase> doGetDatabase() {
return ReactiveMongoDatabaseUtils.getDatabase("one", reactiveMongoDatabaseFactory,
ON_ACTUAL_TRANSACTION);
}
#Override
public MongoDatabase getMongoDatabase() {
return reactiveMongoDatabaseFactory.getMongoDatabase(getDatabaseName());
}
#Override
public MongoCollection<Document> getCollection(String collectionName) {
Assert.notNull(collectionName, "Collection name must not be null!");
try {
return reactiveMongoDatabaseFactory.getMongoDatabase(getDatabaseName())
.getCollection(collectionName);
} catch (RuntimeException e) {
throw potentiallyConvertRuntimeException(e,
reactiveMongoDatabaseFactory.getExceptionTranslator());
}
}
private RuntimeException potentiallyConvertRuntimeException(RuntimeException ex,
PersistenceExceptionTranslator exceptionTranslator) {
RuntimeException resolved = exceptionTranslator.translateExceptionIfPossible(ex);
return resolved == null ? ex : resolved;
}
};
return template;
}
P.S. the provided code was tested with spring-data-mongodb 2.2.4.

Related

Spring Boot: Testing custom MongoTemplate converters

I'm using this custom converters into my Spring Boot service:
#Configuration
public class MongoConfig {
#Bean
public MongoCustomConversions customConversions(){
List<Converter<?,?>> converters = new ArrayList<>();
converters.add(ReferenceWriterConverter.INSTANCE);
return new MongoCustomConversions(converters);
}
#WritingConverter
enum ReferenceWriterConverter implements Converter<Reference, DBObject> {
INSTANCE;
#Override
public String convert(Reference reference) {
//do stuff
}
}
}
Into my controllers, I'm using MontoTemplate in order to talk with MongoDB. So, all converters are already loaded into template.
However, I'd like to test MongoDbTemplate using Spring injection features. I mean, I want to test MongoDbTemplate using custom converters which should already be loaded.
Any ideas on how it can be achieved?
EDIT
public class ModelTest {
private List<Reference> references;
public ModelTest() {
this.references = new ArrayList<Reference>();
}
#Before
public void setUp() {
Reference reference = new Reference();
reference.setId("Ref1");
reference.setTimestamp(new Date());
Metadata met = new Metadata();
met.setId("Mdt1");
met.setUser("user");
met.setCreationTimestamp(new Date());
met.setMetadata("[{'departament': 'JUST'}]");
reference.setMetadata(met);
this.references.add(reference);
ServerAddress serverAddress = new ServerAddress("127.0.0.1", 27017);
MongoClient mongoClient = new MongoClient(serverAddress);
MongoTemplate mongoTemplate = new MongoTemplate(mongoClient, "db");
mongoTemplate.insert(reference);
}
/**
* Assert Office mime type documents.
*/
#Test
public void office() {
fail("Not yet implemented");
}
}
EDIT 2
I also would like to use custom testing properties. I mean, currently, we are setting properties into src/test/resources/application.properties.
spring.data.mongodb.host: localhost
spring.data.mongodb.port: 27017
How could I load these file properties?
Solution 1
If you want to test it with the Spring context, you can annotate your Test class as SpringBootTest and autowire the MongoTemplate. This should then contain your custom conversions for you to test them:
#RunWith(SpringRunner.class)
#SpringBootTest
public class ModelTest {
private List<Reference> references;
#Autowired
private final MongoTemplate mongoTemplate;
public ModelTest() {
this.references = new ArrayList<Reference>();
}
#Before
public void setUp() {
Reference reference = new Reference();
reference.setId("Ref1");
reference.setTimestamp(new Date());
Metadata met = new Metadata();
met.setId("Mdt1");
met.setUser("user");
met.setCreationTimestamp(new Date());
met.setMetadata("[{'departament': 'JUST'}]");
reference.setMetadata(met);
this.references.add(reference);
mongoTemplate.insert(reference);
}
/**
* Assert Office mime type documents.
*/
#Test
public void office() {
fail("Not yet implemented");
}
}
Solution 2
If you just want to test the converter alone, you could make a ReferenceWriterConverterTest like so:
public class ReferenceWriterConverterTest {
private ReferenceWriterConverter converter;
#Before
public void setUp() {
converter = ReferenceWriterConverter.INSTANCE;
}
//test stuff
}

Spring Boot + Mongo DB + Shiro configuration

I am developing a web-based application in Spring boot and Mongo DB. Now I want to use Apache Shiro for Authentication and Authorisation. Can somebody explain to me the procedure and how to establish a mongo db realm and where to mention the permission-user mapping? Thank You.
Basically you need three component
#Component
public class YourMongoConfiguration {
#Bean(name = "mongoTemplate")
#DependsOn({ "lifecycleBeanPostProcessor" })
public MongoTemplate mongoTemplate() throws Exception {
MongoTemplate mt = new MongoTemplate(YOUR_CONFIGURATIOP_HERE);
return mt;
}
}
Then a MongoRealm
#Component("mongoRealm")
public class MongoRealm extends AuthorizingRealm {
private final MongoTemplate mongoTemplate;
#Autowired
public MongoRealm(MongoTemplate mongoTemplate) {
this.mongoTemplate = mongoTemplate;
HashedCredentialsMatcher credentialsMatcher = new HashedCredentialsMatcher();
credentialsMatcher.setHashAlgorithmName(Sha512Hash.ALGORITHM_NAME);
credentialsMatcher.setHashIterations(53);
setCredentialsMatcher(credentialsMatcher);
}
#Override
protected AuthorizationInfo doGetAuthorizationInfo(PrincipalCollection principals) {
// YOUR IMPLEMENTATION
}
#Override
protected AuthenticationInfo doGetAuthenticationInfo(AuthenticationToken token)
throws AuthenticationException {
// YOUR IMPLEMENTATION
}
}
and finally a security manager
#Component("securityManager")
public class SecurityManager extends DefaultWebSecurityManager {
#Autowired
public SecurityManager(Realm mongoRealm, SessionDAO mongoSessionDAO) {
super(mongoRealm);
setRealm(mongoRealm);
SessionManager sessionManager = new SessionManager();
setSessionManager(sessionManager);
sessionManager.setSessionDAO(mongoSessionDAO);
}
}
From now on either Shiro will call your MongoRealm to validate login and permission and you will be able to hadle your collection with classes like
#Service
public class ONE_OF_YOUR_Services {
#Autowired
private MongoTemplate mongoTemplate;
protected List<T> getDocuments(Class<T> clazz, String collection) {
return mongoTemplate.findAll(clazz, collection);
}
}
I hope it helps.
There are a few MongoDB realms up on GitHub. I don't want to link to them as haven't tried them out, but that would be your best place to start.

Not monitor a specific Datasource for Health Check

I would like to know if exist some way to disable the monitoring of a Specific DataSource by SpringBoot Actuator.
Scenario:
One Microservice uses 3 Datasources but for some Business Reason, one Datasource of them, it is not necessary to be monitored by Spring Boot Health Indicator.
How to disable the monitoring of one specific DataSource?
Many thanks in advance
Juan Antonio
I think you'd have to disable the default datasources health indicator, which you can do with this property:
management.health.db.enabled=false
And then configure your own health indicators which only address the datasources you are interested in, something like this perhaps:
#Autowired
private DataSource dataSourceA;
#Autowired
private DataSource dataSourceB;
#Bean
public DataSourceHealthIndicator dataSourceHealthIndicatorA() {
return new DataSourceHealthIndicator(dataSourceA);
}
#Bean
public DataSourceHealthIndicator dataSourceHealthIndicatorB() {
return new DataSourceHealthIndicator(dataSourceB);
}
Or, alternatively write your own 'multiple datasources health indicator' by extending AbstractHealthIndicator and injecting into it only the Datasources you are interested in monitoring. Any Spring bean of type HealthIndicator will be automatically registered with the health actuator so you only have to let Spring create your custom HealthIndicator and it will be exposed by the actuator.
For background, you can see how Spring configures the default datasource health check in: org.springframework.boot.actuate.autoconfigure.DataSourcesHealthIndicatorConfiguration.
Since Spring Boot 2, you can filter datasources from the health check by overriding org.springframework.boot.actuate.autoconfigure.jdbcDataSourceHealthIndicatorAutoConfiguration. In the example below datasources without a pool name is filtered.
#Configuration
public class YourDataSourceHealthIndicatorAutoConfiguration extends DataSourceHealthIndicatorAutoConfiguration {
public NonMigrationDataSourceHealthIndicatorAutoConfiguration(Map<String, DataSource> dataSources, ObjectProvider<DataSourcePoolMetadataProvider> metadataProviders) {
// Filter out datasources without a pool name
super(filterDataSources(dataSources), metadataProviders);
}
private static Map<String, DataSource> filterDataSources(Map<String, DataSource> dataSources) {
return dataSources.entrySet().stream()
.filter(dataSourceEntry -> {
if (dataSourceEntry.getValue() instanceof HikariDataSource) {
HikariDataSource hikariDataSource = (HikariDataSource) dataSourceEntry.getValue();
return hikariDataSource.getPoolName() != null;
} else {
return true;
}
}).collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue));
}
}
I think the easiest way will be the following
#Configuration
public class DatasourceHealthCheckConfig extends DataSourceHealthContributorAutoConfiguration {
public DatasourceHealthCheckConfig(Map<String, DataSource> dataSources, ObjectProvider<DataSourcePoolMetadataProvider> metadataProviders) {
super(dataSources, metadataProviders);
}
#Override
public HealthContributor dbHealthContributor(Map<String, DataSource> dataSources) {
// remove the required datasource from the dataSources map by its name
return super.dbHealthContributor(dataSources);
}
}
Solution for Spring Boot 2.1.2. If you wish to disable healthcheck only for specific datasource. Bit hecky, but worked for me.
#Configuration
public class DatasourceHealthCheckConfig extends DataSourceHealthIndicatorAutoConfiguration {
public DatasourceHealthCheckConfig(ObjectProvider<Map<String, DataSource>> dataSources,
ObjectProvider<DataSourcePoolMetadataProvider> metadataProviders) {
super(filterDataSources(dataSources), metadataProviders);
}
private static ObjectProvider<Map<String, DataSource>> filterDataSources(ObjectProvider<Map<String, DataSource>> dataSources) {
final Map<String, DataSource> map = dataSources.getObject();
map.remove("specificDataSource"); // name of datasource for removal
return new ObjectProvider<>() {
#Override
public Map<String, DataSource> getObject(Object... args) throws BeansException {
return map;
}
#Override
public Map<String, DataSource> getIfAvailable() throws BeansException {
return map;
}
#Override
public Map<String, DataSource> getIfUnique() throws BeansException {
return map;
}
#Override
public Map<String, DataSource> getObject() throws BeansException {
return map;
}
};
}
}

Spring Boot & Mongo: custom MappingMongoConverter doesn't work during DotReplacement

I need to achive dotReplacementKey
I'm using MongoDB with such config:
#Configuration
public class MongoTemplateConfig {
#Value("${adserver_mongo_connection_string}")
private String databaseConnectionString;
#Bean
public MongoDbFactory mongoDbFactory() throws UnknownHostException {
MongoClientURI uri = new MongoClientURI(databaseConnectionString.trim());
return new SimpleMongoDbFactory(uri);
}
#Bean
public MongoTemplate mongoTemplate() throws UnknownHostException {
return new MongoTemplate(mongoDbFactory());
}
#Bean
public MappingMongoConverter mongoConverter(MongoDbFactory mongoFactory, MongoMappingContext mongoMappingContext) throws Exception {
DbRefResolver dbRefResolver = new DefaultDbRefResolver(mongoFactory);
MappingMongoConverter mongoConverter = new MappingMongoConverter(dbRefResolver, mongoMappingContext);
mongoConverter.setMapKeyDotReplacement(".");
return mongoConverter;
}
}
I'm doing upsert as follows:
mongoTemplate.bulkOps(...).upsert(...)
but during runtime via debug I found out that different MappingMongoConverter is used rather then this that was configured as a #Bean
btw, if #Inject wherever MappingMongoConverter I get proper bean from config with keyDotReplacement="."
but looks like Spring Boot uses another one under the hood
P.S. I have seen this question without correct answer, but tried with mongoConverter.afterPropertiesSet() and obviously it doesn't work as well
It's a bit ridiculous but the point is that we should pass this custom mappingMongoConverter into MongoTemplate initialization :
#Bean
public MongoTemplate mongoTemplate(MappingMongoConverter mappingMongoConverter) throws UnknownHostException {
return new MongoTemplate(mongoDbFactory(), mappingMongoConverter);
}

Spring #Autowired annotated object value is null

// My Factory class
#Component
public class UserRewardAccountValidatorFactory {
#Autowired
private VirginAmericaValidator virginAmericaValidator;
private static class SingletonHolder {
static UserRewardAccountValidatorFactory instance = new UserRewardAccountValidatorFactory();
}
public static UserRewardAccountValidatorFactory getInstance() {
return SingletonHolder.instance;
}
private UserRewardAccountValidatorFactory() {}
public PartnerValidator getPartnerValidator(Partner partner){
return virginAmericaValidator;
}
}
// My Validator class
#Service
public class VirginAmericaValidator implements PartnerValidator {
#Override
public void validate(String code) throws InvalidCodeException{
//do some processing if processing fails throw exception
if (code.equals("bad".toString())){
throw new InvalidCodeException();
}
}
}
//Usage:
PartnerValidator pv = UserRewardAccountValidatorFactory.getInstance().getPartnerValidator(partner);
if (pv != null){
try {
pv.validate(userRewardAccount);
} catch (InvalidCodeException e){
return buildResponse(ResponseStatus.INVALID_USER_REWARD_ACCOUNT, e.getMessage());
}
}
My package scan level is at much higher level. Whats happening is my virginAmericaValidator is always empty. Why is #Autowired annotation not working.
Your current approach will not work with Spring as you are ultimately using new UserRewardAccountValidatorFactory to create the instance which essentially bypasses Spring context altogether. Two approaches that should possibly work are these:
a. Using a factory-method and using xml to define your bean:
<bean class="package.UserRewardAccountValidatorFactory" name="myfactory" factory-method="getInstance"/>
This will essentially return the instance that you are creating back as a Spring bean and should get autowired cleanly.
b. Using Java #Configuration based mechanism:
#Configuration
public class MyBeanConfiguration {
#Bean
public UserRewardAccountValidatorFactory myFactory() {
return UserRewardAccountValidatorFactory.getInstance();
}
}

Resources