I am trying to connect to two different buckets in couchbase using spring boot. But in a single spring boot application the database config only takes a single bucket name.
Is it possible to connect to more than one couchbase bucket in spring-boot?
So it seems you want to use Spring Data Couchbase from within a Spring Boot application, and have (at least) two different repositories backed by two different Bucket?
You'll have to customize your Spring Data configuration programmatically (as opposed to letting Spring Boot do all the heavy lifting), but that's possible.
Spring Boot creates a CouchbaseConfigurer through which it creates default Cluster and Bucket (as tuned in the properties file).
If you have a CouchbaseRepository on your classpath, it'll also attempt to configure Spring Data by instantiating a SpringBootCouchbaseDataConfiguration class.
You can customize that by extending the SpringBootCouchbaseDataConfiguration above in your project, marking it as #Configuration
Once you're ready to customize the Spring Data configuration programmatically, what you need is to create a second Bucket bean, a second CouchbaseTemplate that uses that bucket, and then instruct Spring Data Couchbase on which template to use with which Repository.
To that end, there is a configureRepositoryOperationsMapping(...) method. You can use the parameter of this method as a builder to:
link a specific Repository interface to a CouchbaseTemplate: map
say that any repo with a specific entity type should use a given template: mapEntity
even redefine the default template to use (initially the one created by Spring Boot): setDefault.
This second part is explained in the Spring Data Couchbase documentation.
Probably what you are trying to say is that Spring boot provides pre-defined properties that you can modify, such as: couchbase.cluster.bucket that takes single value and you want to connect to two or more buckets.
In case you will not find a better solution, I can point you to a slightly different approach, and that is to setup your own couchbase connection manager that you can inject anywhere you need.
Here is the example of such #Service that will provider you with two connections to different buckets.
You can modify to suite your needs, it is very small.
import java.util.ArrayList;
import java.util.List;
import javax.annotation.PostConstruct;
import javax.annotation.PreDestroy;
import org.apache.log4j.Logger;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import com.couchbase.client.java.Bucket;
import com.couchbase.client.java.Cluster;
import com.couchbase.client.java.CouchbaseCluster;
import com.couchbase.client.java.env.CouchbaseEnvironment;
import com.couchbase.client.java.env.DefaultCouchbaseEnvironment;
#Service
public class CouchbaseConnectionManager {
private static final int TIMEOUT = 100000;
#Value("#{configProp['couchbase.nodes']}")
private List<String> nodes = new ArrayList<String>();
#Value("#{configProp['couchbase.binary.bucketname']}")
private String binaryBucketName;
#Value("#{configProp['couchbase.nonbinary.bucketname']}")
private String nonbinaryBucketName;
#Value("#{configProp['couchbase.password']}")
private String password;
private Bucket binaryBucket;
private Bucket nonbinaryBucket;
private Cluster cluster;
private static final Logger log = Logger.getLogger(CouchbaseConnectionManager.class);
#PostConstruct
public void createSession() {
if (nodes != null && nodes.size() != 0) {
try {
CouchbaseEnvironment env = DefaultCouchbaseEnvironment.builder().connectTimeout(TIMEOUT).build();
cluster = CouchbaseCluster.create(env, nodes);
binaryBucket = cluster.openBucket(binaryBucketName, password);
nonbinaryBucket = cluster.openBucket(nonbinaryBucketName, password);
log.info(GOT_A_CONNECTION_TO_COUCHBASE_BUCKETS + binaryBucket + " " + nonbinaryBucket);
} catch (Exception e) {
log.warn(UNABLE_TO_GET_CONNECTION_TO_COUCHBASE_BUCKETS);
}
} else {
log.warn(COUCH_NOT_CONFIGURED);
}
}
#PreDestroy
public void preDestroy() {
if (cluster != null) {
cluster.disconnect();
log.info(SUCCESSFULLY_DISCONNECTED_FROM_COUCHBASE);
}
}
public Bucket getBinaryBucket() {
return binaryBucket;
}
public Bucket getNonbinaryBucket() {
return nonbinaryBucket;
}
private static final String SUCCESSFULLY_DISCONNECTED_FROM_COUCHBASE = "Successfully disconnected from couchbase";
private static final String GOT_A_CONNECTION_TO_COUCHBASE_BUCKETS = "Got a connection to couchbase buckets: ";
private static final String COUCH_NOT_CONFIGURED = "COUCH not configured!!";
private static final String UNABLE_TO_GET_CONNECTION_TO_COUCHBASE_BUCKETS = "Unable to get connection to couchbase buckets";
}
I followed Simon's approach and extended the org.springframework.data.couchbase.config.AbstractCouchbaseConfiguration for the #Configuration instead of SpringBootCouchbaseDataConfiguration.
Also, a point worth mentioning is that for some reason having separate Repository packages and having its own #Configuration doesn't really work. I struggled a great deal to try and make it work and eventually settled on having all the Repositories in a single package and ended up having something like the below to map the Entities and Templates.
baseMapping.mapEntity(Prime.class, noSQLSearchDBTemplate())
.mapEntity(PrimeDetailsMaster.class, noSQLSearchDBTemplate())
.mapEntity(HostDetailsMaster.class, noSQLSearchDBTemplate())
.mapEntity(Events.class, eventsTemplate())
.mapEntity(EventRulesMaster.class, eventsTemplate());
Related
I have mounted one volume which contained username and password inside pod. If I do:
kubectl exec -it my-app -- cat /mnt/secrets-store/git-token
{"USERNAME":"usernameofgit","PASSWORD":"dhdhfhehfhel"}
I want to read this USERNAME and PASSWORD using Spring Boot.
Assuming:
the file (git_token) format is fixed (JSON).
the file may not have an extension suffix (.json).
... we have some Problems!
I tried 2.3.5. Importing Extensionless Files like:
spring.config.import=/mnt/secrets-store/git-token[.json]
But it works only with YAML/.properties yet!(tested with spring-boot:2.6.1))
Same applies to 2.8. Type-safe Configuration Properties. ;(;(
In Spring-Boot we can (out-of-the box) provide JSON-config (only) as SPRING_APPLICATION_JSON environment/command line property, and it has to be the json string, and cannot be a path or file (yet).
The proposed (baeldung) article shows ways to "enable JSON properties", but it is a long article with many details, shows much code and has decent lacks/outdates (#Component on #ConfigurationProperties is rather "unconventional")..
I tried the following (on local machine, under the mentioned assumptions):
package com.example.demo;
import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.Data;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
#SpringBootApplication
public class DemoApplication {
public static void main(String[] args) {
SpringApplication.run(DemoApplication.class, args);
}
#Value("""
#{#jacksonObjectMapper.readValue(
T(java.nio.file.Files).newInputStream(
T(java.nio.file.Path).of('/mnt/secrets-store/git-token')),
T(com.example.demo.GitInfo)
)}""" // watch out with #Value and text blocks! (otherwise: No converter found capable of converting from type [com.example.demo.GitInfo] to type [java.lang.String])
)
GitInfo gitInfo;
#Bean
CommandLineRunner runner() {
return (String... args) -> {
System.out.println(gitInfo.getUsername());
System.out.println(gitInfo.getPassword());
};
}
}
#Data
class GitInfo {
#JsonProperty("USERNAME")
private String username;
#JsonProperty("PASSWORD")
private String password;
}
With (only) spring-boot-starter-web and lombok on board, it prints the expected output.
Solution outline:
a pojo for this
the upper case is little problematic, but can be handled as shown.
a (crazy) #Value - (Spring-)Expression, involving:
(hopefully) auto-configured #jacksonObjectMapper bean. (alternatively: custom)
ObjectMapper#readValue (alternatives possible)
java.nio.file.Files#newInputStream (alternatives possible)
java.nio.file.Path#of
When you have your volume mounted, then all you need to do is to read a JSON file from the Spring Boot application. I recommend reading Load Spring Boot Properties From a JSON File.
In short, you can create a class corresponding to your JSON file, something like this one.
#Component
#PropertySource("file:/mnt/secrets-store/git-token")
#ConfigurationProperties
public class GitToken {
private String username;
private String password;
// getters and setters
}
Then, you need to add it to componentScan and you can autowire your class.
Quarkus using Rest Client, explains how to use the MicroProfile REST Client. For Base URL application.properties can be used.
org.acme.restclient.CountriesService/mp-rest/url=https://restcountries.eu/rest #
With above approach, cant have dynamic base URL.
Able to achieve it by using RestClientBuilder as explained in MicroProfile Rest Client. Downside of this approach is not having auto-negotiation capability.
SimpleGetApi simpleGetApi = RestClientBuilder.newBuilder().baseUri(getApplicationUri()).build(SimpleGetApi.class);
Is there other or better way to achieve this? Thanks.
While it is true, that the MP Rest CLient does not allow you to set the BaseUri dynamically when you use declarative/Injected clients, there are some (albeit hacky) ways how to achieve that.
One is to use standard ClientRequestFilter which can modify the URL:
#Provider
#Slf4j
public class Filter implements ClientRequestFilter {
#Inject RequestScopeHelper helper;
#Override
public void filter(ClientRequestContext requestContext) throws IOException {
if (helper.getUrl() != null) {
URI newUri = URI.create(requestContext.getUri().toString().replace("https://originalhost.com", helper.getUrl()));
requestContext.setUri(newUri);
}
}
}
Where RequestScopeHelper is some help class (e.g. request scoped bean) through which you can pass the dynamic url, for example:
#Inject
RequestScopeHelper helper;
#Inject
#RestClient
TestIface myApiClient;
public void callSomeAPIWithDynamicBaseUri(String dynamic) {
helper.setUrl(dynamic);
myApiClient.someMethod();
}
Second is to use MP rest client SPI, namely the RestClientListener which allows you to modify the rest clients after they are built.
For this to work, you have to set the scope of your rest client to RequestScoped so that new instance is created for each request(if you use singleton for example, then the client is only created once and your listener will only be called once). This you can do via quarkus properties:
quarkus.rest-client."com.example.MyRestIface".scope=javax.enterprise.context.RequestScoped
public class MyListener implements RestClientListener {
#Override
public void onNewClient(Class<?> serviceInterface, RestClientBuilder builder) {
String newUri = //obtain dynamic URI from somewhere e.g. again request scope bean lookup, or maybe dynamic config source (create new in-memory ConfigSource, before you invoke your rest client set the corresponding rest client url property to your dynamic value, then inside this listener use ConfigProvider.getConfig().getProperty...)
builder.baseUri(URI.create(newUri));
}
}
Don't forget to register this listener as service provider(META-INF/services/org.eclipse.microprofile.rest.client.spi.RestClientListener)
Another option is to use custom CDI producer that would produce the Rest client instances for you; then you could control all client config yourself. You can use the RestClientBase from Quarkus rest client which is exactly what Quarkus uses under the hood during deployment phase to construct client instances. You will however have to duplicate all the logic related to registration of handlers, interceptors etc.
Do keep in mind, that any of these solutions will make the debugging and problem analysis more challenging - because you will now have multiple places, where the URI is controlled(MP config/quarkus properties, env vars, your custom impl...), so you need to be careful with your approach and maybe add some explicit log messages when you override the URI manually.
MicroProfile REST Client in Quarkus does allow you to use dynamic base URL with that simple "hack" :
Just put an empty String in #Path annotations for you API interface like that :
import javax.ws.rs.GET;
import javax.ws.rs.Path;
#Path("")
public interface SimpleGetApi {
#Path("")
#GET
String callWithDynmamicUrl(); //it can be String or any return type you want
}
After that you are ready to call your dynamic base URL :
import org.eclipse.microprofile.rest.client.RestClientBuilder;
import java.net.URI;
public class Example {
public static void main(String[] args) {
URI anyDynamicUrl = URI.create("http://restcountries.eu/rest/some/dynamic/path");
SimpleGetApi simpleGetApi = RestClientBuilder.newBuilder().baseUri(anyDynamicUrl)
.build(SimpleGetApi.class);
simpleGetApi.callWithDynmamicUrl();
}
}
I'm trying to set up a project with two data sources, one is MongoDB and the other is Postgres. I have repositories for each data source in different packages and I annotated my main class as follows:
#Import({MongoDBConfiguration.class, PostgresDBConfiguration.class})
#SpringBootApplication(exclude = {
MongoRepositoriesAutoConfiguration.class,
JpaRepositoriesAutoConfiguration.class
})
public class TemporaryRunner implements CommandLineRunner {
...
}
MongoDBConfiguration:
#Configuration
#EnableMongoRepositories(basePackages = {
"com.example.datastore.mongo",
"com.atlassian.connect.spring"})
public class MongoDBConfiguration {
...
}
PostgresDBConfiguration:
#Configuration
#EnableJpaRepositories(basePackages = {
"com.example.datastore.postgres"
})
public class PostgresDBConfiguration {
...
}
And even though I specified the base packages as described in documentation, I still get those messages in the console:
13:10:44.238 [main] [] INFO o.s.d.r.c.RepositoryConfigurationDelegate - Multiple Spring Data modules found, entering strict repository configuration mode!
13:10:44.266 [main] [] INFO o.s.d.r.c.RepositoryConfigurationExtensionSupport - Spring Data MongoDB - Could not safely identify store assignment for repository candidate interface com.atlassian.connect.spring.AtlassianHostRepository.
I managed to solve this issue for all my repositories by using MongoRepository and JpaRepository but AtlassianHostRepository comes from an external lib and it is a regular CrudRepository (which totally makes sense because the consumer of the lib can decide what type of DB he would like to use). Anyway it looks that basePackages I specified are completely ignored and not used in any way, even though I specified com.atlassian.connect.spring package only in #EnableMongoRepositories Spring Data somehow can't figure out which data module should be used.
Am I doing something wrong? Is there any other way I could tell spring data to use mongo for AtlassianHostRepository without changing the AtlassianHostRepository.class itself?
The only working solution I found was to let spring data ignore AtlassianHostRepository (because it couldn't figure out which data source to use) then create a separate configuration for it, and simply create it by hand:
#Configuration
#Import({MongoDBConfiguration.class})
public class AtlassianHostRepositoryConfiguration {
private final MongoTemplate mongoTemplate;
#Autowired
public AtlassianHostRepositoryConfiguration(final MongoTemplate mongoTemplate) {
this.mongoTemplate = mongoTemplate;
}
#Bean
public AtlassianHostRepository atlassianHostRepository() {
RepositoryFactorySupport factory = new MongoRepositoryFactory(mongoTemplate);
return factory.getRepository(AtlassianHostRepository.class);
}
}
This solution works fine for a small or limited number of repositories used from a library, it would be rather cumbersome to create all the repositories by hand when there are more of them, but after reading the source code of spring-data I see no way to make it work with basePackages as stated in documentation (I may be wrong though).
I am trying to cache a simple web page in Ehcache. Thanks to some help from another SO post I discovered that I need to implement my own filter based on Ehcache 2 code. When I look at the filter I don't understand it. Where does it ever call the cache to return a value? Here is my implementation (quite possibly wrong):
package com.sentiment360.pulse.cache;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.xml.bind.Element;
import org.ehcache.Cache;
import org.ehcache.CacheManager;
import org.ehcache.config.Configuration;
import static org.ehcache.config.builders.CacheManagerBuilder.newCacheManager;
import org.ehcache.core.Ehcache;
import org.ehcache.event.CacheEvent;
import org.ehcache.event.CacheEventListener;
import org.ehcache.xml.XmlConfiguration;
import javax.servlet.http.HttpServletRequest;
public class SimplePageCachingFilter implements CachingFilter {
public static final String DEFAULT_CACHE_NAME = "SimplePageCachingFilter";
private Logger LOG = Logger.getLogger(this.getClass().getName());
private String cacheName="basicCache";
protected String getCacheName() {
if (cacheName != null && cacheName.length() > 0) {
LOG.log(Level.INFO,"Using configured cacheName of {}.", cacheName);
return cacheName;
} else {
LOG.log(Level.INFO,"No cacheName configured. Using default of {}.", DEFAULT_CACHE_NAME);
return DEFAULT_CACHE_NAME;
}
}
protected CacheManager getCacheManager() {
return CacheManager.getInstance();
}
protected String calculateKey(HttpServletRequest httpRequest) {
StringBuffer stringBuffer = new StringBuffer();
stringBuffer.append(httpRequest.getMethod()).append(httpRequest.getRequestURI()).append(httpRequest.getQueryString());
String key = stringBuffer.toString();
return key;
}
}
See in the super class.
But you do implements CachingFilter ?! Where is that interface? It does look like you were trying to "copy" the previous Ehcache's SimplePageCachingFilter, right? You would also need to port that abstract super class (and maybe read a little about javax.servlet.Filter, in case these aren't entirely clear...)
Now, you may also want to ping the dev team on the Ehcache Dev Google group about this. They should be able to provide pointers and then help with the implementation. Looks like a good idea for a future pull request! :)
I am trying to write a custom writer of elasticsearch which would index data in a spring batch implementation.
I could find the below code as Java config for the elasticsearch.
Anyone who has used this, Can please share where to call this configuration?
#Configuration
#EnableElasticsearchRepositories(basePackages = "org/springframework/data/elasticsearch/repositories")
static class Config {
#Value("${esearch.port}") int port;
#Value("${esearch.host}") String hostname;
#Bean
public ElasticsearchOperations elasticsearchTemplate() {
return new ElasticsearchTemplate(client());
}
#Bean
public Client client(){
TransportClient client= new TransportClient();
TransportAddress address = new InetSocketTransportAddress(hostname, port);
client.addTransportAddress(address);
return client;
}
}
The code that you listed above is basically your implementation details of a - Transport Client element pointing to an instance of Elasticsearch Server i.e. this defines your persistent layer by using Spring Data.
This code will be used by your elasticsearch repositories i.e. repositories that you define by extending - ElasticsearchRepository from Spring Data.
You need to edit #EnableElasticsearchRepositories in code listed by you to actually point to package where you are keeping your repository definitions - no other call would be needed.
When you are going to write/index data to elasticsearch, you work with ElasticsearchRepository interface and you need to define your own repositories and these repositories work with instances as per listing in your code.
Hope it helps !!