I am trying to use spring cache abstraction with Redis cache. I am unable to see the values in cache. Please help me if I am missing something in config :
As I am making the call multiple times actual fetch is happening. I tried connecting to same redis host port, I cant find there any keys as well.
PFB the implementation details.
CacheUtils.java :
#Slf4j
public class CacheUtils {
private final CustomerManagementClient customerManagementClient;
#Autowired
public CacheUtils(CustomerManagementClient customerManagementClient) {
this.customerManagementClient = customerManagementClient;
}
#Cacheable(value = "merchant-details", key = "#merchantEntityId")
public MerchantDetails getOrFetchMerchantDetails(OrderItemStatusChangeEvent event, MerchantType merchantType, String merchantEntityId) {
if (BUYER == merchantType) {
log.info("test - get buyer details");
CustomerDetails customerDetails =
customerManagementClient.getData(merchantEntityId);
String businessId = customerDetails.getBusinessId();
String phoneNumber = customerDetails.getPhoneNumber();
return MerchantDetails
.builder()
.merchantEntityId(merchantEntityId)
.businessId(businessId)
.businessName(customerDetails.getBusinessName())
.merchantType(merchantType)
.contactNumber(phoneNumber)
.build();
}
throw new InvalidInputException();
}
}
MainClass.java
#Slf4j
#Component
public class MainClass implements LogisticsPlanningService {
private final CacheUtils cacheUtils;
#Autowired
public LogisticsPlanningServiceImpl(CacheUtils cacheUtils) {
this.cacheUtils = cacheUtils;
}
private Set<LogisticsPlanningRequest> testMethod(Event event) {
MerchantDetails senderDetails = cacheUtils.getOrFetchMerchantDetails(event, SELLER, orderItem.getSellerId());
MerchantDetails receiverDetails = cacheUtils.getOrFetchMerchantDetails(event, BUYER, orderItem.getBuyerId());
}
}
RedisConfiguration.java
#Configuration
#EnableCaching
public class RedisConfiguration {
private String hostName;
private int port;
#Autowired
MarketPlaceServiceProperties properties;
#PostConstruct
public void init() {
hostName = properties.getRedisHostName();
port = Integer.parseInt(properties.getRedisPort());
}
#Bean
protected JedisConnectionFactory jedisConnectionFactory() {
RedisStandaloneConfiguration configuration = new RedisStandaloneConfiguration(hostName, port);
JedisConnectionFactory factory = new JedisConnectionFactory(configuration);
factory.afterPropertiesSet();
return factory;
}
public RedisCacheConfiguration getTestCacheConfig() {
RedisCacheConfiguration cacheConfiguration = RedisCacheConfiguration.defaultCacheConfig();
cacheConfiguration.prefixCacheNameWith("marketplace");
cacheConfiguration.disableCachingNullValues();
return cacheConfiguration;
}
// #Bean
// public RedisTemplate<String, Object> redisTemplate() {
// final RedisTemplate<String, Object> redisTemplate = new RedisTemplate<String, Object>();
// redisTemplate.setKeySerializer(new StringRedisSerializer());
// redisTemplate.setHashKeySerializer(new GenericToStringSerializer<>(Object.class));
// redisTemplate.setHashValueSerializer(new JdkSerializationRedisSerializer());
// redisTemplate.setValueSerializer(new JdkSerializationRedisSerializer());
// redisTemplate.setConnectionFactory(jedisConnectionFactory());
// return redisTemplate;
// }
}
service.properties :
redisHostName: redis.domain.prod.xyz.com
redisPort: 5400
I have implemented Redis session management using LettuceConnectionFactory on my Spring Boot java application. I couldn't figure out a way to capture Redis connection failure. Is there a way to capture the connection failure?
Spring Boot version: 2.2.6
Code:
#Configuration
#PropertySource("classpath:application.properties")
#EnableRedisHttpSession
public class HttpSessionConfig extends AbstractHttpSessionApplicationInitializer {
Logger logger = LoggerFactory.getLogger(HttpSessionConfig.class);
#Value("${spring.redis.host}")
private String redisHostName;
#Value("${spring.redis.port}")
private int redisPort;
#Value("${spring.redis.password}")
private String redisPassword;
private #Value("${spring.redis.custom.command.timeout}")
Duration redisCommandTimeout;
private #Value("${spring.redis.timeout}")
Duration socketTimeout;
#Bean
LettuceConnectionFactory lettuceConnectionFactory() {
final SocketOptions socketOptions = SocketOptions.builder().connectTimeout(socketTimeout).build();
final ClientOptions clientOptions = ClientOptions.builder()
.socketOptions(socketOptions)
.build();
LettuceClientConfiguration clientConfig = LettuceClientConfiguration.builder()
.commandTimeout(redisCommandTimeout)
.clientOptions(clientOptions)
.readFrom(ReadFrom.SLAVE_PREFERRED)
.build();
RedisStandaloneConfiguration serverConfig = new RedisStandaloneConfiguration(redisHostName,
redisPort);
final LettuceConnectionFactory lettuceConnectionFactory = new LettuceConnectionFactory(serverConfig,
clientConfig);
return lettuceConnectionFactory;
}
#Bean
public RedisTemplate<Object, Object> sessionRedisTemplate() {
final RedisTemplate<Object, Object> template = new RedisTemplate<>();
template.setConnectionFactory(lettuceConnectionFactory());
return template;
}
#Bean
public ConfigureRedisAction configureRedisAction() {
return ConfigureRedisAction.NO_OP;
}
}
I want to set values Spring SolrDocument Collection based on application.yml value.
#Data
#SolrDocument(collection = #Value("${solr.core}"))
public class SearchableProduct {
}
Hoi Michela,
Ok, I had the same Problem and I found a solution: SpEL
it is described in details here:Spring Data for Apache Solr
you have to add the EL-expression to the Annotation
#SolrDocument(collection = "#{#serverSolrContext.getCollectionName()}")
public class SOLREntity implements Serializable {
.....
}
you have to provide a the serverSolrContext Bean with the method getCollectionName().
#Value("${solr.core}")
private String core;
public String getCollectionName() {
return core;
}
you have to write in our application.properties the following core entry.
solr.core=myOwnCoreName
That's it actually, BUT
if you get the following Exception, so as I did:
org.springframework.expression.spel.SpelEvaluationException: EL1057E: No bean resolver registered in the context to resolve access to bean
You have to have the following in your Configuration Bean
#Configuration
#EnableSolrRepositories(basePackages = { "de.solr.db" })
#Profile("default")
#PropertySource("classpath:application.properties")
public class ServerSolrContext extends AbstractSolrConfiguration {
private static final Logger logger = LoggerFactory.getLogger(ServerSolrContext.class);
#Resource
private Environment environment;
#Value("${solr.core}")
private String core;
public String getCollectionName() {
return core;
}
#PostConstruct
public void init() {
System.out.println(core);
}
#Bean
public SolrClient solrClient() {
String url = environment.getProperty("solr.server.url");
String user = environment.getProperty("solr.server.user");
String password = environment.getProperty("solr.server.password");
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT),
new UsernamePasswordCredentials(user, password));
SSLContext sslContext = null;
try {
sslContext = ReportConfiguration.getTrustAllContext();
}
catch (KeyManagementException | NoSuchAlgorithmException e) {
e.printStackTrace();
}
LayeredConnectionSocketFactory sslSocketFactory = new SSLConnectionSocketFactory(sslContext);
HttpClient httpClient = HttpClientBuilder.create().setSSLSocketFactory(sslSocketFactory)
.addInterceptorFirst(new PreemptiveAuthInterceptor()).setDefaultCredentialsProvider(credentialsProvider)
.build();
SolrClient client = new HttpSolrClient.Builder().withHttpClient(httpClient).withBaseSolrUrl(url).build();
return client;
}
#Bean
#ConditionalOnMissingBean(name = "solrTemplate")
public SolrTemplate solrTemplate(#Qualifier("mySolrTemplate") SolrTemplate solrTemplate) {
return solrTemplate;
}
#Bean("mySolrTemplate")
public SolrTemplate mySolrTemplate(SolrClient solrClient, SolrConverter solrConverter) {
return new SolrTemplate(new HttpSolrClientFactory(solrClient), solrConverter);
}
#Override
public SolrClientFactory solrClientFactory() {
return new HttpSolrClientFactory(solrClient());
}
}
The last 3 Methods are doing the Trick, that cost me a while to find the right solution:
it is here, so actually I was lucky to find this:
Allow PropertyPlaceholders in #SolrDocument solrCoreName
I have looked everywhere for this and it seems like I cannot find a solution that works. I am using spring boot 1.5.10-RELEASE. I am trying to configure two different mongodb instances in the same application. Here is my code:
Main Application:
#SpringBootApplication(exclude = {MongoAutoConfiguration.class})
#ComponentScan("com.reef.reports")
public class MainApplication
{
public static void main(String[] args)
{
SpringApplication.run(MainApplication.class, args);
}
}
1st Instance
#Configuration
#EnableMongoRepositories(basePackages = {"com.reef.repository.mongousa"} , mongoTemplateRef = "USAMongo")
public class MongoUsaConfig
{
#Value("${usa.mongodb.host}")
private String host;
#Value("${usa.mongodb.database:reef}")
private String database;
#Value("${usa.mongodb.port:27017}")
private int port;
#Value("${usa.mongodb.username:}")
private String username;
#Value("${usa.mongodb.password:}")
private String password;
#Value("${usa.mongodb.authdb:}")
private String authdb;
private final List<MongoCredential> credentials = new ArrayList<>();
private final List<ServerAddress> hosts = new ArrayList<>();
/**
* Method that creates MongoDbFactory
* Common to both of the MongoDb connections
*/
public MongoDbFactory mongoDbFactory()
{
return new SimpleMongoDbFactory(getMongoClient(), database);
}
/**
* Method that creates MongoClient
*/
#Bean(name = "USAClient")
public MongoClient getMongoClient()
{
if ((null != username)&&(!username.isEmpty()))
{
hosts.add(new ServerAddress(host, port));
credentials.add(MongoCredential.createMongoCRCredential(username, authdb, password.toCharArray()));
return new MongoClient(hosts, credentials);
}
else
{
return new MongoClient(host, port);
}
}
#Primary
#Bean(name = "USAMongo")
public MongoTemplate getMongoTemplate()
{
return new MongoTemplate(mongoDbFactory());
}
}
2nd Instance
#Configuration
#EnableMongoRepositories(basePackages = {"com.reef.repository.mongocan"} , mongoTemplateRef = "CANMongo")
public class MongoCanConfig
{
#Value("${can.mongodb.host}")
private String host;
#Value("${can.mongodb.database:reef}")
private String database;
#Value("${can.mongodb.port:27017}")
private int port;
#Value("${can.mongodb.username:}")
private String username;
#Value("${can.mongodb.password:}")
private String password;
#Value("${can.mongodb.authdb:}")
private String authdb;
private final List<MongoCredential> credentials = new ArrayList<>();
private final List<ServerAddress> hosts = new ArrayList<>();
/**
* Method that creates MongoDbFactory
* Common to both of the MongoDb connections
*/
public MongoDbFactory mongoDbFactory()
{
return new SimpleMongoDbFactory(getMongoClient(), database);
}
/**
* Method that creates MongoClient
*/
#Bean(name = "CANClient")
public MongoClient getMongoClient()
{
if ((null != username)&&(!username.isEmpty()))
{
hosts.add(new ServerAddress(host, port));
credentials.add(MongoCredential.createMongoCRCredential(username, authdb, password.toCharArray()));
return new MongoClient(hosts, credentials);
}
else
{
return new MongoClient(host, port);
}
}
#Bean(name = "CANMongo")
public MongoTemplate getMongoTemplate()
{
return new MongoTemplate(mongoDbFactory());
}
}
When I run the application, it will run the configuration for the first instance. However, it will not pick up the second instance. I have put in breakpoints to debug and it never hits the breakpoint in the configuration. The repositories in this package get loaded correctly:
com.reef.repository.mongousa
The errors happen with the repositories in this package:
com.reef.repository.mongocan
Please let me know what I am missing. Why does one config work and the other does not?
Refer to this blog post https://medium.com/#joeclever/using-multiple-datasources-with-spring-boot-and-spring-data-6430b00c02e7
I tried it out for two MySQL DataBases and it is working fine.
Can anybody provide me with a code sample to access the rest service URL secured with HTTPS using the Spring Rest template?
I have the certificate, username and password. Basic Authentication is used on the server-side and I want to create a client that can connect to that server using a provided certificate, username and password (if needed).
KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
keyStore.load(new FileInputStream(new File(keyStoreFile)),
keyStorePassword.toCharArray());
SSLConnectionSocketFactory socketFactory = new SSLConnectionSocketFactory(
new SSLContextBuilder()
.loadTrustMaterial(null, new TrustSelfSignedStrategy())
.loadKeyMaterial(keyStore, keyStorePassword.toCharArray())
.build(),
NoopHostnameVerifier.INSTANCE);
HttpClient httpClient = HttpClients.custom().setSSLSocketFactory(
socketFactory).build();
ClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory(
httpClient);
RestTemplate restTemplate = new RestTemplate(requestFactory);
MyRecord record = restTemplate.getForObject(uri, MyRecord.class);
LOG.debug(record.toString());
Here is some code that will give you the general idea.
You need to create a custom ClientHttpRequestFactory in order to trust the certificate.
It looks like this:
final ClientHttpRequestFactory clientHttpRequestFactory =
new MyCustomClientHttpRequestFactory(org.apache.http.conn.ssl.SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER, serverInfo);
restTemplate.setRequestFactory(clientHttpRequestFactory);
This is the implementation for MyCustomClientHttpRequestFactory:
public class MyCustomClientHttpRequestFactory extends SimpleClientHttpRequestFactory {
private final HostnameVerifier hostNameVerifier;
private final ServerInfo serverInfo;
public MyCustomClientHttpRequestFactory (final HostnameVerifier hostNameVerifier,
final ServerInfo serverInfo) {
this.hostNameVerifier = hostNameVerifier;
this.serverInfo = serverInfo;
}
#Override
protected void prepareConnection(final HttpURLConnection connection, final String httpMethod)
throws IOException {
if (connection instanceof HttpsURLConnection) {
((HttpsURLConnection) connection).setHostnameVerifier(hostNameVerifier);
((HttpsURLConnection) connection).setSSLSocketFactory(initSSLContext()
.getSocketFactory());
}
super.prepareConnection(connection, httpMethod);
}
private SSLContext initSSLContext() {
try {
System.setProperty("https.protocols", "TLSv1");
// Set ssl trust manager. Verify against our server thumbprint
final SSLContext ctx = SSLContext.getInstance("TLSv1");
final SslThumbprintVerifier verifier = new SslThumbprintVerifier(serverInfo);
final ThumbprintTrustManager thumbPrintTrustManager =
new ThumbprintTrustManager(null, verifier);
ctx.init(null, new TrustManager[] { thumbPrintTrustManager }, null);
return ctx;
} catch (final Exception ex) {
LOGGER.error(
"An exception was thrown while trying to initialize HTTP security manager.", ex);
return null;
}
}
In this case my serverInfo object contains the thumbprint of the server.
You need to implement the TrustManager interface to get
the SslThumbprintVerifier or any other method you want to verify your certificate (you can also decide to also always return true).
The value org.apache.http.conn.ssl.SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER allows all host names.
If you need to verify the host name,
you will need to implement it differently.
I'm not sure about the user and password and how you implemented it.
Often,
you need to add a header to the restTemplate named Authorization
with a value that looks like this: Base: <encoded user+password>.
The user+password must be Base64 encoded.
This is a solution with no deprecated class or method :
(Java 8 approved)
CloseableHttpClient httpClient = HttpClients.custom().setSSLHostnameVerifier(new NoopHostnameVerifier()).build();
HttpComponentsClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory();
requestFactory.setHttpClient(httpClient);
RestTemplate restTemplate = new RestTemplate(requestFactory);
Important information : Using NoopHostnameVerifier is a security risk
One point from me. I used a mutual cert authentication with spring-boot microservices. The following is working for me, key points here are
keyManagerFactory.init(...) and sslcontext.init(keyManagerFactory.getKeyManagers(), null, new SecureRandom()) lines of code without them, at least for me, things did not work. Certificates are packaged by PKCS12.
#Value("${server.ssl.key-store-password}")
private String keyStorePassword;
#Value("${server.ssl.key-store-type}")
private String keyStoreType;
#Value("${server.ssl.key-store}")
private Resource resource;
private RestTemplate getRestTemplate() throws Exception {
return new RestTemplate(clientHttpRequestFactory());
}
private ClientHttpRequestFactory clientHttpRequestFactory() throws Exception {
return new HttpComponentsClientHttpRequestFactory(httpClient());
}
private HttpClient httpClient() throws Exception {
KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance("SunX509");
KeyStore trustStore = KeyStore.getInstance(keyStoreType);
if (resource.exists()) {
InputStream inputStream = resource.getInputStream();
try {
if (inputStream != null) {
trustStore.load(inputStream, keyStorePassword.toCharArray());
keyManagerFactory.init(trustStore, keyStorePassword.toCharArray());
}
} finally {
if (inputStream != null) {
inputStream.close();
}
}
} else {
throw new RuntimeException("Cannot find resource: " + resource.getFilename());
}
SSLContext sslcontext = SSLContexts.custom().loadTrustMaterial(trustStore, new TrustSelfSignedStrategy()).build();
sslcontext.init(keyManagerFactory.getKeyManagers(), null, new SecureRandom());
SSLConnectionSocketFactory sslConnectionSocketFactory =
new SSLConnectionSocketFactory(sslcontext, new String[]{"TLSv1.2"}, null, getDefaultHostnameVerifier());
return HttpClients.custom().setSSLSocketFactory(sslConnectionSocketFactory).build();
}
Here is what I ended up with for the similar problem. The idea is the same as in #Avi's answer, but I also wanted to avoid the static "System.setProperty("https.protocols", "TLSv1");", so that any adjustments won't affect the system. Inspired by an answer from here http://www.coderanch.com/t/637177/Security/Disabling-handshake-message-Java
public class MyCustomClientHttpRequestFactory extends SimpleClientHttpRequestFactory {
#Override
protected void prepareConnection(HttpURLConnection connection, String httpMethod) {
try {
if (!(connection instanceof HttpsURLConnection)) {
throw new RuntimeException("An instance of HttpsURLConnection is expected");
}
HttpsURLConnection httpsConnection = (HttpsURLConnection) connection;
TrustManager[] trustAllCerts = new TrustManager[]{
new X509TrustManager() {
public java.security.cert.X509Certificate[] getAcceptedIssuers() {
return null;
}
public void checkClientTrusted(X509Certificate[] certs, String authType) {
}
public void checkServerTrusted(X509Certificate[] certs, String authType) {
}
}
};
SSLContext sslContext = SSLContext.getInstance("SSL");
sslContext.init(null, trustAllCerts, new java.security.SecureRandom());
httpsConnection.setSSLSocketFactory(new MyCustomSSLSocketFactory(sslContext.getSocketFactory()));
httpsConnection.setHostnameVerifier((hostname, session) -> true);
super.prepareConnection(httpsConnection, httpMethod);
} catch (Exception e) {
throw Throwables.propagate(e);
}
}
/**
* We need to invoke sslSocket.setEnabledProtocols(new String[] {"SSLv3"});
* see http://www.oracle.com/technetwork/java/javase/documentation/cve-2014-3566-2342133.html (Java 8 section)
*/
private static class MyCustomSSLSocketFactory extends SSLSocketFactory {
private final SSLSocketFactory delegate;
public MyCustomSSLSocketFactory(SSLSocketFactory delegate) {
this.delegate = delegate;
}
#Override
public String[] getDefaultCipherSuites() {
return delegate.getDefaultCipherSuites();
}
#Override
public String[] getSupportedCipherSuites() {
return delegate.getSupportedCipherSuites();
}
#Override
public Socket createSocket(final Socket socket, final String host, final int port, final boolean autoClose) throws IOException {
final Socket underlyingSocket = delegate.createSocket(socket, host, port, autoClose);
return overrideProtocol(underlyingSocket);
}
#Override
public Socket createSocket(final String host, final int port) throws IOException {
final Socket underlyingSocket = delegate.createSocket(host, port);
return overrideProtocol(underlyingSocket);
}
#Override
public Socket createSocket(final String host, final int port, final InetAddress localAddress, final int localPort) throws IOException {
final Socket underlyingSocket = delegate.createSocket(host, port, localAddress, localPort);
return overrideProtocol(underlyingSocket);
}
#Override
public Socket createSocket(final InetAddress host, final int port) throws IOException {
final Socket underlyingSocket = delegate.createSocket(host, port);
return overrideProtocol(underlyingSocket);
}
#Override
public Socket createSocket(final InetAddress host, final int port, final InetAddress localAddress, final int localPort) throws IOException {
final Socket underlyingSocket = delegate.createSocket(host, port, localAddress, localPort);
return overrideProtocol(underlyingSocket);
}
private Socket overrideProtocol(final Socket socket) {
if (!(socket instanceof SSLSocket)) {
throw new RuntimeException("An instance of SSLSocket is expected");
}
((SSLSocket) socket).setEnabledProtocols(new String[] {"SSLv3"});
return socket;
}
}
}
You need to configure a raw HttpClient with SSL support, something like this:
#Test
public void givenAcceptingAllCertificatesUsing4_4_whenUsingRestTemplate_thenCorrect()
throws ClientProtocolException, IOException {
CloseableHttpClient httpClient
= HttpClients.custom()
.setSSLHostnameVerifier(new NoopHostnameVerifier())
.build();
HttpComponentsClientHttpRequestFactory requestFactory
= new HttpComponentsClientHttpRequestFactory();
requestFactory.setHttpClient(httpClient);
ResponseEntity<String> response
= new RestTemplate(requestFactory).exchange(
urlOverHttps, HttpMethod.GET, null, String.class);
assertThat(response.getStatusCode().value(), equalTo(200));
}
from: Baeldung