Spring data elasticsearch - query - elasticsearch

I'm new to elasticsearch, trying to retrieve indexed data from elasticsearch by using query,date histogram,facets. I have elasticsearch and kibana running properly on server. Now I want to pull the specific indexed data out of elasticsearch and plot it as graphs in another home grown application(Spring web application). So thought of using spring data elasticsearch but found sample applications using elasticsearch repositories over internet.
https://github.com/BioMedCentralLtd/spring-data-elasticsearch-sample-application
Please assist me the way to just pull the data out of elasticsearch using spring data elasticsearch or if there any other better way to do this. (I don't want to use the objects/repositories as in sample, just need to get the data as JSON string).

Finally I have used plain Elasticseach java client to work with. The below code may be useful.
<bean id="esConnection" class="com.es.connection.ESConnection" scope="singleton" autowire="byName">
<property name="host" value="${es.host}" />
<property name="port" value="${es.port}" />
<property name="clusterName" value="${es.cluster}" />
</bean>
import javax.annotation.PostConstruct;
import org.elasticsearch.client.Client;
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.settings.ImmutableSettings;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.transport.InetSocketTransportAddress;
public class ESConnection {
TransportClient client;
private String host;
private int port;
private String clusterName;
public ESConnection() {
}
public ESConnection(String host,int port,String clusterName) {
this.host = host;
this.clusterName = clusterName;
this.port = port;
}
#PostConstruct
public void connect() {
Settings settings = ImmutableSettings.settingsBuilder()
.put("cluster.name",clusterName)
.build();
client = new TransportClient(settings);
client.addTransportAddress(new InetSocketTransportAddress(host,port));
}
public void setHost(String host) {
this.host = host;
}
public void setPort(int port) {
this.port = port;
}
public void setClusterName(String clusterName) {
this.clusterName = clusterName;
}
public Client getClient() {
return (Client) client;
}
public void close() {
if (client != null) {
client.close();
}
}
#Override
public String toString() {
return String.format("%s, Host: %s, Port: %s, Cluster: %s", super.toString(), host, port, clusterName);
}
}
In Start up listener,
public class StartupListener implements ServletContextListener {
#Autowired
ESConnection esConnection;
public void contextInitialized(ServletContextEvent sce) {
try {
ServletContext context = sce.getServletContext();
context.setAttribute("esConnection", esConnection);
} catch (SchedulerException se) {
} catch (Exception e) {
}
}
public void contextDestroyed(ServletContextEvent sce) {
ServletContext context = sce.getServletContext();
if (this.esConnection != null) {
this.esConnection.close();
context.removeAttribute("esConnection");
}
}
}

Related

How to ByPass "No subject alternative names present" in Spring Integration?

I have a project which uses Spring Integration communicates over TCP with my client. My client requests me to use SSL at this TCP communication. When I tried a TCP connection over SSL, I got No subject alternative names present error.
I added this code block to bypass that check:
import org.springframework.integration.ip.tcp.connection.DefaultTcpSSLContextSupport;
import javax.net.ssl.*;
import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;
import java.security.cert.X509Certificate;
public class CustomTCPSSLContextSupport extends DefaultTcpSSLContextSupport {
public CustomTCPSSLContextSupport(String keyStore, String trustStore, String keyStorePassword, String trustStorePassword) {
super(keyStore, trustStore, keyStorePassword, trustStorePassword);
}
#Override
public SSLContext getSSLContext() throws NoSuchAlgorithmException, KeyManagementException {
TrustManager[] trustAllCerts = new TrustManager[]{new X509TrustManager() {
public java.security.cert.X509Certificate[] getAcceptedIssuers() {
return null;
}
public void checkClientTrusted(X509Certificate[] certs, String authType) {
}
public void checkServerTrusted(X509Certificate[] certs, String authType) {
}
}
};
SSLContext sc = SSLContext.getInstance("SSL");
sc.init(null, trustAllCerts, new java.security.SecureRandom());
return sc;
}
}
I defined it like this:
<bean id="sslContextSupport" class="CustomTCPSSLContextSupport">
<constructor-arg value="${keystore.path}"/>
<constructor-arg value="${cacerts.path}"/>
<constructor-arg value="changeit"/>
<constructor-arg value="changeit"/>
</bean>
I used this sslContextSupport as ssl-context-support in my tcp-connection-factory but no affect. It still gives same error.
How can I bypass subject alternative check completely in Spring Integration?
It would be great to see the whole stack trace to be sure that your CustomTCPSSLContextSupport is really in use.
Although I see that typically they implement "trust all" like this:
static class TrustAllX509TrustManager implements X509TrustManager {
TrustAllX509TrustManager() {
}
public void checkClientTrusted(X509Certificate[] x509Certificates, String s) {
}
public void checkServerTrusted(X509Certificate[] x509Certificates, String s) {
}
public X509Certificate[] getAcceptedIssuers() {
return new X509Certificate[0];
}
}
An empty array for new X509Certificate[0], not null like in your case.
The sc.init() looks OK to me.

How to configure spring integration adapters of a merely connecting client and a server sending messages

I'm trying to implement the following scenario using Spring Integration:
I need a client to connect to a server via TCP IP and wait to receive messages within 30 seconds.
I need a server to send 0 to n messages to the client which had connected.
I need a way to start and stop channel transfer without loss of messages.
I need to change the port the server is listening between stop and start.
This is my config so far:
#Configuration
public class TcpConfiguration {
private static Logger LOG = LoggerFactory.getLogger(TcpConfiguration.class);
#Value("${port}")
private Integer port;
#Value("${so-timeout}")
private Integer soTimeout;
#Value("${keep-alive}")
private Boolean keepAlive;
#Value("${send-timeout}")
private Integer sendTimeout;
#Bean
public AbstractServerConnectionFactory getMyConnFactory() {
LOG.debug("getMyConnFactory");
TcpNetServerConnectionFactory factory = new TcpNetServerConnectionFactory(port);
LOG.debug("getMyConnFactory port={}", port);
factory.setSoTimeout(soTimeout);
LOG.debug("getMyConnFactory soTimeout={}", soTimeout);
factory.setSoKeepAlive(true);
LOG.debug("getMyConnFactory keepAlive={}", keepAlive);
return factory;
}
#Bean
public AbstractEndpoint getMyChannelAdapter() {
LOG.debug("getMyChannelAdapter");
TcpReceivingChannelAdapter adapter = new TcpReceivingChannelAdapter();
adapter.setConnectionFactory(getMyConnFactory());
adapter.setOutputChannel(myChannelIn());
adapter.setSendTimeout(sendTimeout);
LOG.debug("getMyChannelAdapter adapter={}", adapter.getClass().getName());
return adapter;
}
#Bean
public MessageChannel myChannelIn() {
LOG.debug("myChannelIn");
return new DirectChannel();
}
#Bean
#Transformer(inputChannel = "myChannelIn", outputChannel = "myServiceChannel")
public ObjectToStringTransformer myTransformer() {
LOG.debug("myTransformer");
return new ObjectToStringTransformer();
}
#ServiceActivator(inputChannel = "myServiceChannel")
public void service(String in) {
LOG.debug("service received={}", in);
}
#Bean
public MessageChannel myChannelOut() {
LOG.debug("myChannelOut");
return new DirectChannel();
}
#Bean
public IntegrationFlow myOutbound() {
LOG.debug("myOutbound");
return IntegrationFlows.from(myChannelOut())
.handle(mySender())
.get();
}
#Bean
public MessageHandler mySender() {
LOG.debug("mySender");
TcpSendingMessageHandler tcpSendingMessageHandler = new TcpSendingMessageHandler();
tcpSendingMessageHandler.setConnectionFactory(getMyConnFactory());
return tcpSendingMessageHandler;
}
}
Please advice!
To change the server port I would shutdown the application context and restart it after configuring the new port in a remote configuration server.
Can I just close the application context without corrupting the current message transfer?
I don't know how to handle the connect-only client thing.
Use dynamic flow registration; just get the connection to open it without sending.
#SpringBootApplication
public class So62867670Application {
public static void main(String[] args) {
SpringApplication.run(So62867670Application.class, args);
}
#Bean
public ApplicationRunner runner(DynamicTcpReceiver receiver) {
return args -> { // Just a demo to show starting/stopping
receiver.connectAndListen(1234);
System.in.read();
receiver.stop();
System.in.read();
receiver.connectAndListen(1235);
System.in.read();
receiver.stop();
};
}
}
#Component
class DynamicTcpReceiver {
#Autowired
private IntegrationFlowContext context;
private IntegrationFlowRegistration registration;
public void connectAndListen(int port) throws InterruptedException {
TcpClientConnectionFactorySpec client = Tcp.netClient("localhost", port)
.deserializer(TcpCodecs.lf());
IntegrationFlow flow = IntegrationFlows.from(Tcp.inboundAdapter(client))
.transform(Transformers.objectToString())
.handle(System.out::println)
.get();
this.registration = context.registration(flow).register();
client.get().getConnection(); // just open the single shared connection
}
public void stop() {
if (this.registration != null) {
this.registration.destroy();
this.registration = null;
}
}
}
EDIT
And this is the server side...
#SpringBootApplication
#EnableScheduling
public class So62867670ServerApplication {
public static void main(String[] args) {
SpringApplication.run(So62867670ServerApplication.class, args);
}
#Bean
public ApplicationRunner runner(DynamicTcpServer receiver) {
return args -> { // Just a demo to show starting/stopping
receiver.tcpListen(1234);
System.in.read();
receiver.stop(1234);
System.in.read();
receiver.tcpListen(1235);
System.in.read();
receiver.stop(1235);
};
}
}
#Component
class DynamicTcpServer {
private static final Logger LOG = LoggerFactory.getLogger(DynamicTcpServer.class);
#Autowired
private IntegrationFlowContext flowContext;
#Autowired
private ApplicationContext appContext;
private final Map<Integer, IntegrationFlowRegistration> registrations = new HashMap<>();
private final Map<String, Entry<Integer, AtomicInteger>> clients = new ConcurrentHashMap<>();
public void tcpListen(int port) {
TcpServerConnectionFactorySpec server = Tcp.netServer(port)
.id("server-" + port)
.serializer(TcpCodecs.lf());
server.get().registerListener(msg -> false); // dummy listener so the accept thread doesn't exit
IntegrationFlow flow = f -> f.handle(Tcp.outboundAdapter(server));
this.registrations.put(port, flowContext.registration(flow).register());
}
public void stop(int port) {
IntegrationFlowRegistration registration = this.registrations.remove(port);
if (registration != null) {
registration.destroy();
}
}
#EventListener
public void closed(TcpConnectionOpenEvent event) {
LOG.info(event.toString());
String connectionId = event.getConnectionId();
String[] split = connectionId.split(":");
int port = Integer.parseInt(split[2]);
this.clients.put(connectionId, new AbstractMap.SimpleEntry<>(port, new AtomicInteger()));
}
#EventListener
public void closed(TcpConnectionCloseEvent event) {
LOG.info(event.toString());
this.clients.remove(event.getConnectionId());
}
#EventListener
public void listening(TcpConnectionServerListeningEvent event) {
LOG.info(event.toString());
}
#Scheduled(fixedDelay = 5000)
public void sender() {
this.clients.forEach((connectionId, portAndCount) -> {
IntegrationFlowRegistration registration = this.registrations.get(portAndCount.getKey());
if (registration != null) {
LOG.info("Sending to " + connectionId);
registration.getMessagingTemplate().send(MessageBuilder.withPayload("foo")
.setHeader(IpHeaders.CONNECTION_ID, connectionId).build());
if (portAndCount.getValue().incrementAndGet() > 9) {
this.appContext.getBean("server-" + portAndCount.getKey(), TcpNetServerConnectionFactory.class)
.closeConnection(connectionId);
}
}
});
}
}

How to access 2 Elasticsearch Instance through RestHighLevelClient in SpringBoot

I trying to access 2 Elasticsearch Server instance through HighLevelRestClient but I couldn't through this array object
HttpHost[] httpHost = new HttpHost(hostName[i...], Integer.parseInt(hostName[i..]), "http");
In hostname i have 2 values in the restHighLevelClient = new RestHighLevelClient( RestClient.builder(httpHost));
I'm also unable to access through second array instance.
Can I have 2 configuration Class if such way a how to create 2 instance of HighLevelRestClient
Or is it any possible way through 2 bean instance if such a way how it is possible
Since we need to have 2 different restHighLevelClient instance.
Kindly let me know in case of more information needed.
Code
import org.apache.http.HttpHost;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.beans.factory.config.AbstractFactoryBean;
import org.springframework.context.annotation.Configuration;
#Configuration
public class AppElasticSearchConfiguration extends AbstractFactoryBean<RestHighLevelClient> {
private static final Logger LOG = LoggerFactory.getLogger(AppElasticSearchConfiguration.class);
#Value("${application.elasticsearch.host}")
private String hostName[];
private RestHighLevelClient restHighLevelClient;
#Override
public void destroy() {
try {
if (restHighLevelClient != null) {
restHighLevelClient.close();
}
} catch (final Exception e) {
LOG.error("Error closing ElasticSearch client: ", e);
}
}
#Override
public Class<RestHighLevelClient> getObjectType() {
return RestHighLevelClient.class;
}
#Override
public boolean isSingleton() {
return false;
}
#Override
public RestHighLevelClient createInstance() {
return buildClient();
}
private RestHighLevelClient buildClient() {
try {
HttpHost[] httpHost = null;
if(hostName!=null) {
httpHost = new HttpHost[hostName.length];
for (int i = 0; i < httpHost.length; i++) {
httpHost[i] = new HttpHost(hostName[i].split(":")[0],
Integer.parseInt(hostName[i].split(":")[1]), "http");
}
}
restHighLevelClient = new RestHighLevelClient( RestClient.builder(httpHost));
} catch (Exception e) {
LOG.error(e.getMessage());
}
return restHighLevelClient;
}
//public RestHighLevelClient getAppRestHighLevelClient() { return restHighLevelClient; }
}
Hi Just pass the secondary instance in the constructor of HttpHost.
#Bean(destroyMethod = "close")
public RestHighLevelClient buildClient() {
RestClientBuilder builder = RestClient.builder(new HttpHost(hostPrimary, Integer.valueOf(portPrimary), "http"),
new HttpHost(hostSecondary, Integer.valueOf(portSecondary), "http"));
RestHighLevelClient client = new RestHighLevelClient(builder);
LOG.info("RestHighLevelClient has been created with:{}", client);
return client;
}

Jetty Websocket Compilation Errors

I am trying to do an Jetty Web Socket example .
I copied a example from internet , which was working fine when i deployed directly into server without making any chnages .
But when i copied the Source (the servlet) into Eclipse IDE , it was giving Compilation
Exceptions related to
The method onClose(int, String) of type Html5Servlet.StockTickerSocket must override a superclass method
- The method onOpen(WebSocket.Connection) of type Html5Servlet.StockTickerSocket must override a superclass method
The method onMessage(String) of type Html5Servlet.StockTickerSocket must override a superclass method
This is my servlet , i kept the jars as it is mentioned in that example
package org.ajeesh.app;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Date;
import java.util.List;
import java.util.Timer;
import java.util.TimerTask;
import java.util.concurrent.atomic.AtomicInteger;
import javax.servlet.http.HttpServletRequest;
import org.eclipse.jetty.websocket.WebSocket;
import org.eclipse.jetty.websocket.WebSocketServlet;
public class Html5Servlet extends WebSocketServlet {
private AtomicInteger index = new AtomicInteger();
private static final List<String> tickers = new ArrayList<String>();
static{
tickers.add("ajeesh");
tickers.add("peeyu");
tickers.add("kidillan");
tickers.add("entammo");
}
/**
*
*/
private static final long serialVersionUID = 1L;
public WebSocket doWebSocketConnect(HttpServletRequest req, String resp) {
System.out.println("On server");
return new StockTickerSocket();
}
protected String getMyJsonTicker(){
StringBuilder start=new StringBuilder("{");
start.append("\"stocks\":[");
int counter=0;
for (String aTicker : tickers) {
counter++;
start.append("{ \"ticker\":\""+aTicker +"\""+","+"\"price\":\""+index.incrementAndGet()+"\" }");
if(counter<tickers.size()){
start.append(",");
}
}
start.append("]");
start.append("}");
return start.toString();
}
public class StockTickerSocket implements WebSocket.OnTextMessage{
private Connection connection;
private Timer timer;
#Override
public void onClose(int arg0, String arg1) {
System.out.println("Web socket closed!");
}
#Override
public void onOpen(Connection connection) {
System.out.println("onOpen!");
this.connection=connection;
this.timer=new Timer();
}
#Override
public void onMessage(String data) {
System.out.println("onMessage!");
if(data.indexOf("disconnect")>=0){
connection.close();
timer.cancel();
}else{
sendMessage();
}
}
private void sendMessage() {
System.out.println("sendMessage!");
if(connection==null||!connection.isOpen()){
System.out.println("Connection is closed!!");
return;
}
timer.schedule(new TimerTask() {
#Override
public void run() {
try{
System.out.println("Running task");
connection.sendMessage(getMyJsonTicker());
}
catch (IOException e) {
e.printStackTrace();
}
}
}, new Date(),5000);
}
}
}

How to create an active-mq ExceptionListener?

I have a project coded using Spring-hibernate-activeMq
What I would like to know is if I configured activeMq like I explained below, how should I implement the exception listener class of it? I know you don't understand well now but please give a look to my samples below.
Let me know if I implemented exception listener right or not. If not, please give an example how it must be. Thanks in advance.
Application context: (note that I didn't declare any bean for exception listener except the one the property of connectionFactory)
<bean id="connectionFactory"
class="org.springframework.jms.connection.CachingConnectionFactory"
depends-on="broker">
<constructor-arg ref="amqConnectionFactory"/>
<property name="reconnectOnException" value="true"/>
<property name="exceptionListener" ref="jmsExceptionListener"/>
<property name="sessionCacheSize" value="100"/>
</bean>
Jms exception listener class: (Note that I am trying to inject ConnectionFactory, I am not sure whether it is possible or not.. And the last thing, please check the constructor arguments of it, I am also not sure of it..)
#Component("jmsExceptionListener")
public class JMSExceptionListener implements ExceptionListener {
private final static Logger logger = LoggerFactory.getLogger(JMSExceptionListener.class);
#Autowired
private CachingConnectionFactory connection;
// private Connection connection = null;
private ExceptionListener exceptionListener = null;
public JMSExceptionListener() {
}
public JMSExceptionListener(CachingConnectionFactory connection, ExceptionListener exceptionListener) {
super();
this.connection = connection;
this.exceptionListener = exceptionListener;
}
public void onException(JMSException arg0) {
logger.error("JMS exception has occured.. ", arg0);
if(connection != null){
connection.onException(arg0);
}
if (exceptionListener != null) {
exceptionListener.onException(arg0);
}
}
}
I have manipulated exceptionListener class like the following:
public class JmsExceptionListener implements ExceptionListener {
private final static Logger logger = LoggerFactory.getLogger(JmsExceptionListener.class);
#Autowired
private MailService mailService;
private ExceptionListener exceptionListener = null;
private CachingConnectionFactory cachingConnectionFactory;
public JmsExceptionListener() {
}
public JmsExceptionListener(Connection connection, ExceptionListener exceptionListener) {
super();
this.exceptionListener = exceptionListener;
}
public synchronized void onException(JMSException e) {
logger.error("JMS exception has occurred: ", e);
sendErrorNotificationMail(e);
Exception ex = e.getLinkedException();
if (ex != null) {
logger.error("JMS Linked exception: ", ex);
}
if (exceptionListener != null) {
exceptionListener.onException(e);
}
}
public CachingConnectionFactory getCachingConnectionFactory() {
return cachingConnectionFactory;
}
public void setCachingConnectionFactory(CachingConnectionFactory cachingConnectionFactory) {
this.cachingConnectionFactory = cachingConnectionFactory;
}
private void sendErrorNotificationMail(Exception e) {
try {
mailService.sendJmsExceptionMail(e, ErrorMessageAccessor.get("core.jms.unexpected"));
} catch (ElekBusinessException e1) {
logger.error(ErrorMessageAccessor.get("generic.mailService.exp"), e);
}
}
}

Resources