caching with apache ignite with spring boot using spring data - spring-boot

I am trying to do caching with apache ignite 2.7.5 in spring boot 2. Here application starts and ignite node is up but the caching is not working (i dont have any errors in console).
Here is my config file
#Bean
public Ignite igniteInstance() {
IgniteConfiguration cfg = new IgniteConfiguration();
cfg.setIgniteInstanceName("springDataNode");
cfg.setPeerClassLoadingEnabled(true);
CacheConfiguration studcfg = new CacheConfiguration("studentCache");
CacheConfiguration coursecfg = new CacheConfiguration("courseCache");
studcfg.setIndexedTypes(Long.class,Student.class);
coursecfg.setIndexedTypes(Long.class,Course.class);
cfg.setCacheConfiguration(new CacheConfiguration[] {studcfg,coursecfg});
return Ignition.start(cfg);
}
*************************************************************************
here is my repositories
#RepositoryConfig(cacheName = "studentCache")
public interface StudentRepository extends IgniteRepository<Student,Long> {
List<Student> getStudentByName(String name);
Student getStudentById (Long id);
}
#RepositoryConfig(cacheName = "courseCache")
public interface CourseRepository extends IgniteRepository<Course, Long> {
List<Course> getAllCourseByName (String name);
#Query("SELECT id FROM Breed WHERE id = ?")
List<Long> getById (Long id, Pageable pageable);
}
here is the caching part
public class CacheApp {
private static AnnotationConfigApplicationContext ctx;
#Autowired
private static StudentRepository studentRepository;
#Autowired
private static CourseRepository courseRepository;
public static void main(String[] args)
{
System.out.println( "Spring Data Example!" );
ctx = new AnnotationConfigApplicationContext();
ctx.register(SpringConfig.class);
ctx.refresh();
studentRepository= ctx.getBean(StudentRepository.class);
courseRepository= ctx.getBean(CourseRepository.class);
Student stud1= new Student();
stud1.setId(111);
stud1.setName("ram");
studentRepository.save(stud1);
List<Student> getAllBreeds = studentRepository.getStudentByName("ram");
for(Student student : getAllBreeds){
System.out.println("student:" + student);
}
Course course1= new Course();
course1.setId(1);
course1.setName("maths");
courseRepository.save(course1);
List<Course> courses = courseRepository.getAllCourseByName("maths");
for(Course cor : courses){
System.out.println("courses:"+ cor);
}
}
}
here is my console log
01:32:56] Ignite node started OK (id=2eb50680, instance name=springDataNode)
2020-04-25 01:32:56.427 INFO 5056 --- [ main] o.a.i.i.IgniteKernal%springDataNode : Data Regions Configured:
2020-04-25 01:32:56.427 INFO 5056 --- [ main] o.a.i.i.IgniteKernal%springDataNode : ^-- default [initSize=256.0 MiB, maxSize=799.3 MiB, persistence=false]
2020-04-25 01:32:56.428 INFO 5056 --- [ main] o.a.i.i.IgniteKernal%springDataNode :
>>> +----------------------------------------------------------------------+
>>> Ignite ver. 2.7.5#20190603-sha1:be4f2a158bcf79a52ab4f372a6576f20c4f86954
>>> +----------------------------------------------------------------------+
..........................................................
[01:32:56] Topology snapshot [ver=1, locNode=2eb50680, servers=1, clients=0, state=ACTIVE, CPUs=4, offheap=0.78GB, heap=0.87GB]
2020-04-25 01:32:56.431 INFO 5056 --- [ main] o.a.i.i.m.d.GridDiscoveryManager : Topology snapshot [ver=1, locNode=2eb50680, servers=1, clients=0, state=ACTIVE, CPUs=4, offheap=0.78GB, heap=0.87GB]
2020-04-25 01:32:56.547 INFO 5056 --- [ main] com.example.demo.IgniteTestApplication : Started IgniteTestApplication in 4.761 seconds (JVM running for 5.566)
hgjjhjhhhjhjhjjjhjhjjh************************************************************************************************************************************************************
You can see that caching part was not working.As I am new to this technology I am unable to figure out want has went wrong here.Can anyone please help?

You need to use #EnableIgniteRepositories To enable Apache Ignite backed repositories in Spring Data.
see: https://apacheignite-mix.readme.io/docs/spring-data#spring-data-and-apache-ignite-configuration
Take a look at: https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/springdata/SpringDataExample.java

Related

Thymeleaf [# th:each] is not getting parsed

I have a thymeleaf (3.0.11.RELEASE) TEXT template with iteration as follows:
[# th:each="sei : ${specificInfoElements}"]
[(${sei?.elementLabel})] : [(${sei?.elementValues})]
[/]
The above is not getting evaluated by template engine and its coming as follows in output:
[# th:each="sei : ${specificInfoElements}"]
:
[/]
Can anybody help me understand what I am doing wrong?
Note: I am using spring boot.
#Autowired
private SpringTemplateEngine thymeleafTemplateEngine;
Context thymeleafContext = new Context();
thymeleafContext.setVariables(templateModel);
String outputText = thymeleafTemplateEngine.process(emailTemplateString,
thymeleafContext);
I just tested this with Spring Boot and it works fine. What I did exactly:
Create a new project via start.spring.io using Spring Boot 2.5.1 with Java 11
Update application.properties with:
spring.thymeleaf.mode=TEXT
spring.thymeleaf.suffix=.txt
Create a file src/main/resources/templates/test.txt containing the template:
[# th:each="sei : ${specificInfoElements}"]
[(${sei?.elementLabel})] : [(${sei?.elementValues})]
[/]
Create a test class that extends from CommandLineRunner so I can just start the app and see some output:
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.CommandLineRunner;
import org.springframework.stereotype.Component;
import org.thymeleaf.context.Context;
import org.thymeleaf.spring5.SpringTemplateEngine;
import java.util.List;
#Component
public class Test implements CommandLineRunner {
#Autowired
private SpringTemplateEngine templateEngine;
#Override
public void run(String... args) throws Exception {
Context context = new Context();
context.setVariable("specificInfoElements", List.of(new SpecificInfoElement("first label", "first value"),
new SpecificInfoElement("second label", "second element")));
String result = templateEngine.process("test", context);
System.out.println("result = " + result);
}
private static class SpecificInfoElement {
private String elementLabel;
private String elementValues;
public SpecificInfoElement(String elementLabel, String elementValues) {
this.elementLabel = elementLabel;
this.elementValues = elementValues;
}
public String getElementLabel() {
return elementLabel;
}
public String getElementValues() {
return elementValues;
}
}
}
Running this outputs:
2021-06-22 08:22:39.229 INFO 13464 --- [ main] com.example.demo.DemoApplication : Started DemoApplication in 1.119 seconds (JVM running for 2.828)
result =
first label : first value
second label : second element
I hope this can help you figure out what you are doing differently.
Since I am storing templates in DB, thymeleaf is using StringTemplateResolver by default. StringTemplateResolver is added to the SpringTemplateEngine upon calling process method for the first time as shown in the code snippet taken from org.thymeleaf.TemplateEngine:
if (this.templateResolvers.isEmpty()) {
this.templateResolvers.add(new StringTemplateResolver());
}
I changed the template mode of StringTemplateResolver before calling the process method as follows:
if (CollectionUtils.isEmpty(springTemplateEngine.getTemplateResolvers())) {
// calling process method will initialize SpringTemplateEngine with StringTemplateResolver.
springTemplateEngine.process("template contents", context);
Set<ITemplateResolver> templateResolvers =
springTemplateEngine.getTemplateResolvers();
StringTemplateResolver stringTemplateResolver = (StringTemplateResolver) templateResolvers.iterator().next();
stringTemplateResolver.setTemplateMode(TemplateMode.TEXT);
}

Is there any way to know the load time for a java application?

I am trying to compare performance of SpringBoot and Micronaut.
I have some applications implemented with both frameworks, and can get some info about JVM with Micrometer, but the information about the time each of these frameworks need to load from scratch and start working is something I am missing.
Is there any way to get it?
Thanks.
Spring boot logs startup time in format:
Started {applicationName} in {time} seconds (JVM running for {jvmTime})
e.g.
2019-05-18 20:50:07.099 INFO 6904 --- [ main] c.e.demo.DemoApplication : Started DemoApplication in 2.156 seconds (JVM running for 3.164)
If you want to have access to startup time programmatically in your application you can JVM running time on ApplicationStartedEvent:
#Component
public class StartupListener {
#EventListener
public void onStartup(ApplicationStartedEvent event) {
double startupTime = ManagementFactory.getRuntimeMXBean().getUptime() / 1000.0;
System.out.println("Application started in: " + startupTime);
}
}
Just to complete the answer with the Micronaut part:
#Singleton
#Requires(notEnv = Environment.TEST)
#Slf4j
public class InitialEventListener implements ApplicationEventListener<ServiceStartedEvent> {
#Getter
private long currentTimeMillis;
#Async
#Override
public void onApplicationEvent(ServiceStartedEvent event) {
currentTimeMillis = System.currentTimeMillis();
log.info("ServiceStartedEvent at " + currentTimeMillis + ":" + event);
}
}

Spring Boot elasticsearch produce warnings: Transport response handler not found of id

I setuped a web app with org.springframework.boot:spring-boot-starter-data-elasticsearch. Everything work well - I can populate indexes with my stand alone Elasticsearch 5. But I continue to receive some weird warnings:
2018-05-08 03:07:57.940 WARN 32053 --- [ient_boss][T#7]] o.e.transport.TransportService : Transport response handler not found of id [5]
2018-05-08 03:08:02.949 WARN 32053 --- [ient_boss][T#8]] o.e.transport.TransportService : Transport response handler not found of id [7]
2018-05-08 03:08:07.958 WARN 32053 --- [ient_boss][T#1]] o.e.transport.TransportService : Transport response handler not found of id [9]
2018-05-08 03:08:12.970 WARN 32053 --- [ient_boss][T#2]] o.e.transport.TransportService : Transport response handler not found of id [11]
...
Simple app to reproduce:
#SpringBootApplication
public class App {
#Configuration
#EnableElasticsearchRepositories(basePackages = "com.test")
public class EsConfig {
#Value("${elasticsearch.host}")
private String esHost;
#Value("${elasticsearch.port}")
private int esPort;
#Value("${elasticsearch.clustername}")
private String esClusterName;
#Bean
public TransportClient client() throws Exception {
Settings esSettings = Settings.builder().put("cluster.name", esClusterName).build();
InetSocketTransportAddress socketAddress = new InetSocketTransportAddress(
InetAddress.getByName(esHost), esPort);
return new PreBuiltTransportClient(esSettings).addTransportAddress(socketAddress);
}
#Bean
public ElasticsearchOperations elasticsearchTemplate(Client client) throws Exception {
return new ElasticsearchTemplate(client);
}
}
public static void main(String[] args) {
SpringApplication.run(App.class, args);
}
}
My compose file for ES
version: "2.3"
services:
elasticsearch:
image: 'docker.elastic.co/elasticsearch/elasticsearch:5.6.8
ports:
- "9200:9200"
- "9300:9300"
environment:
- xpack.security.enabled=false
- ES_JAVA_OPTS=-Xms700m -Xmx700m
- PATH_LOGS="/tmp/el-log"
- cluster.name=dou
cpu_shares: 1024
mem_limit: 1024MB
As a project transitive dependency I have org.elasticsearch.client:transport:5.6.8. Looks like versions of ES instance & library are the same.
So, what does this warning mean & how should we deal with it?

Remove file from remote using streaming inbound channel adapter spring boot implementation

I am trying to remove file from remote by implementing streaming inbound but connection is closing before adviceChain implementing.
CODE:
#Bean
public SessionFactory<LsEntry> sftpSessionFactory() {
DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(true);
factory.setHost(sftpHost);
factory.setPort(sftpPort);
factory.setUser(sftpUser);
factory.setPassword(sftpPwd);
factory.setAllowUnknownKeys(true);
return new CachingSessionFactory<LsEntry>(factory);
}
#Bean
#InboundChannelAdapter(channel = "stream", poller = #Poller(cron = "2 * * * * ?"))
public MessageSource<InputStream> sftpMessageSource() {
SftpStreamingMessageSource messageSource = new SftpStreamingMessageSource(template());
messageSource.setRemoteDirectory(remoteDirecotry);
messageSource.setFilter(new AcceptAllFileListFilter<>());
return messageSource;
}
#Bean
public SftpRemoteFileTemplate template() {
return new SftpRemoteFileTemplate(sftpSessionFactory());
}
#Bean
#Transformer(inputChannel = "stream", outputChannel = "data")
public org.springframework.integration.transformer.Transformer transformer() {
return new StreamTransformer("UTF-8");
}
#ServiceActivator(inputChannel = "data" ,adviceChain = "afterChain")
#Bean
public MessageHandler handler() {
return new MessageHandler() {
#Override
public void handleMessage(Message<?> message) throws MessagingException {
String fileName = message.getHeaders().get("file_remoteFile").toString();
if (!StringUtils.isEmpty(message.toString())) {
else{
log.info("No file found in the Remote location");
}
}
};
}
#Bean
public ExpressionEvaluatingRequestHandlerAdvice afterChain() {
ExpressionEvaluatingRequestHandlerAdvice advice = new ExpressionEvaluatingRequestHandlerAdvice();
advice.setOnSuccessExpression(
"#template.remove(headers['file_remoteDirectory'] + headers['file_remoteFile'])");
//advice.setOnSuccessExpressionString("#template.remove(headers['file_remoteFile'])");
advice.setPropagateEvaluationFailures(true);
return advice;
}
wherever i search every one is suggesting to implement ExpressionEvaluatingRequestHandlerAdvice but it is throwing me below error.
2018-03-27 12:32:02.618 INFO 23216 --- [ask-scheduler-1] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=starsBatchJob]] completed with the following parameters: [{JobID=1522168322277}] and the following status: [COMPLETED]
2018-03-27 12:32:02.618 INFO 23216 --- [ask-scheduler-1] c.f.u.config.ParentBatchConfiguration : Job Status Completed
2018-03-27 12:32:02.618 INFO 23216 --- [ask-scheduler-1] c.f.u.config.ParentBatchConfiguration : Total time tokk for Stars Batch execution: 0 seconds.
2018-03-27 12:32:02.618 INFO 23216 --- [ask-scheduler-1] c.f.u.config.ParentBatchConfiguration : Batch Job lock is released
2018-03-27 12:32:02.633 INFO 23216 --- [ask-scheduler-1] com.jcraft.jsch : Disconnecting from hpchd1e.hpc.ford.com port 22
2018-03-27 12:32:02.633 ERROR 23216 --- [ask-scheduler-1] o.s.integration.handler.LoggingHandler : org.springframework.messaging.MessagingException: Dispatcher failed to deliver Message; nested exception is org.springframework.messaging.MessagingException: Failed to execute on session; nested exception is org.springframework.core.NestedIOException: Failed to remove file: 2: No such file; nested exception is 2
I had this problem. My path to the remote file was incorrect. I needed a trailing /. It is a little difficult to see since the path is being created inside a Spel Expression. You can see the path using the following in the handleMessage() method.
String remoteDirectory = (String) message.getHeaders().get("file_remoteDirectory");
String remoteFile = (String) message.getHeaders().get("file_remoteFile");
I did have to use the advice.setOnSuccessExpressionString("#template.remove(headers['file_remoteFile'])"); that is commented out above instead of advice.setOnSuccessExpression"#template.remove(headers['file_remoteDirectory'] + headers['file_remoteFile'])");
It is incorrect in the documentation https://docs.spring.io/spring-integration/reference/html/sftp.html#sftp-streaming which is why I believe people who struggle with this lose faith in the doc. But this seems to be the only error.

My tcp client using spring integration not able to get response

I have created tcp client using spring integration I am able to receive response for my send message . But when I uses localDateTime.now() to log time I am not able to receive the response of send message . I know this can be solved using time setting to make thread wait. As I am new to spring integration So kindly help me how to do it.
#Configuration
#ComponentScan
#EnableAutoConfiguration
public class Test
{
protected final Log logger = LogFactory.getLog(this.getClass());
// **************** Client **********************************************
#Bean
public MessageChannel replyChannel()
{
return new DirectChannel();
}
#Bean
public MessageChannel sendChannel()
{
MessageChannel directChannel = new DirectChannel();
return directChannel;
}
#EnableIntegration
#IntegrationComponentScan
#Configuration
public static class config
{
#MessagingGateway(defaultRequestChannel = "sendChannel", defaultReplyChannel = "replyChannel")
public interface Gateway
{
String Send(String in);
}
}
#Bean
AbstractClientConnectionFactory tcpNetClientConnectionFactory()
{
AbstractClientConnectionFactory tcpNetClientConnectionFactory = new TcpNetClientConnectionFactory("localhost",
9999);
tcpNetClientConnectionFactory.setSerializer(new UCCXImprovedSerializer());
tcpNetClientConnectionFactory.setDeserializer(new UCCXImprovedSerializer());
tcpNetClientConnectionFactory.setSingleUse(true);
tcpNetClientConnectionFactory.setMapper(new TcpMessageMapper());
return tcpNetClientConnectionFactory;
}
#Bean
#ServiceActivator(inputChannel = "sendChannel")
TcpOutboundGateway tcpOutboundGateway()
{
TcpOutboundGateway tcpOutboundGateway = new TcpOutboundGateway();
tcpOutboundGateway.setConnectionFactory(tcpNetClientConnectionFactory());
tcpOutboundGateway.setReplyChannel(replyChannel());
return tcpOutboundGateway;
}
public static void main(String args[])
{
// new LegaServer();
ConfigurableApplicationContext applicationContext = SpringApplication.run(Test.class, args);
String temp = applicationContext.getBean(Gateway.class).Send("kksingh");
System.out.println(LocalDateTime.now()+"output" + temp);
applicationContext.stop();
}
}
My custom serialzer and deserialser UCCXImprovedSerializerclass
after updating as per #Garry
public class UCCXImprovedSerializer implements Serializer<String>, Deserializer<String>
{
#Override
public String deserialize(InputStream initialStream) throws IOException
{
System.out.println("deserialzier called");
StringBuilder sb = new StringBuilder();
try (BufferedReader rdr = new BufferedReader(new InputStreamReader(initialStream)))
{
for (int c; (c = rdr.read()) != -1;)
{
sb.append((char) c);
}
}
return sb.toString();
}
#Override
public void serialize(String msg, OutputStream os) throws IOException
{
System.out.println(msg + "---serialize---" + Thread.currentThread().getName() + "");
os.write(msg.getBytes());
}
}
My server at port 9999 code
try
{
clientSocket = echoServer.accept();
System.out.println("client connection established..");
is = new DataInputStream(clientSocket.getInputStream());
os = new PrintStream(clientSocket.getOutputStream());
String tempString = "kksingh";
byte[] tempStringByte = tempString.getBytes();
byte[] temp = new byte[tempString.getBytes().length];
while (true)
{
is.read(temp);
System.out.println(new String(temp) + "--received msg is--- " + LocalDateTime.now());
System.out.println(LocalDateTime.now() + "sending value");
os.write(tempStringByte);
break;
}
} catch (IOException e)
{
System.out.println(e);
}
}
My log file for tcp client
2017-06-04 23:10:14.771 INFO 15568 --- [ main] o.s.i.endpoint.EventDrivenConsumer : started org.springframework.integration.endpoint.EventDrivenConsumer#1f12e153
kksingh---serialize---main
pool-1-thread-1---deserialize----
pool-1-thread-1---deserialize----
pool-1-thread-1---deserialize----
pool-1-thread-1---deserialize----
2017-06-04 23:10:14.812 ERROR 15568 --- [pool-1-thread-1] o.s.i.ip.tcp.TcpOutboundGateway : Cannot correlate response - no pending reply for localhost:9999:57622:bc98ee29-8957-47bd-bd8a-f734c3ec3f9d
2017-06-04T23:10:14.809output
2017-06-04 23:10:14.821 INFO 15568 --- [ main] o.s.c.support.DefaultLifecycleProcessor : Stopping beans in phase 0
My log file for server side
client connection established..
kksingh--received msg is--- 2017-06-04T23:10:14.899
2017-06-04T23:10:14.899sending value
when I removed the localdatetime.now() from server and tcpclient I am able to get response as outputkksingh
o.s.i.endpoint.EventDrivenConsumer : Adding {logging-channel-adapter:_org.springframework.integration.errorLogger} as a subscriber to the 'errorChannel' channel
2017-06-05 12:46:32.494 INFO 29076 --- [ main] o.s.i.channel.PublishSubscribeChannel : Channel 'application.errorChannel' has 1 subscriber(s).
2017-06-05 12:46:32.495 INFO 29076 --- [ main] o.s.i.endpoint.EventDrivenConsumer : started _org.springframework.integration.errorLogger
2017-06-05 12:46:32.746 INFO 29076 --- [ main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat started on port(s): 8080 (http)
2017-06-05 12:46:32.753 INFO 29076 --- [ main] o.s.i.samples.tcpclientserver.Test : Started Test in 2.422 seconds (JVM running for 2.716)
2017-06-05 12:46:32.761 INFO 29076 --- [ main] o.s.i.endpoint.EventDrivenConsumer : Adding {bridge:null} as a subscriber to the 'replyChannel' channel
2017-06-05 12:46:32.762 INFO 29076 --- [ main] o.s.integration.channel.DirectChannel : Channel 'application.replyChannel' has 1 subscriber(s).
2017-06-05 12:46:32.763 INFO 29076 --- [ main] o.s.i.endpoint.EventDrivenConsumer : started org.springframework.integration.endpoint.EventDrivenConsumer#1f12e153
kksingh---serialize---main
pool-1-thread-1---deserialize----kksingh
outputkksingh
2017-06-05 12:46:32.837 INFO 29076 --- [ main] o.s.c.support.DefaultLifecycleProcessor : Stopping beans in phase 0
2017-06-05 12:46:32.839 INFO 29076 --- [ main] o.s.i.endpoint.EventDrivenConsumer : Removing {bridge:null} as a subscriber to the 'replyChannel' channel
2017-06-05 12:46:32.839 INFO 29076 --- [
Your deserializer is deserializing multiple packets...
pool-1-thread-1---deserialize----
pool-1-thread-1---deserialize----
pool-1-thread-1---deserialize----
pool-1-thread-1---deserialize----
Which produces 4 reply messsages; the gateway can only handle one reply which is why you see that ERROR message.
You deserializer needs to be smarter than just capturing "available" bytes. You need something in the message to indicate the end of the data (or close the socket to indicate the end).

Resources