I have a Spring boot application with Prometheus Pushgateway using Micrometer, mainly based on this tutorial: https://luramarchanjo.tech/2020/01/05/spring-boot-2.2-and-prometheus-pushgateway-with-micrometer.html
pom.xml has following related dependencies:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-core</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
<dependency>
<groupId>io.prometheus</groupId>
<artifactId>simpleclient_pushgateway</artifactId>
<version>0.16.0</version>
</dependency>
And application.properties file has:
management.metrics.export.prometheus.pushgateway.enabled=true
management.metrics.export.prometheus.pushgateway.shutdown-operation=PUSH
management.metrics.export.prometheus.pushgateway.baseUrl=localhost:9091
This works fine if I leave the application running however with my particular Spring boot application, sometimes it looses the metrics sent just before the shutdown.
I can view the following logs which indicates the PrometheusPushGatewayManager is successfully calling the shutdown() method before the application shuts down which has configured with PUSH operation in the application.properties file as above:
level":"INFO","message":"Shutting down ExecutorService","file":"ExecutorConfigurationSupport.java","line_number":"208","thread_name":"Thread-1","#version":1,"logger_name":"org.springframework.boot.actuate.metrics.export.prometheus.PrometheusPushGatewayManager$PushGatewayTaskScheduler","class":"org.springframework.scheduling.concurrent.ExecutorConfigurationSupport"
I have tried to invoke the shutdown() method on PrometheusPushGatewayManager from my application code but still having the same issue where metrics are not appearing consistently in the Pushgateway/Prometheus (randomly).
Related
I am unable to set group id in Spring cloud stream kafka consumer config using below :-
spring.cloud.stream.default-binder=kafka
spring.cloud.stream.kafka.binder.brokers=${kafka.bootstrap.servers}
spring.cloud.stream.bindings.INPUT.binder=kafka
spring.cloud.stream.bindings.INPUT.destination=datapipeline.ingestion.decision.topic
spring.cloud.stream.bindings.INPUT.content-type=application/json
spring.cloud.stream.bindings.INPUT.group=input-group-1
All above property are getting set except group and getting below in console log while starting service :
group.id = anonymous.cce5a71a-66fa-49c9-874b-09d5685713f7
Kindly help on this as i think because of this my consumer unable to read from where it left.
I am using Spring boot 2.1.3.RELEASE.
Few important dependency related to this, Also I am using Spring integration starter as well :-
<spring-cloud.version>Greenwich.RELEASE</spring-cloud.version>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream-binder-kafka</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream-binder-kafka-streams</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka</artifactId>
</dependency>
I am using elasticsearch7.5. I have turned on the following properties
xpack.security.enabled: true
xpack.security.authc.api_key.enabled: true
I have generated API keys to connect our springboot to elasticsearch.I am using
spring-boot-starter-data-elasticsearch
All my requests are from "RestHighClient" and able to trigger the requests.Only problem is during server startup, they are some errors where it is not able to connect to Elasticsearch.
org.elasticsearch.transport.RemoteTransportException: [ADMIN-PC][127.0.0.1:9300][cluster:monitor/nodes/liveness]
Caused by: org.elasticsearch.ElasticsearchSecurityException: missing authentication credentials for action [cluster:monitor/nodes/liveness]
In pom.xml
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>transport</artifactId>
<version>6.8.3</version>
</dependency>
in application properties:
spring.elasticsearch.server=localhost:9200
spring.data.elasticsearch.cluster-name=elasticsearch
spring.data.elasticsearch.cluster-nodes=127.0.0.1:9300
spring.data.elasticsearch.repositories.enabled=true
Can someone suggest me ,How I can fix it.
Remove these lines from your configuration:
spring.data.elasticsearch.cluster-name=elasticsearch
spring.data.elasticsearch.cluster-nodes=127.0.0.1:9300
These make Spring Boot to configure the transport client.
Written simple POC to prove and test Spring Boot and log4j2 compatibility. Once successful I will move it to real application.
Please refer my POC source code:
https://github.com/Dennyss/SpringBootLog4j2POC
I know/read about Spring version and log4j2 compatibility from:
How to set up Spring Boot and log4j2 properly?
Found and tried recommendations described here:
Spring-Boot logging with log4j2?
But still not working. The problem is that both application logs and Spring logs are printing to console only.
Please refer maven dependencies below (from POC):
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.6.2</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.6.2</version>
</dependency>
</dependencies>
If I don't exclude Spring's logback and don't add boot-starter-log4j2 then application logs are printing to application file but Spring logs are not printing at all. I feel the problem somewhere with dependencies. Appreciate any help.
According to the Spring Boot Logging documentation, the location of the logging configuration file can be specified using the logging.config property. I noticed that your start.sh script is passing -Dlog4j.configurationFile. Normally, this would be sufficient for direct Log4J 2 integration, but Spring Boot uses logging.config.
After switching to this, it writes to the log files:
java -Dlogging.config=/share/LogPOC/log4j2.xml -cp poc.jar:libs/* com.poc.logger.Application
I get spring security to work with a LDAP server running on my local machine, then, I move on to finish this tutorial https://spring.io/guides/gs/authenticating-ldap/ (not step by step, since I am not using Spring boot)
The goal is to run spring security with an "embedded" LDAP server, but I am having trouble setting things up.
to summarize
locally running LDAP(get it to work) vs. embedded LDAP(having trouble)
To configure AuthenticationManagerBuilder, I did
auth.ldapAuthentication()
.contextSource()
.root("dc=oreilly,dc=com") // I didn't set url() here, so embedded server can be used
.ldif("classpath:spring-security.ldif")
.managerDn("uid=admin,ou=system")
.managerPassword("secret")
.and()
.userSearchFilter("uid={0}");
in my pom.xml, I included the following libraries
<dependency>
<groupId>com.unboundid</groupId>
<artifactId>unboundid-ldapsdk</artifactId>
</dependency> <!-- This is the ldap server-->
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-config</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-ldap</artifactId>
</dependency>
I won't include other code here, since I get them to work with a local LDAP.
When I run, I get the following error:
java.lang.NoClassDefFoundError:
org/apache/directory/server/core/partition/Partition
I think I may miss to include some libraries, but not sure which..
After trying different setting, this works for me:
<dependency>
<groupId>org.apache.directory.server</groupId>
<artifactId>apacheds-protocol-ldap</artifactId>
<version>1.5.5</version>
</dependency>
<dependency>
<groupId>org.apache.directory.shared</groupId>
<artifactId>shared-ldap</artifactId>
<version>0.9.15</version>
</dependency>
I'm trying to integrate hawtio in a spring-boot application using apache camel. I followed Spring-Boot Embedded Wars and added HawtioConfiguration from How to run hawt.io in spring boot application with embedded tomcat (except for the kubeservice and kubepod which are not in io.hawt.web package)
So, that works, up to the point where I try to manualy send a message to a direct endpoint from the hawtio interface ( http://localhost:8080/hawtio/index.html#/camel/sendMessage?tab=camel&nid=root-org.apache.camel-camel-1-endpoints-%22direct:%2F%2Fdummy%22 ) . The following warning appears, and no message is sent:
Camel does not support sending to this endpoint.
So, did I forget anything ? Here is my set up : springboot 1.3.3.RELEASE with the following dependencies :
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-spring-boot-starter</artifactId>
<version>2.17.0</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jersey</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-actuator</artifactId>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.hawt</groupId>
<artifactId>hawtio-springboot</artifactId>
<version>1.4.64</version>
</dependency>
<dependency>
<groupId>io.hawt</groupId>
<artifactId>hawtio-core</artifactId>
<version>1.4.64</version>
</dependency>
and the Application.java :
#SpringBootApplication
#EnableHawtio
#EnableAutoConfiguration(exclude={DataSourceAutoConfiguration.class})
public class Application {
#Autowired
private ServletContext servletContext;
public static void main(String[] args) {
System.setProperty(AuthenticationFilter.HAWTIO_AUTHENTICATION_ENABLED, "false");
SpringApplication.run(Application.class, args);
}
#PostConstruct
public void init() {
final ConfigManager configManager = new ConfigManager();
configManager.init();
servletContext.setAttribute("ConfigManager", configManager);
}
}
Thanks !
edit: using hawtio as a standalone app and connecting to springboot works fine
edit2: moving on, I used hawtio as a war on another project (same version) , deployed on a tomcat 7. Same issue, cannot send to a direct endpoint.
go figure.
The option of using hawtio as war works very well with application using spring boot and camel, I am already using it successfully.
check the hawtio github code example, it contains good samples to try
https://github.com/hawtio/hawtio
Also I will share my github link with sample's of using hawtio wat or as maven plug-in.
I had the same problem: I could not use hawt.io to send to a direct endpoint (nor any other endpoint). Maybe it is a general Bug/ unimplemented feature in hawt.io?
However, the following was possible:
Copy the Endpoint URL
Select the context node in hawt.io
Click on "Operations"
Use the method sendStringBody(java.lang.String,java.lang.String) to send a string to that endpoint
First parameter is the endpoint URI that you can paste in