I use Spring Boot + Kafka. This is my current, pretty simple configuration for Kafka:
#Configuration
#EnableKafka
public class KafkaConfig {
}
This configuration works pretty fine and is able to connect to Kafka instance on default Kafka port: 9092
Right now I need to change the port, let's say on 9093.
How to update this Kafka configuration in order to be able to connect on 9093?
I think something like this in your properties file will do the trick
spring.kafka.bootstrap-servers=localhost:9093
you can specify comma separated list of host:port
Related
How to set up partition count in kafka in spring boot using application.yml.
kafka:
zookeeper:
host: localhost:2181
groupId: group1
topic: test_kafkaw
bootstrap:
servers: localhost:9092
If you are using Spring Cloud Stream, you can specify partition count per Kafka topic in application.yml/application.properties:
spring.cloud.stream.bindings.<binding-name>.producer.partition-count
The Kafka binder uses the ‘partitionCount’ setting of the producer to create a topic with the given partition count.
If you are using Spring for Apache Kafka, you can use TopicBuilder to configure that — something like:
#Bean
public NewTopic topic() {
return TopicBuilder.name("topic1")
.partitions(10)
.replicas(1)
.build();
}
TopicBuilder reference: https://docs.spring.io/spring-kafka/docs/current/api/org/springframework/kafka/config/TopicBuilder.html
ActiveMQ Artemis configuration file in Spring Boot below:
spring:
artemis:
host: localhost
port: 61616
user: admin
password: admin123
There is no properties for broker-url so that I can set consumerWindowSize like
tcp://localhost:61616?consumerWindowSize=0`
How can i configured consumerWindowSize in a Spring Boot application.
Based on the Spring Boot documentation (which references ArtemisProperties) I don't believe you can set the broker's actual URL or any of the properties associated with it. This is a pretty serious short-coming of the Artemis Spring Boot integration as it really limits the configuration. There is already an issue open to (hopefully) address this.
Added below configuration to solve this issue:
#Bean("connectionFactory")
public ConnectionFactory connectionFactory(AppProperties appProperties) {
ActiveMQConnectionFactory cf = new ActiveMQConnectionFactory($brokerUrl);
cf.setUser($user);
cf.setPassword($password);
return cf;
}
Morning all,
I've been struggling lately with the spring-boot-artemis-starter.
My understanding of its spring-boot support was the following:
set spring.artemis.mode=embedded and, like tomcat, spring-boot will instanciate a broker reachable through tcp (server mode). The following command should be successful: nc -zv localhost 61616
set spring.artmis.mode=native and spring-boot will only configure the jms template according to the spring.artemis.* properties (client mode).
The client mode works just fine with a standalone artemis server on my machine.
Unfortunatelly, I could never manage to reach the tcp port in server mode.
I would be grateful if somebody confirms my understanding of the embedded mode.
Thank you for tour help
After some digging I noted that the implementation provided out of the box by the spring-boot-starter-artemis uses org.apache.activemq.artemis.core.remoting.impl.invm.InVMAcceptorFactory acceptor. I'm wondering if that's not the root cause (again I'm by no means an expert).
But it appears that there is a way to customize artemis configuration.
Therefore I tried the following configuration without any luck:
#SpringBootApplication
public class MyBroker {
public static void main(String[] args) throws Exception {
SpringApplication.run(MyBroker.class, args);
}
#Autowired
private ArtemisProperties artemisProperties;
#Bean
public ArtemisConfigurationCustomizer artemisConfigurationCustomizer() {
return configuration -> {
try {
configuration.addAcceptorConfiguration("netty", "tcp://localhost:" + artemisProperties.getPort());
} catch (Exception e) {
throw new RuntimeException("Failed to add netty transport acceptor to artemis instance");
}
};
}
}
You just have to add a Connector and an Acceptor to your Artemis Configuration. With Spring Boot Artemis starter Spring creates a Configuration bean which will be used for EmbeddedJMS configuration. You can see this in ArtemisEmbeddedConfigurationFactory class where an InVMAcceptorFactory will be set for the configuration. You can edit this bean and change Artemis behaviour through custom ArtemisConfigurationCustomizer bean which will be sucked up by Spring autoconfig and be applied to the Configuration.
An example config class for your Spring Boot application:
import org.apache.activemq.artemis.api.core.TransportConfiguration;
import org.apache.activemq.artemis.core.remoting.impl.netty.NettyAcceptorFactory;
import org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnectorFactory;
import org.springframework.boot.autoconfigure.jms.artemis.ArtemisConfigurationCustomizer;
import org.springframework.context.annotation.Configuration;
#Configuration
public class ArtemisConfig implements ArtemisConfigurationCustomizer {
#Override
public void customize(org.apache.activemq.artemis.core.config.Configuration configuration) {
configuration.addConnectorConfiguration("nettyConnector", new TransportConfiguration(NettyConnectorFactory.class.getName()));
configuration.addAcceptorConfiguration(new TransportConfiguration(NettyAcceptorFactory.class.getName()));
}
}
My coworker and I had the exact same problem as the documentation on this link (chapter Artemis Support) says nothing about adding an individual ArtemisConfigurationCustomizer - Which is sad because we realized that without this Customizer our Spring Boot App would start and act as if everything was okay but actually it wouldn't do anything.
We also realized that without the Customizer the application.properties file is not beeing loaded so no matter what host or port you mentioned there it would not count.
After adding the Customizer as stated by the two examples it worked without a problem.
Here some results that we figured out:
It only loaded the application.properties after configuring an ArtemisConfigurationCustomizer
You don't need the broker.xml anymore with an embedded spring boot artemis client
Many examples showing the use of Artemis use a "in-vm" protocol while we just wanted to use the netty tcp protocol so we needed to add it into the configuration
For me the most important parameter was pub-sub-domain as I was using topics and not queues. If you are using topics this parameter needs to be set to true or the JMSListener won't read the messages.
See this page: stackoverflow jmslistener-usage-for-publish-subscribe-topic
When using a #JmsListener it uses a DefaultMessageListenerContainer
which extends JmsDestinationAccessor which by default has the
pubSubDomain set to false. When this property is false it is
operating on a queue. If you want to use topics you have to set this
properties value to true.
In Application.properties:
spring.jms.pub-sub-domain=true
If anyone is interested in the full example I have uploaded it to my github:
https://github.com/CorDharel/SpringBootArtemisServerExample
The embedded mode starts the broker as part of your application. There is no network protocol available with such setup, only InVM calls are allowed. The auto-configuration exposes the necessary pieces you can tune though I am not sure you can actually have a TCP/IP channel with the embedded mode.
I have followed Spring Cloud Netflix's guide to configure Turbine. After enabling Hystrix in two microservices I have verified that /hystrix.stream endpoints generate the correct output.
Now in a hystrix dashboard project I have configured Turbine to get the aggregated results of all the services. However all I get is a succession of:
: ping
data: {"reportingHostsLast10Seconds":0,"name":"meta","type":"meta","timestamp":1448552456486}
This is my config:
HystrixDashboard + Turbine Application:
#EnableHystrixDashboard
#EnableTurbine
#SpringBootApplication
public class HystrixDashboardApplication {
public static void main(String[] args) {
SpringApplication.run(HystrixDashboardApplication.class, args);
}
}
HystrixDashboard + Turbine application.yml:
spring:
application:
name: hystrix-dashboard
server:
port: 10000
turbine:
appConfig: random-story-microservice,storyteller-api
instanceUrlSuffix: /hystrix.stream
logging:
level:
com.netflix.turbine: 'TRACE'
UPDATE
Following kreel's directions I have configured Turbine this way:
turbine:
appConfig: random-story-microservice,storyteller-api
instanceUrlSuffix: /hystrix.stream
clusterNameExpression: new String("default")
It doesn't fail with an exception anymore and in the logs I see that Turbine finds the two candidate hosts/microservices:
[ Timer-0] c.n.t.discovery.InstanceObservable : Retrieved hosts from InstanceDiscovery: 2
However only one of them is finally registered. In InstanceObservable.run() only one of the hosts is added because they have the same hashcode so they are considered the same when added to newState.hostsUp. The com.netflix.turbine.discovery.Instance hashcode is calculated based on the hostname ("myhost" in both cases), and cluster ("default"):
// set the current state
for(Instance host: newList) {
if(host.isUp()) {
newState.hostsUp.add(host);
} else {
newState.hostsDown.add(host);
}
}
What do we have to do when the same host offers two different microservices? Only the first instance is registered in this case.
I think I have an answer but first, to be sure, why are you expecting "default" ?
In fact I think you are misunderstanding the doc :
The configuration key turbine.appConfig is a list of eureka serviceIds that turbine will use to lookup instances. The turbine stream is then used in the Hystrix dashboard using a url that looks like: http://my.turbine.sever:8080/turbine.stream?cluster=<CLUSTERNAME>; (the cluster parameter can be omitted if the name is "default"). The cluster parameter must match an entry in turbine.aggregator.clusterConfig. Values returned from eureka are uppercase, thus we expect this example to work if there is an app registered with Eureka called "customers":
turbine:
aggregator:
clusterConfig: CUSTOMERS
appConfig: customers
In your case :
turbine:
aggregator:
clusterConfig: MY_CLUSTER
appConfig: random-story-microservice,storyteller-api
So it will return "random-story-microservice,storyteller-api" in uppercase.
So, I think you need to apply this part :
The clusterName can be customized by a SPEL expression in turbine.clusterNameExpression with root an instance of InstanceInfo. The default value is appName, which means that the Eureka serviceId ends up as the cluster key (i.e. the InstanceInfo for customers has an appName of "CUSTOMERS"). A different example would be turbine.clusterNameExpression=aSGName, which would get the cluster name from the AWS ASG name. Another example:
turbine:
aggregator:
clusterConfig: SYSTEM,USER
appConfig: customers,stores,ui,admin
clusterNameExpression: metadata['cluster']
In this case, the cluster name from 4 services is pulled from their metadata map, and is expected to have values that include "SYSTEM" and "USER".
To use the "default" cluster for all apps you need a string literal expression (with single quotes):
turbine:
appConfig: customers,stores
clusterNameExpression: 'default'
Spring Cloud provides a spring-cloud-starter-turbine that has all the dependencies you need to get a Turbine server running. Just create a Spring Boot application and annotate it with #EnableTurbine.
Add this in your config :
clusterNameExpression: 'default'
I am currently trying to create a web application with Spring Boot. I need to host my application to localhost:8081. How do I change the port?
Actually you want to change server.port and you can change it in many different ways as described http://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#boot-features-external-config
Examples:
in your application.properties (in or outside the jar)
command line
java -Dserver.port=$PORT -jar target/demo-0.0.1-SNAPSHOT.jar
and much more
By default spring boot uses port 8080, BUT you can change the port just by adding the following code line in your main() like this:
System.getProperties().put( "server.port", *YOUR_PORT_NUMBER_GOES_HERE* );
e.g
#SpringBootApplication
public class MyClass {
public static void main(String[] args) {
System.getProperties().put( "server.port", 8181 ); //8181 port is set here
SpringApplication.run(MyClass.class, args);
}
OR
You can configure it in your application.properties file like so:
server.port=8181
If you DON'T have an application.properties file in your spring-boot application, you can go ahead and create one. Right-click on the src/java/resources folder and go to New-> Other-> General and choose 'File' then name as: application.properties
Any other configurations you might need are listed here https://docs.spring.io/spring-boot/docs/current/reference/html/common-application-properties.html. These properties are also configured in the application.properties file.
Actually you want to change server.port and you can change it in many different ways as described
http://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#boot-features-external-config
Put server.port=9000 in your application.properties
In your application.properties file, just add one line
server.port = 8080
And for more configurations you can refer Spring Boot documentation on port
If you are using the embedded tomcat server, you can configure the EmbeddedServletContainerFactory bean yourself in your Application class annotated with #SpringBootApplication.
This will give you options to customize your tomcat server, example configuration
#Bean
public EmbeddedServletContainerFactory servletContainer() {
TomcatEmbeddedServletContainerFactory factory = new TomcatEmbeddedServletContainerFactory();
factory.setPort(9000);
factory.setSessionTimeout(10, TimeUnit.MINUTES);
factory.addErrorPages(new ErrorPage(HttpStatus.NOT_FOUND, "/notfound.html"));
return factory;
}
You could also do the same for Jetty, using the JettyEmbeddedServletContainerFactory bean, or for Undertow using the UndertowEmbeddedServletContainerFactory .
Official documentation found here : http://docs.spring.io/spring-boot/docs/current-SNAPSHOT/reference/htmlsingle/
If you're using STS, you can do it following below steps:
Go to Boot Dashboard view, you'll see your Boot app, say myApp1
Right click and click on Open Config. This should open Run Time
Configuration section.
Go to Argument tab and add parameter server.port=, like in the example below, a custom port 9091 is added.
Start the app and if everything is good, you'll see the desired port
on Boot dashboard.
go to your application.properties file and type server.port=8081
see this image