I have a spring boot test to check if a kafka consumer listens for a message in specific topic. The kafka listener is triggered when using #SpringBootTest. But I just don't want to load all the classes and I only supplied the listener class like this #SpringBootTest(classes={KafkaConsumerTest.class}).
When only loading the consumer class, the listener has stopped to trigger. Is there something I am missing?
Here is the KafkaTestConsumer class
#Service
public class KafkaTestConsumer {
private static final Logger LOGGER = LoggerFactory.getLogger(KafkaTestConsumer.class);
private CountDownLatch latch = new CountDownLatch(1);
private String payload;
#KafkaListener(topics = {"topic"})
public void receive(ConsumerRecord<?, ?> consumerRecord) {
payload = consumerRecord.toString();
latch.countDown();
}
public CountDownLatch getLatch() {
return latch;
}
public void resetLatch() {
latch = new CountDownLatch(1);
}
public String getPayload() {
return payload;
}
}
It would be great to see what is your KafkaConsumerTest, but perhaps you just override the whole auto-configuration with a plain #Configuration.
See more in docs: https://docs.spring.io/spring-boot/docs/current/reference/html/features.html#features.testing.spring-boot-applications.detecting-configuration
If you want to customize the primary configuration, you can use a nested #TestConfiguration class. Unlike a nested #Configuration class, which would be used instead of your application’s primary configuration, a nested #TestConfiguration class is used in addition to your application’s primary configuration.
Related
My Spring Boot project uses JUnit 5. I'd like to setup an integration test which requires a local SMTP server to be started, so I implemented a custom extension:
public class SmtpServerExtension implements BeforeAllCallback, AfterAllCallback {
private GreenMail smtpServer;
private final int port;
public SmtpServerExtension(int port) {
this.port = port;
}
#Override
public void beforeAll(ExtensionContext extensionContext) {
smtpServer = new GreenMail(new ServerSetup(port, null, "smtp")).withConfiguration(GreenMailConfiguration.aConfig().withDisabledAuthentication());
smtpServer.start();
}
#Override
public void afterAll(ExtensionContext extensionContext) {
smtpServer.stop();
}
}
Because I need to configure the server's port I register the extension in the test class like this:
#SpringBootTest
#AutoConfigureMockMvc
#ExtendWith(SpringExtension.class)
#ActiveProfiles("test")
public class EmailControllerIT {
#Autowired
private MockMvc mockMvc;
#Autowired
private ObjectMapper objectMapper;
#Value("${spring.mail.port}")
private int smtpPort;
#RegisterExtension
// How can I use the smtpPort annotated with #Value?
static SmtpServerExtension smtpServerExtension = new SmtpServerExtension(2525);
private static final String RESOURCE_PATH = "/mail";
#Test
public void whenValidInput_thenReturns200() throws Exception {
mockMvc.perform(post(RESOURCE_PATH)
.contentType(APPLICATION_JSON)
.content("some content")
).andExpect(status().isOk());
}
}
While this is basically working: How can I use the smtpPort annotated with #Value (which is read from the test profile)?
Update 1
Following your proposal I created a custom TestExecutionListener.
public class CustomTestExecutionListener implements TestExecutionListener {
#Value("${spring.mail.port}")
private int smtpPort;
private GreenMail smtpServer;
#Override
public void beforeTestClass(TestContext testContext) {
smtpServer = new GreenMail(new ServerSetup(smtpPort, null, "smtp")).withConfiguration(GreenMailConfiguration.aConfig().withDisabledAuthentication());
smtpServer.start();
};
#Override
public void afterTestClass(TestContext testContext) {
smtpServer.stop();
}
}
The listener is registered like this:
#TestExecutionListeners(value = CustomTestExecutionListener.class, mergeMode = MERGE_WITH_DEFAULTS)
When running the test the listener gets called but smtpPort is always 0, so it seems as if the #Value annotation is not picked up.
I don't think you should work with Extensions here, or in general, any "raw-level" JUnit stuff (like lifecycle methods), because you won't be able to access the application context from them, won't be able to execute any custom logic on beans and so forth.
Instead, take a look at Spring's test execution listeners abstraction
With this approach, GreenMail will become a bean managed by spring (probably in a special configuration that will be loaded only in tests) but since it becomes a bean it will be able to load the property values and use #Value annotation.
In the test execution listener you'll start the server before the test and stop after the test (or the whole test class if you need that - it has "hooks" for that).
One side note, make sure you mergeMode = MergeMode.MERGE_WITH_DEFAULTS as a parameter to #TestExecutionListeners annotation, otherwise some default behaviour (like autowiring in tests, dirty context if you have it, etc) won't work.
Update 1
Following Update 1 in the question. This won't work because the listener itself is not a spring bean, hence you can't autowire or use #Value annotation in the listener itself.
You can try to follow this SO thread that might be helpful, however originally I meant something different:
Make a GreenMail a bean by itself:
#Configuration
// since you're using #SpringBootTest annotation - it will load properties from src/test/reources/application.properties so you can put spring.mail.port=1234 there
public class MyTestMailConfig {
#Bean
public GreenMail greenMail(#Value(${"spring.mail.port"} int port) {
return new GreenMail(port, ...);
}
}
Now this configuration can be placed in src/test/java/<sub-package-of-main-app>/ so that in production it won't be loaded at all
Now the test execution listener could be used only for running starting / stopping the GreenMail server (as I understood you want to start it before the test and stop after the test, otherwise you don't need these listeners at all :) )
public class CustomTestExecutionListener implements TestExecutionListener {
#Override
public void beforeTestClass(TestContext testContext) {
GreenMail mailServer =
testContext.getApplicationContext().getBean(GreenMail.class);
mailServer.start();
}
#Override
public void afterTestClass(TestContext testContext) {
GreenMail mailServer =
testContext.getApplicationContext().getBean(GreenMail.class);
mailServer.stop();
}
}
Another option is autowiring the GreenMail bean and using #BeforeEach and #AfterEach methods of JUnit, but in this case you'll have to duplicate this logic in different Test classes that require this behavour. Listeners allow reusing the code.
I created a spring boot application that sends Messages through a PublishSubscribeChannel. This Channel is "autowired" as SubscribableChannel interface.
I am only subscribing one MessageHandler to this channel, a KafkaProducerMessageHandler.
My problem is that one additional MessageHandler is subscribed and this is an LoggingHandler. It is instantiated with ERROR level. So i see every message logged es error.
I want to know why and where this LoggingHandler is wired (instantiated) and why it is subscribed to the channel - i want to disable it.
(
I debugged around a bit but (was not really helpful):
The LoggingHandler is instantiated and subscribed after the KafkaHandler.
I see this chain EventdrivenConsumer.doStart()<-- ``ConsumerEndpointFactoryBean.initializeEndpoint()<-- ... until reflective calls
)
EDIT
As suggested in comments here is some code (i can't share the whole project). My problem is that the code can't explain the behavior. The LoggingHandler is beeing subscribed to my PublishSubscribeChannel for some unknown reason and it is instantiated with error as level for some unknown reason.
The class that subscribes the KafkaHandler:
#Component
public class EventRelay {
#Autowired
private EventRelay( SubscribableChannel eventBus, #Qualifier( KafkaProducerConfig.KAFKA_PRODUCER ) MessageHandler kafka ) {
eventBus.subscribe( kafka );
}
}
The class that send events is implementing an proprietary interface with many callback methods:
public class PropEvents implements PropClass.IEvents {
private SubscribableChannel eventBus;
private final ObjectMapper om;
private final String userId;
public PropEvents( SubscribableChannel eventBus, ObjectMapper om, String userId ) {
this.eventBus = eventBus;
this.om = om;
this.userId = userId;
}
#Override
public void onLogin( ) {
eventBus.send( new OnLoginMessage(... ) ) );
}
//many other onXYZ methods
}
Here is the Factory that produces instances of PropEvents:
#Configuration
public class EventHandlerFactory {
private final ObjectMapper om;
private final SubscribableChannel eventBus;
#Autowired
public EventHandlerFactory( ObjectMapper om, SubscribableChannel eventBus){
this.om = checkNotNull( om );
this.eventBus = checkNotNull( eventBus );
}
#Bean
#Scope( SCOPE_PROTOTYPE)
public IEvents getEvantHandler(String userId){
if(Strings.isNullOrEmpty(userId)){
throw new IllegalArgumentException( "user id must be set." );
}
return new PropEvents(eventBus, om, userId);
}
}
I appreciate any help with debugging or use tooling (e.g. Eclipse Spring tools does not show any hint to a LoggingHandler Bean) to find where and why a LoggingHandler is instantiated and subscribed to my autowired Channel.
My current workaround is to disable logging for LoggingHandler.
My question at a glance
Why spring instantiates an LoggingHandler with error level and subscribes it to my SubscribableChannel(provided by PublishSubscribeChannel)? How to disable this?
When you #Autowired SubscribableChannel, there should be one in the application context. That might be confusing a bit and mislead, but Spring Integration provides a PublishSubscribeChannel for the global errorChannel: https://docs.spring.io/spring-integration/docs/5.0.2.RELEASE/reference/html/messaging-channels-section.html#channel-special-channels
This one has a LoggingHandler to log error as a default subscriber.
I don't think that it is OK to make your logic based on the errorChannel.
You should consider to declare your own MessageChannel bean and inject it by the particular #Qualifier.
I'm connecting to PubNub in a Spring Boot application. From the documentation, it's ok to re-use PubNub objects but it's better to have one per thread. What's the appropriate method to store and retrieve one object per thread in Spring Boot?
This is how you'd store and retrieve an object per thread in Spring using ThreadLocal, this example is based on Spring's own ThreadLocalSecurityContextHolderStrategy which is used to store SecurityContext per thread.
Also, take a look at InheritableThreadLocal especially if your code spins up new thread, e.g. Spring's #Async annotation, it has mechanisms to propagate existing or create new thread local values when creating child threads.
import org.springframework.util.Assert;
final class ThreadLocalPubNubHolder {
private static final ThreadLocal<PubNub> contextHolder = new ThreadLocal<PubNub>();
public void clearContext() {
contextHolder.remove();
}
public PubNub getContext() {
PubNub ctx = contextHolder.get();
if (ctx == null) {
ctx = createEmptyContext();
contextHolder.set(ctx);
}
return ctx;
}
public void setContext(PubNub context) {
Assert.notNull(context, "Only non-null PubNub instances are permitted");
contextHolder.set(context);
}
public PubNub createEmptyContext() {
// TODO - insert code for creating a new PubNub object here
return new PubNubImpl();
}
}
You can use Java ThreadLocal support as mentioned above by #SergeyB. Another way to do it is to use Thread Scope for your beans:
#Configuration
public class AppConfig {
//Register thread scope for your application
#Bean
public BeanFactoryPostProcessor beanFactoryPostProcessor() {
return beanFactory -> beanFactory.registerScope("thread", new SimpleThreadScope());
}
}
Then you can create a bean with a thread scope (proxy mode will be explained below):
#Scope(value = "thread", proxyMode = ScopedProxyMode.TARGET_CLASS)
#Component
public class PubSubContext {
private PubSub pubSub;
public PubSub getPubSub() {
return pubSub;
}
public void setPubSub(PubSub pubSub) {
this.pubSub = pubSub;
}
#PostConstruct
private void init() {
// TODO: your code for initializing PubSub object
log.info("RequiredMessageHeaders started in thread " + Thread.currentThread().getId());
}
#PreDestroy
private void destroy() {
// TODO: your code for cleaning resources if needed
log.info("RequiredMessageHeaders destroyed in thread " + Thread.currentThread().getId());
}
}
The last step is to inject PubSubContext where you need it:
#Controller
public class YourController {
// Spring will inject here different objects specific for each thread.
// Note that because we marked PubSubContext with proxyMode = ScopedProxyMode.TARGET_CLASS we do not need to use applicationContext.get(PubSubContext.class) to obtain a new bean for each thread - it will be handled by Spring automatically.
#Autowired
private PubSubContext pubSubContext;
#GetMapping
public String yourMethod(){
...
PubSub pubSub = pubSubContext.getPubSub();
...
}
}
With this approach, you could go even further and mark your PubSubContext as #Lazy, so it won't be created until it's requested inside yourMethod :
#Controller
public class YourController {
#Lazy
#Autowired
private PubSubContext pubSubContext;
...
}
As you see PubSubContext does basically what ThreadLocal does but leveraged by Spring capabilities.
Hope it helps!
First of all,
As it is safe to use single PubNub object in multiple threads,
You need multiple PubNub objects ONLY if you need performance increase
If that is your case - my suggestion will be to organize pool of PubNub objects (the use case is quite close to DB connection use case).
I'm trying to disable a Zuul route to a microservice registered with Eureka at runtime (I'm using spring boot).
This is an example:
localhost/hello
localhost/world
Those two are the registered microservices. I would like to disable the route to one of them at runtime without shutting it down.
Is there a way to do this?
Thank you,
Nano
Alternatively to using Cloud Config, custom ZuulFilter can be used. Something like (partial implementation to show the concept):
public class BlackListFilter extends ZuulFilter {
#Override
public String filterType() {
return "pre";
}
...
#Override
public Object run() {
RequestContext ctx = RequestContext.getCurrentContext();
String uri = ctx.getRequest().getRequestURI();
String appId = uri.split("/")[1];
if (blackList.contains(appId)) {
ctx.setSendZuulResponse(false);
LOG.info("Request '{}' from {}:{} is blocked",
uri, ctx.getRequest().getRemoteHost(), ctx.getRequest().getRemotePort());
}
return null;
}
}
where blackList contains list of application IDs (Spring Boot application name) managed for example via some RESTful API.
After a lot of efforts I came up with this solution. First, I used Netflix Archaius to watch a property file. Then I proceeded as follows:
public class ApplicationRouteLocator extends SimpleRouteLocator implements RefreshableRouteLocator {
public ApplicationRouteLocator(String servletPath, ZuulProperties properties) {
super(servletPath, properties );
}
#Override
public void refresh() {
doRefresh();
}
}
Made the doRefresh() method public by extending SimpleRouteLocator and calling its method in the overridden one of the interface RefreshableRouteLocator.
Then I redefined the bean RouteLocator with my custom implementation:
#Configuration
#EnableConfigurationProperties( { ZuulProperties.class } )
public class ZuulConfig {
public static ApplicationRouteLocator simpleRouteLocator;
#Autowired
private ZuulProperties zuulProperties;
#Autowired
private ServerProperties server;
#Bean
#Primary
public RouteLocator routeLocator() {
logger.info( "zuulProperties are: {}", zuulProperties );
simpleRouteLocator = new ApplicationRouteLocator( this.server.getServletPrefix(),
this.zuulProperties );
ConfigurationManager.getConfigInstance().addConfigurationListener( configurationListener );
return simpleRouteLocator;
}
private ConfigurationListener configurationListener =
new ConfigurationListener() {
#Override
public void configurationChanged( ConfigurationEvent ce ) {
// zuulProperties.getRoutes() do something
// zuulProperties.getIgnoredPatterns() do something
simpleRouteLocator.refresh();
}
}
}
Every time a property in the file was modified an event was triggered and the ConfigurationEvent was able to deal with it (getPropertyName() and getPropertyValue() to extract data from the event). Since I also Autowired the ZuulProperties I was able to get access to it. With the right rule I could find whether the property of Zuul
zuul.ignoredPatterns
was modified changing its value in the ZuulProperties accordingly.
Here refresh context should work (as long as you are not adding a new routing rule or removing a currently existing one), if you are adding or removing routing rules, you have to add a new bean for ZuulProperties and mark it with #RefreshScope, #Primary.
You can autowire refreshEndpoint bean for example and apply refreshEndpoint.refresh() on the listener.
Marking a custom RouteLocator as primary will cause problems as zuul already has bean of same type marked as primary.
I am using Spring AMQP's MessageListenerContainer for recieving messages from RabbitMq Broker . Though I am able to receive message inside the listener , autowiring is not working inside listener .
Here is how I have configured my Listener
#Bean
public SimpleMessageListenerContainer listenerContainer() {
SimpleMessageListenerContainer listenerContainer = new SimpleMessageListenerContainer();
listenerContainer.setConnectionFactory(this.inputQueueMgr
.getRabbitConnectionFactory());
JsonMessageConverter converter = new JsonMessageConverter();
listenerContainer.setMessageConverter(converter);
listenerContainer.setMessageListener(new InputQueueEventDispatcher());
listenerContainer.setQueueNames("queue1");
listenerContainer.setAcknowledgeMode(AcknowledgeMode.AUTO);
listenerContainer.setPrefetchCount(1);
return listenerContainer;
}
Here is the class where I am listening to the messages from rabbitMq
#Component(value = "inputMessageListner")
public class InputQueueEventDispatcher implements Serializable, MessageListener {
private static final long serialVersionUID = -5391659256992655430L;
#Autowired
private volatile InputQueueManagerImpl inputQueueMgr;
#Autowired
private volatile NotificationQueueManagerImpl notificationManager;
#Value("${input.rabbitmq.events.queue}")
private String queueName;
#Autowired
private SubscriptionRepository subRepository;
#SuppressWarnings({ "unchecked" })
#Override
public void onMessage(Message message) {
try {
String messageContent = new String(message.getBody());
.....
}
The problem is inside onMessage(Message message) all the autowire components are coming as null .
PS-> Please note that I have declared all the autowire instances as #Component and doing appropriate ComponentScan to scan their packages appropriately . Infact these components do get autowired in normal flow but since onMessage(Message message) gets executed on a seperate thread , these values are showing null . Is there any way to enable autowiring here inside listener .
Thanks
You've set a #Component annotation on your listener, but you don't get this bean from the context. Instead, you're creating the instance yourself, using new. So Spring has no way to know that this instance has been created and must be autowired.
Remove that #Component annotation, and change your code to
#Bean
public SimpleMessageListenerContainer listenerContainer() {
SimpleMessageListenerContainer listenerContainer = new SimpleMessageListenerContainer();
listenerContainer.setConnectionFactory(this.inputQueueMgr
.getRabbitConnectionFactory());
JsonMessageConverter converter = new JsonMessageConverter();
listenerContainer.setMessageConverter(converter);
listenerContainer.setMessageListener(inputMessageListener());
listenerContainer.setQueueNames("queue1");
listenerContainer.setAcknowledgeMode(AcknowledgeMode.AUTO);
listenerContainer.setPrefetchCount(1);
return listenerContainer;
}
#Bean
public InputQueueEventDispatcher inputMessageListener() {
return new InputQueueEventDispatcher();
}
Now, since the bean is returned from a #Bean-annotated method, Spring will make is a Spring bean and autowire it.
Most probably the other thread you mentioned on your question is not getting that instance from spring's application context. So no injection happens for it.
You should use
#RabbitListener(queues = "queue_name")
on your onMessage method.
But you should also change onMessage method syntax and and
onMessage(Message<String> originalMessage)
that way spring will automatically call that method with message