Implementing worker processes in a Spring Boot application - spring

Intro
I am currently running a Spring-Boot application through Heroku on a single web dyno. Due to the large number of intensive background tasks (fetching resources from 3rd party APIs, sending mails, etc.), I would like to move all these "heavy jobs" on a second worker dyno/process. However, I am facing several difficulties in properly exposing the application components (e.g. #Repositories) to the second worker process.
What I have attempted so far
I've created a second main class (BackgroundWorker) which I specify in the Procfile as a worker process. The following class is then called in order to initialize the background tasks.
#Service
#EnableMongoRepositories("com.a.viz.db")
#ComponentScan("com.a.viz.db")
#EntityScan("com.a.viz.model")
public class TaskHandler {
#Autowired
UProductRepository productRepository;
public void initScheduler()
{
Runnable fetchProducts = () -> {
Scheduler.fetchProducts(productRepository);
};
}
}
While the main class looks like this:
public class BackgroundWorker {
static Logger logger = LoggerFactory.getLogger(BackgroundWorker.class);
public static void main(String[] args) {
AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext();
context.scan("com.a.viz.workers");
context.refresh();
TaskHandler handler = context.getBean(TaskHandler.class);
handler.initScheduler();
}
}
Upon running the above snippet, I get an unsatisfied dependency error for bean MongoTemplate which I inject in the concrete implementation of UProductRepository, called UProductRepositoryImpl.
public class UProductRepositoryImpl implements UProductRepositoryCustom {
private final MongoTemplate mongoTemplate;
#Autowired
public UProductRepositoryImpl(MongoTemplate mongoTemplate) {
this.mongoTemplate = mongoTemplate;
}
}
org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'org.springframework.data.mongodb.core.MongoTemplate'
How may I expose MongoTemplate to the second worker process? Moreover, what would be a good way to approach something like this? Should I try to organize my components such that only the relevant ones are exposed to the worker process? Thanks for your attention!

Solution
Since the worker process must also be a Spring application (in order to allow for injecting repositories and such), its application context must be initialized as such. The web parameter is to prevent a proper web server being set up, since that is not necessary.
// Other configs..
#EnableAutoConfiguration
public class BackgroundWorker implements ApplicationRunner {
#Autowired
// Repositories..
public static void main(String[] args)
{
new SpringApplicationBuilder(BackgroundWorker.class)
.web(WebApplicationType.NONE)
.run(args);
}

Related

How to execute code in a SpringBootTest before the Application is run?

I have a SpringBoot based command line application. The application creates or deletes some records in a database. It does so not directly via JDBC but rather through a special API (instance variable dbService).
The application class looks like this:
#SpringBootApplication
public class Application implements CommandLineRunner {
#Autowired
private DbService dbService;
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
#Override
public void run(String... args) {
// Do something via dbService based on the spring properties
}
}
Now I'd like to create a SpringBoot test that would run the whole application with a configuration specially prepared for the test.
I run the test with an in-memory DB (H2) which is empty at the test start. Hence I'd like to insert some records into the DB -- as the setup for the test. The code for inserting the records must be executed
After the Spring context has been loaded -- so that I can use the bean dbService.
Before the Application is run -- so that the application runs with the prepared DB.
Somehow I fail to implement the above two points.
What I have so far is this:
#SpringBootTest
#DirtiesContext(classMode = ClassMode.AFTER_CLASS)
#ActiveProfiles("specialtest")
public class MyAppTest {
#Autowired
private DbService dbService;
private static final Logger logger = LoggerFactory.getLogger(MyAppTest.class);
// The expectation is that this method is executed after the spring context
// has been loaded and all beans created, but before the Application class
// is executed.
#EventListener(ApplicationStartedEvent.class)
public void preparedDbForTheTest() {
// Create some records via dbService
logger.info("Created records for the test");
}
// This test is executed after the application has run. Here we check
// whether the DB contains the expected records.
#Test
public void testApplication() {
// Check the DB contents
}
}
My problem is that the the method preparedDbForTheTest does not seem to get executed at all.
According to the SpringBoot docs, the event ApplicationReadyEvent is sent exactly when I want to execute the setup code. But somehow the code is not executed.
If I annotate the method with #Before... (I tried several variants of it) then it gets executed, but after the Application class has run.
What am I doing wrong?
Test classes aren't Spring-managed beans so things like #EventListener methods will be ignored.
The most conventional solution to your problem would be to add some #TestConfiguration that declares the #EventListener:
#SpringBootTest
#DirtiesContext(classMode = ClassMode.AFTER_CLASS)
public class MyAppTest {
private static final Logger logger = LoggerFactory.getLogger(MyAppTest.class);
#Test
public void testApplication() {
}
#TestConfiguration
static class DatabasePreparation {
#EventListener(ApplicationStartedEvent.class)
public void preparedDbForTheTest() {
logger.info("Created records for the test");
}
}
}
A #TestConfiguration is additive so it'll be used alongside your application's main configuration. The preparedDbForTheTest method will now be called as part of refreshing the application context for the tests.
Note that, due to application context caching, this method won't be called for every test. It will only be called as part of refreshing the context which may then be shared among several tests.

How can I run a specific class / utility in a Spring Boot application with wiring?

I have my standard Spring Boot application working. I have situations where I want to run a "job" which is basically some specific method normally run via a user doing something in their browser but I want to run it from command line.
I'm able to run an arbitrary class with gradlew;
./gradlew -PmainClass=kcentral.backingservices.URLMetaExtractor execute
However when run this way none of the "autowiring" works. What is a better way to execute an arbitrary class (that has a main method) such that it also works with any Autowiring?
EDIT:
I got some advice to use a CommandLineRunner and some args, which work to execute the command via:
./gradlew bootRun -Pargs=--reloadTestData
However, the Autowiring of my Repo is failing. What I have is:
#EnableAutoConfiguration
#EnableMongoAuditing
#EnableMongoRepositories(basePackageClasses=KCItemRepo.class)
#ComponentScan(basePackages = {"kcentral"})
public class ReloadTestData implements CommandLineRunner {
#Autowired
AddItemService addItemService;
#Autowired
KCItemRepo itemRepo;
#Autowired
KCItemRatingRepo itemRatingRepo;
private static final Logger log = LoggerFactory.getLogger(ReloadTestData.class);
public void reloadData(){
log.info("reloadData and called");
if (itemRepo == null){
log.error("Repo not found");
return;
}
long c = itemRepo.count();
log.warn("REMOVING ALL items "+c);
itemRepo.deleteAll();
log.warn("REMOVING ALL ratings");
itemRatingRepo.deleteAll();
}
itemRepo is always null even though I wire the same way in my 'regular' spring boot app without an issue. What do I need to do to have it wire properly?
The fact that you say you want to run a "job" suggests that you might want to use a scheduled task within your application, rather than trying to run it through the command line. e.g. Scheduling tasks in Spring
#Scheduled(fixedRate = 5000)
public void reportCurrentTime() {
log.info("The time is now {}", dateFormat.format(new Date()));
}
If you want to make a command line application work with Autowiring, you can make a command line application by making your Application class implement the CommandLineRunner interface, e.g. Spring Boot Console App
#SpringBootApplication
public class SpringBootConsoleApplication
implements CommandLineRunner {
public static void main(String[] args) {
SpringApplication.run(SpringBootConsoleApplication.class, args);
}
#Override
public void run(String... args) {
}
}
And add spring.main.web-application-type=NONE to the properties file.
If you want to stop the application after running you can use SpringApplication.exit(ctx). Don't know about your auto-wiring problem though, maybe try printing out the list of available beans which might give some insight. Example:
#Component
public class DoThenQuit implements CommandLineRunner {
#Autowired
private ApplicationContext ctx;
#Override
public void run(String[] args) {
// do some other stuff before quitting
String[] beanNames = ctx.getBeanDefinitionNames();
Arrays.stream(beanNames).forEach(System.out::println);
// then quit the application
SpringApplication.exit(ctx);
}
}

Stateless Service Layer in Spring

These days im working on a Web project and i just want to clarify couple of things regarding Spring bean scopes and best practices for Spring based developments. Here i am using a scenario using a sample code
I have a Web Controller as below
#Controller
Public class JobController{
private JobService jobService;
#Autowired
public void setJobService(JobService jobService ) {
this.jobService = jobService ;
}
public void run(){
Job job = new Job();
-- Setting the properties for the Object
jobService.run(job);
}
}
Then I have the Service as below
#Service
Public class JobService {
public void run(Job job){
-- perform the business logic
}
}
In Here i want to make the JobService class stateless so i can define JobService as singleton hence reduce the unnecessary object creation. As per my understanding in-order make a class stateless we do not want to keep instance properties.In This scenario i pass different Job objects to the service. Does this make this JobService statefull because JObservice process different different job objects? Can you please help me to understand
Thanks,
Keth
Passing different objects does not make your service stateful.
Consider this for example.
#Service
Public class JobService {
private Job currentJob;
public void setJob(Job job) {
currentJob = job;
}
public void run(){
-- perform the business logic on currentJob
}
}
This would make the bean 'stateful' and cause unexplained behavior.
The execution of the method in your singleton by multiple controller/threads will not collide and can be assumed to be safe.

Use spring application as library in non-spring application

I implemented spring-boot application and now I want to use it as a lib for non-spring application.
How can I initialize lib classes so autowired dependencies work as expected?Obviously if I create class instance with 'new', all autowired dependencies will be null.
The theory is that you need to instantiate an application context for your Spring Boot dependency to live in, then extract a bean from there and make use of it.
In practice, in your Spring Boot dependency you should have an Application.java class or similar, in which a main method starts the application. Start by adding there a method like this:
public static ApplicationContext initializeContext(final String[] args) {
return SpringApplication.run(Application.class, args);
}
Next step, in you main application, when you see fit (I'd say during startup but might as well be the first time you need to use your dependency) you need to run this code:
final String[] args = new String[0]; // configure the Spring Boot app as needed
final ApplicationContext context = Application.initializeContext(args); // createSpring application context
final YourBean yourBean = (YourBean)context.getBean("yourBean"); // get a reference of your bean from the application context
From here you can use your beans as you see fit.
I'm not sure how you will handle to wait until the context is fully loaded before you try to access some beans from your Constructor etc. But if you just want to access context without creating components manually try one of those:
1) Simply Inject ApplicationContext
#Inject
private ApplicationContext context;
or
2) Implement ApplicationContextAware
public class ApplicationContext implements ApplicationContextAware {
private ApplicationContext context;
#Override
public void setApplicationContext(ApplicationContext context) {
this.context = context;
}
// just quick example, better to set it to your custom singleton class
public static ApplicationContext getContext() {
return context;
}
}
Then use context.getBean(SomeServiceFromLibrary.class);

Accessing Spring context from non-spring component that is loaded at the same time with Spring

The cool enterprise app I'm working on is in the process of going Spring. That's very cool and exciting exercise to all the team, but also a huge source of stress. What we do is we gradually move legacy components to Spring context. Now what we have is a huuuge, I mean it, huuuuge component that is not piece of cake to spring-ify, and at the same time it needs to get access to some of the Spring beans.
Now here comes the problem: this component is being loaded at application startup (or bootstrap, whatever you prefer!). That means that there is a race condition between this guy and a Spring itself, so sometimes when I access the context from within that non-spring monstrosity, I get sweet and nice NPE. Which basically means that at the time we need that context, it's not yet initialized!
You might be curious how exactly we're accessing the context: and the answer is - it's a standard AppContextProvider pattern.
public class ApplicationContextProvider implements ApplicationContextAware {
private static ApplicationContext ctx;
public void setApplicationContext(ApplicationContext applicationContext) {
ctx = applicationContext;
}
public static ApplicationContext getApplicationContext() {
return ctx;
}
}
The ideal workaround for me in this case would be to tell Spring to notify that non-spring component "Okay, I'm up!", and perform all actions that require the context only after that. Is this actually possible?
Thanks in advance!
The correct way to make the application context available to non-spring beans is to use the ContextSingletonBeanFactoryLocator.
Take a look at this answer for more details.
Take a look at the mechanism of context events.
Perhaps you can block getApplicationConext() until receiving of ContextRefreshedEvent (if it wouldn't create deadlocks):
public class ApplicationContextProvider implements ApplicationListener<ContextRefreshedEvent> {
private static ApplicationContext ctx;
private static Object lock = new Object();
public void onApplicationEvent(ContextRefreshedEvent e) {
synchronized (lock) {
ctx = e.getApplicationContext();
lock.notifyAll();
}
}
public static ApplicationContext getApplicationContext() {
synchronized (lock) {
while (ctx == null) lock.wait();
return ctx;
}
}
}

Resources