I have a Java-Configuration with this annotaions:
#EnableAsync
#EnableCaching
#Configuration
#EnableScheduling
#ComponentScan("XXX")
public class AppConfig implements AsyncConfigurer {
...
#Override
public Executor getAsyncExecutor() {
SimpleAsyncTaskExecutor executor = new SimpleAsyncTaskExecutor("Rm-");
executor.setConcurrencyLimit(50);
executor.setThreadPriority(Thread.MAX_PRIORITY);
return executor;
}
}
Then I have 3 Schedulers
/**
* Schedul on application start.
*/
#Transactional
#Scheduled(fixedDelay = Long.MAX_VALUE, initialDelay = 200)
public void initialReplicate() {
replicateFromTime(0l);
}
/**
* Every minute check for synchronization.
*/
#Transactional
#Scheduled(cron = "0 * * * * * ")
public void replicate() {
long minutesSinceTimeRecord = System.currentTimeMillis() / 1000 / 60;
replicateFromTime(minutesSinceTimeRecord);
}
#Transactional
#Scheduled(fixedDelay = 1000)
public void build() {
LOG.info("Check if somethings to build.");
}
What the replicate-from-time does:
It calculates if the day is over, if so, it replicate once per day (and once at appstart) the schema of a database.
For every table in the database one connection is created.
#Async
#Transactional
public Future<Void> readTable(TableModel table, String jdbcurl, Driver driver, Properties props, String schema,
Set<Throwable> errors, String catalogName) {
Connection connect = connectionService.connect(jdbcurl, errors, driver, props);
final DatabaseMetaData metadata;
try {
metadata = connect.getMetaData();
} catch (SQLException e) {
errors.add(e);
try {
connect.close();
} catch (SQLException e2) {
errors.add(e2);
}
return new AsyncResult<Void>(null);
}
...
There are 804 tables of any kind in the database (views, tables, temporary-tables, ...). For every table one connection is created. Makes 804 threads.
The database is on the machin as the app. I test from localhost.
Problem:
On the remote-server, on appstart it replicates successfully. But no other scheduler are executed anymore except the public void build()-Scheduler. But only 48 times (see the limit of 50 threads in the configuration).
From local-server, on appstart it replicates successfully. All 3 schedulers are working as expected.
The tomcat on the server should run all schedulers, but it runs only one. The tomcat on my local machine run all schedulers.
Confusion
I expected to have more problems in a local environment, connected to a remote server than inside the remote-server itself.
Question
Any idea why the remote server have problems but my local machine not?
Technical details
Local is a apache-tomcat-8.5.27.
Remote is a apache-tomcat-8.5.15.
Database to replicate schema is a postgres.
Related
I am experiencing an odd behavior of my spring boot websocket set-up.
Sometimes it works, sometimes it doesn't, it just feels random.
I have tried the several setups, none proved solid: I moved the last piece of code in a commandlinerunner inside the primary class of the application and the last choice was a different class with #Component annotation.
My setup is the following: I use a jdbc driver (pgjdbc-ng) to use the listen notify function of postgres.I have a function and a trigger that listens to a specific postgres table for inserations. If any occur, notifications are sent through the websocket. The other and is an angular app that uses ng2-stompjs to listen to /topic/notificari for notifications. I am not posting the code because the notifications don't get out of spring, the angular is not the problem.
Kind regards,
This is my WebSocketConfiguration
Override
public void configureMessageBroker(MessageBrokerRegistry registry) {
registry.enableSimpleBroker("/topic", "/queue", "/user", "/notificari");
registry.setApplicationDestinationPrefixes("/app");
registry.setUserDestinationPrefix("/user");
}
#Override
public void registerStompEndpoints(StompEndpointRegistry registry) {
registry.addEndpoint("/socket").setAllowedOrigins("*")
.setHandshakeHandler(new CustomHandshakeHandler());
}
I am using a class ListenNotify and the JDBC driver pgjdbc-ng to connect to the postgresql db and use listen notify functionality
public class ListenNotify {
private BlockingQueue queue = new ArrayBlockingQueue(20);
PGConnection connection;
public ListenNotify() {
PGNotificationListener listener = new PGNotificationListener() {
#Override
public void notification(int processId, String channelName, String payload) {
queue.add(payload);
}
};
try {
PGDataSource dataSource = new PGDataSource();
dataSource.setHost("localhost");
dataSource.setDatabase("db");
dataSource.setPort(5432);
dataSource.setUser("user");
dataSource.setPassword("pass");
connection = (PGConnection) dataSource.getConnection();
connection.addNotificationListener(listener);
Statement statement = connection.createStatement();
statement.execute("LISTEN n_event");
statement.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
public BlockingQueue getQueue() {
return queue;
}
}
And finally this is the code that instantiate the ListenNotify object and listens to postgres for events that might trigger notifications that have to be send using websocket.
#Component
public class InstantaNotificari {
#Autowired
SimpMessagingTemplate template;
#EventListener(ApplicationReadyEvent.class)
public void runn() {
System.out.println("invocare met");
ListenNotify ln = new ListenNotify();
BlockingQueue queue = ln.getQueue();
System.out.println("the que ies "+ queue);
while (true) {
try {
String msg = (String) queue.take();
System.out.println("msg " + msg);
template.convertAndSend("/topic/notificari", msg);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
I didn't use Spring so I can't test your code. Here is my tested version. I think this summarizes the differences -
Change to a try with resources block. This will close the connection on destruction of the class.
Move your while(true) into the try block on the Listener so that the
lines inside the try block doesn't ever get out of execution scope.
The while(true) is blocking, so it needs to be on another thread. ListenNotify extends Thread
I'm sure there are other ways of implementing and welcome corrections to any of my assumptions.
My tested, running code is in this answer JMS Websocket delayed delivery.
I have created spring boot thread pool project which has thread that needs to run 24x7 once spawned but when I need to stop the app in server for some maintenance it should shutdown after completing its current task and not taking up any new task.
My code for the same is:
Config class
#Configuration
public class ThreadConfig {
#Bean
public ThreadPoolTaskExecutor taskExecutor(){
ThreadPoolTaskExecutor executorPool = new ThreadPoolTaskExecutor();
executorPool.setCorePoolSize(10);
executorPool.setMaxPoolSize(20);
executorPool.setQueueCapacity(10);
executorPool.setWaitForTasksToCompleteOnShutdown(true);
executorPool.setAwaitTerminationSeconds(60);
executorPool.initialize();
return executorPool;
}
}
Runnable class
#Component
#Scope("prototype")
public class DataMigration implements Runnable {
String name;
private boolean run=true;
public DataMigration(String name) {
this.name = name;
}
#Override
public void run() {
while(run){
System.out.println(Thread.currentThread().getName()+" Start Thread = "+name);
processCommand();
System.out.println(Thread.currentThread().getName()+" End Thread = "+name);
if(Thread.currentThread().isInterrupted()){
System.out.println("Thread Is Interrupted");
break;
}
}
}
private void processCommand() {
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
public void shutdown(){
this.run = false;
}
}
Main class:
#SpringBootApplication
public class DataMigrationPocApplication implements CommandLineRunner{
#Autowired
private ThreadPoolTaskExecutor taskExecutor;
public static void main(String[] args) {
SpringApplication.run(DataMigrationPocApplication.class, args);
}
#Override
public void run(String... arg0) throws Exception {
for(int i = 1; i<=20 ; i++){
taskExecutor.execute(new DataMigration("Task " + i));
}
for (;;) {
int count = taskExecutor.getActiveCount();
System.out.println("Active Threads : " + count);
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
e.printStackTrace();
}
if (count == 0) {
taskExecutor.shutdown();
break;
}
}
System.out.println("Finished all threads");
}
}
I need help to understand if I need to stop my spring boot application it should stop all the 20 threads running which runs (24x7) otherwise after completing there current loop in while loop and exit.
I would propose couple of changes in this code to resolve the problem
1) since in your POC processCommand calls Thread.sleep, when you shutdown the executor and it interrupts workers InterruptedException get called but is almost ignored in your code. After that there is if(Thread.currentThread().isInterrupted()) check which will return false for the reason above. Similar problem is outlined in the post below
how does thread.interrupt() sets the flag?
the following code change should fix the problem:
private void processCommand() {
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
shutdown();
}
}
2) Also because of ThreadConfig::taskExecutor executorPool.setWaitForTasksToCompleteOnShutdown(true) Spring will call executor.shutdown instead of executor.shutdownNow. According to javadoc ExecutorService.shutdown
Initiates an orderly shutdown in which previously submitted tasks are
executed, but no new tasks will be accepted.
So I would recommend to set
executorPool.setWaitForTasksToCompleteOnShutdown(false);
Other things to improve in this code: although DataMigration is annotated as a component the instances of this class are creared not by Spring. You should try using factory method similar to ThreadConfig::taskExecutor in order to make Spring initiate instances of DataMigration for example to inject other bean into DataMigration instances.
In order to shutdown executor when running jar file on linux environment you can for example add actuator module and enable shutdown endpoint:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
in application.properties:
endpoints.shutdown.enabled=true
It will enable JMX shutdown endpoint and you can call shutdown on it.
If you want current job cycle of the task to be finished you should set
executorPool.setWaitForTasksToCompleteOnShutdown(true);
In order to connect to your jvm process on linux env remotely you have to specify an RMI Registry port.
Here is a detailed article:
How to access Spring-boot JMX remotely
If you just need to connect to JMX from local env you can run jsoncole or command-line tools : Calling JMX MBean method from a shell script
Here is an example uf using one of these tools - jmxterm
$>run -d org.springframework.boot: -b org.springframework.boot:name=shutdownEndpoint,type=Endpoint shutdown
#calling operation shutdown of mbean org.springframework.boot:name=shutdownEndpoint,type=Endpoint with params []
#operation returns:
{
message = Shutting down, bye...;
}
I am trying o schedule a method call. I want to schedule this method call as soon as server starts and then after every 30 seconds.
Below code:
#Configuration
#EnableScheduling
#EnableTransactionManagement
public class Schedular implements SchedulingConfigurer {
#Override
public void configureTasks(ScheduledTaskRegistrar taskRegistrar) {
taskRegistrar.setScheduler(poolScheduler());
taskRegistrar.addTriggerTask(new Runnable() {
#Override
public void run() {
testScheduling();
}
}, new Trigger() {
#Override
public Date nextExecutionTime(TriggerContext triggerContext) {
Calendar nextExecutionTime = Calendar.getInstance();
nextExecutionTime.add(Calendar.SECOND, <some value from database>);
return nextExecutionTime.getTime();
}
});
}
#Bean
public TaskScheduler poolScheduler() {
ThreadPoolTaskScheduler scheduler = new ThreadPoolTaskScheduler();
scheduler.setThreadNamePrefix("poolScheduler");
scheduler.setPoolSize(10);
return scheduler;
}
public void testScheduling(){
System.out.println("Scheduling Testing");
}
}
The code below schedule the method code after 30 seconds after the server started BUT NOT just after server started. I know I need to do some other config to schedule the method call just after server start and then after every 30 seconds (or whatever time I want to).
I am using spring boot. Could anyone please suggest.
Also, is it possible to get both initial and fixeddelay/fixedrate value from database. I want to set the initial value as well from database
Thanks in advance.
let me know if this worked for you
#PostConstruct
#Scheduled(fixedDelay=30000)
public void testScheduling(){
System.out.println("Scheduling Testing");
}
Use the #PostConstuct annotation to start the method after the application starts.
You can use like below. I had used Spring Boot version v2.2.7
#Scheduled(fixedRateString = "${echo.interval(milliseconds)}", initialDelayString = "${echo.initialDelay(milliseconds)}")
The properties should be mentioned in "application.properties" file for the Spring Boot to detect and inject the values of the fixed rate and initial delay into the Scheduler.
I use #Schedule to have the system run tasks at different time. But recently, I found that some of the tasks are being postponed. For example, there is one task which should run on 5:00 every morning, now runs sometimes in the noon or even later in the afternoon. Any ideas? Thanks.
Scheduler service:
#Service
public class DailyReminderTasks extends AbstractTask {
private final static Logger logger = LoggerFactory
.getLogger(DailyReminderTasks.class);
private SendAuditReminderNotificationSerivce sendAuditReminderNotificationSerivce;
#Scheduled(cron = "0 0 5 * * ?")
// run at 5:00 am every day
public void sendAuditReponseReminderLetter() {
try {
sendAuditReminderNotificationSerivce.sendAuditReponseReminder();
} catch (Exception ex) {
logger.error("failed to send reminder: ", ex);
}
}
public SendAuditReminderNotificationSerivce getSendAuditReminderNotificationSerivce() {
return sendAuditReminderNotificationSerivce;
}
#Autowired(required = true)
public void setSendAuditReminderNotificationSerivce(
SendAuditReminderNotificationSerivce sendAuditReminderNotificationSerivce) {
this.sendAuditReminderNotificationSerivce = sendAuditReminderNotificationSerivce;
}
With the information available here, the most likely explanation is that the background thread pool handling task scheduling does not keep up and thus between the time the task is put on the queue and the time it is executed, many hours can pass.
I have a spring boot application which is using gradle as build tool and jooq for dao class generation and db connection. Previously my application was connecting to single mysql instance. Below are the configuration we used for connecting to single db instance:
spring.datasource.username=user
spring.datasource.password=password
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.datasource.name=ds_name
spring.datasource.schema=ds_schema
spring.jooq.sql-dialect=MYSQL
Current project structure is
a) Main application project MainApp having application.properties with above key-value pairs.
b) Separate application project as DBProject which has jooq's generated DAO classes. MainApp include DBProject as a jar.
I am using gradle as build tool for this.
Everything is working fine till here. But now I have to connect to one more instance of MySQL. So, I have created another db project as DBProject2 which also contains dao classes generated by jooq using another mysql schema. I have created DBProject2 exactly as DBProject is created.
Now, my question is if I include both DBProjects in MainApp as jar then both will use same db configuration as in application.properties. How I can make separate db jars to point to their respective db schemas. I googled alot about this but couldn't find helpful solution.
This is what I do to connect to multiple (additional) data sources in my Play app. I am not sure if it is the best approach, but it works great for me. I have changed names below to be generic.
// In my application.conf
// default data source
db.default.driver=com.mysql.jdbc.Driver
db.default.url="jdbc:mysql://localhost:3306/myDb?useSSL=false"
db.default.username=myuser
db.default.password="mypassword"
// additional data source
db.anothersource.driver=com.mysql.jdbc.Driver
db.anothersource.url="jdbc:mysql://localhost:3306/myothersource?useSSL=false"
db.anothersource.username=myuser
db.anothersource.password="mypassword"
// Then in Java, I create a JooqContextProvider class to expose both connections.
public class JooqContextProvider {
#Inject
Database db;
#Inject
play.Configuration config;
public JooqContextProvider(){}
/**
* Creates a default database connection for data access.
* #return DSLConext.
*/
public DSLContext dsl() {
return DSL.using(new JooqConnectionProvider(db), SQLDialect.MYSQL);
}
/**
* Creates an anothersource database connection for data access.
* #return DSLConext for anothersource.
*/
public DSLContext anotherDsl() {
return DSL.using(
new JooqAnotherSourceConnectionProvider(
config.getString("db.anothersource.url"),
config.getString("db.anothersource.username"),
config.getString("db.anothersource.password")),
SQLDialect.MYSQL);
}
}
// Then I needed to implement my JooqAnotherSourceConnectionProvider
public class JooqAnotherSourceConnectionProvider implements ConnectionProvider {
private Connection connection = null;
String url;
String username;
String password;
public JooqAnotherSourceConnectionProvider(String url, String username, String password){
this.url = url;
this.username = username;
this.password = password;
}
#Override
public Connection acquire() throws DataAccessException {
try {
connection = DriverManager.getConnection(url, username, password);
return connection;
}
catch (java.sql.SQLException ex) {
throw new DataAccessException("Error getting connection from data source", ex);
}
}
#Override
public void release(Connection releasedConnection) throws DataAccessException {
if (connection != releasedConnection) {
throw new IllegalArgumentException("Expected " + connection + " but got " + releasedConnection);
}
try {
connection.close();
connection = null;
}
catch (SQLException e) {
throw new DataAccessException("Error closing connection " + connection, e);
}
}
}
// Then in Java code where I need to access one or the other data sources...
jooq.dsl().select().from().where()...
jooq.anotherDsl().select().from().where()...