Why does resque-scheduler try to run a job that used to exist, but no longer does?
I used to have an old resque-scheduler job a few weeks ago, but have removed it:
EmailRelevantQuestions:
every: 15m
class: EmailRelevantQuestions
queue: email_queue
description: "Iterates through users and emails them relevant questions."
But for some reason this job is still firing every 15 minutes, and the class no longer exists in my codebase, or my scheduler?? My logs state:
2013-09-18 15:18:42.267177+00:00 app scheduler.1 - - 2013-09-18 10:18:42 queueing EmailRelevantQuestions (EmailRelevantQuestions)
2013-09-18 15:18:42.600357+00:00 app scheduler.1 - - 2013-09-18 10:18:42 queueing FrequentWorker (FrequentWorker)
2013-09-18 15:18:43.817671+00:00 app resque.1 - - *** Checking daily_queue
2013-09-18 15:18:43.820166+00:00 app resque.1 - - *** Checking email_queue
2013-09-18 15:18:43.826799+00:00 app resque.1 - - *** Found job on email_queue
2013-09-18 15:18:43.826914+00:00 app resque.1 - - *** got: (Job{email_queue} | EmailRelevantQuestions | [])
2013-09-18 15:18:43.832159+00:00 app resque.1 - - *** resque-1.25.0: Processing email_queue since 1379517523 [No Name]
2013-09-18 15:18:43.933346+00:00 app resque.1 - - *** Running after_fork hooks with [(Job{email_queue} | EmailRelevantQuestions | [])]
2013-09-18 15:18:43.944943+00:00 app resque.1 - - *** (Job{email_queue} | EmailRelevantQuestions | []) ***FAILED***: #<NameError: uninitialized constant EmailRelevantQuestions>
Is there some sort of cache?
ps. I recently updated my resque form 1.24 to 1.25...not sure if this caused the issue
It looks like this was causing old jobs to come up from past creation:
Resque::Scheduler.dynamic = true
Related
I run my app in Xcode through the simulator and it builds with no crashes. When I run on real device it successfully builds but I get crash errors. Someone suggested it might be the cordova screen lock but I tried it without and still have problems.
Running on real device: a blinking screen or it keeps reloading the first screen, never to make it to the next one. Please help!
Here are the different lines I am seeing when compared to the simulator:
Metal API Validation Enabled
<UMP SDK> To enable debug mode for this device, set: UMPDebugSettings.testDeviceIdentifiers = #[ #"___________________________________" ];
THREAD WARNING: ['CDVOrientation'] took '44.751953' ms. Plugin should use a background thread.
[Process] 0x10f003740 - [PID=1655] WebProcessProxy::didClose: (web process 0 crash)
[Process] 0x10f003740 - [PID=1655] WebProcessProxy::processDidTerminateOrFailedToLaunch: reason=4
[ProcessSuspension] 0x10e0042a0 - ProcessAssertion: Failed to acquire RBS Background assertion 'ConnectionTerminationWatchdog' for process because PID 0 is invalid
[ProcessSuspension] 0x10e0042a0 - ProcessAssertion::acquireSync Failed to acquire RBS assertion 'ConnectionTerminationWatchdog' for process with PID=0, error: (null)
0x150842818 - [pageProxyID=13, webPageID=14, PID=1655] WebPageProxy::processDidTerminate: (pid 1655), reason 4
[assertion] Error acquiring assertion: <Error Domain=RBSServiceErrorDomain Code=1 "target is not running or doesn't have entitlement com.apple.runningboard.assertions.webkit" UserInfo={NSLocalizedFailureReason=target is not running or doesn't have entitlement com.apple.runningboard.assertions.webkit}>
[ProcessSuspension] 0x10e004300 - ProcessAssertion::acquireSync Failed to acquire RBS assertion 'WebProcess Background Assertion' for process with PID=1655, error: Error Domain=RBSServiceErrorDomain Code=1 "target is not running or doesn't have entitlement com.apple.runningboard.assertions.webkit" UserInfo={NSLocalizedFailureReason=target is not running or doesn't have entitlement com.apple.runningboard.assertions.webkit}
[Loading] 0x150842818 - [pageProxyID=13, webPageID=14, PID=1655]
WebPageProxy::dispatchProcessDidTerminate: reason=Crash
[XPC] Handle connection with error: Connection interrupted
[Process] 0x10f004110 - [PID=0] WebProcessProxy::didFinishLaunching: Invalid connection identifier (web process failed to launch)
[Process] 0x10f004110 - [PID=0] WebProcessProxy::processDidTerminateOrFailedToLaunch: reason=4
[Process] 0x150842818 - [pageProxyID=13, webPageID=14, PID=0] WebPageProxy::processDidTerminate: (pid 0), reason 4
[ServicesDaemonManager] interruptionHandler is called. -[FontServicesDaemonManager connection]_block_invoke
[Process] 0x10f0048f0 - [PID=1682] WebProcessProxy::didClose: (web process 0 crash)
[Process] 0x10f0048f0 - [PID=1682] WebProcessProxy::processDidTerminateOrFailedToLaunch: reason=4
THREAD WARNING: ['AdMob'] took '30.550049' ms. Plugin should use a background thread.
THREAD WARNING: ['Consent'] took '10.710205' ms. Plugin should use a background thread.
We saw this issue on Xcode 14.2 where web views were not loading.
Our WKWebViews had Content Rule List set.
The root cause was it compiled the Content Rule List to the underlying chipset (non-Intel), instead of respecting the Rosetta.
This issue was not seen when Xcode was launched without Rosetta.
I integrated apartment-sidekiq with my multi tenant rails app, but its logging following issue in sidekiq.log. Custom emails are perfectly handled but seems its issue with active_storage job, why is it trying to find the directory which doesn't exists? I manually checked the directory and its not there, (sometimes, worker try to find a record before it is saved, seems thats not the case here), Is it trying to find a path in other tenant?.
I am using Rails 5.2, ruby 2.3.1
2019-08-15T06:58:14.857Z 12629 TID-137g5x ActiveStorage::AnalyzeJob JID-2555d469bbee6a7b618446d3 INFO: start
2019-08-15T06:58:15.968Z 12629 TID-137g5x ActiveStorage::AnalyzeJob JID-2555d469bbee6a7b618446d3 INFO: fail: 1.121 sec
2019-08-15T06:58:15.973Z 12629 TID-137g5x WARN: {"context":"Job raised exception","job":{"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"ActiveStorage::AnalyzeJob","queue":"default","args":[{"job_class":"ActiveStorage::AnalyzeJob","job_id":"30a39eff-b93f-41b3-8d0f-aae6e070c4c7","provider_job_id":null,"queue_name":"default","priority":null,"arguments":[{"_aj_globalid":"gid://flap/ActiveStorage::Blob/5"}],"executions":0,"locale":"en"}],"retry":true,"jid":"2555d469bbee6a7b618446d3","created_at":1565789876.7082293,"apartment":"tenant_name","enqueued_at":1565852294.8079991,"error_message":"No such file or directory # rb_sysopen - /abc/flapp-cap/releases/20190814172709/storage/Ma/xS/MaxSwEqwUMV5WFG35Kz7qMLy","error_class":"Errno::ENOENT","failed_at":1565789876.718125,"retry_count":12,"retried_at":1565831257.7109427},"jobstr":"{\"class\":\"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper\",\"wrapped\":\"ActiveStorage::AnalyzeJob\",\"queue\":\"default\",\"args\":[{\"job_class\":\"ActiveStorage::AnalyzeJob\",\"job_id\":\"30a39eff-b93f-41b3-8d0f-aae6e070c4c7\",\"provider_job_id\":null,\"queue_name\":\"default\",\"priority\":null,\"arguments\":[{\"_aj_globalid\":\"gid://flapp/ActiveStorage::Blob/5\"}],\"executions\":0,\"locale\":\"en\"}],\"retry\":true,\"jid\":\"2555d469bbee6a7b618446d3\",\"created_at\":1565789876.7082293,\"apartment\":\"tenant_name\",\"enqueued_at\":1565852294.8079991,\"error_message\":\"No such file or directory # rb_sysopen - /abc/flapp-bc/cap/releases/20190814172709/storage/Ma/xS/MaxSwEqwUMV5WFG35Kz7qMLy\",\"error_class\":\"Errno::ENOENT\",\"failed_at\":1565789876.718125,\"retry_count\":12,\"retried_at\":1565831257.7109427}"}
2019-08-15T06:58:15.973Z 12629 TID-137g5x WARN: Errno::ENOENT: No such file or directory # rb_sysopen - /abc/flapp/storage/cW/J7/cWJ7vKtzyfDNMHmtYV8ckErR
I'm using Pacemaker + Corosync in Centos7
When one of my resource failed/stopped I/m getting a failed action message:
Master/Slave Set: myoptClone01 [myopt_data01]
Masters: [ pcmk01-cr ]
Slaves: [ pcmk02-cr ]
myopt_fs01 (ocf::heartbeat:Filesystem): Started pcmk01-cr
myopt_VIP01 (ocf::heartbeat:IPaddr2): Started pcmk01-cr
ServicesResource (ocf::heartbeat:RADviewServices): Started pcmk01-cr
Failed Actions:
* ServicesResource_monitor_120000 on pcmk02-cr 'unknown error' (1): call=141, status=complete, exitreason='none',
last-rc-change='Mon Jan 30 10:19:36 2017', queued=0ms, exec=142ms
Is there a way to clean automatically the failed actions after X sec/min/hour?
Look into the 'failure-timeout' resource option. This will automatically cleanup the failed action if no further failures for the particular resource has occurred within the value of failure-timeout.
I believe the failure-timeout is calculated during the cluster-recheck-interval. Which means that even if you have the failure-timeout configured to 1 minute it may still take up to 15 minutes and 59 seconds to clear the failed action with Pacemaker's default 15 minute cluster-recheck-interval.
More information:
http://clusterlabs.org/doc/en-US/Pacemaker/1.1/html/Pacemaker_Explained/s-failure-migration.html
http://clusterlabs.org/doc/en-US/Pacemaker/1.1/html/Pacemaker_Explained/s-resource-options.html
I'm trying to use sidekiq to process some requests in background (it's integrated with padrino framework).
My expectation was that once the worker got called, the request handler would return immediately the answer to user.
In order to be sure it was working properly I implemented a worker that would print some messages and sleep for some 44s prior to finish it's processing (to simulate a long processing in background).
For my surprise, the request got stuck until the worker had the job finished. Only after that the request handler could return an answer to user.
At first I thought that the sleep function could be the problem, so I replaced the sleep function by a busy while but I have the same behavior: the request handler got hanged until the worker finished its task.
Any idea why is this happening?
You can see the following :
request handler:
get :hardworker, map: '/hardworker' do
logger.info "I'll call HardWorker"
HardWorker.perform_async(44)
logger.info "HardWorker was called"
return "The job should still be running in background."
end
Sidekiq worker:
class HardWorker
include Sidekiq::Worker
def perform(count)
logger.info "Doing hard work"
Sidekiq.redis { |c| logger.info "redis location: [#{c.client.location }]" }
redis_info = Sidekiq.redis { |conn| conn.info }
logger.info "connected clients: [#{redis_info['connected_clients']}]"
sleep count
logger.info "hard work was done"
end
end
The redis server is running:
ps -ef | grep redis
returns
redis 1232 1 0 16:54 ? 00:00:09 /usr/bin/redis-server 127.0.0.1:6379
as well as sidekiq server:
bundle exec sidekiq -r ./config/boot.rb -v
2015-06-06T20:31:26.207Z 3343 TID-8614g INFO: Booting Sidekiq 3.3.4 with redis options {:url=>"redis://127.0.0.1:6379/0", :concurrency=>25}
Also, from logs I put in the worker we can see that client apparently is connected to redis:
INFO - redis location: [127.0.0.1:6379]
INFO - connected clients: [3]
sidekiq version: 3.3.4
redis server: 2.8.4
ruby: 2.1.2p95
Note: I mounted sidekiq web tool on my padrino and it shows 0 for all stats (Processed, Failed, Busy, Enqueued, Retries, Scheduled, Dead) both prior and after the worker has executed.
The sample project can be downloaded from here: https://github.com/plicatibu/sidekiq-problem.git
Well I found your bug, in your boot.rb file you have this line:
require 'sidekiq/testing/inline' if RACK_ENV == 'development'
This bit of code, uses Sidekiq's Testing framework which bypasses redis and runs it almost as if it's a ruby class (ie it doesn't ever get queued on redis). Remove that line and only use it in your test suite (if you need it).
You'll also need to remove that pid file from your YML file, you don't need that and if you do, it's probably only for production. My 2cents - remove it.
Once you do that, you're good to go:
Here's your rackup with a get request to /hardworker:
[2015-06-07 07:54:25] INFO WEBrick 1.3.1
[2015-06-07 07:54:25] INFO ruby 2.2.2 (2015-04-13) [x86_64-darwin14]
[2015-06-07 07:54:25] INFO WEBrick::HTTPServer#start: pid=74128 port=9292
INFO - I'll call HardWorker
INFO - HardWorker was called
DEBUG - GET (0.0131s) /hardworker - 200 OK
::1 - - [07/Jun/2015:07:55:32 -0400] "GET /hardworker HTTP/1.1" 200 46 0.0291
Here's sidekiq processing that job in the background:
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
ss
sss sss ss
s sss s ssss sss ____ _ _ _ _
s sssss ssss / ___|(_) __| | ___| | _(_) __ _
s sss \___ \| |/ _` |/ _ \ |/ / |/ _` |
s sssss s ___) | | (_| | __/ <| | (_| |
ss s s |____/|_|\__,_|\___|_|\_\_|\__, |
s s s |_|
s s
sss
sss
INFO - Running in ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-darwin14]
INFO - See LICENSE and the LGPL-3.0 for licensing details.
INFO - Upgrade to Sidekiq Pro for more features and support: http://sidekiq.org/pro
INFO - Starting processing, hit Ctrl-C to stop
DEBUG -
INFO -
INFO - Doing hard work
INFO - redis location: [127.0.0.1:6379]
INFO - connected clients: [3]
INFO - hard work was done
INFO -
I'm just trying out play framework 2.0 on my mac. I've been using Play 1.x for a while without any issues.
I've downloaded the new framework, updated my PATH to point to the new version of play and created a simple Java project ('play new testapp')
When I call play run I get the following error:
log4j:WARN No appenders could be found for logger (akka.event.slf4j.Slf4jEventHandler).
log4j:WARN Please initialize the log4j system properly.
[error] java.lang.NoSuchMethodError: org.slf4j.spi.LocationAwareLogger.log(Lorg/slf4j/Marker;Ljava/lang/String;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V
[error] Use 'last' for the full log.
As per the error above I've run the 'last' command and below is an extract from the stack trace that is produced from this:
java.lang.NoSuchMethodError: org.slf4j.spi.LocationAwareLogger.log(Lorg/slf4j/Marker;Ljava/lang/String;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V
at org.slf4j.bridge.SLF4JBridgeHandler.callLocationAwareLogger(SLF4JBridgeHandler.java:203)
at org.slf4j.bridge.SLF4JBridgeHandler.publish(SLF4JBridgeHandler.java:278)
at java.util.logging.Logger.log(Logger.java:478)
at java.util.logging.Logger.doLog(Logger.java:500)
at java.util.logging.Logger.logp(Logger.java:616)
at org.jboss.netty.logging.JdkLogger.debug(JdkLogger.java:36)
at org.jboss.netty.logging.InternalLoggerFactory$1.debug(InternalLoggerFactory.java:80)
at org.jboss.netty.channel.socket.nio.NioProviderMetadata.<clinit>(NioProviderMetadata.java:95)
at org.jboss.netty.channel.socket.nio.NioWorker.<clinit>(NioWorker.java:58)
at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.<init>(NioServerSocketPipelineSink.java:58)
at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:133)
at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:104)
at play.core.server.NettyServer.<init>(NettyServer.scala:43)
I fired up an Ubuntu VM and completed the same process and the app runs fine.
Any suggestions?
EDIT: I ran play classpath to get my classpath for the app and this was the output (replaced the common part of the directory with ...):
Here is the computed classpath of your application:
- .../testapp/target/scala-2.9.1/classes
- ...play-latest/framework/sbt/boot/scala-2.9.1/lib/scala-library.jar
- ...play-latest/framework/../repository/local/play/play_2.9.1/2.0/jars/play_2.9.1.jar
- ...play-latest/framework/../repository/local/play/templates_2.9.1/2.0/jars/templates_2.9.1.jar
- ...play-latest/repository/local/com.github.scala-incubator.io/scala-io-file_2.9.1/0.2.0/jars/scala-io-file_2.9.1.jar
- ...play-latest/repository/local/com.github.scala-incubator.io/scala-io-core_2.9.1/0.2.0/jars/scala-io-core_2.9.1.jar
- ...play-latest/repository/local/com.github.jsuereth.scala-arm/scala-arm_2.9.1/0.3/jars/scala-arm_2.9.1.jar
- ...play-latest/framework/../repository/local/play/anorm_2.9.1/2.0/jars/anorm_2.9.1.jar
- ...play-latest/framework/../repository/local/io.netty/netty/3.3.0.Final/bundles/netty.jar
- ...play-latest/framework/../repository/local/org.slf4j/slf4j-api/1.6.4/jars/slf4j-api.jar
- ...play-latest/framework/../repository/local/org.slf4j/jul-to-slf4j/1.6.4/jars/jul-to-slf4j.jar
- ...play-latest/framework/../repository/local/org.slf4j/jcl-over-slf4j/1.6.4/jars/jcl-over-slf4j.jar
- ...play-latest/framework/../repository/local/ch.qos.logback/logback-core/1.0.0/jars/logback-core.jar
- ...play-latest/framework/../repository/local/ch.qos.logback/logback-classic/1.0.0/jars/logback-classic.jar
- ...play-latest/framework/../repository/local/com.typesafe.akka/akka-actor/2.0/jars/akka-actor.jar
- ...play-latest/framework/../repository/local/com.typesafe.akka/akka-slf4j/2.0/jars/akka-slf4j.jar
- ...play-latest/framework/../repository/local/com.google.guava/guava/10.0.1/jars/guava.jar
- ...play-latest/framework/../repository/local/com.google.code.findbugs/jsr305/1.3.9/jars/jsr305.jar
- ...play-latest/framework/../repository/local/org.avaje/ebean/2.7.3/jars/ebean.jar
- ...play-latest/framework/../repository/local/org.hibernate.javax.persistence/hibernate-jpa-2.0-api/1.0.1.Final/jars/hibernate-jpa-2.0-api.jar
- ...play-latest/framework/../repository/local/com.h2database/h2/1.3.158/jars/h2.jar
- ...play-latest/framework/../repository/local/org.scala-tools/scala-stm_2.9.1/0.4/jars/scala-stm_2.9.1.jar
- ...play-latest/framework/../repository/local/com.jolbox/bonecp/0.7.1.RELEASE/bundles/bonecp.jar
- ...play-latest/framework/../repository/local/org.yaml/snakeyaml/1.9/bundles/snakeyaml.jar
- ...play-latest/framework/../repository/local/org.hibernate/hibernate-validator/4.2.0.Final/jars/hibernate-validator.jar
- ...play-latest/framework/../repository/local/javax.validation/validation-api/1.0.0.GA/jars/validation-api.jar
- ...play-latest/framework/../repository/local/org.springframework/spring-context/3.0.7.RELEASE/jars/spring-context.jar
- ...play-latest/framework/../repository/local/org.springframework/spring-core/3.0.7.RELEASE/jars/spring-core.jar
- ...play-latest/framework/../repository/local/org.springframework/spring-beans/3.0.7.RELEASE/jars/spring-beans.jar
- ...play-latest/framework/../repository/local/joda-time/joda-time/2.0/jars/joda-time.jar
- ...play-latest/framework/../repository/local/org.joda/joda-convert/1.1/jars/joda-convert.jar
- ...play-latest/framework/../repository/local/javassist/javassist/3.12.1.GA/jars/javassist.jar
- ...play-latest/framework/../repository/local/commons-lang/commons-lang/2.6/jars/commons-lang.jar
- ...play-latest/framework/../repository/local/com.ning/async-http-client/1.7.0/jars/async-http-client.jar
- ...play-latest/framework/../repository/local/oauth.signpost/signpost-core/1.2.1.1/jars/signpost-core.jar
- ...play-latest/framework/../repository/local/commons-codec/commons-codec/1.3/jars/commons-codec.jar
- ...play-latest/framework/../repository/local/com.codahale/jerkson_2.9.1/0.5.0/jars/jerkson_2.9.1.jar
- ...play-latest/framework/../repository/local/org.codehaus.jackson/jackson-core-asl/1.9.5/jars/jackson-core-asl.jar
- ...play-latest/framework/../repository/local/org.codehaus.jackson/jackson-mapper-asl/1.9.5/jars/jackson-mapper-asl.jar
- ...play-latest/framework/../repository/local/org.reflections/reflections/0.9.6/jars/reflections.jar
- ...play-latest/framework/../repository/local/javax.servlet/javax.servlet-api/3.0.1/jars/javax.servlet-api.jar
- ...play-latest/framework/../repository/local/javax.transaction/jta/1.1/jars/jta.jar
- ...play-latest/framework/../repository/local/tyrex/tyrex/1.0.1/jars/tyrex.jar
- ...play-latest/framework/../repository/local/jaxen/jaxen/1.1.3/jars/jaxen.jar
- ...play-latest/framework/../repository/local/net.sf.ehcache/ehcache-core/2.5.0/jars/ehcache-core.jar