I'm attempting to resolve this weird problem with our OS X application.
As part of it's normal operation, it starts a long running background process — specifically a Web server, written in node.js.
After some time running, the background process shows up in the activity monitor as "Not Responding".
It's still functioning normally though, responding to requests from the web browser and not eating a high percentage of CPU or anything.
I would like to understand what exactly might be causing this status to be displayed, since it's not even a GUI application in the first place. What exactly does OS X use to determine if a command line application is not responding, if it doesn't have a Cocoa event loop?
Is it possible to flag it somehow as a background process so that the system does not query it for responses?
Thanks in advance.
Update: Just checked, the same seems to happen when launching the task from the command line. What makes a background app "Not respond" on the activity monitor?
Okay so as sergio pointed out in his comment, it was indeed something specific to Node.JS.
The dev responsible for Node.JS portion of our app was able to trace back the source of the problem to the usage of the process.title property in Node.JS.
It looks like the implementation of process.title in libuv does a lot of Core Foundation tricks to set the process title, so these are probably getting the process flagged as a GUI app by whatever heuristics the Activity Monitor uses to tell Cocoa apps apart from CLI apps.
We have filed a bug report on libuv about this, but for now we're skipping process.title as simply setting it causes the server to show up as "not responding" on the Activity Monitor after a couple minutes.
Update: This is now fixed in libuv as of this commit. It still hasn't propagated to a node release yet.
According to Apple's Process Manager Reference, GetCurrentProcess, GetFrontProcess (et al.) are deprecated in OS X 10.9 and causes the "not responding" issue in Activity Monitor (my observation).
Related
API Monitor has a feature to automatically watch for a new process starting and ask if you want to monitor it. However I have not been able to get this to actually work. The only option in the program I can find which seems to be related is the File menu "Pause Process Notifications" option. However, this is disabled which gives me the impressions that it can't be turned off but also that it is supposed to work automatically "out of the box". But whenever I start a new process, nothing happens.
Specifically I'm referring to the feature described here:
Process Notification
API Monitor intercepts process creation and allows you to select the
process for monitoring. Each time a process is created by the system,
a notification window is displayed with options to monitor, skip or
terminate the process. This is especially useful for monitoring
processes with a short lifespan or processes that are automatically
launched in the background. Process Notification can also be used to
monitor applications such as consent.exe (UAC prompt), which run on a
different desktop.
The following screenshot shows an example of the Process Notification
window that is displayed when launching an application that requires
elevation
I've tried both the 32-bit and 64-bit versions of API Monitor (Version 2.0 Alpha-r13) running both as regular user and as admin; makes no difference.
How can this feature be activated?
The specific reason I'd like to use this feature is that I have process A which starts process B, and it is B I need to monitor. A and B each run for only a few seconds so I can't manually get it to monitor fast enough.
Finally after reading through API Monitor forums I found some information. Unfortunately (for now at least) it seems the answer is that this feature no longer works (since Windows 8.1).
As posted on http://www.rohitab.com/discuss/topic/40418-process-notification-on-81/?p=10093378
rohitabPosted 11 October 2013 - 03:38 AM
Due to security related changes in Windows 8.1, the Process
Notifications feature of API Monitor does not work. I will try to
resolve this issue as soon as possible and post a hotfix.
But a later update in 2014 indicated that it hadn't been fixed yet, and seems not to have been since then either.
It was implied that running in a Windows 7 (or 8.0?) virtual machine might be a workaround, or obviously finding another tool which has this capability.
I built an application using openframeworks that is live 24/7 on a kiosk. Every now and then (every few weeks) it will randomly go unresponsive and I still can't get to the bottom of it because it's so random and infrequent it is hard to debug.
I wrote a ruby script that looks for the application running and if it doesn't exist it will start it up. This works in all cases where the application name doesn't show up in activity monitor. Meaning if the app crashes and completely force quits itself or something. It works just fine.
However, if the app just freezes and goes unresponsive (red highlight in activity monitor) the app doesn't quit out completely unless I force quit manually. Is there some kind of script I can write to look for all "unresponsive apps/processes" every few seconds and force quits them automatically? That way my app launcher script will be able to detect that the app isn't running and boot it up again.
I suggest you look at Monit because it's solid, well tested, well documented, and easy to learn.
If you still want to write your own monitoring script, Monit is a good example to follow.
The most reliable way to detect an unresponsive app is to have a "vital sign" which is a generic term for a signal that an app emit to prove it's healthy. Some people call this a "pulse" or "heartbeat" or "brainwave". Your external script watches the vital sign. If your external script sees the vital sign flatline, then the script takes action to cure the app or restart it.
An alternate way is to have a "ping" which is a generic term for your external script sending a signal to the app or the system, then listening for a reply. You can use tools such as the Unix ps command for processes, or AppleScript Activity Monitor. As far as I know, these do a pretty good job of catching common cases, but have trouble catching apps that are soaking up resources, such as being caught in endless loops.
I'd like to detect when someone terminates a suspended debugged process without informing the debugger. (For example, get to a breakpoint in a console app, and close the app's console window.) The process goes into a zombie-like state and cannot be interacted with further until the debugger releases its hold.
This state appears to set the PROCESS_EXTENDED_BASIC_INFORMATION::IsProcessDeleting flag when gathering information on the process via NtQueryInformationProcess, but both the flag, structure, and function are effectively undocumented and marked "do not use" on MSDN.
Is testing this flag reliable? Is there a better, "official" API I can use?
(Yes, I know IsProcessDeleting is also set when the process is (surprise, surprise) shutting down normally. This is not a problem from my perspective.)
Nope, not that I can see. NtQueryInformationProcess isn't going away anytime soon though, if that function was removed hundreds of apps would be broken by it.
I have a console application (written in VB6 ) which is behaving strangely on my machine. I kick it off from the command line and what should be a two minute job drops straight back to the prompt - if I run this on another machine the executable will sit and wait until the job finishes before returning control back to the prompt. If I check process explorer I can see that the executable is running as a background process and other than this strange background-ness is running as expected.
Any thoughts on why this could be happening? (Running on 32-bit Windows XP Pro SP3.)
It's totally unclear whether this is an application you wrote and have the source code for. If that's the case, you need to get in and start debugging. At the least, use OutputDebugString to send information about what's going on to any number of potential viewers. Taking that a step further, consider rewiring the app using the Console module I wrote, along with vbAdvance to recompile. This combination will allow you the full power of the VB6 IDE to debug within. No more guessing about what's going on.
Then again, if it's not your app, I'm not sure what VB6 has to do with it and wish you the best of luck trying to figure out what's up.
It sounds to me as though the app isn't being recognised as a console app on one of your machines. Console apps weren't officially supported in VB6, although there are some well-known hacks for creating them (particularly the free add-in vbAdvance). Possibly your console app is a bit unreliable? If Windows thinks your app is a GUI rather than a console app, it won't wait for it to finish.
As a pragmatic workaround: try launching with start /wait rather than just using the exename. That forces the command prompt to wait for the program to finish, whether it's a GUI app or a console app.
Sounds like an error is occurring that is being 'swallowed' by the application. Do you have the source code?
Errors in VB6 apps are often due to some COM component not installed and/or registered.
Download SysInternals Process Monitor and this will show up accesses to ProgIDs that fail (uninstalled/unregistered COM components).
Check out: Process Monitor - Hands-On Labs and Examples.
Have you checked permissions? Is the application accessing any network based resources?
How to reload a crashed process on Windows? Of course, I can run a custom monitoring Win service process. But, for example, Firefox: it doesn't seem to install such a thing, but still it can restart itself when it crashes.
On Vista and above, you can use the RegisterApplicationRestart API to automatically restart when it crashes or hangs.
Before Vista, you need to have a top level exception filter which will do the restart, but be aware that running code inside of a compromised process isn't entirely secure or reliable.
Firefox constantly saves its state to the hard disk, every time you open a tab or click a link, or perform some other action. It also saves a flag saying it shut down safely.
On startup, it reads this all back, and is able to "restore" based on that info.
Structured exception handling (SEH) allows you to catch program crashes and to do something when it happens.
See: __try and __except
SEH can be very dangerous though and could lead to your program hanging instead. Please see this article for more information.
If you write your program as an NT service then you can set the first, second and subsequent failure actions to "Restart the service".
For Windows 2008 server and Windows Vista and Windows 7 you can use the Win32 API RegisterApplicationRestart
Please see my answer here for more information about dealing with different types of program crashes.
If I recall correctly Windows implements at least some subset of POSIX and so "must" have the signal interface (things like SIGKILL, SIGSEGV, SIGQUIT etc.).
I've never done this but on linux, but you could try setting the unexpected termination trap with signal() (signal.h).
From quick scan of docs it seems that very few things can be done while handling signal, it may be possible that even starting a new process is on forbidden list.
Now that I've thought about it, I'd probably go with master/worker pattern, very simple parent thread that does nothing but spawns the worker (that does all the UI / other things). If it does not set a specific "I'm gonna die now" bit but still dies (parent process always gets message / notification that spawned process died) then master respawns the worker. The main theme is keep master very simple and hard to die due to own bugs.