I'm doing some research about the way launchd load it's services from plist files under /Library/LaunchDaemons/ or via the command launchctl load
So far I've managed to gather some various sources and compose the following vague picture as I understand it:
Upon Service loading (launchctl load) The process launchctl send the launchd an appropriate XPC message, and then the launchd is forked into new process with the context of xpcproxy.
This generic process, is waiting for another XPC call from the launchd to run it's real process context according to the launchDaemon plst.
Is this explanation sounds right ? perhaps anybody can help me make it more accurate ?
thanks
This is actually a bit more complicated. The kernel is composed of two parts, BSD and the mach kernel; the latter being responsible for management of memory and process scheduling.
Each mach process has one or more mach tasks (really task port rights!). When an application is first launched, it has just one right, the bootstrap port, allowing communication with launchd. Note that a task port right is uni-directional, so a launching process that has the right to communicate with launchd must give a right for launchd to communicate back to it.
When an XPC message is received by launchd, it depends upon the Launch Daemon as to what action it takes. It's possible that the message is for a service that runs with a network port that may or may not be running. If running, it forwards any arguments from the calling process to the running service. If not running, it can provide the service on demand by launching the process first.
More specifically you asked about launchctl load. Since the source code for launchd is no longer open source, the next best resource is the reverse engineering work by Jonathan Levin; Author of Mac OS X and iOS Internals and more recently, his newer self-published books on *OS Internals.
You'll find his slides about launchd here, but probably more useful to you is his version of launchctl, jlaunchctl which is open source.
Finally, if you want to view content of XPC messages between processes, disable SIP and use Jonathan's invaluable XPoCe tool.
Related
I'm using the Kernel Control API (SYSPROTO_CONTROL) for a user-land application to request information from a kernel extension, based on the code in Apple's documentation.
All works as expected with a single connected client. If a 2nd client tries to connect whilst the first is connected, it fails with the message: -
Error 16 (Resource busy).
The first client is then automatically disconnected.
Is it possible for two clients to be connected using the Kernel Control API and if not, is the best solution to keep trying to connect if the resource is busy?
I don't know if it is possible but the recommend way is to always only have one user space client that talks to one kernel extension, usually a background daemon running in user space and started by launchd. If you want multiple other apps or processes to access data from your kernel extension or somehow interact with it, then these would talk to the user space daemon and not directly to the kernel extension, which can also cache data in user space as crossing the users space/kernel space bridge is always expensive for data, so when 10 processes all want the same data, it would be better to just pull it once from the kernel and then distribute it 10 times in user space using any IPC mechanism of your choice.
This setup is recommend as you should limit kernel control to root processes (using the CTL_FLAG_PRIVILEGED flag), so your daemon would run as such a root process, whereas normal apps and processes run with the privileges of the current user. Such a root helper daemon can be bundled inside an app bundle and using the SMJobBless API, the daemon is automatically copied to /Library/PrivilegedHelperTools and it's embedded plist (see SMJobBless documentation, which is only available in its header file AFAIK) is copied to /Library/LaunchDaemons and registered with launchd. Within such a plist you can use various triggers when the daemon shall be started by launchd, e.g. when your application tries to connect to a specific IPC UNIX socket. Then all you have to do in your app is trying to open that socket, what launchd will detect and start the daemon for you, which can then use launchd's API to get hold of that connected socket and immediately start talking with your app. So the whole thing is only installed once and then brings up itself each time your app is launched.
For a SMJobBless sample project, see here.
Our group has long running processes which run daily. The processes are typically started at 9pm on any given day and run until 7pm the next day. Thus they typically run 22hrs/day. They are started by scheduled tasks on servers under a particular generic user ID, and they start and run regardless of whether or not that user ID is logged on. Thus, they are windowless console executables.
The tasks orchestrate computations running on a large server farm. Generally these controlling tasks run uninterrupted for the full 22hrs/day. However, we often have a need to stop and restart these processes. Because they control a multitude of tasks running on our server farm, it is important that they be shut down cleanly, so that they can stop and shut down all the server farm processes. Which brings me to our problem.
The controlling process has been programmed to respond to ctrl-C and ctrl-break signals. This works fine when the process is manually started in a console where we have access to the console and can "type" ctrl-c or ctrl-break in the console window. However, as mentioned, the processes typically run as windowless scheduled tasks. Hence we cannot "type" anything into a non-existent console window. Because they are console processes that execute without a logon process, the also must be able to execute in a completely windowless environment. So, how do we set up the process to listen for a shut-down signal?
While the process does indeed listen for a ctrl-C and ctrl-break signal, I can see no way to send that signal to a process. This seems to be a fundamental problem in Windows, or am I wrong? I am aware of SendSignal.exe, but so far have been unable to get it to work. It fails as follows:
>SendSignal 26320
Sending signal to process 26320...
CreateRemoteThread failed with 0x00000005.
StartRemoteThread failed with 0x00000005.
0x00000005 == Access is denied.
Trying "taskkill" without -F results in:
>taskkill /PID 24840
ERROR: The process with PID 24840 could not be terminated.
Reason: This process can only be terminated forcefully (with /F option).
All other "kill" functions kill the process immediately rather than sending a signal.
One possible solution would be a file-watch based solution: create a watch for some modification of a specific file. But this is a hack and we would prefer to do it with appropriate signaling. Has anyone solved this issue? It seems to be so very basic a functionality, and it is certainly trivial to do it in a Unix environment. Surely Microsoft has provided SOME mechanism to allow clean shut down of a windowless executable?
I am aware of the thread below, whose question is virtually identical (save for the specification of why the answer is necessary, i.e. why one needs to be able to do this for a windowless, console-less process), but there is no answer there excpet for "use SendSignal", which, as I said, does not work for us:
Can I send a ctrl-C (SIGINT) to an application on Windows?
There are other similar questions, but no answers as yet.
Any help appreciated.
[Upgrading #Anon's comment to an answer for visibility]
windows-kill worked perfectly and managed to resolve access denial issues faced with SendSignal. A privileged user would have to run it as well of course.
windows-kill also supports both ctrl-c and ctrl-break signals.
When an application process launches an XPC helper process, it doesn't actually do the fork()/exec() itself in the classic UNIX style. Instead, it sends a message to launchd, which does the dirty work for it. Thus, if you query the parent process on the XPC process, it comes back as the launchd process.
However, if you open Activity Monitor in the hierarchical process view, the XPC helper processes are all shown below the original application that requested them, for example:
In the software I'm working on, knowing this relationship between processes would be extremely useful. So far we've been using the regular BSD parent process information, but as everything moves towards XPC, this isn't much use anymore.
So:
Where is the "original" parent process information stored for XPC processes?
How does Activity Monitor access it?
There is a kext involved, so I'd be happy to pull this information straight out in the kernel instead of userspace, but I can't seem to even figure out where it's stored.
Update: Discussion on Apple's darwin-kernel mailing list: http://lists.apple.com/archives/darwin-kernel/2015/Mar/msg00001.html
I imagine that launchd knows what you are looking for.
The Service Management framework has a method that might give you what you are looking for easily.
CFDictionaryRef SMJobCopyDictionary(CFStringRef domain, CFStringRef jobLabel); function.
I am developing an application on MAC OS . It has 2 parts -- a UI element and a daemon (which needs to run continuously and must restart on being killed). Currently I am using launchctl to restart the daemon.
But there is another issue. I need the 2 parts of my application to communicate with each other . For this I am using distibuted objects for the same (as given here) . However this does not work when I launch the daemon with launchctl. Can anyone suggest some alternative???
I use NSDistributedNotifications to handle this pretty well in one app, even on 10.7. You have to do your own handshaking since this can be lossy (i.e. include an ack notification and resend in case of timeouts). A side effect of this approach is that if there are multiple clients running (particularly under fast user switching), all of them receive the notifications. That's good in the particular case of this app. It's also extremely simple to implement.
For another app, I use two FIFOs. The server writes to one and reads from the other. The client does the opposite. You can of course also use a network socket to achieve the same thing. I tend to prefer FIFOs because you don't have to do deal with locking down a network socket.
That said, what problem are you seeing using distributed objects under launchd? Are you just seeing problems on 10.7 (which changed the rules around the launchd context)?
Are you using launchd to lazy-load the daemon when the port is accessed (this is the normal way to do it). Have you considered using a launchagent instead of a launchdaemon?
EDIT:
Ah... the bootstrap server. Yes. You need to execute things in the correct bootstrap context in order to talk to them. The bootstrap context for the login session is rooted to the windowserver process. LaunchDaemons run in a different context, so they can't directly communicate with the login sessions. Some background reading:
Starting/stopping a launchd agent for all users with GUI sessions
How can you start a LaunchAgent for the first time without rebooting, when your code runs as a LaunchDaemon?
launch agent from daemon in user context
I am not aware of anyway to get processes into the correct context without using launchctl bsexec. Launchd technically has an API (launchctl uses it), but it is not well documented. You can pull the source from opensource.apple.com.
Even if you stay with NSDistributedObject, I would try to use something other than the bootstrap service if you can. As I mentioned, I tend to use other tools and avoid NSDistributedObject. In my opinion, for the same reasons that REST is better than SOAP, simple protocols are usually better than remote objects. (YMMV)
If you are launching your daemon using sudo launchctl; You should not use CFMessagePort and Distributed object for IPC. CFMessagePort and Distributed object are implemented using the bootstrap service(Many Mac OS X subsystems work by exchanging Mach messages with a central service. For such a subsystem to work, it must be able to find the service. This is typically done using the Mach bootstrap service, which allows a process to look up a service by name).
If you will use DO or CFMessagePort; you will run into bootstrap namespace problem.
when you will launch your daemon using sudo launchctl ; your service is register in root bootstrap namespace so your clients(running in user mode) will not able to use that services.
you can check bootstrap service using
$ launchctl bslist
$ sudo launchctl bslist // If you are using sudo lunchctl
You should use UNIX Domain Sockets. UNIX domain sockets are somewhat like TCP/IP sockets, except that the communication is always local to the computer. You access UNIX domain sockets using the same BSD sockets API that you'd use for TCP/IP sockets. The primary difference is the address format. For TCP/IP sockets, the address structure (that which you pass to bind, connect, and so on) is (struct sockaddr_in), which contains an IP address and port number. For UNIX domain sockets, the address structure is (struct sockaddr_un), which contains a path.For an example of using UNIX domain sockets in a client/server environment, see Sample Code 'CFLocalServer'.
Take a look at this Technical Note TN2083 Daemons and Agents
Daemon IPC Recommendations
Mach Bootstrap Basics
Each user has a separate Mach namespace .You cannot communicate
between namespaces. You'll need to use sockets (NSSocketPort)
instead, which are not limited in such ways.[1]
Anyone can please provide sample code to launch a daemon from cocoa application using AuthorizationRef.
A daemon is generally defined as a long-running system service that has no user interaction. If that is what you mean by daemon then this is the wrong way to solve your problem most likely. Daemons should be started through launchctl and should have a preferences plist in /Library/LaunchDaemons or ~/Library/LaunchDaemons depending on whether they are system-wide or user-specific. Communication between your user application and the daemon can be done any way you please but I've found NotificationCenter (CFNotificationCenterRef) to be useful for such things.
However, if what you really need is to launch a tool to perform a specific task and that tool needs special privileges then check out the sample code here :
http://developer.apple.com/library/mac/#documentation/Security/Conceptual/authorization_concepts/03authtasks/authtasks.html%23//apple_ref/doc/uid/TP30000995-CH206-TPXREF33