I need to interpose (get my functions called instead of the original functions) some OS X system calls to overcome a flaw in a piece of closed-source software.
Preferably, the resulting solution would work under 10.5 (Leopard) and newer, but I might be able to require 10.6 (Snow Leopard) if the argument were strong enough.
Preferably, the resulting solution would be an executable, but I might settle for a script.
Preferably, the resulting solution would be able to interpose ("steal the vectors") even after the target application is running, but I could settle for a technology that must inject itself as the application is loading.
Preferably, the resulting solution would be developed in C or C++, but I could settle for Objective-C or something else.
So far, I've experimented with:
1) DTrace scripting, which has taught me a lot, but the limitations of the D language (limited flow control, etc.) make it a major pain for what I'm doing, not to mention that the result would be a script, which isn't as tidy and self-contained as what I'm shooting for.
2) DYLD_INSERT_LIBRARIES interposition, which is slick in many ways, but perhaps due to namespace flattening (I won't pretend to deeply understand what this means), it works nicely against simpler executables, but makes my target application choke, even when I build a do-nothing library that doesn't actually interpose any calls.
My latest idea is to experiment with mach_star (https://github.com/rentzsch/mach_star), but I'm stopping here first, to ask the Stack Overflow community which invariably knows more than do I...
...should I be looking at something besides mach_star next?
I think you've made the right choice looking at mach_star.
If you actually want to learn how the darwin link-loader works, etc., I'd put more time into your DYLD insertion problems. But obviously you're looking for a quick solution, not an in-depth learning experience. And I doubt anyone's going to be able to figure out the problems you're having without having access to your project. So, this is probably a dead end. Besides, Mach overriding and injection are more fun anyway.
The basics of Mach injection aren't actually that hard, but there are a ton of things you have to get right, most of which aren't well documented. You're going to get 11 things wrong before you get something that works on your system, and then it won't work for the next function you try, and then it won't work on 10.5 or 10.8, and… The mach_star library wraps up all that stuff for you. So, why not use it?
I should mention that I haven't used mach_star since pre-Intel days. But it looks like it's still being updated regularly-ish, with changes for x86_64 and 10.7 and Xcode 4 and so on.
Related
I'm trying to build an automated test to check that methods called outside of the current deployment target are handled correctly. Here's a concrete example:
My app is set to use the Mac OS X 10.8 sdk and the deployment target is set to 10.6.
I often use the setTimingFunciton: method. It was introduced in 10.7.
[[NSAnimationContext currentContext] setTimingFunction:[CAMediaTimingFunction functionWithName:kCAMediaTimingFunctionEaseIn]];
In most cases I correctly bounded this method call with a check to ensure the method exists or that the OS was the correct version or later. But I missed one. And it's in a rarely executed code path, so even beta testing didn't notice it. It only came back to bite me much later.
I'd like to build an automated way to test for things like this that can be deployed on an integration server or other such scripted regression test. It seemed like something there must be a way to check for automatically.
Recompiling with the 10.6 sdk unfortunately results in many errors, most due to "modern obj-c". These, of course, are not errors when using the 10.8 sdk. Being careful, I can pick through the errors and find the method calls -- but that's overly complicated and manual.
Does anyone know of a way to statically check these sorts of things that doesn't involve a human picking through hundreds of lines of false-errors?
P.S. I've read this: How to check if not available methods are used if deployment target < base sdk?
Which recommends another manual flow for digging through them. The problem is, the human method failed me. I missed one. I'd like something a bit more certain.
I've used the technique from this answer and it has worked great. Totally automatic and it doesn't miss anything.
Here is an idea. undef and redefine the availability macros so they contain something that Clang will object to but will otherwise not affect program flow. Run Clang.
Would be usefull to see how things work but not sure on the legality of it
Most Mac apps are written using Cocoa in Objective-C; which, while it is a compiled language, means that there is a fair bit of information left over that could be used by a decompiler.
I'm not sure if there are a lot of decompilers out there that leverages this information, at least I haven't heard of any.
However, there are also another option; F-Script.
F-Script can be used to attach to an executable and explore its interfaces, while not as good as source, it can give you a pretty clear idea of how the executable is built, and how it operates.
As for the legality issue:
IANAL, but as far as I know, reverse-engineering for the purposes of compatibility is legal in many jurisdictions, and I can't imagine that decompiling an executable to look at its code is illegal, unless the specific EULA specifically prohibits it.
Edit: WRT Steam specifically, it is probably NOT written in Cocoa, but C# with some manner of .NET compatibility layer; and it's probably not a good place to start if you want to learn how to make applications for Mac OS X.
By far, the best Mac OS X disassembler I've used is Hopper available here:
http://www.hopperapp.com/
It will also convert the assembly to C pseudo code as best it can. It will generate code flow diagrams with blue lines (true blue, love it) for true and red for false paths.
It's The Mac OS reverse engineering tool. There are even Youtube videos that will show you how to use it.
If it's an open-source app, yes. Otherwise it's possible through decompilation but the output will be a real pain in the ass to look at. If you just want the protocols and the interfaces of categories and classes, have a look at class-dump.
I'm not aware of a nib decompiler.
Whether decompilation is legal: ask a lawyer. This may (and probably does) differ per jurisdiction.
Is it possible to view the source of a mac app?
Realistically, no. Sure, you might be a able to use a decompiler to get a peek, but the kind of output you'll get won't be easy to read. If you're asking this question, this route probably isn't going to be helpful to you.
Specifically interested in GUI and how the steam app for mac works
It's a good bet that it works about the same way that most other applications work. It might use custom controls to look different from a typical application that mostly uses the standard Cocoa controls. But underneath, just about any GUI application written for MacOS X will use the run loops, responder chain, and view hierarchy that Cocoa provide. The main exceptions would be applications that are built mostly using an alternate framework like OpenGL or WebKit.
Figure out what, specifically, the Steam application does that you'd like to do. Take a look at the tools that Cocoa provides to see if you can figure it out yourself; if not, ask about it here.
Can the Ruby language be used to create an entire new mobile operating system or desktop operating system i.e. can it be used in system programming?
Well there are a few operating systems out there right now which use higher-level languages than C. Basically the ruby interpreter itself would need to be written in something low-level, and there would need to be some boot-loading code that loaded a fully-functional ruby interpreter into memory as a standalone kernel. Once the ruby interpreter is bootstrapped and running in kernel-mode (or one of the inner rings), there would be nothing stopping you from builing a whole OS on top of it.
Unfortunately, it would likely be very slow. Garbage collection for every OS function would probably be rather noticeable. The ruby interpreter would be responsible for basic things like task scheduling and the network stack, which using a garbage-collecting framework would slow things down considerably. To work around this, odds are good that the "performance critical" pieces would still be written in C.
So yes, technically speaking this is possible. But no one in their right mind would try it (queue crazy person in 3... 2...)
For all practical purposes: No.
While the language itself is not suited for such a task, it is imaginable (in some other universe ;-) that there a Ruby run-time developed with such a goal in mind.
The only "high level" -- yes, the quotes are there for a reason, I don't consider C very "high level" these days -- language I know of designed for Systems Programming is BitC. (Which is quite unlike Ruby.)
Happy coding.
Edit: Here is a list of "Lisp-based OSes". While not Ruby, the dynamically-typed/garbage-collected nature of (many) Lisp implementations makes for a favorable comparison: if those crazy Lispers can do/attempt it, then so can some Ruby fanatic ... or at least they can wish for it ;-) There is even a link to an OCaml OS on the list...
No, not directly
In the same way that Rails is built on top of Ruby, Ruby is built on top of the services that lower layers .. the real OS .. provide.
I suppose one could subset Ruby until it functionally resembled C and then build an OS out of that, but it wouldn't be worth it. Sure, it would have a nice if .. end but C syntax is perfectly usable and we already have C language systems. Also, operating systems don't handle character data very much, so all of the Ruby features to manipulate it wouldn't be as valuable in a kernel.
If we were starting from scratch today we might actually try (as various experimental projects have) to use garbage collected memory allocation in a kernel but we already have OS kernels.
People are making investments at the higher layers rather than redoing work already done. After all, with all the upper level software to run these days, a new kernel would need to present a compatible interface and the question would then be asked "why not just run the nice kernels we already have?".
Now, the application API for a mobile OS could indeed be done for Ruby. So, just as Android apps are written in Java, RubyPhone apps could be written in Ruby. But Ruby might not be the best possible starting point for a rich application platform. Its development so far has been oriented to server-side problems. There exist various graphical interface gems but I don't think they are widely used.
basically yes, but with a big disclamer .. which is basically Chris' answer but a different spin on it. Since for kernel performance it would kinda suck to use ruby, you'd probably want to build around a linux-ish kernel and just not load any of the rest of the operating system. This is basically what Android does: the kernel is a fork from Linux (and is maintained close to linux), the console is a webkit screen, and the interpreter is Java with some Android specific libraries. IE, Android is Java masquerading as an OS, .. you could do about the same thing with Ruby instead of Java and only a smallish hit to performance from java
While building a whole OS from scratch in Ruby seems like
a multi-billion project (think of all the drivers), a
linux kernel module that runs simple ruby scripts does
make sense for me - even it was only for prototyping
new linux drivers.
After a long evaluation period of mainstream toolkits Qt, WxWidget, GTK i came to the conclusion that it does not make sense to religiously equalize the different platform. Now more then ever before.
In the days before Java portability meant, that platform dependent code was located in known places and should be small but not none. No write once, run everywhere marketing.
Is there any GUI tookit - experimental or commerical (even outdated or dead projects) - that is going this way. It would need to give me native widgets of course. I would like to have a look at them before i start my own toolkit
If you think this is a stupid idea tell me why (cause everybody else is doing it differently there may be a reason i don't see)
As someone who zealously believes that GUIs should be drawn with native widgets, I must say I've been very impressed with wxWidgets, although you mention that at the beginning.
As far as I know, wxWidgets uses native widgets whenever possible, and indeed it does also include platform-specific features that are neither possible nor relevant on other platforms. A good example of this is how you can use the native Mac OS X menu, while on other systems it won't do squat. There are other and better examples.
I must mention wxPython in particular, in case you either don't want to or just don't have the time to write a C++ program. wxPython astonished me when I tried it out first, and I even managed to make a stand-alone .exe file from it that worked on a fresh install of Windows 2000 without any updates or particular runtime environments.
If wxWidgets doesn't suit your needs, I wonder what would. :) Hurray for wxWidgets!
AFAIK, wxWidgets uses the platform's native widgets whenever possible.
Have a look at SWT, it tries to use the widgets of the underlying platform as much as possible.
My company is in the process of rewriting our software from scratch, and I'm the one who is going to be doing most of the work in rewriting the Mac client (The core of our software is Windows based, and the Mac client communicates with it through a webservice).
This isn't a real heavy app, mainly does some background work tracking stuff and a UI component for the user to enter information.
I'm trying to decide how hard I should argue for dropping support for 10.4 and going with pure 10.5+/Obj-C 2.0 code.
My main motivations for this are:
It would be easier to code, I could use all the features of Obj-C 2.0 such as synthesized properties and fast enumeration.
It would give me access to several classes, and methods in existing classes, that don't exist in 10.4 (Just in mocking up a UI I've come across NSPathControl and NSTreeNode, both of which I would otherwise be very happy to use.
Preparing for the conversion to 64 bit coming in Snow Leopard. It seems like most of the techniques for preparing for the move to 64 bit (NSInteger, etc) are only available in 10.5+, and it would not be possible to use these if writing for 10.4.
The downside would of course be that we'd no longer be supporting an operating system that was only a year out of date.
My boss is himself supportive of this move, but of course has our customers to consider and doesn't want to cause any more issues for them than are justified. The director of support would like to support 10.4. I suspect the other execs will be marginally against it at first, just due to the not being able to support some customers thing. Everybody would be open to persuasion by a good argument from either side.
I'm trying to talk to some of the support people and get an idea of how many of our customers are actually still using 10.4, but I don't have that data yet.
Some kind of hybrid solution might be possible, such as rewriting parts of the old client to use the new webservice, or writing the client in 10.5 and backporting it to 10.4 if enough people made a fuss, but quite frankly those sound like they're likely to be even more trouble than giving up the 10.5 features and writing the code in 10.4 to begin with.
So I guess my questions are as follows:
Given the information above, do you think making a case for the adoption of 10.5+ only is the right thing to do? Do you have any suggestions as to how this might be presented positively to the rest of the company?
I don't know as much about the coming 64 bit transition as I'd like. Does anybody have any good references on what will be different, and do you think that supporting only 10.5+ would make this transition easier for us?
If it were I doing the update, I would target 10.5, especially since 10.6 is just around the corner and 10.5 did come out with a lot of great, new things (especially Objective-c 2.0). However, I think you really need to answer this question based on what you think your target customer group will be using. If they are slow to adopt new technology, it may be that you have to support 10.4 or risk losing a portion of your customer base.
On the other hand, you can actually target 10.4 and write using the 10.5 SDK. That way you can take advantage of all the preparations for 64-bit added to the SDK. You just have to ensure that you don't use any classes or features of the frameworks that didn't exist in 10.4. You can also do weak linking to the 10.5 frameworks and programatically decide whether you can use a new feature or not (while this is a bit of extra work up front, you can easily phase 10.4 support out of your code in the future and take full advantage of 10.5 improvements for users that actually are running 10.5).
There are a lot of blogs and write-ups about doing the cross-platform stuff out on the web. The other thing to keep in mind is that if you do target 10.4 make sure you have a 10.4 machine available to do a lot of testing (especially if you compile from the 10.5 SDK to take advantage of the 64-bit ready features). Also check the docks for any feature you may want to use from the 10.5 SDK. Many features were actually available in 10.4 but undocumented and the new documentation usually states which features you can safely use when deploying to 10.4
Do you need 64-bit? Unless your application is very CPU-intensive, it won't make any difference.
Tiger can run 64-bit applications, but without GUI. If you need 64-bit, you can create 64-bit CLI executable that does heavy lifting and provide 32-bit font-end for it (using NSTask and NSPipe).
You can also have separate .nib files for Leopard and Tiger:
-(id)init
{
BOOL tiger = floor(NSAppKitVersionNumber) <= NSAppKitVersionNumber10_4;
NSString nibname = (tiger ? #"WindowTiger" : #"WindowLeopard");
if (self = [super initWithWindowNibName:nibname])
…
You really need to find out what your customers are using, and the support person is probably best positioned to know, or the product manager. That said there's nothing wrong with making the technical arguments clear now even if 90%+ of your user base were pre-Leopard; that way the issues will be known (and hopefully understood) so you'll have more support as the environment does change.
I never wrote production code in Objective-C and its hard to keep up, but as far as i am aware NSInteger and friends are in 10.4, it's just that Cocoa isn't 64 bit in 10.4 whereas in 10.5 most of it is (so no more need for seperate 64bit worker process under a 32bit UI).
I don't know what your product is, or who your customers are, but from my experience, Mac users are early adopters (relatively speaking) I've never used an OS X version longer than two weeks before the next upgrade was out, and in my circle I am a late adopter. Ofcourse I'm not just a business Mac user and that may well make a big difference.
What makes 64bits a requirement in your code? There's not much of a reason to not compile a universal binary holding as many architectures as you wish you could have one binary run on G4, G5, IA32 and IA64 no problem, and have it be native on all of them. If you're just doing 64bits because you can there's no reason (that I can imagine) not to keep supporting 32bits, but if you want stuff like CoreAnimation you don't have much choice.
I don't think it's wrong to demand 10.5 for new development, but it wouldn't make much business sense to force a whole new OS on customers just to keep using your existing product. So if you can, stay compatible, maybe backport your new features/patches for a time. There is a good reason for forking in version control and this might be it.
edit-
Since I posted this I learned that I was wrong and NSInteger did not exist before 10.5. I think I assumed too much having used similar types (like NSDecimal) earlier.