I am working on a Qt app with some library dependencies, for which I will have to make an installer.
From everything I read, seems like the best way is to make a bundle app with all library dependencies, and the required Qt frameworks, inside myapp.app/Contents/Frameworks
There are other applications created in parallel... that will get to be deployed on mac as well. They will have the same library dependencies and will be built using the same Qt version.
In that case it makes sense for the libraries and Qt to be installed OUTSIDE the bundle... so both (all) apps have access without having multiple copies of the same libraries.
Does that seem reasonable - and do-able with mac osx concept of bundle islands ? And how would I create such an installer that places libraries outside the app bundle ?
The simplest method of deploying Qt for OS X is to use the macdeployqt command line tool, and you have identified correctly that the normal method is to place the frameworks inside the app bundle, but multiple apps will each have copies of the frameworks.
It is reasonable to suggest moving the Qt frameworks to a separate, external location and linking to that instead. However, you will need to manage the framework carefully, especially when it comes to providing updates and be aware that if the framework is removed or altered, all your applications will fail to load. This, however is the same for any framework dependent application.
The thing to consider is where to place the framework. Normally, external frameworks reside in /Library/Frameworks, but if we all start to use that for Qt, problems may occur when your app is installed and another developer installs their app's frameworks with a different version of the libraries.
Apple defines various 'key directories' for applications and initially, the most likely location would appear to be the "Application Support Directory", but the documentation states that this is for: -
any type of file that supports the app but is not required for the app to run
This location is often used for support files, such as templates for the user to select.
If your application is to be deployed via the Apple Store, I wouldn't be surprised if it is rejected if you use this location. However, you're not using the Apple Store, then you could deploy the frameworks here.
If the Apple Store is your method of deployment, then /Library/Frameworks is probably the only place acceptable for the Qt framework to reside, with the possibility of the problems I've mentioned above.
Alternatively, consider just how many applications you're developing and is it really an issue to bundle the frameworks multiple times against the advantages that it brings, such as allowing the user to cleanly remove the application and all of its dependencies, as well as reducing problems of the framework being altered or removed accidentally?
If you choose to move them externally, you can refer to the answer to this question, which comprehensively explains how to make installer packages, after having updated your binary dependencies on the frameworks with the install_name_tool, as outlined here.
Related
I'm developing an electron application which depends on a bunch of modules that use native (C/C++) code.
Due to the way electron includes a patched node.js, these native modules have to be recompiled from source (everything is OSS) when we prepare a release.
One problem we have is that for a small number of users those native modules fail to load with pretty unspecific error messages. The modules don't have any further dependencies and we do validate at startup that all files shipped with the application are there.
So the only reason I can think of why the modules don't load is that some Anti-Virus may interfere with the loading of those modules (they are dlls but with a different file extension).
Since the modules were recompiled, even if the prebuild ones had been signed, the ones we ship are not.
So my question is: Does it make sense for us to sign all those dlls we ship with our own certificate, even though they aren't developed by us? Are there potential (legal) ramifications for using a certificate like that?
Is my theory about AV interfering with the loading of DLLs even plausible?
I'm using Alamofire in a Swift project, and part of their manual installation instructions are to add Alamofire under Embedded Binaries in the General tab for my application target.
What are Embedded Binaries?
Embedded binaries are binary files that are copied to your application bundle when you build the project. Use embedded binaries when your application relies on third-party frameworks so people can use your application without needing those frameworks installed on their machine. Embedded binaries keep users from having to manually install third-party frameworks. Your application uses the framework you embedded.
In your Alamofire example your application relies on Alamofire. If you didn't embed the Alamofire framework, nobody would be able to use your application unless they installed Alamofire manually. By embedding Alamofire with your application everyone can run your application.
"Binary" means: compiled code — as opposed to "source code", which is what you are working with when you write code as text.
They could have given you source code and let you compile it, but they didn't; they are keeping the source code secret, so they've given it all to you after compilation, so that you can't read it.
"Embedded" means: to be included inside your app bundle, by copying them into it at build time.
So, they are handing you some compiled code (frameworks) and telling you how to include them inside your app bundle. These frameworks, unlike Cocoa's frameworks, do not already exist on the device, so if you don't include them inside the app, they won't be present and your app would be unable to call into them.
Contrast this to Cocoa's frameworks. They, too, are compiled code. But they do already exist on the device. Therefore they are not embedded inside your app; they are merely linked (and, if they appeared, would appear in the next group, Linked Frameworks and Libraries).
Embedding binaries copies the entire framework to the target.
A framework is a hierarchical directory that encapsulates a dynamic
library, header files, and resources, such as storyboards, image
files, and localized strings, into a single package. Apps using
frameworks need to embed the framework in the app's bundle.
So, when you embed a framework in your app, it increases the size of your app because it is copied to you app bundle. In most of the scenarios we will be using this sections when we are using third party framework.
When we add a framework to Embedded Binaries it automatically adds that framework to Linked Frameworks and Libraries also.
Refer to apple documentation for more details: https://developer.apple.com/library/archive/technotes/tn2435/_index.html
I have written a simple application using the GPhoto2 Framework, and this works so long as the framework is in the location where it was originally compiled. I would like to move this inside the app bundle, though, so it does not need separate installation, so I need to make it work relative to this main executable.
Unfortunately the framework is not an Xcode project. It uses a script to build, so I cannot simply change the installation directory build setting, which is the solution that I have frequently seen while searching for an answer. Being quite new to Xcode and Mac programming it is also beyond my abilities to know how to convert the framework into an Xcode project.
The other advice I came across was to use install_name_tool to update the library ID and dependencies, replacing the absolute paths with ones of the form "#executable_path/../Frameworks/GPhoto2.framework". The framework is not a single binary, but contains a number of .dylib and .so libraries, but updating all of these has only been half successful.
I have set Xcode to copy the framework into the app bundle when it builds it. Then if I remove the framework from its originally compiled location the application fails to load, with the report generated by OS X saying the libgphoto2 library can no longer be found, as to be expected.
If I then use install_name_tool to update the references in all of the framework libraries inside the app bundle, and also in the app binary itself, then the application will load but fails to find any camera connected. Using otool I am able to verify that all references have correctly been changed.
But if I replace a copy of the framework to its original location it then works properly again, recognizing connected cameras, regardless of whether that framework uses relative or absolute locations. Clearly it is still looking at this location despite loading. I have even tried removing each of the individual library files from the framework in its original location in turn to see if the problem was just the result of a dependency in of these, but no matter which is missing the app will not work.
Incidentally, if I build the app using an updated version of the framework, it fails saying it cannot find the library "#executable_path/../Frameworks/GPhoto2.framework/prefix/lib/libgphoto2.2.dylib"
Am I doing something wrong or missing a step, or is what I am trying to do impossible for frameworks created outside Xcode?
In case someone comes across this future, the answer to my question was that I was doing nothing wrong. The problem was that the .so files were being loaded by libtool ltdl, at it requires absolute paths so these were being set at build time.
I patched the files gphoto2-abilities-list.c and gphoto2-port-info-list.c so that at runtime it would combine the relative library paths with the executable location. As a result I also needed to increase the FILENAME_MAX constant to allow it the mail application to run from, for example, the Desktop. But this, along with the use of install_name_tool allowed me to add GPhoto2 as a framework inside my application without needing any external dependencies.
The final problem of not being able to build my app in XCode with the framework after using install_name_tool remained, but for that I just used the original framework build, then after compilation I updated the references in the copied framework at the same time as I updated the ones in the main executable.
I am realtivly new to the concept of dynamic loading and shared libraries. While I fully understand how dlopen() could be used to reference symbols in a shared library I have yet to fullt grasp what MacOS does behind the scenes when I don't statically link against something. When adding a framework to Xcode I have the option to load it into my project or I can just provide some form of symlink to it(the actual implementation is obfuscated be the easy to use interface).
There after all I seem to need to do is import the header files that porvide and API to these frameworks and I can just invoke their symbols free of hassle. Can someone explain to me what I am actually doing, because it make no sense to me.
The sheet you're referring to has nothing to do with the actual linking of the framework. The copy vs. link choice refers to how you want to include the framework in your Xcode project, not your app binary.
For system frameworks there really isn't anything you need to do but import the headers.
For custom frameworks (your own or third-party) the framework must reside at the load path directory when your app launches. Typically the load path will point to your app bundle's Frameworks (sub)directory, so you must add a Copy Files build phase that copies the framework to your app bundle's Frameworks directory.
Remember to check out Apple's Framework Programming Guide, especially the section on frameworks embedded in your app bundle.
I have a custom framework that, following the advice in Apple's Framework Programming Guide >> Installing your framework I install in /Library/Frameworks. I do this by adding a Run Script build phase with the following script:
cp -R build/Debug/MyFramework.framework /Library/Frameworks
In my projects I then link against /Library/Frameworks/MyFramework and import it in my classes like so:
#import <MyFramework/MyFramework.h>
This works very well, except that I always see the following message in my debugger console:
Loading program into debugger…
sharedlibrary apply-load-rules all
warning: Unable to read symbols for "/Users/elisevanlooij/Library/Frameworks/MyFramework.framework/Versions/A/MyFramework" (file not found).
warning: Unable to read symbols from "MyFramework" (not yet mapped into memory).
Program loaded.
Apparently, the compiler first looks in /Users/elisevanlooij/Library/Frameworks, can't find MyFramework, then looks in /Library/Frameworks, does find MyFramework and continues on its merry way. So far this has been more of an annoyance than a real problem, but when runnning unit tests, gdb stops on the (file not found) and refuses to continue. I have solved the problem by adding an extra line to the Run Script Phase
cp -R build/Debug/MyFramework.framework ~/Library/Frameworks
but it feels like sello-taping something that shouldn't be broken in the first place. How can I fix this?
In the past months, I've learned a lot more about frameworks, so I'm rewriting this answer. Please note that I'm talking about installing a framework as part of the development workflow.
The preferred location for installing a public framework (i.e. a framework that will be used by more than one of your apps or bundles) is /Library/Frameworks[link text] because "frameworks in this location are discovered automatically by the compiler at compile time and the dynamic linker at runtime."[Framework Programming Guide]. The most elegant way to do this is in the Deployment section of the Build settings.
As you work on your framework, there are times when you do want to update the framework when you do a build, and times when you don't. For that reason, I change the Deployment settings only in the Release Configuration. So:
Double-click on the framework target to bring up the Target info window and switch to the Build tab.
Select Release in the Configuration selectbox.
Scroll down to the Deployment section and enter the following values:
Deployment Location = YES (click the checkbox)
Installation Build Products Location = /
Installation Directory = /Library/Frameworks
The Installation Build Products Location serves as the root of the installation. Its default value is some /tmp directory: if you don't change it to the system root, you'll never see your installed framework since it's hiding in the /tmp.
Now you can work on your framework as you like in the Debug configuration without upsetting your other projects and when you are ready to publish all you need to do is switch to Release and do a Build.
Xcode 4 Warning
Since switching to Xcode 4, I've experienced a number of problems with my custom framework. Mostly, they are linking warnings in GDB that do not really interfere with the usefulness of the framework, except when running the built-in unit-test. I have submitted a technical support ticket to Apple a week ago, and they are still looking into it. When I get a working solution I will update this answer since the question has proven quite popular (1 kViews and counting).
There's not much reason to put a framework into Library/Frameworks, and it's a lot of work: You'd need to either do it for the user in an Installer package, which is a tremendous hassle to create and maintain, or have installation code in your app (which could only install to ~/L/F, unless you expend the time and effort necessary to make your app capable of installing to /L/F with root powers).
Much more common is what Apple calls a “private framework”. You'll bundle this into your application bundle.
Even frameworks intended for general use by any applications (e.g., Sparkle, Growl) are, in practice, built to be used as private frameworks, simply because the “right” way of installing a single copy of the framework to Library/Frameworks is such a hassle.
The conventional way to do this is to have your framework project and its clients share a common build directory. Xcode will search for framework headers and link against framework binaries in the build folder first, before any other location. So an app project that compiles and links against the header will pick up the most-recently-built one, rather than whatever's installed.
You can then remove the cp -r and instead use the Install Location build setting to place your build product in the final location, using xcodebuild install DSTROOT=/ at the command line. But you'll only need to do this when you're finished, not every time you rebuild the framework.
Naturally, when you distribute your framework it should be installed in /Library/Frameworks; however it seems odd to me that you're doing that with the test/debug versions of your framework.
My first instinct would be to install test versions under ~/Library, as it just makes setting up your test and debug environment that much simpler. If possible, I would expect the debug/test framework to be located in the build tree of the version I'm testing, in which case it's installed as a Private Framework for testing purposes. That would make your life much simpler when it comes time to deal with multiple versions of your framework.
Ultimately, it doesn't matter where the framework is located as long as your application or test suite loads the correct version. Choose the location that makes testing/debugging/development easiest.