We want to improve the reproducibility of the analyses at our institute. To this effect, we contemplate on implementing a system based on Singularity. The idea is that at the beginning of the analysis, the user can choose a machine configuration (later amendments must be possible) that sticks with them until the project is complete. Then, the image is archived with the analysis. Ideally, the user doesn't have to issue system admin commands (install packages etc.) in the process.
She just makes a request like "I need R with tidyverse and Python 3 and this and that in-house packages" and she gets a command that she can use to ssh into a singularity container that has those features. When she makes a new request, she gets the newest version of the programs but once the container has been deployed those versions don't change anymore.
It gets tricky when I think of the fact that multiple users will need different combinations of software. Do I need to provide an image for every combination of Software and software extension packages? If I only think of a scenario where users can choose of an arbitrary combination of {R, Julia, Python, r-tidyverse, r-data.table, r-whatever-genomic-analysis-package-on-bioconductor, python-...}
Is there a feature selection method in the veins of
singularity pull library://alpine:3.7 +r:3.2.1 +python3:3.7 +r-package:1.2.3
such that the user can
ssh cluster01 -- singularity shell project-abc.simg
and start/continue working?
If not, is there an alternative approach to supplying custom machine configurations to users using singularity?
I could find Singularity Compose, but this seems to just run multiple containers as services next to each other. So the images can stay separate. I have to merge them.
Yes, with Singularity, a dedicated image must be provided for each possible combination of packages.
Selecting a set of applications per-user is possible by changing your server configuration to the package managers Nix or GUIX, a fork of nix. The concept here is that each application/library lives within its own directory, whose name is a hash of the app! Therefore, multiple application versions can coexist and each application can link to another version of the same library.
A user can select a set of those directories as a user profile. This is a folder of symlinks into binaries in the proper application folders. From the Nix manual:
So, each user can setup their environment as they like, down to bitwise reproducability.
After the analysis, the profile can be turned into an image. I know its possible with GUIX using guix pack (tar, Docker, Singularity).
For Nix, I'm not sure. There is a project on GitHub, datakurre/nix-build-pack-docker, but it's dormant since 2015. Maybe it's enough to copy the needed subset of /nix/store into a folder, pull a NixOS image, and bind /nix/store of that image to your own folder?
Related
I'm trying to collaborate with with a colleague on a Gate app. He uses a Mac, while I am on Windows. We are both on Gate 8.5.1. He has uploaded the current project to a Bitbucket.org account.
My method is to download the entire folder from - https://bitbucket.org/our username/our project name/downloads/ - save it across to my Gate folder in Program Files, and then open the GAPP file. This opens GATE, but with none of the configuration (Gazzateers, JAPE rules etc), just a standard unmodified version of GATE.
I'm not sure if the File paths which he has chosen are not mapping across to my system, or there is some other element of his set it up which is different.
Is there an easy way to drag and drop a configured Gate application from one operating system (OSX) to the another (Windows)?
Thanks
You should check the gate manual on "Saving Applications and Language Resources". I think for beginners it's best to export the whole pipeline and resources in a zip file using the method described here.
In general, you don't need the GATE installation and binaries in your bitbucket project. You only need the configuration and code, specific to your application.
Keep in mind that if you use / edit any of the GATE PRs, like ANNIE gazetteers, japes, etc., you need to make sure they're all properly exported / updated within your team and working environments. (That's why I recommend the export for cloud option, because it exports all resources and updates paths so that they're all put in a common folder).
In my scenario, I have two people that do work on the same code base. Their only available workspace is a shared dev environment (where the files built are used to host the dev version of the site to boot). As such, they perform their work directly in that location. I've recently introduced source control to the project, and turned that location into a Git repository.
Let me preface by saying: Yes, I would love it if the dev host spot was a deploy-to spot, and these people had their own local copies of the source code. But that isn't feasible right now.
My question: Is it possible for two different Windows users/Git users (they have separate accounts that they can use to interact with GitHub/etc. with) to share the same folder? My hope would be that SourceTree (our weapon of choice) or Git, at least, wouldn't have a problem with this: Just show diffs of what's changed, and use the currently-logged-in user's information when making commits/other actions.
It looks like that while SourceTree has separate installation directories, it still embeds some account information in the .git folder itself. When I try to interact with Git (via a pull for example), it first tries to prompt for new credentials/etc., but shortly thereafter it says "please enter password for {other-user}" without an option to hop usernames.
It looks like we'll just have to do things the right way after all. Painful (for them) but no choice.
Over the years our InstallShield code has grown un-manageable and messy. We're about to design an InstallShield 2015 installation from scratch for a new major release, and I was wondering if there's any way to automatically capture a set of system changes to use as a cleaner starting point for developing a new install package. Ideally, I would like to turn on some capturing software, install the oldest release from which we support an upgrade, install the latest service packs, and then apply the manual changes that will get the new release running on the system. Then I would turn off the capturing software, and it would provide an InstallShield project pre-loaded with all the files and registry entries (GAC changes, .NET assemblies, etc) that were created as part of that install. Then I could add steps to delete those that we no longer needed and do some other clean-up and refinements. Does such a thing exist?
Yes, this is called a repackager most often, in the enterprise world where sys admin and packagers prepare applications for deployment on the company machines.
Flexera has one that can create a project for InstallShield, but it is sold together with their AdminStudio solution, which is not cheap.
We (at Caphyon) have a repackager included in Advanced Installer (the architect edition) which also requires you purchase license. And of course the generated project will be compatible only with Advanced Installer, where you can configure your installers as you wish.
I think there is also a free repackager, AppDeploy from Dell, but I never used it, so I don't know how accurate it is and if you can use the results created into InstallShield or if it generates directly an MSI.
If you started looking more careful you will probably find other repackager tools, but you should know that building such a tool is not an easy task so choose carefully. Also probably only the one from AdminStudio will be able to generate a project that can be read by InstallShield.
If you already have the source projects from the older editions I would personally not try using a repackager. Instead I would go for cleaning up all the configurations which you do not understand and re-build them from scratch.
A repackager as good as it can be still has some problems. It can capture incomplete data, for example if you have a custom action that runs different code based on the OS where the installer is running the repackager will capture only its effects on the OS where you run it. On another one might run differently and have another output.
Also if your installer has prerequisites and you run the repackager on a machine where those prerequisites are installed then the repackager will not capture anything related to this, so by accident you can forget to include required prerequisites in the new package.
There is also the things like meta-information which few repackagers can detect. For example files associations which are actually a set of file and registry entries connected together or environment variables, scheduled tasks, etc...
Most repackagers capture all this data and simply show it to you as configuration files and registry entries, instead of creating the correct entities in your projects, i.e. files associations, environment variables or scheduled tasks in their correspondent views.
Are local pre-commit hooks a function of TortoiseSVN only? It seems that the svn command line client does not support them. If the answer is yes, is there any way to script their installation? My situation is that there are a lot of committers, two or three hundred maybe.
We have several release managers who each have multiple VMs with multiple branches checked out on each VM. I'd like to have a script we can lay down and run on each VM instead of manually adding (through the tortoise GUI) one hook per release manager per VM per working copy. Adding it server-side would result in a lot of unnecessary work and slowdown for the server. We are all on windows. Thank you!
Yes, client-side hooks are a TortoiseSVN-only feature (the settings shown here are global to the user's TortoiseSVN client). You can configure them for for specific projects via the tsvn:*hookscript properties.
For distribution, you may want to keep the hook scripts on a centralized file share and reference them via UNC path in those properties. That way, everyone should be pointed to the same hook scripts all the time. You'll need to ensure that your hooks do not depend upon specific paths either for working copies or other tools that your scripts might be dependent upon.
However, you should not become completely dependent upon client-side hook scripts. You can't control which client(s) a user may use and if they don't use TortoiseSVN, that functionality won't exist.
I'm researching the best approach to automating our build process. I've got my own ideas (through experience on a previous non-iOS project) but need good arguments for and against various possibilities.
Objective: A single xcode project with a single target (think white-label) needs to be built in 1..N different flavours (concrete brandings) with minimum user interaction and minimum technical knowledge. For AdHoc and/or AppStore.
Essentially, that will mean specifying per build; a folder containing Icons + Splashscreen, a bundle containing brand specific resources and (presumably?) the Info.plist, specifying appname, bundle-id, etc.
Issues that need to be respected or clarified;
Manual build of a single brand via Idiot-Proof GUI (choose a git
branch/tag, specify a certain brand, configure the app e.g.
IAP-enabled, server-domainname, etc - will be written to the
info.plist)
In previous manual tests, setting the executable name in
the plist didn't work? Sorry, have forgotten the exact problem..
perhaps was only an Xcode Debug buildconfig problem, not relevant to
a distribution build?
Code-Signing?!? Can the profile be specified
on-the-fly? Some brands need to be built with the customer's own
profile.
My personal feeling: Hudson or CruiseControl + Xcode plugin.
There seems to be plenty of documentation around for an Xcode solution and I've seen this in action on a Flex project I worked on, with almost exactly the same white-label/branding requirements. Of course that was using Ant script though and there was NO behavioral config to respect. That's my only uncertainty here... I suspect it would have to be hardcoded somewhere, but that's not the answer that's going to please some people. There is a wish to be able to specify the various app-config settings (server url, is function Foo supported, is the view X displayed, etc, etc) via a GUI form, when building manually. I'm not sure how easy it would be to shoehorn that into a typical Hudson or CC config?
And hence one suggestion that has been made is to write an OSX app for building our clients. The theory being, nice clean non-tech UI for entering all the necessary meta data & app setting and a big shiny green button labelled "Build". But personally I'm skeptical that this approach is any more flexible or easier to implement than a classic CI solution.
So the question is basically, what's preferable; a classic server based, version control integrated, CI approach or a custom OSX utility?
Whichever we go for it'll almost certainly be a requirement to get it up and running in 2 or 3 days (definately less than one week).
IMHO you can resolve all issues using different targets of XCode.
Every target will share the code but it could:
be signing with diferent profiles
use diferent plist: this implies having different names..
use diferent brand images. You only have to name the image with the same name and select the correct target in file inspector.
Build with one click in XCode.
I hope this helps
An extremely later reply, but the approach I would take would be to create the white label IPA, and then create a script to:
1. Unzip it (change the .ipa file extension to .zip).
2. Change assets.
Update the info.plist (using Plistbuddy command)
Zip it again.
Resign the code.
See this script as a starting point: https://gist.github.com/catmac/1682965
Very late answer. But I would go with different .xcconfig files and multiple schemes. The scheme names could be a combination of target/brand.