How to keep Firely.Terminal from trashing the FHIR package cache? - hl7-fhir

One of the brilliant aspects of Firely.Terminal is its ability to interoperate with the local FHIR package cache (~/.fhir) in a way that is fully compatible with HAPI tools using the cache. Sadly, that no longer seems to be the case.
Today I updated Firely.Terminal to version 2.4.2 and it seems that the new version walks all over the FHIR package cache, changing files without having been asked to.
It used to be that the only thing Firely.Terminal changed in existing packages was the generation of a missing .index.json. For newly installed packages, the only difference to a HAPI-installed package was some additional fields in .index.json (presence of some fields containing null which would normally be suppressed, and the addition of a fhirVersion field).
When the new Firely.Terminal is told to add a package to a scope (fhir install) it automatically 'bakes' it, which seems to involve things like snapshotting all StructureDefinition resources and expanding all ValueSet resources. Even resources whose content remains unscathed get their timestamps trashed. The same fate befalls all packages that are listed as dependencies in the manifest of the package being added to the scope.
There is an 'unbake' command (e.g. fhir unbake --package kbv.ita.for#1.0.1) but this does not operate recursively. What's more, when it says 'Bake successfully removed from KBV.ITA.FOR#1.0.1' (note the erroneous capitalisation) then that is an outright lie - the contents of the package directory are completely unchanged, except for the removal of the file .bake.json.
Hence the only way of restoring the package cache to working order is to identify all trashed packages, delete them all, and then reference them with some HAPI tool in order to get them re-cached.
I wouldn't mind so much if Firely.Terminal trashed its own cache. But what it destroys is the global HAPI package cache for the current user, and that is simply not acceptable.
Is there any way of suppressing the destructive behaviour of Firely.Terminal? Ideally globally (with machine-wide effect), but a secret command switch would do in a pinch. If that is not possible: does anyone know which of the older versions is the newest that still works, and where to get it?
Note: if the cached packages are write-protected then Firely.Terminal doesn't take the hint - it tries to clobber the files anyway and spews out oodles of 'access denied' messages. What's more, it doesn't even stop when an error occurs; instead it continues on its merry way and trashes everything that one might have forgotten to write-protect.
Background: one of the properties of the FHIR package cache that is important for our work is that the files in the cache are exactly the same as those in the (normative) published packages. In particular, we need profiles published without snapshots to not contain snapshots, value sets published without expansions to not contain expansions and so on. For one thing, this makes it possible to verify that the cached files are exactly the same as those contained in the published packages (or fixed versions thereof). For another, we need to control the context in which profiles are snapshotted, value sets expanded and so on because it may be necessary to supply dependencies that are different from those declared in the package manifest. The latter is sometimes necessary because the profile/package version management in the context of electronic prescriptions in Germany is a bit, erm, peculiar and can diverge from FHIR standards. For this to work at all the resources must be snapshotted/expanded dynamically (depending on the use context), not statically on disk. Things are moving in a more standards-compliant direction but we are not quite there yet.

Latest version without bake (on install)
From some quick testing of the latest versions of Firely Terminal, it seems 2.2.0 is the latest without bake functionality (and auto-bake on install). Installation instructions:
> dotnet tool uninstall --global Firely.Terminal
> dotnet tool install --global Firely.Terminal --version 2.2.0
Baking
The bake functionality has been introduced to provide packages with snapshots, because not all downstream tooling (most notably sushi) are able to generate these themselves.
Currently bake might be a little too aggressive by default, also recalculating snapshots for packages that already have them. In principle, this should not be a problem since snapshots are a just a cache for the calculation of all the layered differentials. Since snapshot logic still evolves it might even be desirable now and then to recalculate. But in newer versions we will look to:
Change the default to not recalculate when already provided
Provide a global setting to change that default to never calculate/always (re)calculate snapshots
This should prevent Firely Terminal from touching any files that don't need touching in the package cache. I'm not sure from your question if there was anything broken in the state of the shared FHIR cache after 'baking', given your use of 'thrashing' and 'destroying'?
Unbaking
The unbake command is intended to remove snapshots from a folder of packages. I see in my testing that it's not doing that, which I'll take as an issue to fix.

Related

Installshield not updating related DLLs on minor upgrade

I'm currently working with InstallShield to deploy a .NET Winforms app. I am new to InstallShield and have not enjoyed the learning curve. The Winforms app has three related DLL's which are not getting updated during a minor upgrade. For a minor upgrade I am changing the version from 1.0.001 to 1.0.002 for example. The package code is being changed for each build automatically.
I have tried adding the dll's to the [INSTALLDIR] and setting the property to "always overwrite". For some reason this causes the upgrade to also not update the main exe.
Tried changing the product code to force a major upgrade. This installed a new version alongside the old version, but the new version still had the old dll's.
Tried changing ReinstallMode from "omus" to "vomus". This had no effect at all.
Tried using REINSTALL=ALL, REINSTALLMODE=vomus. This did not update the dll's and also caused new installs to fail with message that application "is not marked for installation".
Tried changing the version from 1.0.00x to 1.1.00x. dll's still not updated.
I notice that when I view properties of these dll files, they have File Version = 1.0.0.0 and Product Version 1.0.0.0. Do I need to manually increase these versions in order for InstallShield to recognize that they have been updated?
Use one component per file and set each file to be keyfile in its own component. This avoid all sorts of component referencing and file replacement issues. Be aware that multi-file assemblies must share the same component as they are intended as one "atomic" file system unit.
In addition you must also increment the version number for each build or set REINSTALLMODE to emus instead of the default omus. Never use amus.
My advice: go with the file version updates - it is much more reliable. Like you state the File Version is used, it must be incremented. I like to auto increment the build version number (last digit). It has been a while, but I think you just replace the number with * and it auto increments. I think you can do this from the Visual Studio project property view.
Maybe read up on the file versioning rules as well. Essentially versioned files are version compared, and for unversioned files the create and modify date stamps are compared and the file is replaced if it is unchanged on disk. More sample info.
Remove the "always overwrite" flag you enabled for all the files you enabled it for. This flag may work poorly with patches if you ever need them and also with other features.
When a major upgrade creates two side-by-side installations it hasn't worked. What you are left with are two different products installed at the same time. There is good inline help in Installshield itself with regards to how a major upgrade is set up. Which version of Installshield are you using? The version bundled with Visual Studio may not feature this help material.
A note on major upgrades and "reverted files":
A warning on a classic major upgrade issue: be aware that changed, unversioned files not set to be permanent on original install may be uninstalled during a major upgrade and then reinstalled yielding the impression that they have been replaced, but they are actually deleted and recreated. These are typically important settings files like XML files or similar - and people struggle with this issue a lot. Major upgrades are essentially a sequence. The old product is uninstalled, and then the new one is installed or vice versa. In the former case the files may be uninstalled first and then recreated. This does not happen in the latter case if component referencing is done right because the files that are matching between products are not uninstalled, but retained and then overwritten if need be (according to the file replacement / versioning rules).

How can I safely remove old InnoSetup installations?

I use InnoSetup 5.5.1 (a) for my Delphi 6 software installations. A user just informed me that the old uninstall entries pile up in the Control Panel Remove Programs list if not uninstalled manually. I'm thinking of changing my install to remove old entries automatically but I don't want to remove old entries that are valid. Some users like to keep an old version or two around in case they don't like a later version or for fear a later version will break something.
For clarity's sake, I am referring specifically to the typical situation where a user usually accepts the default installation directory, overwriting the existing version. However, with the exception that some users may install one or more versions to a different directory just to preserve them, while choosing to overwrite most of the time. I found this SO post on removing old versions:
InnoSetup: How to automatically uninstall previous installed version?
But did not see any mention on knowing how to detect which of the old versions are superfluous or not.
Therefore, during an install, how can I automatically remove old uninstall entries automatically from the installed program list without removing any that the user actually wants to keep?
As long as you keep your AppId the same between different versions of your application, there will only ever be one Add/Remove entry for it, no matter how many times the user runs the installer. This is the normal and recommended design for a typical application where the user only has one copy of it and wants to keep it up to date.
If, however, the user chooses alternate installation locations in one or more of the later installs, then it's possible for the older location to get "orphaned" -- running the uninstall will remove the newer copy but leave the older one behind, with no Add/Remove entry. (It will still be possible to uninstall it by running the uninstaller manually.) It's normally recommended to avoid this situation by including these options in your [Setup] section:
DisableDirPage=auto
DisableProgramGroupPage=auto
Using these options will make Inno skip asking these questions for an upgrade install, which helps to prevent the user accidentally making orphan copies. (If the user does intentionally want to move the installation, they can still do it by uninstalling first.)
Note that I've sidestepped your question a bit, since as written it doesn't make sense -- if there are multiple copies, there's no possible way to automatically determine which are "superfluous", since that's purely a judgement call on the part of the user. What I've tried to explain here is that your design should aim to discourage this happening accidentally.

Best Practice: Removing obsolete artifacts from UCM ClearCase

We have a stream in ClearCase UCM. We create Views on this stream and fetch code for Build purpose. The total data copied is 10 GB. This is a huge codebase. I decided to investigate what makes it huge.
I found:
1) Multiple versions of Third Party applications are stored in
ClearCase
2) But only the latest Third Party applications are used by our
application
3) Lots of obsolete and redundant code is available
I proposed:
1) Removal of old versions of Third Party applications using rmname
(NOT rmelement) which will ensure the availability of element history
2) Removal of all redundant code
A total of 5 GB of obsolete data has been detected.
My Logic:
I think this is the best way to keep a stream of development clean. That is, the best way to organize a stream of development is to have the best, the cleanest and the leanest source code available.
Also, since all HISTORY will be available always in ClearCase, there is no need to panic about the deletion of elements.
I feel old, redundant and obsolete code and artifacts belong in HISTORY and not in the current stream of development.
Lastly, I feel ClearCase operations like making a baseline etc will take more time if we have bloat in the VOB. Since we do an incremental baseline for nightly builds, I do not think these obsolete items are baselined. But I feel all ClearCase operations are affected by bloat.
Is my LOGIC proper? Is my understanding of UCM ClearCase proper?
*Please let me know the best practice in such cases.*
People at my work place do not want to delete the obsolete files although 5 GB data is obsolete in the current stream.
Any help would be appreciated.
The best practice is actually separate from UCM in this case.
I too started by storing third-party binaries in ClearCase. It didn't scale well and the Vob started to get bloated, and simply too large to be managed (ie backed up) easily.
I now prefer storing third-parties in an artifact repository like Nexus, and add a little maven script to my build process in order to download the right binaries at the right versions, as declared in a pom.xml file.
Note that to remove old versions of a binaries from a vob, rmelem or rmver are really not advisable (risk of hyperlink corruption), but I used to do:
cleartool rmver -data aLargeBinary#/main/.../branch/OldVersion
That would keep the version in ClearCase, but would remove the version content (ie the large binary itself): that allowed for the Vob to get much smaller.
That being said, I agree with your general policies (especially regarding redundant code)

TYPO3 extensions and symlinks

Can I create a symlink to the local extension from aonther project folder? I have a common local-server and i need to implement same extension on all local project-installations. I tried to put the symlink, but some times i do not get expected output. I get it only after clearing the cache of that perticular project.
Your scenario is a common one I guess. But as Omar said, linking to the same code base of the extension through several typo3 instances is not a good practice.
But we have the same structure as yours, we realize this through SVN. All of our projects got a SVN repository and common extensions have their own repository. Through svn:externals the extensions are linked into the concrete project. This has the advantage that you can change the extension in the concrete project and after committing all other projects (that do have to update from svn though) contribute from it. I Think this would fit your needs, too.
If I understand your question correctly you have several Typo3 sites on the same server and want to share an extension between them using a symlink. I don't think that is a very great idea because many extensions use tables and every site normally has it's own database so you would have to do a lot of tinkering to get that to work.
Instead you could make all the modifications to the extension files in the typo3conf/ext/extension_name folder and then export the extension to a t3x file (Ext Manager in the Backend). This t3x file can be installed as a extension (Import extension) on all your other sites.
If you extension does not use a database and you are planning to make frequent changes then I guess you should be able to make that work (the symlink). Otherwise I recommend you use the first approach I described.
I have not tried this, but you should be able to install extensions globally in Typo3. What this means is that the given extension is placed inside '(typo3_src/)typo3/ext/' instead of 'typo3conf/ext/', presuming both sites use the same Typo3 Core/Source (and thus typo3_src is a symlink to the location of the core).
You can enable installing global extensions via the Install Tool. Once inside the tool, click on 'All Configuration', then search for allowGlobalInstall. Or put the following line into your localconf.php:
$TYPO3_CONF_VARS['EXT']['allowGlobalInstall'] = '1';
At last, but not least, you need to make sure the 'typo3/ext/' directory is writeable.
Hope this will be to some help. If you have any further questions, let me know :)
As Björn mentioned, I'd sugegst to install them globally. Mind you, updating the source will require to move the extensions accordingly..
As for "expected output": be aware that the code in these folders is cached in various ways (mainly page content and config settings), and hence not always run. This is the reason a change done from "outside" the current installation is likely not to propagate to your output without clearing these caches (as you have observed).
When you actually install an extension via the extension manager, the cache should (if correctly configured) be cleared (interested parties may search for clearCacheOnLoad in class.em_index.php to reveal a clear_cacheCmd('all')). There is a small checkbox, which is normally checked, during the installation process to accomplish this.
Omar's first approach is therefore, as I see it, the more easy way to get "expected output" and less jumbling around with global extensions.

Lightweight version control for small projects (prototypes, demos, and one-offs) [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
Background
I work on a lot of small projects (prototypes, demos, one-offs, etc.). They are mostly coded in Visual Studio (WPF or ASP.NET with code written in C#). Usually, I am the only coder. Occasionally, I work with one other person. The projects come and go, usually in a matter of months, but I have a constantly evolving set of common code libraries that I reuse.
The problem
I've tried to use source control software before (SourceGear Vault), but it seemed like a lot of overhead when working on a small project, especially when I was the only programmer. Still, I would like some of the features that version control offers.
Here's a list of features I'd like to have:
Let me look at any file in an older version of my project instantly. Please don't force me through the rigmarole of (1) checking in my current work, (2) reverting my local copy to the old version, and (3) checking the current version back out so I can once again work on it.
In fact, if I'm the only one on the project, I don't ever want to check out. The only thing I want to be able to do is say, "Please save what I have now as version 2.5."
Store my data efficiently. If I have 100 Mb of media in my project, I don't want that to get copied with every new version I release. Only copy what changes.
Let me keep my common library code files in a single location on my hard drive so that all my current projects can benefit from any bug fixes or improvements I make to my library. I don't want to have to keep copying my library to other projects every time I make a change.
However, do let me go back in time to any version of any project and see what the source code (including the library code) looked like at the time that version was released.
Please don't make me store a special database server on my machine that makes my computer take longer to start up and/or uses resources when I'm not even programming.
Does this exist?
If not, how close can I get?
Edit 1: TortoiseSVN impressions
I did some experimenting with Subversion. A couple observations:
Once you check something in to a repository, it does stuff to your files. It puts these hidden .svn folders inside your project folders. It messes with folder icons. I'm still yet to get my project back to "normal". Unversion a working copy got me part of the way there, but I still have folders with blue question mark icons. This makes me grumpy :-/ Update: finally got rid of the folder icons by manually creating new folders and copying the folders over. (Not good.)
I installed the open source plugin for Visual Studio (AnkhSVN). After creating a fresh repository in my hard drive, I attempted to check in a solution from Visual Studio. It did exact what I was afraid it would do. It checked in only the folders and files that are physically (from the POV of the file system) inside my solution folder. In order to accomplish item #5 above, I need all source code used by solution to be check in. I attempted to do this by hand, but it wasn't a user friendly process (for one thing, when I selected multiple library projects at once and attempted to check them in, it only appeared to check in the first one). Then, I started getting error dialogs when I tried to check in subsequent projects.
So, I'm a little frustrated with SVN (and its supporting software) at this point.
Edit 2: TortoiseHG impressions
I'm trying out Mercurial now (TortoiseHG). It was a little bit difficult to figure out at first, no better or worse than TortoiseSVN I'd say. I noticed an RPC Server on startup (relates to item 6). I figure it should be possible to turn this off if I'm not sharing anything with anyone, but it wasn't something I could figure out just by looking at the options (will check out the help later).
I do appreciate having my local repository as just a single .hg folder. And, simply throwing the folder in the Recycle Bin seemed to be all I needed to do to return everything back to normal (i.e., unversion my project). When I check in (commit), it seems to offer a simple comment window only. I thought maybe there would be a place to put version numbers.
My (probably not very clever) attempt to add a Windows shortcut (a folder aliasing my library projects) failed, not that I really thought it would work :) I thought maybe this would be a sneaky way to get my library projects (currently located elsewhere) included in the repository. But no. Maybe I'll try out "subrepos", but that feature is under construction. So, iffy that I'll be able to do items 4 and 5 without some manual syncing.
Any of the distributed source control solutions seem to match your requirements. Take a look at bazaar, git or mercurial (already mentioned above). Personally I have been using bazaar since v0.92 and have no complaints.
Edit: Heck, after looking at it again, I'm pretty sure any of those 3 solutions handles all 6 of your requested features.
Distributed Version Control Systems (Mercurial, Bazaar, Git) are nice in that they can be completely self-contained in a single directory (.hg, .bzr, .git) in the top of the working copy, where Subversion uses a separate repository directory, in addition to .svn directories in every directory of your working copy.
Mercurial and Subversion are probably the easiest to use on Windows, with TortoiseHG and TortoiseSVN; the Bazaar GUIs have also been improving. Apparently there is also TortoiseGit, though I haven't tried it. If you like the command line, Easy Git seems to be a bit nicer to use than the standard git commands.
I'd like to address point 4, common libraries, in more detail. Unfortunately I don't think any of them will be too easy to use, since I don't think they're directly supported by GUIs (I could be wrong). The only one of these I've actually used in practice is Subversion Externals.
Subversion is reasonably good at this job; you can use Externals (see the chapter in the SVN book), but to associate versions of a project with versions of a library you need to "pin" the library revision in the externals definition (which is itself versioned, as a property of the directory).
Mercurial supports something similar, but both solutions seem a bit immature: subrepository support built-in to the latest version and the "Forest Extension".
Git has "submodule" support.
I haven't seen anything like sub-respositories or sub-modules for Bazaar, unfortunately.
I think Fog Creek's new product, Kiln, will get you pretty close. In response to your specific points:
This is easily done through the web interface -- you don't need to touch your local copy or update. Just find the file you want, click the revision you want to see, and your code will be in front of you.
I'm not sure you can do things exactly like "Please save this as version 2.5", but you can add unique tags to changesets that allow you to identify a special revision (where "special" can mean whatever it wants to you).
Mercurial does a great job of this already (which Kiln uses in the back end), so there shouldn't be any problems in this regard.
By creating different repositories, you can easily have one central 'core' section which is consistent across various projects (though I'm not entirely sure if this is what you're talking about).
I think most version control systems allow you to do this...
Kiln is hosted, so there's no hit on performance to your local machine. The code you commit to the system is kept safe and secure.
Best of all, Kiln is free for up to two licenses by way of their Student and Startup Edition (which also gets you a free copy of FogBugz).
Kiln is in public beta right now -- you can request your account at my first link -- and users are being let as more and more problems are already resolved. (For some idea of what current beta users are saying, take a look at the Kiln Knowledge Exchange site that's dedicated to feedback.)
(Full Disclosure: I am an intern currently working at Fog Creek)
For your requirements I would recommend subversion.
Let me look at any file in an older version of my project instantly. Please don't force me through the rigmarole of (1) checking in my current work, (2) reverting my local copy to the old version, and (3) checking the current version back out so I can once again work on it.
You can use the repository browser of Tortoise Svn to navigate to every existing version easily.
In fact, if I'm the only one on the project, I don't ever want to check out. The only thing I want to be able to do is say, "Please save what I have now as version 2.5."
This is done by svn copy . svn://localhost/tags/2.5.
Store my data efficiently. If I have 100 Mb of media in my project, I don't want that to get copied with every new version I release. Only copy what changes.
Given by subversion.
Let me keep my common library code files in a single location on my hard drive so that all my current projects can benefit from any bug fixes or improvements I make to my library. I don't want to have to keep copying my library to other projects every time I make a change.
However, do let me go back in time to any version of any project and see what the source code (including the library code) looked like at the time that version was released.
Put your libraries into the same svn repository as your remaining code and you'll have global revision numbers to switch back all to a common state.
Please don't make me store a special database server on my machine that makes my computer take longer to start up and/or uses resources when I'm not even programming.
You only have to start svnserve to start a local server. If you only work on one machine you can even do without this and use your repository directly.
I'd say that Mercurial along with TortoiseHg will do what you want. Of course, since you don't seem to be requiring much, subversion with TortoiseSvn should serve equally well, if you only ever work alone, though I think mercurial is nicer for collaboration.
Mercurial:
hg cat --rev 2.5 filename (or "Annotate Files" in TortoiseHg)
hg commit ; hg tag 2.5
Mercurial stores (compressed) diffs (and "keyframes" to avoid having to apply ten thousand diffs in a row to find a version of a file). It's very efficient unless you're working with large binary files.
Symlink the library into all the projects?
OK, now that I read this point I'm thinking Mercurial's Subrepos are closer to what you want. Make your library a repository, then add it as a subrepository in each of your projects. When your library updates you'll need to hg pull in the subrepos to update it, unfortunately. But then when you commit in a project Mercurial will record the state of the library repo, so that when you check out this version later to see what it looked like you'll get the correct version of the library code.
Mercurial doesn't do that, it stores data in files.
Take a look on fossil, its single exe file.
http://www.fossil-scm.org
As people have pointed out, nearly any DVCS will probably serve you quite well for this. I thought I would mention Monotone since it hasn't been mentioned already in the thread. It uses a single binary (mtn.exe), and stores everything as a SQLite database file, nothing at all in your actual workspace except a _MTN directory on the top level (and .mtn-ignore, if you want to ignore files). To give you a quick taste I've put the mtn commands showing how one carries out your wishlist:
Let me look at any file in an older version of my project instantly.
mtn cat -r t:1.8.0 readme.txt
Please save what I have now as version 2.5
mtn tag $(mtn automate heads) 2.5
Store my data efficiently.
Monotone uses xdelta to only save the diffs, and zlib to compress the deltas (and the first version of each file, for which of course there is no delta).
Let me keep my common library code files in a single location on my hard drive so that all my current projects can benefit from any bug fixes or improvements I make to my library.
Montone has explicit support for this; quoting the manual "The purpose of merge_into_dir is to permit a project to contain another project in such a way that propagate can be used to keep the contained project up-to-date. It is meant to replace the use of nested checkouts in many circumstances."
However, do let me go back in time to any version of any project and see what the source code (including the library code) looked like at the time that version was released.
mtn up -r t:1.8.0
Please don't make me store a special database server on my machine
SQLite can be, as far as you're concerned, a single file on your disk that Monotone stores things in. There is no extra process or startup craziness (SQLite is embedded, and runs directly in the same process as the rest of Monotone), and you can feel free to ignore the fact that you can query and manipulate your Monotone repository using standard tools like the sqlite command line program or via Python or Ruby scripts.
Try GIT. Lots of positive comments about it on the Web.

Resources