We've got a repo on GitHub with several tags with quotes " in the name. Occasionally, some Windows hosts will fail to fetch with the error below. I don't know why some Windows hosts fail to fetch and others don't, if we solved that that would be great.
fatal: Unable to create 'C:/src/code/tags/some_tag_"with_quotes".lock': Invalid argument
As a workaround, we deleted the tags on GitHub, but inevitably someone who has this tag locally runs a git push and pushes the tag back into GitHub. Having everyone reclone will be painful.
These tags cannot be deleted locally (on Windows), the same error is generated.
Note: This answer has been torn apart and rewritten from its original version
I wonder if you're having issues with the filesystem's permitted characters in filenames. For example, according to Wikipedia, FAT does not permit double-quotes ("). According to Wikipedia, NTFS will accept double-quotes only if a filename is in the Posix namespacem, but I cannot seem to find that information anywhere else. Further, Microsoft seems to recommend avoiding double-quotes in filenames. Posix appears to only allow A-Za-z_.- in "portable" filenames¹. See Wikipedia on filesystem limitations and filesystem capabilities for more.
Perhaps you might try the Cygwin version of Git. It appears that Cygwin manages to support quotes in filenames on Windows.
Related
I am trying to checkout a git repo (this) which requires the creation of a folder with unicode characters (the name is uni¢𐍈d€), where the missing character (I don't know how to render it here) is the Hwair gothic character. On my workstation, it works fine, but when I try to do it on a server (RedHatEnterpriseServer, where I do NOT have sudo powers), I get the error
fatal: cannot create directory at 'python/ycm/tests/testdata/uni¢𐍈d€': Permission denied
As a side note, I have tried to create the folder manually, and the hwair character is the only one that gives problems, meaning, that the folder uni¢d€ can be successfully created.
The LANG variable is set to en_US.UTF-8 on both systems. I have tried to find the differences between my workstation fonts and the server fonts. Grepping 'hwair', I found a font on my workstation that was missing on the server, so I copied the font folder in ~/.fonts on the server, and ran fc-cache, but that didn't work.
Google-ing hwair and grepping here and there on my workstation and the server, I found out that the 'hwair' character should be in the Code2001 font, which I think should be installed on both systems, since both system have the file /etc/fonts/conf.d/69-unifont.conf, which are identical, and contain the line
<family>Code2001</family> <!-- plane1 and beyond -->
I never really dealt with fonts at this fine level, so I'm not really sure if this is something solvable by adding some fonts locally or not (perhaps the system inhibits such particular fonts). I don't even know if copying fonts files to ~/.fonts/ and running fc-cache is enough or if there is something more to it.
So I guess the questions are:
Is the problem related to missing fonts on the server?
If yes, how can I add the missing font?
If no, is it related to something I cannot fix (perhaps it requires sudo privileges)?
Edit: the folder that cannot be created is part of the git repository. The error appears when cloning the repo (or anytime you try to checkout the master branch). I don't think there is an issue with permissions, otherwise it would fail also on my workstation (the permissions of the 'host' folder are the same). Also, trying to do mkdir $( printf 'uni¢\xF0\x90\x8D\x88d€' ) does not work either. The bash seems indeed to interpret correctly the unicode encoding, since when printing 'cannot create...', the name of the folder DOES show the hwair symbol.
Edit2: if you think the question has some flaws, please, add a comment rather than just voting for closing. I'm happy to change/edit/improve if need be.
I had the exact same issue!
It seems that for some - unknown to me - reason it is not possible to have that Hwair character in filenames on some filesystems as that other guy has already mentioned.
So as a workaround I suggest that you look at the available filesystems with something like:
df -Th
and then e.g. use a symbolic link to a filesystem which uses ext4.
FYI: The filesystem where it didn't work for me is of type nfs4.
I've got a Jenkins instance running on Windows and for one item, I need to check out from a CVS repository. As it is, the built-in CVS plugin for Jenkins is checking it out with Windows CRLF line endings. However, some of what needs to be done requires that the files have Unix style LF line endings.
Running a dos2unix like utility over the entire repository seems painful to do safely and reliably.
I could use Cygwin's cvs to do it by running a shell script build step, but I would prefer something better integrated with Jenkins.
So, short of writing up a script to call an external CVS, or fixing it afterwards with dos2unix, can Jenkins check out a CVS repository with Unix line endings on Windows?
Edit:
I've since come across this bug report indicating that this issue has been known since 2012 and not much has been done about it. I'll leave the question open in case someone does know of an alternative to what I've already mentioned.
I've added dos2unix steps pretty often in my build scripts for windows (automake chain is very sensitive to this stuff), it might seem a bit ugly/fragile, but it's ok I think for a Jenkins build : if it breaks because of new files you'll get build issues.
In my experience, there are always a couple of platform specific steps in the build, so a bit of custom code to support windows seems ok.
An alternative might be to tag your files as "binary" on the CVS server
(-kb flag) and make sure they are encoded correctly up there, this will avoid any transcoding at checkout time, but it seems more intrusive to me. I haven't actually used this for sources, I think it might mess the "diff" command.
A last idea might be to look more precisely at what tools are expecting the LF style endings on your win box, there might be mingW or windows versions you could deploy that support CRLF.
My vote still goes to the dos2unix solution overall.
If you don't want to make all your files binary in the repository, you can do each checkout or update as binary by adding "-kb" to the command.
Background: this is running on a Windows 2008 Server.
https://www.mercurial-scm.org/wiki/Workflows#Feature_separation_through_named_branches
I'm a Mercurial newbie, and am trying to follow the advice above where it says to "Merge default into your feature as often as possible" -- and I've done this a couple of times previously today, already, with other files.
However, this newest change just won't merge.
When I do the 'hg merge default' I get the error "abort: Access is denied". After googling around, I see that some people reported actually having permissions problems on the files in question. There's nothing special about the permissions on the file in question, at least not that I can see. I'm a Linux person by training, not a Windows person, so fundamentally I don't really understand Windows file permissions. Cygwin claims that the file in question is 644 (i.e., I can write to the file), which is the same set of permissions as every other file that has previously been successfully touched by the hg merge process in the past.
I took a look at the DOS 'attrib' command and it doesn't show a 'read-only' flag next the file in question, either.
If this is less a Mercurial question than it is a Windows permissions question, I'm happy to modify the tags further as well.
I'm assuming that this "abort: Access is denied" error refers to the file in the changeset that needs to be merged, and not to one of the .hg/ files, but it's a very cryptic error message -- it doesn't say which file has an access issue (and there is only one single file in the changeset that was changed -- I purposely tried to be VERY simple with this test).
UPDATE:
Did an 'hg rollback' and tried the same test of commands in the same order, leaving the files open, only with a completely different file (lib/blort.html instead of blah/foo.html).
The 'hg merge' worked just fine.
So there's probably something particularly hinky about the permissions on the specific file I started with (foo.html), or the directory that is its parent.
Another Edit
Definitely something off about the parent directory, as I had the same problem w/ another file in the directory, but the problem does not manifest elsewhere in the directory structure, only in this one directory.
Just checked the permissions on lib/ versus blah/ and they both seem the same in Cygwin, but that's only a vague approximation of the Windows ACLs. When examining those by right-clicking each directory and examining the 'Properties' Security tabs, they again seem the same for both directories, but I think the crux of the problem is that I really don't quite grok ACLs.
Is there some DOS command-line tool, like 'attrib' only more powerful, that, like 'ls' in linux, would give me more information than the confusing jumble of checkmarks I get in the 'Properties'?
Hopefully the final edit
How to diff Windows permissions
'iCacls' seems to be the correct tool for the job, and seems to have solved the problem, but I'd like to duplicate this before accepting/closing.
SUMMARY
When in doubt, use --debug on your Mercurial commands. This is a little bit 'duh' now, but thanks to #lazy-badger for pointing it out.
You can fix your Server 2008 permissions using a command like icacls path /grant domain\user:(OI)(CI)F (see e.g. http://www.petri.co.il/forums/showthread.php?t=23207), but be sure to do this from an escalated CMD window ('Run As Adminstrator'). Again 'duh' to Windows people, not so obvious to a Linux person (instead I went googling around for some DOS equivalent to sudo).
You may get an access denied error on Windows if some other program has the file open. You can try closing programs that might have any files in that tree open, or rebooting the computer (which would, of course, close all programs).
Just to note "Access denied" on merge can have two different roots: mentioned by #emil and wrong or nonexistent rights for user, under which credentials hg was started, to create temporary files in $TEMP folder
I've been using Subversion for code control with TortoiseSVN to interface with the server for the past few months, and in general it's been going great! However, occasionally my FoxPro IDE will change the case of a file extension without warning where "program.prg" becomes "program.PRG") TortoiseSVN apparently takes this to mean the first file was removed, becoming flagged as "missing" and the second name comes up as "non-versioned", wreaking havoc on my ability to track changes to the file. I understand that Subversion has it origins in the case-sensitive world of *nix but, is there any way to control this behavior in either Subversion or TortoiseSVN to be file name case-insensitive when used with Windows?
Unfortunately, Subversion is case-sensitive. This is due to the fact that files from Subversion can be checked out on both case-sensitive file systems (e.g., *nix) and case-insensitive file systems (e.g., Windows, Mac).
This pre-commit hook script may help you avoid problems when you check in files. If it doesn't solve your problem, my best suggestion is to write a little script to make sure that all extensions are lowercase and run it every time before you check in/check out. It'll be a PITA, but maybe your best bet.
Windows does support case sensitivity, but you must send it the correct POSIX flags on CreateFile from the Windows API! A registry key may need changed (SFU/Tools for Unix and Ultimate Windows 7 has this registry entry already set so windows supports case sensitive file names).
Windows is designed off of Unix, but things such as Explorer.exe and other programs are designed to disallow case sensitivity for backwards compatibility and security (mostly when dealing with dos executing notepad.exe vs. NOTEPAD.EXE, where all caps is a virus or malware).
But Vista+ has security attributes which makes this obsolete.
TortiousSVN just doesn't support passing this posix flag while making and renaming files.
I use TortoiseSVN with VFP, and it mostly-seamlessly handles the case flipping. The only time it doesn't is if I have the file open in the IDE when I try to do the commit: the file lock VFP holds confuses it. Is this where your problem comes in, or are there other issues?
I did a presentation at FoxForward last year about using VFP with Subversion: most of the presentation dealt with the command line, but there are a couple of slides at the end that have links to tools that help you work with Subversion in VFP. http://docs.google.com/Presentation?id=dfxkh6x4_3ghnqc4
Kit, you comment above that VFP's binary-based source files are tough to work with in Subversion. The link I gave above mentions a couple of tools to make it easier, but the one I work with is Christof Wollenhaupt's TwoFox utility -- it converts a VFP project to text-only. You have to run it manually, but I don't have a problem with that.
http://www.foxpert.com/docs/cvs.en.htm
I believe the random upper and lower case on the extensions isn't random at all.
I remember testing on this. If you modify a program from the project manager.
By clicking on the modify button let's say. And then save the changes the extension is lower case. If you do a modify command from the command window and save the changes the extension is upper case. Apparently the coders at Microsoft didn't worry about the extension case being the same.
TortoiseSVN has a Repairing File Renames feature. It requires manual intervention and it actually issues a file rename operation to be committed but nonetheless addresses current use case by keeping file history.
Nope you sure can't. SVN is case-sensitive unless you were to rewrite the code somehow ... it is open-source.
We had a similar problem and I found a better solution than the ones exposed here, so I'm sharing it now:
For commits done manualy, now TortoiseSVN fixes the case of the file names automatically: it renames the local files to match the case of the versioned files (just by opening the commit window in that path), so there should be no problem with that.
For automated commits you cannot use TortoiseSVN, as it requires you to manually confirm the commit (it opens the commit window with a specific message, but you still have to click ok). But if you directly use Subversion (svn) to make an automated commit, then you will have the case-sensitive issue on that commit, as Subversion is still case-sensitive...
How to solve this for automated commits? Well, I tried a mixed approach: creating a batch file called FixCaseSensitiveFileNames.bat that you may call passing the path you want to fix before the commit, for example: call FixCaseSensitiveFileNames.bat C:\MyRepo. The batch file opens TortoiseSVN for a manual commit, and that automatically fixes the file names, but then it closes the commit window after a predefined pause, so you can continue with the automated commit with the case-sensitive file names already fixed. The pause is emulated with a local ping, and you can change the duration by changing the -n argument, which is the number of tries. If you don't make a long enough pause, it exist the risk to close the TortoiseSVN window before it makes its magic fix. Here it is the code of the batch file:
#echo off
REM *** This BAT uses TortoiseSVN to fix the case-sensitive names of the files in Subversion
REM *** Call it before an automated commit. The Tortoise commit fixes this issue for manual commits,
REM *** so the trick is opening the commit window and close it automatically after a pause (with ping).
REM *** %1 = path to be fixed
start TortoiseProc.exe /command:commit /path:"%1"
ping localhost -n 10 >nul
taskkill /im TortoiseProc.exe
This totally solved the issue for our automated daily build process. The only problem I see is a window will open for a few seconds, which was not a problem for our daily build, but if that is a problem for you there could be workarounds too...
How do you store file permissions in a repository? A few files need to be read-only to stop a third party program from trashing it but after checking out of the repository they are set to read-write.
I looked on google and found a blog post from 2005 that states that Subversion doesn't store file-permissions. There are patches and hook-scripts listed (only one url still exists). Three years later does Subversion still not store file permissions and are hooks the only way to go about this? (I've never done hooks and rather use something that is native to Subversion.)
SVN does have the capability of storing metadata (properties) along with a file. The properties are basically just key/value pairs, however there are some special keys like the 'svn:executable', if this property exists for a file, Subversion will set the filesystem's executable bit for that file when checking the file out. While I know this is not exactly what you are looking for it might just be enough (was for me).
There are other properties for line ending (svn:eol-style) and mime type(svn:mime-type).
There's no native way to store file permissions in SVN.
Both asvn and the patch from that blog post seem to be up (and hosted on the official SVN repository), and that's a good thing, but I don't think they will have such metadata handling in the core version any time soon.
SVN has had the ability to handle symbolic links and executables specially for a long while, but neither work properly on Win32. I wouldn't hold my breath for another such non-portable feature (though it wouldn't be too hard to implement on top of the already existing metadata system.)
I would consider writing a shell script to manually adjust file permissions, then putting it in the repository.
One possible solution would be to write a script that you check in with the rest of your code and which is run as the first step of your build process.
This script runs through your copy of the codebase and sets read permissions on certain files.
Ideally the script would read the list of files from a simple input file.
This would make it easy to maintain and easy for other developers to understand which files get marked as read-only.
Since this wasn't fully said in previous responses yet. I hate to resurrect zombied threads though.
Since adding permission support for SVN would have to accommodate multiple OS's and permission types, NFS, POSIX, ARWED, and RACF
This would make SVN bloated, possibly clash with conflicting permission types like NFS and POSIX, or open up possible exploits/security vulnerabilities.
There are a couple of workarounds.
pre-commit, post-commit, start-commit are the more commonly used, and are a part of the Subversion system.
But will allow you to control the permissions with what ever programming language you like.
The system I implemented is what I call a packager, that validates the committed files of the working copy, then parses a metadata file, which lists out the default permissions desired for files/folders, and any changes to them you also desire.
Owner, Group, Folders, Files
default: <user> www-user 750 640
/path/to/file: <user> non-www 770 770
/path/to/file2: <user> <user> 700 700
You can also expand upon this and allow things such as automated moving, renaming them, tagging revision by types, like alpha, beta, release candidate, release
As far as supporting clients to checkout your repository files with permissions attached to them. You are better off looking into creating an installer of your package and offering that as a resource.
Imagine people setting their repositories with an executable in it set with permissions of root:www-user 4777
This is the updated link for SVN patch which handles unix style file permissions correctly. I have tested out on fedora12 and seems to work as expected:
I just saved it /usr/bin/asvn and use asvn instead of svn command if i need permissions handled correctly.
Many answers have stated that svn does not store file permissions. This may be true, but I was able to solve a dll file without execute permissions problem simply by these steps:
chmod 755 badpermission.dll
mv badpermission.dll ../
svn update
svn rm badpermission.dll
svn commit badpermission.dll -m "Remove dll to fix permissions"
mv ../badpermission.dll .
svn add badpermission.dll
svn commit badpermission.dll -m "Add the dll back to fix permissions"
rm badpermission.dll
svn update
badpermission.dll comes back with execute permissions
#morechilli:
The asvn wrapper from my earlier post and the blog in the OP's post seems to do what you're suggesting. Though it stores the permissions in the corresponding files' repository properties as opposed to a single external file.
I would recommend to generate permissions map using mtree utility (FreeBSD has it by default), store the map in the repository, and, as was mentioned above, run a script that would restore proper file permissions from the map as the first step of the build process.
Locking would not solve this problem. Locking stops others from editing the file. This is a third party application which gets run as part of the build process that tries to write to a file - changing it - which breaks the build process. Therefore we need to stop the program from changing the file which is simply marking the file read-only. We would like that information to be held in the repository and carried across checkins, branches, etc.
Graham, svn doesn't store permissions. Your only option is to wrap your call to svn in a script. The script should call svn with its arguments, then set the permissions afterward. Depending on your environment, you might need to call your script svn and tweak your PATH to ensure it gets called.
I quite like morechilli's idea to have the list of files and permissions checked into the repository itself.
We created a batch file to do this for us. Would prefer actual support in subversion though...
Consider using svn lock to disallow others from writing to the file.