Illegal Byte sequence in a file name while checking out git branch - bash

I am using git in cli to change the current branch:
git checkout dev
and it produces:
fatal: cannot create directory at 'app/src/androidTest/java?com': Illegal byte sequence
As answered in this question and this one, I tried:
LC_ALL=C git checkout dev
or
LC_CTYPE=C git checkout dev
but I am getting the same error as shown above.
Running:
git status
shows that some of the files were changed by the checkout, but I am still on the master branch.
How can I remove the file causing the problems or how can I checkout the branch without getting this error?

The locale only affects how things are displayed. If the file name contains a character which isn't allowed by the file system, no amount of locale tweaking can fix that.
I can't think of a way to force a file system to let you create a file which then cannot be used, or a good reason to want to be able to do that.
Probably as a workaround, create a virtualized host with a bare-bones Linux system formatted to permit old-style 8-bit file names (Latin-1 or CP1252 if you can live with the unsavory Windows flavor of that), check out the file there, rename and commit the rename back to git. You still won't be able to check out versions of the source tree from before the rename.

I have found a few files in various github projects that are not compatible with one or another operating system. Files with a ".nul" or ".con" extension are a real pain on windows, for example. It isn't a problem exclusive to git. For example Subversion will abort nastily if it can't restore a file for local naming reasons.
In some cases the file may have been uploaded in error. If that is the case for your own projects it should be possible to use the git tools to list the archive and perform a delete of the file from the archive without actually instancing the file locally.
In other cases perhaps that particular file is not significant, and perhaps can be ignored. Perhaps a test will fail if it is missing?
One trick I have used is to stop the whole folder containing that file from being synced by manually creating the directory path, but for the last element, create an empty file instead of a folder. Of course, now the whole test suite will fail.
When the version control tries to do the checkout, it will simply fail to restore the folder, rather than giving a fatal error.
Of course, that only works if the folder is non-critical, e.g. some test files.
The alternative is to piecemeal check-out all but the problem file, but that can be a tedious sequence of checkouts. But you can use this attack to restore the rest of the folder that you omitted using the above technique. Alternatively, locally drag the files from the zip download if they are non-critical.

Related

Git - fatal: unable to checkout working tree

I uploaded my Laravel project to a shared server, now I am trying to clone the project from a local computer and I am getting the following error:
error: invalid path 'public/C:\xampp\htdocs\pfuxelacolab\storage\logs/laravel.log'
fatal: unable to checkout working tree
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'
Help Please...
You have a repository with a commit that you, personally, cannot use on your system. A Linux user can use this commit, but you can't.
The problem here is the file's name:
public/C:\xampp\htdocs\pfuxelacolab\storage\logs/laravel.log
In a Git commit,1 there are no folders: files simply have long names like this one. Each forward slash / in a committed file's name—and only the forward slashes—will, on a Windows system, be turned into the appropriate backwards slash \ and Git will break up the path into pieces such as public and laravel.log and so on.2 Git will then do what your OS demands: treat each of the various pieces as folder-and-file parts, and make any folders needed so that it can create your OS's idea of a file named laravel.log in your OS's idea of a folder. Git itself doesn't hold with such silly notions: you just have a file with a long name with forward slashes in it, but OSes have these dumb and silly folder-and-file requirements, so, okay, Git will deal with it.
In any case, this file's name includes C: right in the middle—not up front, where some DOS/Windows software might be able to deal with it, but in the middle, after public. So Git is trying to create a new folder, named either C: or C:\xampp\htdocs\pfuxelacolab\storage\logs, within a public folder, and that is never going to work: Windows says you can't do that. Linux, by contrast, says Oh you'd like to create that as a folder name? Sure, done!
So, someone on a Linux or similar system can check out this commit. Having done so, they can fix the file's name or remove the file entirely and make a new commit that doesn't have this bizarro name with the colon and backslashes inside it. Remember, Git demands forward slashes. It will make forward slashes for its own commits, from the OS's folders and files. But it will also take any file names with embedded colons and backslashes found on a Linux system, and make a commit from those, that can't be extracted on a Windows system. That's what someone did: they created a folder named C:\xampp\htdocs\pfuxelacolab\storage\logs in a folder named public, and created a file named laravel.log in that, and then added that to Git's index and committed it, presumably on a Linux system.3
The easiest way for you to deal with this will be to boot up a Linux system (perhaps in a VM) and check out the commit in question and make a corrected one to use instead. Whether you can do this at all in WSL, I have no idea.
1Technically, this claim is true not of the commits themselves, but of Git's index. But commits move "through" the index on the way out to your working tree, and are made from the index, so the condition applies anyway.
2Since I don't use Windows, I don't know whether Git will try to break apart the backslashed pieces here too—but it won't matter because of the C: part. It would be interesting to create a file named test\path/file on a Linux system, commit it, and try to check that commit out on Windows, though. Does Windows Git make a test folder? If not, does it (whether intentionally or merely accidentally) use an existing test folder and make a path folder? But Windows forbids colons in path names, and the embedded C: part, with or without any subsequent part, is the first and fatal problem here.
3While I keep saying "Linux" here, any OS that allows this will suffice, so Solaris or one of the BSDs would do the trick too. You could even do this on macOS.

Git Issue: "cannot update the ref 'HEAD': unable to append to '.git/logs/HEAD': Bad file descriptor"

I use Git on Windows 10 to track version control for projects I keep on my company's shared network drive. I have a project for which I have been using Git for months with no trouble. I recently created a new branch 'development' and have been using it as a testing environment for new changes before merging the changes back to master. This arrangement has been working fine for the last few commits until now.
I made changes in the testing branch then went to commit my changes but received the below error:
$ git commit -m "this is a commit description"
fatal: cannot update the ref 'HEAD': unable to append to '.git/logs/HEAD': Bad file descriptor
I now notice this error with many different operations, including git checkout <branch>. The error in this case is:
$ git checkout branch-name
error: unable to append to '.git/logs/HEAD': Bad file descriptor
I'm relatively new to this stuff so apologies if there is an obvious solution here. Any help is appreciated!.
The Bad file descriptor error is coming from your operating system. In C programs (which this particular Git program is), to write to a file, the program is required to first open the file, which produces a file descriptor: a number, usually a small integer such as 3 or 4. Then, to write to that file, the program should make an OS call of the form write(fd, data, len) where fd is an int variable holding the number returned earlier by open.
The Bad file descriptor error means that the number being passed to write is incorrect in some way: perhaps it is out of range or was never returned by an open call, or perhaps it refers to a file that is open, but not for writing. In general, if there are no programming mistakes, this should not happen.
You did, however, mention a shared drive. On Unix-like operating systems, when using networks to access files, things can go wrong that simply cannot happen if the drive is local.1 On these Unix-like systems you would normally get an ESTALE error ("Stale network file handle") rather than an EBADF ("Bad file descriptor") error here, but perhaps this is what is going wrong. In general it's unwise to attempt to use Git across any sort of sharing system: Git really wants full and total control over everything it does, and putting it on any kind of shared system—network drives, Dropbox, OneDrive, Google Drive, etc.—tends to be a bad idea.
This arrangement has been working fine for the last few commits until now.
This behavior is highly characteristic of attempts to use shared file systems with Git: it works great, until it doesn't. If you're lucky, the "doesn't" is merely a minor malfunction like this; if not, it wrecks the entire repository, and since you've been relying on the sharing instead of making clones, there may not be any backup clones from which to fix things.
1Technically, similar things can go wrong, if, e.g., the drive catches on fire. But usually by then the program has stopped running for other reasons. 😅
I was able to resolve this issue by performing the below actions:
Create a local desktop directory
$ cd '<local-directory>'
Clone the repo into this directory
$ git clone '<origin-directory>' .
Make and commit the backlogged changes
$ git commit -m "commit desc"
Break link to remote origin source
$ git remote remove origin
Delete everything in original directory
Copy the contents of the local directory back to the original directory
It seems to work for now but if this happens again I'll just repeat these steps.

How to solve a Mercurial case-folding collision?

I use Mercurial as source control with the main repository managed on KILN.
At one point in time I changed my iOS project name from WeatherTimeMachine to weathertimemachine.
This resulted in a case change of several files and folders:
WeatherTimeMachine.xcode
WeatherTimeMachine_Prefix.pch
WeatherTimeMachine-Info.plist
In the meantime I've added a tag to a revision in KILN... So I now have:
a head in KILN
a head on my local repo with case changes
When trying to merge I get the following error message: "Mercurial case-folding collision"
How can I fix this?
If you're on Mac OS X, you don't need to export your repository to Linux or another foreign case-sensitive file system as suggested by the Mercurial documentation. Just use Disk Utility to create a case-sensitive, journaled disk image slightly bigger than your repository, copy your repo there, then remove the conflicting files and commit.
I have found some information here: FixingCaseCollisions, but somehow this did not work for me. Here is how I managed to solve this issue:
Make a copy of your existing repository folder (for safety). For example:
cp -r WeatherTimeMachine WeatherTimeMachineCopy
Fool mercurial into thinking the problematic revision is the current tip:
hg debugsetparents <bad revision>
hg debugrebuildstate
Remove the files which are causing the problem (-f is required to force the removal). Here is an example:
hg rm -A -f WeatherTimeMachine-Info.plist
Once all problematic files have been removed, commit the changes
hg ci -m "fixed collision-folding issue" -u michael
Then restore mercurial to the proper revision
hg debugsetparents tip
hg debugrebuildstate
After this the merge is possible and the problem is gone.
And now I can happily resume working with MacHg to manage my Mercurial repository and push my change sets to KILN.
This is a no-programming no-hg answer, but it solved my mercurial case-folding problems once and for all! For now anyway..
I gave up trying to avoid and "fix" the case-collision problems. That just looks ugly and you can never really "solve" the problem, only can do a workaround.
The only way (that I can think of) to really solve the problem is to have a case-sensitive filesystem. No need to reformat your entire disk, a single partition will do the job nicely.
I used the Disk Utility app that comes with the os, pretty straightforward, just remember to select Mac OS Extended (Case-sensitive, Journaled) when creating new partition. Also, note that the Disk Utility can resize partitions only by moving the end (not the beginning) of partition.
You can probably create a symlink to where your old source code lived, so no need to change the IDE settings and stuff (but I haven't tried this, just happy with the new partition).
We resolved this without resorting to a case-sensitive filesystem by issuing HG rename commands. Say you are having trouble because "Foo.txt" needs to be called "foo.txt":
> hg rename Foo.txt Foo.txt.renamed
> hg rename Foo.txt.renamed foo.txt
We encountered this problem when a file was deleted and then later re-created in the main repository with the same name, but different case. A branch repository that was created before these changes could not then be merged in, despite the changesets from the main repository having been pulled.
For Mac OS X, what worked for me is to simply copy the folder (duplicate works - cmd-D) and continue working on it, from the new path.
OSX same I wanted to clone a repo and I had case-folding error preventing me top clone.
If the repository has multiple commit simply clone an earlier revision with this command:
hg clone -r 7
Then add anything conflicting to the .hgignore file and update.

Subversion control No Such file or directory. Can't open file

Error message :
"svn: Can't open file '/Users/username/Projects/myproject/trunk/project/.svn/text-base/filetoupdate.h.svn-base': No such file or directory"
Question:
I have an issue I've replaced a file in a project (in Xcode) with a new file (For reference and if this makes a difference, the new file has the same name as the one I deleted previously).
Now when I try to commit my changes in Xcode I get the error message detailed above and am unable to commit the changes (i.e. adding the new file).
In the file system view (in Xcode on the left hand side of the screen) the file has an R next to it (indicating Replaced in the repository).
Does anyone know how to fix it so I can commit the files?
Thanks
There is a bug or limitation in Subversion when using case-insensitive filesystems:
https://superuser.com/questions/303348/cant-checkout-in-subversion
This bug normally shows up when checking out a repository that contains two files whose names differ only in case. Of course, these cannot exist at the same time in the same directory on a case-insensitive filesystem. SVN could give a much more helpful error message, but it can't really solve the problem.
Your issue is a bit different because I assume the file filetoupdate.h (with the old case) no longer exists in your filesystem. So it's not a case conflict in the working directory. But I guess that SVN is trying to create the file in .svn/text-base with the new case, while the old one still exists, and that is failing (for the same reason).
You could try deleting the file from Subversion first, keeping the local copy (untested). The new copy must be removed from SVN control for the commit to succeed:
svn rm --keep-local --force FileToUpdate.h
And the old copy must be removed as well, to allow us to add the new copy later:
svn rm --keep-local filetoupdate.h
Commit this change:
svn commit
Now hopefully you can add the new file to version control:
svn add FileToUpdate.h
If that doesn't work, you might need to blow away the whole checkout and start again with a fresh one.
Are you on a Mac or Windows? Those have case-insensitive filesystems which causes the above problem when
a file currently exists with the same name but with different cases.
To fix , checkout out the tree on a Linux machine, then "svn rm" one of the files.
Maybe your local version has permission issues. Check if your user have the permissions to write for the .svn directories.
good luck
It looks like something got confused somewhere. To fix, I simply copied the offending files, saved them under a new name. Removed the originals from the project and the added the copied (renamed) version of the file to the project.
It seems to be that SVN doesn't like it if you add and remove a file with the same name. I tried cleaning the SVN through terminal, but it had no affect on this issue. But changing the name did work for me.

How to recover from git-svn putting a different cased dulplicate file in the repository?

Git-svn allowed for a duplicate filename, just with different case, to be added to our subversion repository.
On Windows this meant that subversion could not checkout the file, complaining of a duplicate.
Another developer deleted the incorrectly cased version from the repository. Now when trying to do a git-svn rebase I get a "could not detach HEAD" message and a complaint about the file name in question being untracked and needing to be overwritten. Deleting the file makes git-svn complain that the file needs to be updated.
Is the only solution to copy the repository to a machine with a case sensitive filesystem do the rebase then move it back?
I understand that git-svn isn't ready for real world work on Windows but I'd like to recover from this mess it has created.
Best current answer (I don't know if this existed back when this was originally posed):
git config core.ignorecase true
Then redo the rebase. It will proceed without error -- and will even properly handle the case change in the filename.
This isn't specific to git-svn. It can happen in a straight 'git rebase'.
I hope that helps the next person....
I ran into a similar problem with Git (alone, not even with git-svn) getting very confused about changing the case of filenames when using a case-insensitive Mac filesystem. I didn't find a solution to fix the repository on the case-insensitive filesystem, but instead created a new case-sensitive volume in a .dmg file, mounted that, and used Git on that volume instead.
You may find that some of the Git "plumbing" commands might be helpful. For example, git checkout-index has a -f flag to force overwriting of existing files (also use -a with that command otherwise it might not do anything). The plumbing commands are generally more powerful (and more dangerous) than the porcelain commands, but may give you the flexibility you need to fix this.
The same way you always solve this problem under Windows. You have to rename one of the files, then you can restore the other one.

Resources