.gitattributes don't work properly on mac and windows - windows

On my project i use computers with different OS, one is Mac second is with windows. When I use git every change is shown as whole document change. The reason is different end-of-line in these two OS. I read this https://help.github.com/articles/dealing-with-line-endings/ and made a .gitattributes file in the root folder but the problem still exists. This is my .gitattributes file:
# Set the default behavior, in case people don't have core.autocrlf set.
* text=auto
# Explicitly declare text files you want to always be normalized and converted
# to native line endings on checkout.
*.css text
*.html text
*.js text
# Declare files that will always have CRLF line endings on checkout.
*.sln text eol=crlf
# Denote all files that are truly binary and should not be modified.
*.png binary
*.jpg binary
I have no idea why it's not working because I was try a lot of configurations of this file before.

The .gitattributes file should be added with the first commit. If you add it a few commits in, you need to normalize all the existing files explicitly.
$ rm .git/index # Remove the index to force Git to
$ git reset # re-scan the working directory
$ git status # Show files that will be normalized
$ git add -u
$ git commit -m "Introduce end-of-line normalization"
See https://git-scm.com/docs/gitattributes

If .gitattributes file was not added with the first commit, the following should be performed to apply attributes locally:
Go to the root of the repository
Check status:
git status
If it says "nothing to commit, working tree clean", perform:
git rm --cached -r .
git reset --hard
The answer is based on https://dev.to/deadlybyte/please-add-gitattributes-to-your-git-repository-1jld

Related

Cannot prevent git from modifying files after rm --cached

I am attempting to run 'git rm -rf --cached .' along with 'git add .' to remove cached files that are now listed in the .gitignore. I use Visual Studio on a windows computer, and prefer to leave line endings just as they are for this particular situation.
I tried setting core.autocrlf to false using git config command. I tried creating a .gitattributes with the line '* -text', rm'ing the .git/index, and running git reset. So far, every time I add the files back, I get a huge list of modified files.
EDIT: The change in the files is not actually line endings, it is changes in file permissions which I did not request.
Edit: the remaining problem is that the file modes are apparently not stored properly in Windows systems (see also What is git's "filemode"?). To save and restore them, one will need a script, plus the original data:
git ls-files --stage > /tmp/original
To recover the modes, this rather crude pipeline should work:
< /tmp/original \
awk -F$'\t' '/^100755 / { print "git update-index --chmod=+x \"" $2 "\"" }' |
sh
This will attempt to chmod +x files that have been removed by the below sequence, so you can expect some error messages if there are any such files. (It also assumes no files have double quotes in their names.)
Assuming you do not already have a .gitattributes file, here is a six step process that should work:
Create that .gitattributes file just as you did
Run rm .git/index
Run git checkout HEAD -- .
Run git rm -r --cached .
Run git add .
Run git rm .gitattributes (you can leave this until after verifying that it all worked). Run git commit afterward.
I do not have (nor use) Windows so cannot test this, but here's the theory behind why it should work, and hence why there are these steps.
Git's actual data storage format is a special, Git-only, compressed (sometimes highly compressed) format. Files stored in this format are mainly useful only to Git itself. This format stores a raw, uninterpreted byte stream: files do not have to be separated into "text" and "data" and so on, they are just raw byte streams (hence treated as "data" / "non-text"). The data, once stored, are read-only and get assigned a hash ID (currently SHA-1 though a future Git may use SHA-256). Git calls a file stored this way a blob, which is a term stolen from the database world.
Your computer's useful-file-storage format is of course different, and may (and does on Windows) make a distinction between "text" and "data". Text may have encodings (such as ISO-8859-1, UTF-8, UTF-16, and so on). These files are generally both readable and writable and anything on your computer can deal with them (to some degree anyway, depending on encoding).
Git has to extract files from commits, turning them from blobs into files that you can work with. These files live in your work-tree. You work with them, and then git add them to give Git a chance to re-blob-ize them.
In between these special Git-only blobs and the work-tree, Git needs a place to store the blobbed data, that—unlike a commit—is writable, but that—like a commit—has the file in the special Git-only format. This "in between" place is Git's index. Various bits of Git documentation sometimes call this the staging area or the cache.
Git uses the index copy of each file (or blob, really) to make new commits. When you run git add, Git reads the work-tree file, encodes it down into the blob form, and saves it—well, its hash ID, really—in the index. When you run git commit, Git simply freezes the index copies into committed copies.
When you run git checkout to switch to some commit, Git extracts the commit into the index (filling in all the blob hash IDs), and also extracts the blobs into the work-tree so that they are in useful format and you can work on them. When you run git add, Git compresses the work-tree file into its blob format and replaces the index entry for the file.
Transforming a blob into a work-tree file, or vice versa, is the ideal place where Git will do any conversions you need, such as turning newlines into CRLF line endings. So that's where Git does it: git checkout fills the index and expands-and-converts into the work-tree, and git add compresses-and-un-converts from the work-tree into the index, ready for the next git commit. (Any files you don't touch, stay compressed and ready to go, safely tucked away in the index.)
You already know that a tracked file is one that is in the index, and an untracked file is one that is in the work-tree but not in the index. Your goal is to use the existing .gitignore to make files that are currently in the index go away from the index if they would be .gitignore-ed. The process you are using is:
git rm -r --cached .: remove everything from the index, so that the entire work-tree is untracked
git add .: produce all new blobs in the index from whatever is in the work-tree, while ignoring any file that is listed in .gitignore.
The issue here is that what's in the work-tree has been converted by the "blob to work-tree" conversions, and will be "un-converted" by the "work-tree to blob" conversions. Creating a .gitattributes file with * -text tells Git: The conversions to do are no conversions at all."
Unfortunately, it's too late: the git checkout you ran earlier, to get this commit into the work-tree, already did some conversions.
So here, we use step 1 to create a .gitattributes file that says do no conversions. Step 2, rm .git/index, removes the index entirely. Git now has no idea what's actually in the work-tree. This step may be unnecessary but I use it to force Git to act in step 3, which tells Git: extract every file from the HEAD commit into the index and the work-tree. This re-creates the index, and re-fills the work-tree, this time doing no conversions.
Steps 4 and 5 are just as before, but this time, the work-tree files all match the blobs in the HEAD commit since step 3 operated with the .gitattributes directive in place. Step 6 is to make sure you do not commit the "do no conversions" directive.

How to re-download whole repository in Git

We are team of users, who work on Windows, our Remote is in Bitbucket (Linux/UNIX) and our application is being deployed to Linux machine. We didn't pay attention to line endings, until one day we found out, that .sh scripts on our laptops have CRLF line endings. We decided to set core.autocrlf to false, so there will be no differences between line endings on Remote and line endings on our Windows laptops. However, this option does not change CRLF in our local source code.
Is there any way, how to tell Git to update all files so CRLF will be changed to LF as it is on Remote? Some kind of re-download, that would download even unchanged files.
No, there is no way to tell git to change all your files for you. Git simply stores the files. But you can clone your repo, update your files locally, and then push the changes back.
From a unix-style shell you could use the commonly available tool dos2unix for easy conversions. Something like this might fly:
git clone bitbucket:path/to/repo foo
cd foo
find . -type f -name \*.sh -exec dos2unix {} {} \;
git add .
git commit -m 'convert line endings on .sh files'
git push
If you can't find dos2unix, you can duplicate the functionality with a number of other tools, for which answers are readily available already here or via your favourite search engine.
UPDATE
If what you're really asking is "how do I refresh a local copy of a repo, overwriting mis-matched files", then I believe the following may work:
git fetch --all
git reset --hard origin/master
Or if you're on a different branch:
git reset --hard origin/branchname
The git fetch re-downloads the latest from remote without a merge or rebase (which would be done if you were to git pull).
The git reset will reset the master branch to whataver was just fetched. The --hard option changes all the files in your working tree to match the files in origin/master (or whatever branch you specify).
NOTE: This overwrites local files. Make sure to stash any local changes, as this will overwrite anything that hasn't yet been pushed. Files that are not in the repo will be left untouched, but anything that is in the repo will be overwritten with whatever is fetched, and local (unpushed) commits will be lost.
I would go for grep + xargs + sed
grep -rlI "\r\n" * | xargs -I{} sed -i.bak -e 's/\r//g' {}
grep will list non binary file that have \r\n and sed will remove the \r
I didn't try, so it may need a bit of tweeking!

On Windows git can't handle files with the same name but in different case properly

I use git on windows. In my project I changed case of filename. After that checkout of previous commits failed (commands are in Git Bash):
mkdir repofolder
cd repofolder
git init # create empty repo
git config core.ignorecase false # turn on case-dependent filenames
# create 'readme.txt'
$ echo "blahblahblah" > readme.txt
$ git add readme.txt
$ git commit -m "+readme.txt"
# rename it to 'README.txt'
$ git mv -f readme.txt README.txt
$ git commit -m "readme.txt => README.txt"
$ git status
On branch master
nothing to commit, working directory clean
$ git checkout HEAD~1
error: The following untracked working tree files would be overwritten by checkout:
readme.txt
Please move or remove them before you can switch branches.
Aborting
Why git doesn't allow to checkout previos commits?
You face with the same problem when delete one file and append another one with the same name, but different case. No matter how many commits you do: one (removing and appending in the same commit) or two commits (in first commit you remove file, in second you add another one).
On Windows git can't handle files with the same name but in different case properly
Git on Windows can't handle it because Windows itself can't handle it (emphasis mine):
As part of the requirements for POSIX compliance, the Windows NT File System (NTFS) provides a case-sensitive file and directory naming convention. Even though NTFS and the POSIX subsystem each handle case-sensitivity well, 16-bit Windows-based, MS-DOS-based, OS/2-based, and Win32-based applications do not.
In truth, Windows does have some level of support for NTFS case-sensitivity, but it's pretty flaky:
However, if you attempt to open one of these files in a Win32 application, such as Notepad, you would only have access to one of the files, regardless of the case of the filename you type in the Open File dialog box.
Other inconsistencies also exist. The Windows NT Command Prompt and File Manager correctly display the names of the files. However, normal commands, such as COPY, fail when you attempt to access one or more filenames that differ only in case.

Git: File Rename

I wanted to rename a folder from "Frameworks" to "frameworks", but git would not let me add the new lowercase name. I guess it treats filenames case insensitive, does it?
A git add frameworks/ -f didn't help
You can try:
"git mv -f foo.txt Foo.txt" (note: this is no longer needed since git 2.0.1)
to set ignorecase to false in the config file.
But the issue of case (on Windows for instance) is described in the msysgit issue 228 (again: this should now -- June 2014 -- work with git 2.0.1)
there is always an option to set ignorecase to false in the config file that will force Unix like Git semantics on top of NTFS.
Git supports this behavior but it is not the default - from NTFS point of view a.txt
and A.txt are the same thing - so Git tries to preserve that as most users would expect
As a better workaround, you can
git mv foo.txt foo.txt.tmp && git mv foo.txt.tmp Foo.txt
, which also changes the case of the file as stored on disk.
This blog post illustrates the same issue on MacOs during a rebase:
The default on Mac OS X file systems is that they are case-insensitive. FFFFFF.gif is the same as ffffff.gif.
If you delete the file in question, just from the file system, not from the Git index, mind you, you can merge the branch in question, and have it restore the file as if nothing happened.
The steps are pretty simple:
$ rm file/in/question.gif
$ git merge trunk
Anyhow, remember what git mv stands for:
mv oldname newname
git add newname
git rm oldname
, so if newname and oldname clash, you need to make them different (even if it is only for a short period of time), hence the git mv foo.txt foo.txt.tmp && git mv foo.txt.tmp Foo.txt
If you happen to host on Github, you can use the rename function on their website. Had to change the casing for 5 files and found it worked really well.
I was having a similar problem and couldn't get a new folder name (different case) to change on remote repos. I found that the easiest solution was just to move the file out of the repo and commit. Triggering a delete action. Then re-add and when I added, it came in with the proper case.

Ignoring directories in Git repositories on Windows

How can I ignore directories or folders in Git using msysgit on Windows?
Create a file named .gitignore in your project's directory. Ignore directories by entering the directory name into the file (with a slash appended):
dir_to_ignore/
More information is here.
By default, Windows Explorer will display .gitignore when in fact the file name is .gitignore.txt.
Git will not use .gitignore.txt
And you can't rename the file to .gitignore, because Windows Explorer thinks it's a file of type gitignore without a name.
Non command line solution:
You can rename a file to ".gitignore.", and it will create ".gitignore"
It seems that for ignoring files and directories there are two main ways:
.gitignore
Placing .gitignore file into the root of your repository besides the .git folder (in Windows, make sure you see the true file extension and then make .gitignore. (with the point at the end to make an empty file extension))
Making the global configuration ~/.gitignore_global and running git config --global core.excludesfile ~/.gitignore_global to add this to your Git configuration
Note: files tracked before can be untracked by running git rm --cached filename
Repository exclude - For local files that do not need to be shared, you just add the file pattern or directory to the file .git/info/exclude. Theses rules are not committed, so they are not seen by other users. More information is here.
To make exceptions in the list of ignored files, see this question.
To ignore an entire directory place a .gitignore of “*” there.
For example,
Example System
/root/
.gitignore
/dirA/
someFile1.txt
someFile2.txt
/dirB/
.gitignore
someFile3.txt
someFile4.txt
Goal
ignore the contents of dirB/
Top Level (/root/.gitignore)
You could just “dirB/“ here
Ignored Directory (/root/dirB/.gitignore)
Or you could “*” here
Git watches for gitignore at every step of the file system. So here I choose dirB/.gitignore as “*” to ignore dirB/, including all files and subdirs within.
Done ☺️
To instruct Git to ignore certain files or folders, you have to create .gitignore file.
But in Windows Explorer you have to provide a name for the file. You just cannot create file with just an extension. The trick is that create a empty text file and go to command prompt and change the name of the file to .gitignore:
ren "New Text Document.txt" .gitignore
Now open the file with your favorite text editor and add the file/folder names you wish you ignore. You can also use wildcards like this: *.txt.
I had some issues creating a file in Windows Explorer with a . at the beginning.
A workaround was to go into the commandshell and create a new file using "edit".
If you want to maintain a folder and not the files inside it, just put a ".gitignore" file in the folder with "*" as the content. This file will make Git ignore all content from the repository. But .gitignore will be included in your repository.
$ git add path/to/folder/.gitignore
If you add an empty folder, you receive this message (.gitignore is a hidden file)
The following paths are ignored by one of your .gitignore files:
path/to/folder/.gitignore
Use -f if you really want to add them.
fatal: no files added
So, use "-f" to force add:
$ git add path/to/folder/.gitignore -f
You can create the ".gitignore" file with the contents:
*
!.gitignore
It works for me.
In Windows there's an extra catch with slashes. Excluding a single directory in .gitignore with
dir_to_exclude/
will possibly work, but excluding all directories with
/
causes problems when you have file names with spaces (like my file.txt) in your directory: Git Bash escapes these spaces with a backslash (like my\ file.txt) and Git for Windows doesn't distinguish between / and \.
To exclude all directories, better use:
**/
Two consecutive asterisks signify directory contents.
Just in case you need to exclude sub folders you can use the ** wildcard to exclude any level of sub directory.
**/build/output/Debug/
Also in your \.git\info projects directory there is an exclude file that is effectively the same thing as .gitignore (I think). You can add files and directories to ignore in that.
When everything else fails try editing the file
/.git/info/exclude
and adding the directories you want to the end of the file, like this:
# git ls-files --others --exclude-from=.git/info/exclude
# Lines that start with '#' are comments.
# For a project mostly in C, the following would be a good set of
# exclude patterns (uncomment them if you want to use them):
# *.[oa]
# *~
assets/
compiled/
I added the folders "assets" and "compiled" to the list of files and directories to ignore.
I've had some problems getting Git to pick up the .gitignore file on Windows. The $GIT_DIR/info/exclude file always seems to work though.
The downside of this approach, however, is that the files in the $GIT_DIR directory are not included in the check-in, and therefore not shared.
p.s. $GIT_DIR is usually the hidden folder named .git
On Unix:
touch .gitignore
On Windows:
echo > .gitignore
These commands executed in a terminal will create a .gitignore file in the current location.
Then just add information to this .gitignore file (using Notepad++ for example) which files or folders should be ignored. Save your changes. That's it :)
More information: .gitignore
I assume the problem is that your working tree is like:
a-cache/foo
a-cache/index.html
b-cache/bar
b-cache/foo
b-cache/index.html
.gitignore
... with the .gitignore you describe. This will give you git status output like:
$ git status
# On branch master
# Untracked files:
# (use "git add <file>..." to include in what will be committed)
#
# .gitignore
# a-cache/
# b-cache/
... if the index.html files have not yet been added to the repository. (Git sees that there are unignored files in the cache directories, but it only reports the directories.) To fix this, make sure that you have added and committed the index.html files:
git add *cache/index.html
git commit -m "Adding index.html files to the cache directories"
... and your git status will then look like:
$ git status
# On branch master
# Untracked files:
# (use "git add <file>..." to include in what will be committed)
#
# .gitignore
nothing added to commit but untracked files present (use "git add" to track)
(Obviously you do want to commit .gitignore as well. I was just being lazy with this test case.)
On Windows and Mac, if you want to ignore a folder named Flower_Data_Folder in the current directory, you can do:
echo Flower_Data_Folder >> .gitignore
If it's a file named data.txt:
echo data.txt >> .gitignore
If it's a path like "Data/passwords.txt"
echo "Data/passwords.txt" >> .gitignore.
I had similar issues. I work on a Windows tool chain with a shared repository with Linux guys, and they happily create files with the same (except for case) names in a given folder.
The effect is that I can clone the repository and immediately have dozens of 'modified' files that, if I checked in, would create havoc.
I have Windows set to case sensitive and Git to not ignore case, but it still fails (in the Win32 API calls apparently).
If I gitignore the files then I have to remember to not track the .gitignore file.
But I found a good answer here:
http://archive.robwilkerson.org/2010/03/02/git-tip-ignore-changes-to-tracked-files/index.html
Just create .gitignore file in your project folder Then add the name of the folder in it for ex:
frontend/node_modules
This might be extremely obvious for some, but I did understand this from the other answers.
Making a .gitignore file in a directory does nothing by itself. You have to open the .gitignore as a text file and write the files/directories you want it to ignore, each on its own line.
so cd to the Git repository directory
touch .gitignore
nano .gitignore
and then write the names of the files and or directories that you want to be ignored and their extensions if relevant.
Also, .gitignore is a hidden file on some OS (Macs for example) so you need ls -a to see it, not just ls.
Temporarily ignore a directory/file that was already in git:
I have a lot of projects in a multi-project gradle project and they can take a long time to delete them, and they're all pretty much the same but different. From time to time I want to remove those from the gradle build by deleting them altogether. git can get them back after all. However I don't want them showing up in git status either. So I use the following simple procedure;
delete files and folders I don't want.
verify build still works
tell git to ignore the deleted files for a bit (we can get them back)
git ls-files --deleted -z | git update-index --assume-unchanged -z
--stdin
go about life without the dirs until you want them back. Then run the same command as before but switch out assume-unchanged for no-assume-unchanged
git ls-files --deleted -z | git update-index --no-assume-unchanged -z
--stdin

Resources