Is it possible to tell Git to ignore symlinks ? I'm working with a mixed Linux / Windows environment and, as you know, symlinks are handled very differently between the two.
Use git version >= 1.6
Git used to treat sym-links the same as regular files, but newer git versions (>= 1.6) check if a file is beyond a symbolic link and will throw a fatal error.
e.g.:
# git init
# mkdir newdir
# touch newdir/foo
# git add newdir/foo
# git commit -m 'add foo'
# mv newdir /tmp/
# ln -s /tmp/newdir
# touch newdir/bar
# git add newdir/bar
fatal: 'newdir/bar' is beyond a symbolic link
# git add/tmp/newdir
fatal: '/tmp/newdir' is outside repository
# git --version
git version 1.7.3.4
No, it is not possible to do this globally. However, if you have lots of symlinks here is a bash script that you can use to easily add them to your repo's .gitignore file:
for f in $(git status --porcelain | grep '^??' | sed 's/^?? //'); do
test -L "$f" && echo $f >> .gitignore; # add symlinks
test -d "$f" && echo $f\* >> .gitignore; # add new directories as well
done
Related
Using git-filter-repo is it possible to combine N repositories into a mono-repository re-writing the commits so that the commits are interwoven, or "zippered" up by date?
Currently, I'm testing this with only 2 repos with each repo having their own subdirectory. After the operation, the commits for each repo are on "top" of each other rather than interwoven. What I really want is to be able to have a completely linear history by authored data without the added merge commits.
rm -rf ___x
mkdir ___x
cd ___x
echo "creating the monorepo"
git init
touch "README.md"
git add .
git commit -am "Hello World!"
declare -A data
data=(
["foo"]="https://github.com/bcanzanella/foo.git"
["bar"]="https://github.com/bcanzanella/bar.git"
)
for d in "${!data[#]}";
do {
REPO_NAME=$d
REPO_REMOTE=${data[$d]}
# since we can use a foo/bar as the repo identifier, replace the / with a -
REPO_DIR_TMP="$(mktemp -d -t "${REPO_NAME/\//-}.XXXX")"
echo "REPO REMOTE: $REPO_REMOTE"
echo "REPO NAME: $REPO_NAME"
echo "REPO TMP DIR: $REPO_DIR_TMP"
echo ""
echo "Cloning..."
git clone "$REPO_REMOTE" "$REPO_DIR_TMP"
echo "filtering into ..."
cd $REPO_DIR_TMP && git-filter-repo --to-subdirectory-filter "$REPO_NAME"
# cat .git/filter-repo/commit-map
## merge the rewritten repo
git remote add "$REPO_NAME" "$REPO_DIR_TMP"
echo "fetching..."
git fetch "$REPO_NAME"
echo "merging..."
git merge --allow-unrelated-histories "$REPO_NAME/master" --no-edit
## delete the rewritten repo
echo "Removing temp dir $REPO_DIR_TMP..."
rm -rf "$REPO_DIR_TMP"
echo "Removing remote $REPO_NAME..."
# git remote rm "$REPO_NAME"
echo "$REPO_NAME done!"
}
done
To emphasize on eftshift0's comment : rebasing and rewriting history can lead to commits being ordered in seemingly absurd chronoogical order.
If you know for a fact that all commits are well ordered (e.g : the commit date of a parent commit is always "older" than the commit date of its child commit), you may be able to generate the correct list of commits to feed in a git rebase -i script.
[edit] after thinking about it, this may be enough for your use case :
Look at the history of your repo using --date-order :
git log --graph --oneline --date-order
If the sequence of commits matches what you expect, you can use git log to generate a rebase -i sequence script :
# --reverse : 'rebase -i' asks for entries starting from the oldest
# --no-merges : do not mention the "merge" commits
# sed -e 's/^/pick /' : use any way you see fit to prefix each line with 'pick '
# (another valid way is to copy paste the list of commits in an editor,
# and add 'pick ' to each line ...)
git log --reverse --no-merges --oneline --date-order |\
sed -e 's/^/pick /' > /tmp/rebase-apply.txt
Then rebase the complete history of your repo :
git rebase -i --root
In the editor, copy/paste the script you created with your first command,
save & close.
Hopefully, you will get a non conflicting unified history.
I am facing merging of a few repositories in to one, with miscellaneous file moved around
Based on some research on SO, SO, how to merge repositories I ended up with following sketch:
user=some_user
new_superproj=new_proj # new repository, will include old repositories
hosting=bitbucket.org # gitgub etc
r1=repo1 # repo 1 to merge
r2=repo2
...
# clone to the new place. These are throw-away (!!!) directory
git clone git#${hosting}:${some_user}/${r1}.git
git clone git#${hosting}:${some_user}/${r2}.git
...
mkdir ${new_superproj} && cd ${new_superproj}
# dummy commit so we can merge
git init
dir > deleteme.txt
git add .
git commit -m "Initial dummy commit"
git rm ./deleteme.txt
git commit -m "Clean up initial file"
# repeat for all source repositories
repo=${r1}
pushd .
cd ../${repo}
# In the throw-away repository, move to the subfolder and rewrite log
git filter-branch --index-filter '
git ls-files -s |
sed "s,\t,&'"${repo}"'/," |
GIT_INDEX_FILE=$GIT_INDEX_FILE.new git update-index --index-info &&
mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE
' HEAD
popd
# now bring data in to the new repository
git remote add -f ${repo} ../${repo}
git merge --allow-unrelated-histories ${repo}/master -m "Merging repo ${repo} in"
# remove remote to throw-away repo
git remote rm ${repo}
So far so good, unless we want to move files around while still preserving log. Git is sucks on move/rename and log rewrite fragment is not quite adapted, hence rewrite done uniform way, recursively for whole directory
Idea is, while files are moving we know there are no other changes in repository but renames and moves. So, how can I rewrite following part to be canonical, per file. Taken from git filter-branch, official documentation
git filter-branch --index-filter \
'git ls-files -s | sed "s-\t\"*-&newsubdir/-" |
GIT_INDEX_FILE=$GIT_INDEX_FILE.new \
git update-index --index-info &&
mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"' HEAD
I have hard time to understand stuff past 'sed' and how it is applied for git filter-branch
I want to run script (bash, python etc), so:
for each file in repository get moved/renamed
...
# in the loop, moved/renamed file found
old_file="..." # e.g. a/b/c/old_name.txt
new_file="..." # e.g. a/b/f/g/new_name.txt, at this point it is known, old_file and new_file is the same file
update_log_paths(old_file, new_file) # <--- this part is needed
Any ideas?
As it turned out to be, hinting from the following command Move file-by-file in git, it is as simple as (pseudocode):
move_files
cd repo_root
git add . # so changes detected as moves, vs added/deleted
repo_moves=collect_moves_data()
git reset HEAD && git checkout . && git clean -df . # undo all moves
Biggest misunderstanding I found is "git log --follow" or other, "stronger" options doesn't work for many in related SO questions:
git log --follow <file>
does not show log until moved, while unchanged, file is committed.
for each_move in repo_moves
old_file, new_file=deduct_old_new_name(each_move)
new_dir=${new_file%/*}
filter="$filter \n\
if [ -e \"${old_file}\" ]; then \n\
echo \n\
if [ ! -e \"${new_dir}\" ]; then \n\
mkdir --parents \"${new_dir}\" && echo \n\
fi \n\
mv \"${old_file}\" \"${new_file}\" \n\
fi \n\
"
git filter-branch -f --index-filter "`echo -e $filter`"
If you need to get back:
git pull # with merge
git reset --hard <hash> # get hash of your origin/master, orignin/HEAD), which will be HEAD~2, but I'd check it manually and copy/paste hash
I am trying to write a shell script that needs to be able to find the .git folder for the current directory, correctly handling all of the following possibilities:
I might be in a bare repo, in which case the .git folder is either . or .. or ../.. or so on.
I might be in a submodule (in which I'll find a .git file that contains the path to the git folder)
$GIT_DIR might be set.
I might not be in a git repo at all
I have this:
seemsToBeGitdir() {
# Nothing special about "config --local -l" here, it's just a git
# command that errors out if the `--git-dir` argument is wrong.
git --git-dir "$1" config --local -l >/dev/null 2>/dev/null
return $?
}
gitdir() {
local cursor relpath
if [ "$GIT_DIR" ]; then
echo "$GIT_DIR"
return 0
fi
cursor="$(pwd)"
while [ -e "$cursor" ] && ! seemsToBeGitdir "$cursor"; do
# Git won't traverse mountpoints looking for .git
if mountpoint -q "$cursor"; then
return 1
fi
# We might be in a submodule
if [ -f "$cursor/.git" ]; then
# If .git is a file, its syntax is "gitdir: " followed by a
# relative path.
relpath="$(awk '/^gitdir:/{print$2}' "$cursor/.git")"
# convert the relative path to an absolute path.
cursor="$(readlink -f "$cursor/$relpath")"
continue
fi
if seemsToBeGitdir "$cursor/.git"; then
echo "$cursor/.git"
return 0
fi
cursor="$(dirname "$cursor")"
done
echo "$cursor"
}
And it works, but seems way too complicated -- clearly, git itself does this sort of calculation every time it's invoked. Is there a way to make git itself tell me where .git is?
Use git rev-parse, which has options specifically for this:
git rev-parse --git-dir
See also:
git rev-parse --absolute-git-dir
(new in Git version 2.13.0), and:
git rev-parse --show-toplevel
and:
git rev-parse --show-cdup
(note that its output is empty if you are already in the top level of the repository). View your own documentation to find out which options your Git supports; most of these have been around since Git 1.7, though.
I've been attempting to use the following script to commit a wordpress plugin to the WP svn repo from my public git repository and keep getting error:
svn: E155013: Can't add '/path/to/file' to a parent directory scheduled for deletion
Whenever there is a new file, not previously known to svn, when adding files, resulting in either an aborted commit, or worse, a broken commit where it commits with missing files.
#! /bin/bash
# A modification of Dean Clatworthy's deploy script as found here: https://github.com/deanc/wordpress-plugin-git-svn
# The difference is that this script lives in the plugin's git repo & doesn't require an existing SVN repo.
# main config
PLUGINSLUG="my-plugin-slug"
CURRENTDIR=`pwd`
MAINFILE="my-plugin.php" # this should be the name of your main php file in the wordpress plugin
# git config
GITPATH="$CURRENTDIR/" # this file should be in the base of your git repository
# svn config
SVNPATH="/tmp/$PLUGINSLUG" # path to a temp SVN repo. No trailing slash required and don't add trunk.
SVNURL="http://plugins.svn.wordpress.org/my-plugin" # Remote SVN repo on wordpress.org, with no trailing slash
SVNUSER="wp_username" # your svn username
# Let's begin...
echo ".........................................."
echo
echo "Preparing to deploy wordpress plugin"
echo
echo ".........................................."
echo
# Check if subversion is installed before getting all worked up
#if [ $(dpkg-query -W -f='${Status}' subversion 2>/dev/null | grep -c "ok installed") != "1" ]
#then
# echo "You'll need to install subversion before proceeding. Exiting....";
# exit 1;
#fi
# Check version in readme.txt is the same as plugin file after translating both to unix line breaks to work around grep's failure to identify mac line breaks
NEWVERSION1=`grep "^Stable tag:" $GITPATH/readme.txt | awk -F' ' '{print $NF}'`
echo "readme.txt version: $NEWVERSION1"
NEWVERSION2=`grep "^ \* Version:" $GITPATH/$MAINFILE | awk -F' ' '{print $NF}'`
echo "$MAINFILE version: $NEWVERSION2"
if [ "$NEWVERSION1" != "$NEWVERSION2" ]; then echo "Version in readme.txt & $MAINFILE don't match. Exiting...."; exit 1; fi
echo "Versions match in readme.txt and $MAINFILE. Let's proceed..."
if git show-ref --tags --quiet --verify -- "refs/tags/$NEWVERSION1"
then
echo "Version $NEWVERSION1 already exists as git tag. Exiting....";
exit 1;
else
echo "Git version does not exist. Let's proceed..."
fi
cd $GITPATH
echo -e "Enter a commit message for this new version: \c"
read COMMITMSG
git commit -am "$COMMITMSG"
echo "Tagging new version in git"
git tag -a "$NEWVERSION1" -m "Tagging version $NEWVERSION1"
echo "Pushing latest commit to origin, with tags"
git push origin master
git push origin master --tags
echo
echo "Creating local copy of SVN repo ..."
svn co $SVNURL $SVNPATH
echo "Clearing svn repo so we can overwrite it"
svn rm $SVNPATH/trunk/*
echo "Exporting the HEAD of master from git to the trunk of SVN"
git checkout-index -a -f --prefix=$SVNPATH/trunk/
echo "Ignoring github specific files and deployment script"
svn propset svn:ignore "deploy.sh
README.md
bower_components
node_modules
.DS_Store
.gitmodules
advanced
assets
gulpfile.js
bower.json
package.json
.git
.gitignore" "$SVNPATH/trunk/"
echo "Changing directory to SVN and committing to trunk"
cd $SVNPATH/trunk/
# Add all new files that are not set to be ignored
echo "Doing the file adding"
# THIS IS WHERE ERROR OCCURS
svn status | grep -v "^.[ \t]*\..*" | grep "^?" | awk '{print $2}' | xargs svn add
echo "Committing"
svn commit --username=$SVNUSER -m "$COMMITMSG"
echo "Creating new SVN tag & committing it"
cd $SVNPATH
svn copy trunk/ tags/$NEWVERSION1/
cd $SVNPATH/tags/$NEWVERSION1
svn commit --username=$SVNUSER -m "Tagging version $NEWVERSION1"
echo "Removing temporary directory $SVNPATH"
rm -fr $SVNPATH/
echo "*** FIN ***"
Could anyone elaborate on what is causing this error and possibly recommend either a different or adapted approach?
Dean's version doesn't contain brainless svn rm $SVNPATH/trunk/*
rm $SVNPATH/trunk/* or checkout with --depth+commit with --set-depth
I want to copy all the *.obj files (ignored by the git, but present in the directory) after commit to a directory.
With this script:
#!/bin/sh
pwd
branch=$(git rev-parse --symbolic --abbrev-ref HEAD)
echo logging objfiles for $branch
mkdir -p ./OBJ_$branch/
cp -u -f -r *.obj ./OBJ_$branch/
Git says after commit, that "Cannot stat *.obj"
Ia aware of different endlines and this file is saved using PSPad with linux endlines.
Thanks for any help!
pwd prints the correct directory.
Solved, the problem is this
http://www.linuxquestions.org/questions/linux-newbie-8/how-to-do-recursive-file-copy-of-directory-for-specific-files-199134/
cp evidently doesnt know how to do a recursion.