We have GIT set up within our windows network (using msysgit & GitExtensions). We each have our own repositories and we push to a remote 'bare' repository on one of our servers. All good.
I'm trying to set up a scheduled job on the server, which will clone a repository from the C drive to an external drive (on F) - having some difficulty getting this to work. I can do this in GIT bash relatively easily, but I'm not sure how to save this into a batch file that I can then scehdule.
What I have so far:
rmdir F:\GitClone /s /q
mkdir F:\GitClone
mkdir F:\GitClone\Repo1
CD /D F:\GitClone\Repo1\
GIT CLONE /c/GIT/Repo1/
I've also tried the following for the last line:
GIT CLONE C:\GIT\Repo1\
But this doesn't work either... I'm a little stumped and would appreciate some help. The C drive contains our bare repositories and the F drive being our external drive that we swap out daily...
Several answers here that have been very useful, thanks. My resulting answer is probably a combination of these, so points for pointing out how to run a bash script and how to script the pull/push.
Need to bring these together to work so that it's happy when various drives are swapped in and out (i.e. clone a repository if it doesn't exist on the external drive and then only pull the differences otherwise), but that should be doable. Thanks to all.
Please note that git itself is excellent at copying only the needed changes to a cloned repository.
If you want a copy of your repo to be regularly updated, do this: You create a bare repository as a backup repository, and then repeatedly push all new changes there (no need to delete the old backup).
Ok, let's start by creating your repo
$ cd /tmp
$ mkdir myrepo && cd myrepo
$ touch hi && git add . && git commit -m "bla"
So, this is your repository. Now we create the clone:
$ cd /tmp
$ mkdir backup && cd backup
$ git --bare init
Initialized empty Git repository in /tmp/backup/
Now, let's set up your repo for regular backups …
$ cd /tmp/myrepo
$ git remote add backup /tmp/backup
$ git config remote.backup.mirror true
Then copy everything to the backup:
$ git push backup
Counting objects: 3, done.
Writing objects: 100% (3/3), 206 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To /tmp/backup
* °new branch§ master -> master
And see if it worked:
$ cd /tmp/backup
$ git log
commit d027b125166ff3a5be2d7f7416893a012f218f82
Author: Niko Schwarz <niko.schwarzàgmail.com>
Date: Fri Dec 11 12:24:03 2009 +0100
hi
Tada, you're set. Therefore, all your script needs to do is to issue git push backup. There's exactly no need to repeatedly throw away the old backup.
The alternative is you can have rsync do it all for you:
rsync -av rsync://rsync.samba.org/ftp/unpacked/rsync /dest/dir/
User Offby adds: Since version 1.5.4, "git remote add" takes a "--mirror" option, which saves you both from having to "git config remote.origin.mirror true", and from having to pass --mirror to "git push".
Because git command is little bit weird you have to use call to execute any git commands from a batch file:
rmdir F:\GitClone /s /q
mkdir F:\GitClone
CD /D F:\GitClone\
call GIT CLONE c/GIT/Repo1/
You could always just schedule bash.exe mybashbackupscript.sh
Anyway, in windows terms:
rmdir F:\GitClone /s /q
mkdir -p F:\GitClone\Repo1
copy c:\GIT\Repo1\.git F:\GitClone\Repo1\.git
git clone doesn't really do anything fancier than that.
edit: as someone else pointed out, it's probably best to just make a new backup repo and just pull/fetch into that. You'll avoid any accesing issues with the .git :)
Sorry, I can't comment posts, but I was thinking too about copying the .git, but what happens if the .git is copyed during a pull ?
Anyway, why copying the whole stuff as you could fetch the deltas from time to time ?
Initialization of backup : (in F:\GitClone\Repo1 empty)
git init
git add remote origin /c/GIT/Repo1
Then your "delta backup script" would just do :
cd /f/GitClone/Repo1
git fetch origin
Why do you delete the clone all the time? What is the point of cloning an existing repository when you just want to copy the file?
Just create the repo on the external drive (using git clone once), and then run git pull on it regularly.
Problem solved:
h: (flash drive)
cd \
mkdir YourProject
cd YourProject
git init
git remote add origin git#github.com:user/YourProject.git
git remote add local C:\w\YourProject
git pull origin master
git push local master
Related
I have a few dozen git repositories under one directory, all of them contain not only source code but also build artifact, developer artifact, blobs and other data normally ignored by .gitignore
I would like to quickly transfer these source code to another computer. Is there an easy way to package everything that is not ignored by git?
To add more context:
I have 50-ish git folders or so, each on different branches with some current changes
I would like to copy all files that are tracked by git, at the current state ( current branch plus current local change )
If by cloning them all, it means to go to each folder, commit the current changes locally, remember the branch name, git clone local branch to another folder, do that 50 times then I am strongly agaist it
One-liner:
$ alias gitzip="git archive HEAD -o ${PWD##*/}.zip"
$ cd mydir
$ gitzip
If you want to include history, use git bundle (or xeyownt/git-subundle if your repositories have submodules)
See this script as an example:
for repository in ${repositories[#]}
do
echo "$repository..."
declare file_name="../$repository-$(git rev-parse --short HEAD).bundle"
cd $repository
if [ -f $file_name ]; then
echo 'Found bundle is current'
else
git bundle create $file_name --all
fi
cd -
done
Each repository gives one file (without anyprivate or ignored files), that you can easily copy and, once copied, cloned from.
You can try this :
for dir in repo1 repo2 repo3; do
git -C "$dir" ls-files | perl -pe "s[^][$dir/]"
done | zip /tmp/test.zip -#
I'd like to write a script that, when run, pushes all the last updated files into the matching GitHub repositories.
Here's what I tried:
myPath=absolute_project_path
for i in $*
do
if [[ -d $myPath/$i ]]
then
cd $myPath/$i
git add ./*
git commit -m "update"
git remote set-url origin https://usr:pwd#github.com/username/$i.git
git push -f -u origin main
fi
done
But this is adding all the folders to all the directories: so myrepo1 in GitHub, gets myrepo1+myrepo2 files, and myrepo2 gets myrepo1+myrepo2.
Does anybody know the reason why?
Notes on the workarounds tested, if needed: It seems that the staging needs to be emptied every time before changing file for another repository.
But i tried this also:
- Did a backup of my folders, then ran git rm -f, git add, git commit, git push (caused local folder to get deleted), re-copied from backup, and re-ran the script, but didn't help
- Also tried to manually remove all the folders on GitHub and re-push them, but caused Git to want me to do a Pull. But once the Pull has been done (which caused the deletion of my local files), a re-copy from backup and a new Push caused Git to ask to make a new Pull again... reason why of the push -f
It is only a suggestion, but for this you can use already existing tools like myrepo.
Simply register your repos:
mr register ~/gitHub/repo_1
mr register ~/gitHub/repo_2
...
then if you want to push all your repos, simply type:
mr push
if you wan to pull them all, type
mr update
an so on... read to doc.
See the answer that I linked to this question:
https://stackoverflow.com/a/8603156/1445967
I could not get this to work at all under the latest Git for windows. (Windows 7 x64)
I used git bash:
<my username> /d/<worktree>
$ git --git-dir=/c/dev/gitrepo/repo.git --work-tree=. init && echo "gitdir: /c/dev/gitrepo/repo.git" > .git
Initialized empty Git repository in c:/dev/gitrepo/repo.git/
Then I tried:
<my username> /d/<worktree>
$ git status
fatal: Not a git repository: /c/dev/gitrepo/repo.git
So I tried something slightly different, thanks to the way windows paths get stored...
<my username> /d/<worktree>
$ git --git-dir=/c/dev/gitrepo/repo.git --work-tree=/d/<worktree> init && echo "gitdir: /c/dev/gitrepo/repo.git" > .git
Initialized empty Git repository in c:/dev/gitrepo/repo.git/
This is copy-paste verbatim except I changed my username and a single directory name to <worktree> for SO.
Then I tried the following:
<my username> /d/<worktree>
$ git status
fatal: Not a git repository: /c/dev/gitrepo/repo.git
Then I looked inside /c/dev/gitrepo/repo.git/config and I saw this:
worktree = d:/<worktree>
Maybe this won't work with the windows path notation. So I changed it:
worktree = /d/<worktree>
Still no luck. Is what I am trying to do possible under git for Windows?
The only progress I have made on this question is the discovery that on the workstation I was using, the SUBST command was used to create drive D. That is to say, C and D are really the same physical drive.
SUBST does not seem to completely break Git though. I have the project on drive D and the git repo on a completely different directory on drive D and everything works.
username#host /d/ProjectName (branch)
$ cat .git
gitdir: d:\gitrepo\ProjectName.git
So the best answer I have is a workaround, where I advise: In Windows there may be an issue:
1. If the repo and working dir are on different drives
2. If (1) is true and you use SUBST
But whichever of those is the case, the work around is one or both of the following:
1. put everything on the same drive letter. It works even if that drive is a 'subst' drive.
2. Use the windows "d:\" notation in the .git file
When I clone the repo using msysgit, all the files with spaces in the filename are not brought down, and then show as deleted in the status.
The filenames looks something like this: styles-ie (1:12:11 6:02 PM).css so it might actually be the colon or brackets?
How can I fetch those files to bring my local repo inline with the origin?
Good news.
Technically, the answer to "how do I clone files with colons in the filename" is to simply use "git clone". Luckily it is only the checkout that fails on Windows (even under msysgit) and there is a rather clean workaround for this shown below.
TL;DR
in Git Bash...
git clone {repo URL}
cd {repo dir}
git ls-tree -r master --name-only | grep -v ":" | xargs git reset HEAD
git commit -m "deleting all files with a colon in the name"
git restore .
... and then
download the Zip of the whole git repo
rename files with colons inside the Zip (without extracting them)
extract just those files you renamed
add those renamed files to your working directory
For insight into those few steps listed above, please keep reading....
I was able to work around this issue while working with a repo with colons in various filenames. The following worked for me:
Do a regular git clone.
$ git clone https://github.com/wdawson/dropwizard-auth-example.git
You should see the following error that notes that the clone succeeded, but the checkout failed.
Cloning into 'dropwizard-auth-example'...
remote: Enumerating objects: 322, done.
remote: Total 322 (delta 0), reused 0 (delta 0), pack-reused 322
Receiving objects: 100% (322/322), 15.00 MiB | 2.88 MiB/s, done.
Resolving deltas: 100% (72/72), done.
error: invalid path 'src/test/resources/revoker/example-ca/certs/root.localhost:9000.cert.pem'
fatal: unable to checkout working tree
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'
Change directories to the new cloned repo
cd dropwizard-auth-example
Check that the git repo working directory is completely empty
ls
Run git-status to find that all the files are staged for deletion
$ git status
Output...
On branch master
Your branch is up to date with 'origin/master'.
Changes to be committed:
(use "git restore --staged <file>..." to unstage)
deleted: .gitignore
deleted: .travis.yml
deleted: LICENSE
deleted: NOTICE
deleted: README.md
deleted: conf.yml
...
Revert the staged deletion of only the files that do not contain a colon in the file name.
$ git ls-tree -r master --name-only | grep -v ":" | xargs git reset HEAD
Output...
Unstaged changes after reset:
D .gitignore
D .travis.yml
D LICENSE
D NOTICE
D README.md
D conf.yml
D java-cacerts.jks
D pom.xml
D src/main/java/wdawson/samples/dropwizard/UserInfoApplication.java
D src/main/java/wdawson/samples/dropwizard/api/UserInfo.java
D src/main/java/wdawson/samples/dropwizard/auth/OAuth2Authenticator.java
D src/main/java/wdawson/samples/dropwizard/auth/OAuth2Authorizer.java
D src/main/java/wdawson/samples/dropwizard/auth/Role.java
...
Run git status again to see that only the files that contain a colon in the file name are now staged for deletion. All other files are still showing as deleted, but not staged for commit. This is what we want at this stage.
$ git status
Output...
On branch master
Your branch is up to date with 'origin/master'.
Changes to be committed:
(use "git restore --staged <file>..." to unstage)
deleted: src/test/resources/revoker/example-ca/certs/root.localhost:9000.cert.pem
deleted: src/test/resources/revoker/example-ca/csr/root.localhost:9000.csr.pem
deleted: src/test/resources/revoker/example-ca/intermediate/certs/intermediate.localhost:9000.cert.pem
deleted: src/test/resources/revoker/example-ca/intermediate/csr/intermediate.localhost:9000.csr.pem
deleted: src/test/resources/revoker/example-ca/intermediate/private/intermediate.localhost:9000.key.pem
deleted: src/test/resources/revoker/example-ca/private/root.localhost:9000.key.pem
Changes not staged for commit:
(use "git add/rm <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
deleted: .gitignore
deleted: .travis.yml
deleted: LICENSE
deleted: NOTICE
deleted: README.md
deleted: conf.yml
deleted: java-cacerts.jks
deleted: pom.xml
Commit all the staged files. That is, commit the deletion of all the files that contain a colon in the file name.
git commit -m "deleting all files with a colon in the name"
Restore everything in the working directory.
$ git restore .
View all the files. What a beautiful site.
$ ls
Output...
conf.yml java-cacerts.jks LICENSE NOTICE pom.xml README.md src
Once you've deleted the offending files from your working directory...
download a Zip of the whole GitHub repo
open it up in 7Zip... Don't unzip it ... just open it for editing (to rename files)
find the files that have a colon in the name
rename each file with a colon replacing the colon with an underscore...or whatever is appropriate
now you can extract those files you just renamed
copy them into the git working directory
PS: All of the above was done in GitBash on Windows 10 using git version 2.25.1.windows.1. Similar steps can be done via the GUI using TortoiseGit on Windows.
If you try doing:
touch "styles-ie (1:12:11 6:02 PM).css"
you will see that you cannot create it on Windows.
Basically, the repo has the file ( the blob and the tree entry ) but you cannot checkout on Windows as git would be unable to create such a file. No other way but to change the filename.
You can clone the repo on a linux environment, tar it up and copy it to windows, and untar it on windows with tools such as 7zip. 7zip will replace the colon with underscore, and preserve all the git information. As long as that file does not change, you'll be all set for a while. Those files tend not to change much anyway (for example, I have a cert file with a colon in the middle).
In support to the answers "using WSL" or "using Linux environment":
Using WSL:
(Windows 11)
1. Enable virtualization:
in BIOS
in Windows ("Turn Windows features on or off" -> "Virtual Machine Platform"/"Windows Subsystem for Linux" -> check)
2. Download and install linux distibutive (e.g. Ubuntu - latest):
in PowerShell:
wsl --install -d Ubuntu
3. Clone repo in WSL linux console
After WSL has been installed - run the application "WSL" - there going to be a linux console available. In that linux console - clone repository as you would normally do**.
** In my case I logged in as root (>sudo su), created ssh keys, added public ssh key to the github repo, navigated to required directory and cloned ssh repo.
As a result, through WSL console I'm able to see files with ":".
Through another file managers, consoles (File Explorer, PowerShell, cmd, git CLI) - in place of colons different symbols displayed.
I tried to put a series of GIT commands that I always use continuously togeter as batch files so that I don't repeat myself too much. For example, I have this batch file called update_repo_branch.bat to update a local repo and synch a branch with the remote branch:
#echo off
if(%1) == () goto end
if(%2) == () goto end
cd %1
git checkout %2
git fetch origin
git merge oring/%2
:end
Good to be lazy, but what I found is that when a GIT command is finished, it seems to send an exit flag back to terminate whatever is running. Therefore, using a batch file to exectute them all in one go simply doesn't work. Any idea how to work around it?
I'm not sure if this is true for all Windows git packages, but at least some use a git.cmd script as a wrapper around the actual git executables (for example git.exe). So when you're batch file uses a git command, Windows is actually running another batch file.
Unfortunately, when one batch file invokes another, by default it 'jumps' to the invoked batch file, never to return (this is for compatibility with ancient MS-DOS command processors or something).
You can solve this problem in a couple ways:
invoke git in your batch files using the call command to run the git.cmd batch file and return back to yours:
call git checkout %2
call git fetch origin
rem etc...
invoke git in your batch file using the .exe extension explicitly to avoid the git.cmd batch file altogether. For this to work, you might need to make sure that you have your path and other environment variables set the way git.exe expects (that seems to be what git.cmd does in msysgit):
git.exe checkout %2
rem etc...
Assuming you are using msysGit as your Git client you might actually want to use Bash scripts for this. You could place a bash function in your ~/.bashrc (~ is usually your C:\Users\- see here) as follows
update_repo_branch() {
if [ $# != "2" ]; then
echo "Usage: update_repo_branch REPO BRANCH" 1>&2
return 1
fi
cd $1
git checkout $2
git fetch origin
git merge origin/$2
}
You can then run update_repo_branch myrepo cool-branch from the mysysGit shell.
Of course, this won't be accessible from cmd.exe. You will only be able to use it within the msysGit cygwin shell.
As i see from your example you're actually trying to sync your local branch 'branchname' with origin/branchname
For this you don't need any additional scripting, you just have to use git pull instead of sequence git checkout branchname; git fetch origin; git merge origin/branchname
take a look at the docs about tracking branches in git and their benefits.
generally speaking if you have a repo layout like this:
git branch -a
...
master
dev1
dev2
remotes/origin/master
remotes/origin/dev1
remotes/origin/dev2
And your dev1 and dev2 branches are tracking branches for origin/dev1 and origin/dev2 correspondingly then you just need to execute in repository:
git pull
This command will effectively sync up all you local tracking branches with remote ones.
for more see here:
Git pull docs
Git remote branches and tracking branches (Progit book)
Create a notepad file and paste the below content and save with .bat extension.
This script can be use for setting up git first time in a project.
!echo off
git init
git add .
git commit -m "first commit"
git branch -M main
git remote add origin <YOUR_REPO_URL>
git push -u origin main
pause