I'm trying to run some ruby scripts for automating exports. Since these run remotely we build them in a Docker container and push them to iron worker.
We use GemFury for hosting some essential private gems for these scripts.
To keep the credentials for GemFury out of Git we use a global bundle config bundle config gem.fury.io MY_SECRET_TOKEN.
How can I set the config for bundle so it will pull in gems from GemFury without having them show in source control?
Set the global bundle config property as an application specific property. Push the changes to the public repository. Update the SECRET_TOKEN value in the bundle-config file ($APP_DIR/.bundle/config) and run the $ git update-index --assume-unchanged <file> command to remove the file from git tracking and prevent updating the actual SECRET_TOKEN value in the public repository.
$ bundle config --local gem.fury.io SECRET_TOKEN
$ git commit -a -m "adding application bundle config properties"
$ git push origin master
$ bundle config --local gem.fury.io d1320f07ac50d1033e8ef5fbd56adf360ec103b2
$ git update-index --assume-unchanged $APP_DIR/.bundle/config
This creates a template file on the public repository. Provide instructions to repository contributors to add the secret token and execute the same --assume-unchanged command.
Example Files
$APP_DIR/.bundle/config file on public github repo:
---
BUNDLE_GEM__FURY__IO: MY_SECRET_TOKEN
$APP_DIR/.bundle/config file a local machine
---
BUNDLE_GEM__FURY__IO: d1320f07ac50d1033e8ef5fbd56adf360ec103b2
See bundle-cofig documentation for clarification and more detail
NOTE: The disadvantage to this approach is two fold:
Developers who clone the repository and need the SECRET_TOKEN value will have to obtain it by some external manual process (good security practices, but a pain to set up)
If you need to add more bundle-config properties, you will have to run git update-index --no-assume-unchanged <file> to enable tracking, and revert all of the private values to their pseudo values. This template method, also risks that contributors forget to disable tracking on the file and push their private values to the public repo (but at least they won't be your secret values)
The advantage of this template approach is that you are giving the developers as much as possible to be able to start contributing to the repository.
Related
Running a Git LFS command such as
GIT_TRACE=1 git lfs locks
reveals that the system credential.helper is used, even though it is overwritten by the local config.
Running
git config --system -l
lists 'credential.helper=manager'
whereas
git config --local -l
only lists 'credential.helper=other'
Running the Git LFS locks command with the tracer enabled shows this line
run-command.c:663 trace: run_command: 'git credential-manager get'
Removing the system wide manager with
git config --system --unset credential.helper
fixes the issue and my local helper 'other' is used correctly.
According to the git configuration documentation, each level trumps the previous level, hence Git LFS is not respecting the git standard. Is there any smart way to make this work without unsetting the system wide helper and by that potentially breaking authentication for other repositories?
Actually, Git LFS is doing the right thing here. It uses Git's git credential command and therefore inherits exactly the behavior that Git itself has.
While you are correct that when a Git option takes a single value, more specific configuration files override more general files, in the case of credential.helper, it's possible to specify multiple values. This can be helpful if you have a custom credential helper for one set of sites (e.g., those in one set of domains) and then a regular helper for other sites. All the credential helpers in this case will be asked for credentials until one is found that provides the required credentials.
If you want to override this for just one repository, you can write something like this in .git/config to first clear the existing list (with the empty entry) and then add a new credential helper:
[credential]
helper =
helper = other
This is pretty tricky to set correctly from git config, so I recommend editing the file by hand.
As part of the development of a CI & CD flow for the company I work at, I am building a command line program (Bash script on OSX) that
creates a new local Git repo
adds some default branches to this repo
Then adds a new repo to Bitbucket using the next code:
gitUserName = Joris <-- provided by the user, this is an example
projectName = TestProject <-- provided by the user, this is an example
git remote add $projectName "https://bitbucket.org/$gitUserName/$projectName.git"
After running this command, I don't see the repository on my Bitbucket account on the website. When I try to re-run this command, it says the repository already exists.
Also, when I run git push $projectName master it says fatal: repository 'https://bitbucket.org/Joris/TestProject.git/' not found
This behavior seems inconsistent, and I have followed the Atlassian guide to set this up so I don't really understand why it doesn't add the repository as expected. I do realize that I can also just go on the BitBucket website and add the repository manually, but the purpose of my program is that it generates a fully set-up repository for a user based on as little commands as possible.
The git remote add documentation says that the command adds a remote to the local repo. This terminology is, IMO, a bit off; it would better to say it adds a remote configuration to the local repository (i.e. configures the repo to access a remote). This does not actually create the remote repo; that must be done separately.
In the case of bitbucket, the "normal" thing to do is to go to the website and create the repo through their UI. Because you're trying to automate things, you don't want to do that; so in that case, you would need to use the BitBucket REST API, which is documented here: https://developer.atlassian.com/server/bitbucket/reference/rest-api/
The "Core API" section talks about repositories and permissions, so you should be able to script out requests to (if necessary) check if the repo exists and set it up if it doesn't. You'll just need a way for your script to send HTTP requests and receive the responses.
In your machine:
Create repo:
git init
Add branches:
git checkout -b branchX
git checkout -b branchY
git checkout -b branchZ
In Bitbucket website:
Create new repository named TestProject, allow write permissions to user Joris in settings and save. Finally copy the url of the repository, this must be something like bitbucket.mydomain:port/nameofproject/testproject.git (Notice this is all in lowcase)
In your machine:
git remote add origin theURL
git push origin *:*
git push origin --tags
The last is the command to push all your local repo, this will overwrite the history and tags in your remote repo, but since is a new repo it doesn't matter.
So I have a private repo that my main.go imports. I'm getting this error when I do a go build:
cannot find module for path
Do I need to do anything special for a private repo? I have been googling and can't find any good information. It works fine with dep.
Do this
git config --global --add url."git#your-repo.com:".insteadOf "https://your-repo.com/"
export GOPRIVATE='your-repo.com'
Make sure your git clone via ssh works.
(Answer duplicated from this SO Question)
I wrote up a solution for this on Medium: Go Modules with Private Git Repositories.
The way we handle it is basically the same as the answer from Alex Pliutau, and the blog goes into some more detail with examples for how to set up your git config with tokens from GitHub/GitLab/BitBucket. It also goes into a working Dockerfile example for using modules with private repos.
The relevant bit for GitLab:
git config --global \
url."https://oauth2:${personal_access_token}#privategitlab.com".insteadOf \
"https://privategitlab.com"
#or
git config --global \
url."https://${user}:${personal_access_token}#privategitlab.com".insteadOf \
"https://privategitlab.com"
I hope it's helpful.
You should use a SSH Key to fetch your repository, check if your SSH key is in system keychain too:
ssh-add -K ~/.ssh/id_rsa
Given that such a private repo is often in active development, I personally simply clone it to the "proper" location in my $GOPATH and use the source management (e.g. git) as you would any other project. Adding the SSH key as in Rodrigo's answer is great, but if you are actively developing the private repo anyway, the extra step to clone it to the right directory isn't by any means a difficult step vs being able to go get it.
So, for example, for a private repo hosted on Github, I would cd to $GOHOME/src/github.com/git-username-for-repo then git clone the-repo
I need to share a large git repository (>3GB in total, .git folder is ~1.1GB) as Windows Shared Folder over LAN with my colleagues. But they found it is really slow to clone and push/pull -- i.e. waits ~30min before the clone starts, and suspended ~5min before any push/pull starts.
Does everyone know any methods to reduce the latency or better way to share the repo?
P.S. I don't want to setup a Gitlab because it's too complicated.
Since your git repo is very big, you can bundle the whole git repo or part of commits among your colleagues sharing.
Bundle the whole repo: git bundle create repo.bundle --all
Bundle a branch: git bundle create repo.bundle branchname
Bundle some commits: git bundle create commits.bundle branchname ^commit
For the one who apply the bundled commits to his local repo, he can verify the bundle file by git bundle verify /path/to/bundle/file.
More details, you can refer Git's Little Bundle of Joy and git bundle.
git daemon works really well in this situation. Its performance is good, but lacks of security on Windows Server.
On server side, run the following command:
cd ~/Documents/All-My-Git-Repos/
git daemon --verbose --reuseaddr --export-all --enable=receive-pack --base-path=.
On client side, clone any repo like this:
git clone git://my-git-server-address/repo-folder-name repo-clone-name
While using this on Windows server, the firewall will prompt for permissions on Port 9418, which should be granted.
I am creating a new app and I want to be able to create a repository from the command line to be able to add commits without having to go to github and create a repo. I am totally noobian. is it possible? and if so how? Thank you!
for github you could try hub - is a command line tool that wraps git in order to extend it with extra features and commands that make working with GitHub easier.
Examples from doc:
$ git create
[ repo created on GitHub ]
> git remote add origin git#github.com:YOUR_USER/CURRENT_REPO.git
# with description:
$ git create -d 'It shall be mine, all mine!'
$ git create recipes
[ repo created on GitHub ]
> git remote add origin git#github.com:YOUR_USER/recipes.git
$ git create sinatra/recipes
[ repo created in GitHub organization ]
> git remote add origin git#github.com:sinatra/recipes.git
If you want to use git, then it is fairly straightforward, but you won't go far without going through some tutorial. I highly recommend Git Book by Scott Chacon. At least go through chapter 2 and perhaps chapter 3. To answer your questions though: Git is a distributed versioning system so you definitely do not need a repo in GitHub - you can create a repo on your own harddrive and then push it to any other repos (i.e. you could create a GitHub repository later and publish your repository there - it will be an exact clone!).
To create a repo you invoke a command: git init or git init repoName - the former creates the repo in current folder, the latter creates a new one named "repoName".
When you are ready to create a first commit invoke git add . to add all files to index (think of it as an area where you prepare what will go into the next commit) and the git commit -m "Commit message". nstead of adding all files you could also choose files to commit individualy by git add path/to/file
You will need at least some basics with git covered though before you can start using it comfortably.