Capistrano deploy fails after I changed the repository URL - ruby

I have a simple deployment via capistrano from a Git repository. At first I was deploying form GitHub, everything worked just fine. But then I moved my repository to BitBucket and now I'm getting
fatal: Could not parse object '9cfb...'.
The problem goes away once I change
set :deploy_via, :remote_cache
to
set :deploy_via, :copy
but that doesn't fix the problem, it only bypasses it. Is there any way I can tell capistrano to just drop the old cache?

Capistrano 2.X
Delete and re-clone the repo using the new address:
cd $deploy_to/shared
rm -rf cached-copy
git clone ssh://git#example.org/new/repo.git cached-copy
Modify your config/deploy.rb to use the new repo:
set :repository, "ssh://git#example.org/new/repo.git"
set :scm, :git
set :deploy_via, :remote_cache
Deploy again:
cap deploy
Capistrano 3.X
Remove the $deploy_to/repo directory
Modify your config/deploy.rb (same as 2.X)
cap deploy

I gotta say I’m not sure, since I haven’t been able to test this but this should work:
cap deploy:cleanup -s keep_releases=0
Since it wipes every release (cache) from the server.
Apparently you will also need to remove shared/cached-copy, because this doesn’t seem to be cleaned by the Capistrano call above according to the comment below.

Capistrano 2 and below
SSH to your server and update the repo in ./shared/cached-copy/.git/config of the deployment folder, or just remove the ./shared/cached-copy
Capistrano 3 and above
SSH to your server and update the repo in ./repo/config of the deployment folder.
Check Fixing Capistrano 3 deployments after a repository change

I solved this with the following in deploy.rb:
namespace :deploy do
task :cope_with_git_repo_relocation do
run "if [ -d #{shared_path}/cached-copy ]; then cd #{shared_path}/cached-copy && git remote set-url origin #{repository}; else true; fi"
end
end
before "deploy:update_code", "deploy:cope_with_git_repo_relocation"
It makes deploys a little slower, so it's worth removing once you're comfortable that all your deploy targets have caught up.

You need to change git origin in your /shared/cached-copy folder
cd /var/www/your-project/production/shared/cached-copy
git remote remove origin
git remote add origin git#bitbucket.org:/origin.git
try cap production deploy

The most simple way is just changing the repo url to the new one in .git/config in the shared/cached-copy directory on the webserver. Then you can do a normal deploy as usual.

Depends on your version Capistrano 3 is different from it's older ancestors:
Read my original answer here and how to fix similar issues Capistrano error when change repository using git

If you need to do a lot of repo's you might want to add a task for it.
For capistrano 3 you add this task in your deploy.rb
desc "remove remote git cache repository"
task :remove_git_cache_repo do
on roles(:all) do
execute "cd #{fetch(:deploy_to)} && rm -Rf repo"
end
end
And then run it once for every stage:
cap testing remove_git_cache_repo

Here's the Capistrano 3 version of what this answer talks about. It might be tedious to do what the answer suggests on each server.
So drop this in deploy.rb and then run cap <environment> deploy:fix_repo_origin
namespace :deploy do
desc 'Fix repo origin, for use when changing git repo URLs'
task :fix_repo_origin do
on roles(:web) do
within repo_path do
execute(:git, "remote set-url origin #{repo_url}")
end
end
end
end

For Capistrano 3.0+
Change the repository URL in your config/deploy.rb
Change the repository URL in the your_project/repo/config file on the server.

Related

Best Simple Way to Share a Git Repository over LAN

I need to share a large git repository (>3GB in total, .git folder is ~1.1GB) as Windows Shared Folder over LAN with my colleagues. But they found it is really slow to clone and push/pull -- i.e. waits ~30min before the clone starts, and suspended ~5min before any push/pull starts.
Does everyone know any methods to reduce the latency or better way to share the repo?
P.S. I don't want to setup a Gitlab because it's too complicated.
Since your git repo is very big, you can bundle the whole git repo or part of commits among your colleagues sharing.
Bundle the whole repo: git bundle create repo.bundle --all
Bundle a branch: git bundle create repo.bundle branchname
Bundle some commits: git bundle create commits.bundle branchname ^commit
For the one who apply the bundled commits to his local repo, he can verify the bundle file by git bundle verify /path/to/bundle/file.
More details, you can refer Git's Little Bundle of Joy and git bundle.
git daemon works really well in this situation. Its performance is good, but lacks of security on Windows Server.
On server side, run the following command:
cd ~/Documents/All-My-Git-Repos/
git daemon --verbose --reuseaddr --export-all --enable=receive-pack --base-path=.
On client side, clone any repo like this:
git clone git://my-git-server-address/repo-folder-name repo-clone-name
While using this on Windows server, the firewall will prompt for permissions on Port 9418, which should be granted.

How can I use private gems(GemFury) in a docker container?

I'm trying to run some ruby scripts for automating exports. Since these run remotely we build them in a Docker container and push them to iron worker.
We use GemFury for hosting some essential private gems for these scripts.
To keep the credentials for GemFury out of Git we use a global bundle config bundle config gem.fury.io MY_SECRET_TOKEN.
How can I set the config for bundle so it will pull in gems from GemFury without having them show in source control?
Set the global bundle config property as an application specific property. Push the changes to the public repository. Update the SECRET_TOKEN value in the bundle-config file ($APP_DIR/.bundle/config) and run the $ git update-index --assume-unchanged <file> command to remove the file from git tracking and prevent updating the actual SECRET_TOKEN value in the public repository.
$ bundle config --local gem.fury.io SECRET_TOKEN
$ git commit -a -m "adding application bundle config properties"
$ git push origin master
$ bundle config --local gem.fury.io d1320f07ac50d1033e8ef5fbd56adf360ec103b2
$ git update-index --assume-unchanged $APP_DIR/.bundle/config
This creates a template file on the public repository. Provide instructions to repository contributors to add the secret token and execute the same --assume-unchanged command.
Example Files
$APP_DIR/.bundle/config file on public github repo:
---
BUNDLE_GEM__FURY__IO: MY_SECRET_TOKEN
$APP_DIR/.bundle/config file a local machine
---
BUNDLE_GEM__FURY__IO: d1320f07ac50d1033e8ef5fbd56adf360ec103b2
See bundle-cofig documentation for clarification and more detail
NOTE: The disadvantage to this approach is two fold:
Developers who clone the repository and need the SECRET_TOKEN value will have to obtain it by some external manual process (good security practices, but a pain to set up)
If you need to add more bundle-config properties, you will have to run git update-index --no-assume-unchanged <file> to enable tracking, and revert all of the private values to their pseudo values. This template method, also risks that contributors forget to disable tracking on the file and push their private values to the public repo (but at least they won't be your secret values)
The advantage of this template approach is that you are giving the developers as much as possible to be able to start contributing to the repository.

How do you force a deploy to Heroku without doing a git commit

I had a case where a push to heroku failed because of a database issue. I fixed it, but the only way I know to deploy is via "git push heroku master". Since I didn't commit anything, it won't push a new deployment. The only way I can get it to deploy to make some minor change and then do it. Is there a way to force a deploy? I'm using play 2.1.2.
You could try a throw-away commit if you concern is to avoid actually saving the 'minor/dummy' commits to the repo permanently:
Heroku Throwaway Commit
See Section: "Automating the throwaway commit"
The author has basically automated the above with a quick bash script; however, as the author indicates use with caution -- you wouldn't want to use this in other situations with un-tested code.
https://www.darraghoriordan.com/2019/03/02/heroku-push-failed-force-rebuild/
heroku plugins:install heroku-releases-retry
heroku releases:retry
# or
heroku releases:retry --app darragh-starter
If you are just trying to reload the dynos you can issue this from the command line:
heroku restart -a appname
More info https://devcenter.heroku.com/articles/application-offline

Capistrano 3 copy strategy equivalent

I updated to Cap 3 and it appears that set :deploy_via, :copy is no longer supported. In the release annoucement there is a link to a video for replicating the copy strategy which currently returns a 404.
I used the :copy strategy because the server did not have access to git or access to the repository because it was behind a firewall.
What is the best way to replicate this functionality with v3?
I ran into the same issue and posted a similar question on the capistrano google group.
See here: https://groups.google.com/forum/#!topic/capistrano/BRa4Vj1_mEo
Short answer: Write your own rake task.
The capistrano maintainer provided some example code, via a blog post on his website, that can be found here: http://lee.hambley.name/2013/06/11/using-capistrano-v3-with-chef.html
In the end, we've decided to go w/ a different strategy entirely, and implement a mirror repository on the same network as our servers.
There is one work in the exact same way
https://github.com/xuwupeng2000/capsitrano-scm-gitcopy
Capistrano 3 :copy
A copy strategy for Capistrano 3, which mimics the :copy scm of Capistrano 2.
This Gem is inspired by and based on https://github.com/wercker/capistrano-scm-copy.
Thank wercker so much.
This will make Capistrano tar the a specific git branch, upload it to the server(s) and then extract it in the release directory.
Usage
cap uat deploy -s branch=(your release branch)
You can use its gem - https://github.com/WildZero/capistrano-scm-tar-copy
set:
set :scm, :copy
set :include_dir, '/User/w1ldzer0/ExampleDir'
and go

GIt Deployment + Configuration Files + Heroku

I'm using Heroku to host a Rails app, which means using Git to deploy to Heroku. Because of the "pure Git workflow" on Heroku, anything that needs to go upstream to the server has to be configured identically on my local box.
However I need to have certain configuration files be different depending on whether I'm in the local setup or deployed on Heroku. Again, because of the deployment method Heroku uses I can't use .gitignore and a template (as I have seen suggested many times, and have used in other projects).
What I need is for git to somehow track changes on a file, but selectively tell git not to override certain files when pulling from a particular repo -- basically to make certain changes one-way only.
Can this be done? I'd appreciate any suggestions!
You can have config vars persistently stored ON each heroku app's local setup so they do not have to be in your code at all! so the same code can run on multiple heroku sites but with different configuration. It very simple, easy, elegant...
It's the approach we used. (We used it for the SAME thing... we have multiple clones of the SAME app at Heroku, but we want only ONE source at github, in our dev local directory we do the PUSH to ORIGIN (github), then when we have it the way we like it, we CD to the prod local directory, which goes to the SAME github repository, and we ONLY PULL from GITHUB into this directory, never push (eg, all pushes to github come from our dev directory, the prod directory is just a staging area for the other heroku app.)
By having the different configs ON the different HEROKU sites (as explained below), the EXACT SAME CODE works on BOTH heroku sites.
So our workflow is: (the key is that BOTH directories point to SAME github repo)
cd myDEVdir
*....develop away....*
git add .
git commit -am "another day, another push"
git push origin *(to our SINGLE github repo)*
git push heroku *(test it out on heroku #1)*
cd ../myPRODdir
git pull *(grabs SAME code as used on other site *)
git push heroku *(now the SAME code runs on Heroku #2)*
that's it!
Now here's how you keep your site-specific config vars ON the heroku site:
http://docs.heroku.com/config-vars
on your local command line, for EACH of your two local directories, do:
$ heroku config:add FIRST_CONFIGVAR=fooheroku1
Adding config vars:
FIRST_CONFIGVAR => fooheroku1
$ heroku config:add SECOND_CONFIGVAR=barheroku1
Adding config vars:
SECOND_CONFIGVAR => barheroku1
to see the ones you have defined:
$ heroku config
FIRST_CONFIGVAR => fooheroku1
SECOND_CONFIGVAR => barheroku1
then cd to your other directory myPRODdir and do the SAME thing, only set the same remote heroku vars to fooheroku2 and barheroku2.
then in your rails app you simple refer to them like so:
a = ENV['FIRST_CONFIGVAR']
One app will read 'fooheroku1' the other app will read 'fooheroku2'
And finally, on your LOCAL directory myDEVdir, where you run in DEV mode, put the same config commands in your config/environment/development.rb file your 'dev' version of the config vars will be set to whatever they should be:
ENV['FIRST_CONFIGVAR'] = "foodev"
ENV['SECOND_CONFIGVAR'] = "bardev"
Easy, elegant. Thanks, Heroku!
Here are a few solutions:
1) If you want to ignore files only on heroku, use slugignore
2) If your changes are minor, stay DRY and use universal config files, inserting switches for server-specific behavior
if Rails.env == "production"
#production server code
elsif Rails.env == "development"
#development server code
else
#test server code
end
3) If your changes are major, write one config file and add a "smudge file" to config/initializers for each additional server. Basically, you would have separate files using the technique in (2).
4) If your changes are SWEEPING (unlikely), then maintain separate branches for each server.
5) This script can do exactly what you requested, but may be overkill.
I hope that helped.

Resources