bash use ruby code and save as variable - ruby

my end goal is to parse a visual studio team services ssh git url and use it to clone origin and my fork. I'm in windows and I use git bash I've made a few shell scripts to help me to clone it. Before when we used gitweb it was easy for me to parse as I could either run git_clone myproject or git_clone myproject.git or git_clone git://ourgitserver.ourcompany.com/myproject.git and the script would clone the above as origin and also add a remote with my user name in the form of ssh://git#outgitserver.ourcompany.com/myproject.git (and it handled name spaces well too). Well we started using vsts and I want to do the same thing.
The git_clone method changed a few times because of how people would tell/im/email me the link for the git project. I wanted to be able to just copy and paste it with minimal changes. thus far I have a simple git_vsts_close which requires two parameters the name of the project and the name of the repository. (in gitweb we would reference the namespace as vsts's project and the project would be vsts's repository). For the time being I'd like to take either the ssh url or the two parameters and do all the git things. in brief this is what i have so far
function git_vsts_clone {
local projectName=$1
local repositoryName=$2
if MISSING_ARG "usage: git_vsts_clone <project name> <repositoryName>\n projectName must be provided\n repositoryName must be provided" $projectName; then return 1; fi;
if MISSING_ARG "usage: git_vsts_clone <project name> <repositoryName>\n projectName must be provided\n repositoryName must be provided" $repositoryName; then return 1; fi;
local gitServer="ssh://mycompany#vs-ssh.visualstudio.com:22/${projectName}/_ssh/${repositoryName}"
local clonePath="/c/git/${projectName}/${repositoryName}"
local user_name=${USER:-${USERNAME}}
if [ ! -d $clonePath ]; then
INFO "Cloning $gitServer"
git clone $gitServer $clonePath || { ERROR "ERROR cloning $gitServer"; return 1;}
pushd $clonePath
INFO "Updating Submodules (gsui)"
git submodule update --init
INFO "adding user fork ${user_name}"
git remote add $user_name $gitServer.$user_name
git fetch $user_name
popd
INFO "Opening $clonePath in vscode"
fi
code $clonePath
}
last time when I tried to parse a url in bash I struggled with the whole split an item into an array. so I decided i'd try to use ruby (since it has a easy split method) so i've tried things like
$ gitServer='ssh://mycompany#vs-ssh.visualstudio.com:22/someProject/_ssh/myRepo'
$ ruby -e "a = '$gitServer'; b=a.split('/'); p b"
["ssh:", "", "mycompany#vs-ssh.visualstudio.com:22", "someProject", "_ssh", "myRepo"]
$ foo=`ruby -e "a = '$gitServer'; b=a.split('/'); p b"`
$ echo "${c[3]}"
$ echo "${c[0]}"
["ssh:", "", "mycompany#vs-ssh.visualstudio.com:22", "someProject", "_ssh", "myRepo"]
so I dunno. I don't have to use ruby it just seemed like a easy solution... now not so much. So how can I get the project and repository name out of the url in either bash or bash using ruby?

Here is a way you can get the values into environment variables using Ruby:
Assuming you have a URL environment variable containing the git repo url, such as created by the line below:
export URL='ssh://mycompany#vs-ssh.visualstudio.com:22/someProject/_ssh/myRepo'
You can do the following to put your desired values into other environment variables:
export PROJECT=`ruby -e "puts ENV['URL'].split('/')[3]"`
export REPO_NAME=`ruby -e "puts ENV['URL'].split('/')[5]"`

Related

Bash - Gitlab CI not converting variable to a string

I am using GitLab to deploy a project and have some environmental variables setup in the GitLab console which I use in my GitLab deployment script below:
- export S3_BUCKET="$(eval \$S3_BUCKET_${CI_COMMIT_REF_NAME^^})"
- aws s3 rm s3://$S3_BUCKET --recursive
My environmental variables are declared like so:
Key: s3_bucket_development
Value: https://dev.my-bucket.com
Key: s3_bucket_production
Value: https://prod.my-bucket.com
The plan is that it grabs the bucket URL from the environmental variables depending on which branch is trying to deploy (CI_COMMIT_REF_NAME).
The problem is that the S3_BUCKET variable does not seem to get set properly and I get the following error:
> export S3_BUCKET=$(eval \$S3_BUCKET_${CI_COMMIT_REF_NAME^^})
> /scripts-30283952-2040310190/step_script: line 150: https://dev.my-bucket.com: No such file or directory
It looks like it picks up the environmental variable value fine but does not set it properly - any ideas why?
It seems like you are trying to get the value of the variables S3_BUCKET_DEVELOPMENT and S3_BUCKET_PRODUCTION based on the value of CI_COMMIT_REF_NAME, you can do this by using parameter indirection:
$ a=b
$ b=c
$echo "${!a}" # c
and in your case, you would need a temporary variable as well, something like this might work:
- s3_bucket_variable=S3_BUCKET_${CI_COMMIT_REF_NAME^^}
- s3_bucket=${!s3_bucket_variable}
- aws s3 rm "s3://$s3_bucket" --recursive
You are basically telling bash to execute command, named https://dev.my-bucket.com, which obviously doesn't exist.
Since you want to assign output of command when using VAR=$(command) you should probably use echo
export S3_BUCKET=$(eval echo \$S3_BUCKET_${CI_COMMIT_REF_NAME^^})
Simple test:
VAR=HELL; OUTPUT="$(eval echo "\$S${VAR^^}")"; echo $OUTPUT
/bin/bash
It dynamically creates SHELL variable, and then successfully prints it

Chef - use env variable with package resource what created with bash resource

I would get the changed project from git commit and install the package based on this.
Here is my code
bash 'get_project' do
code <<-EOH
filelist=$(git diff-tree --no-commit-id --name-only -r $1)
for file in ${filelist[#]}; do
project=$(echo $file | cut -d "/" -f1)
projectList+=($project)
done
for changedProject in $(echo "${projectList[#]}" | sort | uniq); do
INSTALLABLE_RPM=application-$changedProject
done
EOH
environment 'INSTALLABLE_RPM' => '$INSTALLABLE_RPM'
end
zypper_package ENV['INSTALLABLE_RPM']
My idea is to generate the INSTALLABLE_RPM variable with bash and install the package with zypper. Unfortunately it doesn't work. The zypper_package resource cant recognize.
I ran out of ideas :-(
The environment property of the bash resource is to supply existing environment variables to execute the bash command(s).
(These variables must exist for a command to be run successfully.)
Specifying environment variables here will not set them in the shell. Also from within the code block you will not be able to access the Ruby's ENV hash.
There may not be a straight-forward way to do this. One of the options is to write this package (list?) to a file. Then we can read the file contents into variable, and use it with zypper_package resource.
Example:
Since you have used for loop in Shell, I believe you get a list of packages, so I am considering pkg_list as Array. I've set compile_time to true as the variable assignment below bash resource will only run during compile time.
bash 'get_project' do
code <<-EOH
# your code as-it-is
for changedProject in $(echo "${projectList[#]}" | sort | uniq); do
echo "application-$changedProject" >> /tmp/rpm_packages
done
EOH
compile_time true
end
pkg_list = File.read('/tmp/rpm_packages').split
zypper_package pkg_list
# remove the file for good measure :)
file '/tmp/rpm_packages' do
action :delete
end

How to check if master branch pushed with a tag under Travis?

I want to do some additional testing on Travis, but I just want to check if the commit has directly pushed to master branch and the commit comes with a Tag ? Is there anyway to check for tag on Travis inside the yaml file ?
During build travis set some useful environment variable, which can be used in your script also in yml file.
https://docs.travis-ci.com/user/environment-variables#Default-Environment-Variables
Example:
script:
- if [ "$TRAVIS_BRANCH" = "master" -a "$TRAVIS_PULL_REQUEST" = "false" ]; then something on direct push to master; fi
- if [ ! "$TRAVIS_PULL_REQUEST" = "false" ]; then something on pull request; fi
- if [ -n "$TRAVIS_TAG" ]; then something when tag set; fi
Travis run commands in order like defined in script tag, so you can build some logic on build which depends on environment variables
As Slawomir Jaranowski described, travis provides environment variables describing the commit.
Travis’ default convenience environment variables will tell you either the branch name or the tag. Travis does not give you the branch name in tagged builds, but you can get it for yourself if you need both the branch and the tag.

Setting environment variables with puppet

I'm trying to work out the best way to set some environment variables with puppet.
I could use exec and just do export VAR=blah. However, that would only last for the current session. I also thought about just adding it onto the end of a file such as bashrc. However then I don't think there is a reliable method to check if it is all ready there; so it would end up getting added with every run of puppet.
I would take a look at this related question.
*.sh scripts in /etc/profile.d are read at user-login time (as the post says, at the same time /etc/profile is sourced)
Variables export-ed in any script placed in /etc/profile.d will therefore be available to your users.
You can then use a file resource to ensure this action is idempotent. For example:
file { "/etc/profile.d/my_test.sh":
content => 'export MYVAR="123"'
}
Or an alternate means to an indempotent result:
Example
if [[ ! grep PINTO_HOME /root/.bashrc | wc -l > 0 ]] ; then
echo "export PINTO_HOME=/opt/local/pinto" >> /root/.bashrc ;
fi
This option permits this environmental variable to be set when the presence of the
pinto application makes it warrented rather than having to compose a user's
.bash_profile regardless of what applications may wind up on the box.
If you add it to your bashrc you can check that it's in the ENV hash by doing
ENV[VAR]
Which will return => "blah"
If you take a look at Github's Boxen they source a script (/opt/boxen/env.sh) from ~/.profile. This script runs a bunch of stuff including:
for f in $BOXEN_HOME/env.d/*.sh ; do
if [ -f $f ] ; then
source $f
fi
done
These scripts, in turn, set environment variables for their respective modules.
If you want the variables to affect all users /etc/profile.d is the way to go.
However, if you want them for a specific user, something like .bashrc makes more sense.
In response to "I don't think there is a reliable method to check if it is all ready there; so it would end up getting added with every run of puppet," there is now a file_line resource available from the puppetlabs stdlib module:
"Ensures that a given line is contained within a file. The implementation matches the full line, including whitespace at the beginning and end. If the line is not contained in the given file, Puppet appends the line to the end of the file to ensure the desired state. Multiple resources can be declared to manage multiple lines in the same file."
Example:
file_line { 'sudo_rule':
path => '/etc/sudoers',
line => '%sudo ALL=(ALL) ALL',
}
file_line { 'sudo_rule_nopw':
path => '/etc/sudoers',
line => '%sudonopw ALL=(ALL) NOPASSWD: ALL',
}

Disable auto-completion of remote branches in Git Bash?

I'm working on a fairly large git repo with a couple of thousand (remote) branches. I am used to using auto-completion (using [TAB]) in the console (Git Bash in that case), so I unconsciously do that for git commands, too.
e.g. I'd type
git checkout task[TAB]
with the effect that the console stalls for often minutes. Is there a way to limit auto-completion to local branches only?
With Git 2.13 (Q2 2017), you can disable (some of) the branch completion.
git checkout --no-guess ...
# or:
export GIT_COMPLETION_CHECKOUT_NO_GUESS=1
See commit 60e71bb (21 Apr 2017) by Jeff King (peff).
(Merged by Junio C Hamano -- gitster -- in commit b439747, 01 May 2017)
As documented in contrib/completion/git-completion.bash now:
You can set the following environment variables to influence the behavior of the completion routines:
GIT_COMPLETION_CHECKOUT_NO_GUESS
When set to "1", do not include "DWIM" suggestions in git-checkout
completion (e.g., completing "foo" when "origin/foo" exists).
Note: DWIM is short for Do What I Mean, where a system attempts to anticipate what users intend to do, correcting trivial errors automatically rather than blindly executing users' explicit but potentially incorrect inputs.
completion: optionally disable checkout DWIM
When we complete branch names for "git checkout", we also complete remote branch names that could trigger the DWIM behavior. Depending on your workflow and project, this can be either convenient or annoying.
For instance, my clone of gitster.git contains 74 local "jk/*" branches, but origin contains another 147.
When I want to checkout a local branch but can't quite remember the name, tab completion shows me 251 entries. And worse, for a topic that has been picked up for pu, the upstream branch name is likely to be similar to mine, leading to a high probability that I pick the wrong one and accidentally create a new branch.
Note: "picked up for pu": see a What's cooking in git.git: it starts with:
Commits prefixed with '-' are only in 'pu' (proposed updates) while commits prefixed with '+' are in 'next'.
This is part of the Git Workflow Graduation process.
pu (proposed updates) is an integration branch for things that are not quite ready for inclusion yet
This patch adds a way for the user to tell the completion
code not to include DWIM suggestions for checkout.
This can already be done by typing:
git checkout --no-guess jk/<TAB>
but that's rather cumbersome.
The downside, of course, is that you no longer get completion support when you do want to invoke the DWIM behavior.
But depending on your workflow, that may not be a big loss (for instance, in git.git I am much more likely to want to detach, so I'd type "git checkout origin/jk/<TAB>" anyway).
I'm assuming that you are using the git-completion.bash script, and that you only care about git checkout.
To accomplish this, I just changed one line in the definition of the _git_checkout () function in git-completion.bash:
< __gitcomp_nl "$(__git_refs '' $track)"
---
> __gitcomp_nl "$(__git_heads '' $track)"
My understanding is that this only affects the tab-completion action (because of its location within the * case of the switch-case statement).
If you installed git-completion via homebrew, it's located here:
/usr/local/etc/bash_completion.d/git-completion.bash
Following erik.weathers' answer above, I made the following change so autocompletion can work for both local and remote based on the current prefix. By default, it'll only search local, but if I specify origin/… it'll know I want to search remote branches too.
In the _git_checkout () method, change
__gitcomp_nl "$(__git_refs '' $track)"
to:
# only search local branches instead of remote branches if origin isn't specified
if [[ $cur == "origin/"* ]]; then
__gitcomp_nl "$(__git_refs '' $track)"
else
__gitcomp_nl "$(__git_heads '' $track)"
fi
Of course, you can change origin to something else or you can have it search through through a list of remote prefixes if you have more than 1.
You can hack /etc/bash_completion.d/git
You'll need to edit __git_refs ()
Note that the change in behaviour will apply every where (so even with git push/pull where you might not want it to). You could of course, make a copy of the function or pass an extra parameter, but I leave that to you
You could think that you just the local branches with the alias co and all the branches with the complete command checkout.
You could perform the following. In your .bashrc, you redefine the _git_checkout() function. You let this function unchanged, except the end:
if [ $command -eq "co" ]; then
__gitcomp "$(__git_refs_local '' $track)"
else
__gitcomp "$(__git_refs '' $track)"
fi
Then, you just have to define a new function, __git_refs_local, where you remove the remote stuff.
Carey Metcalfe wrote a blog post containing a solution that also edits the auto-completion function, but with slightly newer code than other answers. He also defines an alias checkoutr that keeps the old auto-complete behavior in case it’s ever needed.
In short, first create the checkoutr alias with this command:
git config --global alias.checkoutr checkout
Then find git-completion.bash, copy the _git_checkout function into your shell’s RC file so that it gets redefined, and inside that function, replace this line:
__git_complete_refs $track_opt
with the following lines:
if [ "$command" = "checkoutr" ]; then
__git_complete_refs $track_opt
else
__gitcomp_direct "$(__git_heads "" "$cur" " ")"
fi
See the blog post for more details and potential updates to the code.
Modifying $(brew --prefix)/etc/bash_completion.d/git-completion.bash is not a good idea because it will be overwritten every time you update Git through Homebrew.
Combining all the answers I overwrite only _git_checkout function from the completion file in my .bash_profile after sourcing the completion file:
_git_checkout ()
{
__git_has_doubledash && return
case "$cur" in
--conflict=*)
__gitcomp "diff3 merge" "" "${cur##--conflict=}"
;;
--*)
__gitcomp "
--quiet --ours --theirs --track --no-track --merge
--conflict= --orphan --patch
"
;;
*)
# check if --track, --no-track, or --no-guess was specified
# if so, disable DWIM mode
local flags="--track --no-track --no-guess" track=1
if [ -n "$(__git_find_on_cmdline "$flags")" ]; then
track=''
fi
# only search local branches instead of remote branches if origin isn't
# specified
if [[ $cur == "origin/"* ]]; then
__gitcomp_nl "$(__git_refs '' $track)"
else
__gitcomp_nl "$(__git_heads '' $track)"
fi
;;
esac
}
I'm not using Git Bash myself, but if this is the same as mentioned in
http://tekrat.com/2008/04/30/bash-autocompletion-git-super-lazy-goodness/, you should be able to replace git branch -a with a plain git branch in
_complete_git() {
if [ -d .git ]; then
branches=`git branch -a | cut -c 3-`
tags=`git tag`
cur="${COMP_WORDS[COMP_CWORD]}"
COMPREPLY=( $(compgen -W "${branches} ${tags}" -- ${cur}) )
fi
}
complete -F _complete_git git checkout
(in your .profile or similar) and get what you want.
FWW here is a hack to __git_complete_refs that does the trick
__git_complete_refs ()
{
local remote track pfx cur_="$cur" sfx=" "
while test $# != 0; do
case "$1" in
--remote=*) remote="${1##--remote=}" ;;
--track) track="yes" ;;
--pfx=*) pfx="${1##--pfx=}" ;;
--cur=*) cur_="${1##--cur=}" ;;
--sfx=*) sfx="${1##--sfx=}" ;;
*) return 1 ;;
esac
shift
done
echo cur_ $cur_ > a
if [[ $GIT_COMPLETION_CHECKOUT_NO_GUESS != 1 || $cur_ == "origin"* ]]; then
__gitcomp_direct "$(__git_refs "$remote" "$track" "$pfx" "$cur_" "$sfx")"
else
__gitcomp_direct "$(__git_heads "" "$cur_")"
fi
}

Resources