I made a script on OSX terminal which utilizes the gsed and grep commands. I tried running this on a jenkins job but it seems that these were incompatible in this unix environment. Here is an example of the code I'm trying to run:
line_formatted=$(gsed -r '/name="adhoc.display"/{s#(.*value=")([^"]*)(".*)#\1$R{\2.LABEL}\3#g;}' <<<"$line")
sed -i bak -e 's|'"${line}"'|'"${line_formatted}"'|g' $topicJRXML_source
adhoc_value=$(grep -oE '{[^/]+}' <<<"$line_formatted" | cut -c2- | rev | cut -c2- | rev)
I am able to assign my formatted lines I'm reading from a file into the variables line_formatted and adhoc_value on the OSX terminal but when these are run on jenkins it seems to fail with the error: gsed: command not found
I was wondering if there is a way to run these commands on jenkins, or if there is an equivalent way to express these lines on jenkins?
You have to have the gsed command installed and in the PATH on the jenkins machine. Is the jenkins machine a linux box? You aren't running these commands in Jenkins. Jenkins runs the commands on the local machine in a shell.
Related
I am trying to update contents of a file from a variable with sed during Gitlab CI job. The variable comes from artifacts of the previous stage version. If simplified, my job looks something like this:
build-android-dev:
stage: build
dependencies:
- version
only:
- mybranch
before_script:
- PUBSPEC_VERSION="`cat app-pubspec-version`"
- echo "Pubspec version - $PUBSPEC_VERSION"
script:
- >
sed -i -E "s/^(version: )(.+)$/\1${PUBSPEC_VERSION}/g" pubspec.yaml
- cat pubspec.yaml | grep version
interruptible: true
tags:
- macmini-flutter
Unfortunatelly, the job fails with the following error message:
$ sed -i -E "s/^(version: )(.+)$/\1${PUBSPEC_VERSION}/g" pubspec.yaml
sed: 1: "s/^(version: )(.+)$/\14 ...": \1 not defined in the RE
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: exit status 1
PUBSPEC_VERSION coming from artifacts is the following:
$ echo "Pubspec version - $PUBSPEC_VERSION"
Pubspec version - 4.0.0+2
I am able to execute the command on my local Ubuntu (Linux) machine without any issues:
$ export PUBSPEC_VERSION=4.0.0+2
$ sed -i -E "s/^(version: )(.+)$/\1${PUBSPEC_VERSION}/g" pubspec.yaml
$ cat pubspec.yaml | grep version
version: 4.0.0+2
The remote machine where Gitlab Runner is started is MacOS. Not sure whether it matters.
As you can see, I also use folding style in my CI configuration like proposed here in order to avoid inproper colon interpretation.
I googled for solutions to solve the issue but it seems that I don't need to escape (though I also tried) group parentheses in my regular expression because I use extended regular expression.
So I'm stuck on it...
P.S. I don't have access to the shell of remote MacOS.
is MacOS.
-i takes a suffix argument, so -E is the backup suffix to create. Yuo would want:
- sed -i '' -E 's/...'
I'm trying to understand what's happening here out of curiosity, even though I can just copy and paste the output of the terminal to do what I need to do. The following command does not print anything.
ls /opt/local/var/macports/registry/portfiles -1 | sed 's/-.*//g' | sort -u | parallel "sudo port -N install" {} 2>&1 | grep -Po "Use '\K.*(?=')" | parallel "{}"
The directory I call ls on contains a bunch of filenames starting with the string I want to extract that ends at the first dash (so stringexample-4.2009 pipes stringexample into parallel (like xargs but to run each line separately). After running the command sudo port install <stringexample>, I get error outputs like so:
Unable to activate port <stringexample>. Use 'port -f activate <stringexample>' to force the activation.
Now, I wish to run port -f activate <stringexample>. However, I cannot seem to do anything with the output port -f activate gettext that I get to the terminal.
I cannot even do ... | grep -Po "Use '\K.*(?=')" | xargs echo or ... | grep -Po "Use '\K.*(?=')" >> commands_to_run.txt (the output stream to file only creates an empty file), despite the shorter part of the command:
ls /opt/local/var/macports/registry/portfiles -1 | sed 's/-.*//g' | sort -u | parallel "sudo port -N install {}" 2>&1 | grep -Po "Use '\K.*(?=')"
printing the commands to the terminal. Why does the pipe operator not work here? If the commands I wish to run are outputting to the terminal, surely there's got to be a way to capture them.
I am trying to kill all the existing processes before checkout the code and build it, in order to do, I am using the below command.
sudo ps -ef | grep 'dotnet' | grep -v grep | awk '{print $2}' | xargs -r kill -9;
This is working fine when I run it in the server manually. whereas, using in the jenkins pipeline as Execute Shell script, its not working.
Here is the jenkin's output
**-----------
[CICD] $ /bin/sh -xe /tmp/jenkins6283168714008394634.sh
+ ps -ef
+ grep dotnet
+ grep -v grep
+ + awk {print $2}xargs
-r kill -9
Failed build for hudson.tasks.Shell#178f47d3
**------------
Can someone please help?
I have windows docker installed and when I run this on command line:
docker stop $(docker ps -q)
I get:
unknown shorthand flag: 'q' in -q)
But when running:
docker ps -q
Everything is alright. Any clues?
The $(sub command) is a syntax of the bash shell (along with many other command shells on Linux). If you try to run this from a Windows command prompt, it will not be correctly expanded before running the rest of the command and you'll see the errors you're encountering. Try installing and running the commands on bash for Windows.
I used this in Windows:
powershell docker stop $(docker ps -aq)
You can try a little trick to stop all containers :
docker ps -a -q | xargs -n 1 -P 8 -I {} docker stop {}
It does not contain $ symbol, so should be correct
You can also enable Bash shell within Powershell on recent releases of Windows 10 and the use the $ command
I would like to know what is the difference between the below commands:
ssh vagrant#someipaddress
cd /home/vagrant/
grep -i "something" data.txt
and
ssh vagrant#someipaddress 'cd /home/vagrant; cat data.txt' | grep -i "something"
From this website it mentions, that you can send multiple commands to the remote server. Is the second option actually logging into the server? What is the benefit in this second approach?
Strictly Speaking from the example provided:
The first command:
Logs onto the remote server
Executes a couple commands, and
Stays logged on to the server
The second command runs half on the remote machine, logs out of the remote machine, and then pipes the output to grep on your local machine, all in one command line.
Breaking down what's happening:
ssh vagrant#someipaddress 'cd /home/vagrant; cat data.txt' | grep -i "something"
The section in bold is running on your local PC, based on the output from the ssh session
The 'quotes "contain" the entire command block
the " quotes "contain" the individual arguments within the command block.
You may have meant to do this:
ssh vagrant#someipaddress 'cd /home/vagrant; cat data.txt' | grep -i "something"
Where the bold section runs locally
Or you may have intentionally done this:
ssh vagrant#someipaddress 'cd /home/vagrant/ | grep -i "something" data.txt'
Where the entire command runs on the server.
Either way, the end result:
Is that you automatically log out of the remote machine, and the whole command sequence was executed in one hit.