How to use bash variables in Jenkins multi-line shell script - bash

I have unsuccessfully tried to use bash variables in Jenkins pipeline.
My first attempt
sh """#!/bin/bash
for file in *.map; do
filename=`basename $file .map`
echo "##### uploading ${$filename}"
curl -X POST ${SERVER_URL}/assets/v1/sourcemaps \
-F service_name="${SERVICE_NAME}" \
-F service_version="${revision}" \
-F bundle_filepath="${main_url}${filename}" \
-F sourcemap="#${filename}.map" &
done
wait
"""
Resulted in exception: MissingPropertyException: No such property: file
The second attempt, after seeing this answer https://stackoverflow.com/a/35047530/9590251
sh """#!/bin/bash
for file in *.map; do
filename=`basename \$file .map`
echo "##### uploading \$filename"
curl -X POST ${SERVER_URL}/assets/v1/sourcemaps \
-F service_name="${SERVICE_NAME}" \
-F service_version="${revision}" \
-F bundle_filepath="${main_url}\$filename" \
-F sourcemap="#\$filename.map" &
done
wait
"""
Simply omitted bash variables. So $filename was empty.
How do I need to property encode bash variables in this scenario?

Try this:
sh """#!/bin/bash
set -x
for file in *.map; do
filename="\$(basename "\$file" .map)"
echo "Uploading \$filename"
curl -X POST "${SERVER_URL}/assets/v1/sourcemaps" \
-F service_name="${SERVICE_NAME}" \
-F service_version="${revision}" \
-F bundle_filepath="${main_url}\$filename" \
-F sourcemap="#\${filename}.map" &
done
wait
"""

Related

Environment variables not defined in SSH AuthorizedKeysCommand (Docker)

I'm trying to make the private key SSH connection with LDAP.
/etc/ssh/sshd_config
AuthorizedKeysCommand /etc/ldap_ssh_authorized_keys.sh
AuthorizedKeysCommandUser nobody
Script to get public keys from LDAP server
/etc/ldap_ssh_authorized_keys.sh
#!/bin/bash
USERSLIST=$( ldapsearch -x -D "${LDAP_USER}" -w "${LDAP_PASSWORD}" -H $LDAP_URI -b "${LDAP_BASEDN}" -s sub '(objectClass=posixAccount)' -u 'uid' \
grep '^uid:' | sed -n '/^ /{H;d};/uid:/x;$g;s/\n *//g;s/uid: //gp' \
)
while IFS= read -r line; do
exists=$(ldapsearch -x -D "${LDAP_USER}" -w "${LDAP_PASSWORD}" -H $LDAP_URI -b "${LDAP_BASEDN}" \
-s sub "(&(objectClass=posixGroup)(cn=sysadmin)(memberUid=${line}))" | grep "^# numEntries:")
if [[ ! -z $exists ]]
then
ldapsearch -x -D "${LDAP_USER}" -w "${LDAP_PASSWORD}" -H $LDAP_URI -b "${LDAP_BASEDN}" \
-s sub "(&(objectClass=posixAccount)(uid=${line}))" \
-u 'sshPublicKey' \
| sed -n '/^ /{H;d};/sshPublicKey:/x;$g;s/\n *//g;s/sshPublicKey: //gp'
echo -e "";
fi;
done <<< "$USERSLIST"
When I'm running script with /bin/bash it's working well and return my public keys.
All environment variables defined normally.
LDAP_URI
LDAP_BASEDN
LDAP_USER
LDAP_PASSWORD
The script also running normally when trying to make an SSH connection. But environment variables not available.
I'm trying also with AuthorizedKeysCommandUser as root. But nothing changed.
I solved this problem by getting the environment variables from /proc/1/environ.
Reference

Curl Works in Command Line, but Not Shell Script

I'm trying to automate a post-build process to get closer to a one-click release. However, I'm having problems with a script that.
Expected: The script to work exactly like the command line.
Results: Everything works, including grabbing the versionCodes except I am getting a curl: (26) Failed to open/read local data from file/application
The script I wanted to imitate with a more automated approach.
react-native bundle \
--platform android \
--dev false \
--entry-file index.js \
--bundle-output android-release.bundle \
--sourcemap-output android-release.bundle.map &&
curl https://upload.bugsnag.com/react-native-source-map \
-F apiKey=API-KEY \
-F appVersion=10.6.7 \
-F appVersionCode=4515 \
-F dev=false \
-F platform=android \
-F sourceMap=#android-release.bundle.map \
-F bundle=#android-release.bundle
react-native bundle \
--platform ios \
--dev false \
--entry-file index.js \
--bundle-output ios-release.bundle \
--sourcemap-output ios-release.bundle.map &&
curl https://upload.bugsnag.com/react-native-source-map \
-F apiKey=API-KEY \
-F appVersion=10.6.7 \
-F appBundleVersion=4515 \
-F dev=false \
-F platform=ios \
-F sourceMap=#ios-release.bundle.map \
-F bundle=#ios-release.bundle
So I wrote a script to fix this, which is
#!/bin/sh
# Ensure this is run on the commit used to generate the published app!
# Comment this line out to enable bundling and publishing
# debug=echo
bundle () {
platform=$1
$debug yarn react-native bundle \
--platform "$platform" \
--dev false \
--entry-file index.js \
--bundle-output "$platform-release.bundle" \
--sourcemap-output "$platform-release.bundle.map"
}
upload () {
platform=$1
version=$2
version_code=$3
# See https://docs.bugsnag.com/api/rn-source-map-upload/#uploading-source-maps
curl --http1.1 https://upload.bugsnag.com/react-native-source-map \
-F apiKey="$BUGSNAG_API_KEY" \
-F appVersion="$version" \
-F appVersionCode="$version_code" \
-F dev=false \
-F platform="$platform" \
-F sourceMap=#"$platform-release-bundle.map" \
-F bundle=#"$platform-release.bundle" \
}
read_version () {
local tag=$1
local file=$2
echo "$(grep -m 1 $tag $file | grep -o '[0-9]\+\(\.[0-9]\+\)*')"
}
BUGSNAG_API_KEY=$(plutil -p ios/RVLife/Info.plist | grep BugsnagAPIKey | awk '{ print $3 }' | sed 's/"//g')
if [[ -z $BUGSNAG_API_KEY ]]; then
echo "Couldn't find bugsnag API key"
exit 1
fi
IOS_VERSION=$(read_version MARKETING_VERSION ios/RVLife.xcodeproj/project.pbxproj)
echo "iOS app version: $IOS_VERSION"
IOS_VERSION_CODE=$(read_version CURRENT_PROJECT_VERSION ios/RVLife.xcodeproj/project.pbxproj)
echo "iOS version code: $IOS_VERSION_CODE"
if [[ -z $IOS_VERSION || -z $IOS_VERSION_CODE ]]; then
echo "Couldn't get iOS app versions"
exit 1
fi
bundle ios
upload ios "$IOS_VERSION" "$IOS_VERSION_CODE"
echo ""
ANDROID_VERSION=$(read_version versionName android/app/build.gradle)
echo "Android app version: $ANDROID_VERSION"
ANDROID_VERSION_CODE=$(read_version versionCode android/app/build.gradle)
echo "Android version code: $ANDROID_VERSION_CODE"
if [[ -z $ANDROID_VERSION || -z $ANDROID_VERSION_CODE ]]; then
echo "Couldn't get Android app versions"
exit 1
fi
bundle android
upload android "$ANDROID_VERSION" "$ANDROID_VERSION_CODE"
I'm assuming I messed up some syntax or permissions or something somewhere, but can't seem to find the area I did. Any help is appreciated.
Difference between bash -x myScript and set -x on command line
Command Line
omz_termsupport_preexec:1> [[ '' == true ]]
+omz_termsupport_preexec:3> emulate -L zsh
+omz_termsupport_preexec:4> setopt extended_glob
+omz_termsupport_preexec:7> local -a cmdargs
+omz_termsupport_preexec:8> cmdargs=( curl https://upload.bugsnag.com/react-native-source-map -F -F -F -F -F - )
+omz_termsupport_preexec:10> [[ curl = fg ]]
+omz_termsupport_preexec:44> local CMD=curl
+omz_termsupport_preexec:45> local LINE='curl https://upload.bugsnag.com/react-native-source-map -F -F -F -F -F -'
+omz_termsupport_preexec:47> title '$CMD' '%100>...>$LINE%<<'
+title:1> emulate -L zsh
+title:2> setopt prompt_subst
+title:4> [[ '' == *term* ]]
+title:8> : '%100>...>$LINE%<<'
+title:10> case xterm-256color (cygwin | xterm*)
+title:12> print -Pn '\e]2;%100\>...\>\$LINE%\<\<\a'
+title:13> print -Pn '\e]1;\$CMD\a'
+-zsh:75> curl https://upload.bugsnag.com/react-native-source-map -F 'apiKey=API-KEY' -F 'appVersion=10.6.7’ -F 'appBundleVersion=4515' -F 'dev=false' -F 'platform=ios' -F 'sourceMap=#ios-release.bundle.map' -F 'bundle=#ios-release.bundle'
OK%
SH Script
+ upload ios 10.6.7 4515
+ platform=ios
+ version=10.6.7
+ version_code=4515
+ curl --http1.1 https://upload.bugsnag.com/react-native-source-map -F apiKey=API-KEY -F appVersion=10.6.7 -F appVersionCode=4515 -F dev=false -F platform=ios -F sourceMap=#ios-release-bundle.map -F bundle=#ios-release.bundle
curl: (26) Failed to open/read local data from file/application
+ echo ''

How to rewrite grep in #!/bin/sh?

I have the following shell script, that does not compile:
#!/bin/sh
.....
.....
clean_up() {
echo "Clean up"
docker stop $KC_TEST_SVC
docker stop $KC_NAME
docker stop $POSTGRES_NAME
docker network rm $KC_NETWORK
}
# Check network if exists, if not then create
docker network inspect $KC_NETWORK
if [ $? -eq 1 ]; then
docker network create $KC_NETWORK
if [ $? -eq 1 ]; then
exit 1
fi
fi
docker run -d --rm --name $POSTGRES_NAME \
-e POSTGRES_DB=$POSTGRES_DB \
-e POSTGRES_USER=$POSTGRES_USER \
-e POSTGRES_PASSWORD=$POSTGRES_PW \
--network=$KC_NETWORK \
postgres:12.3
if [ $? -eq 1 ]; then
exit 1
fi
docker build --build-arg STAGE=int --tag $KC_TAG .
if [ $? -eq 1 ]; then
exit 1
fi
docker run -d --rm --name $KC_NAME \
-e DB_VENDOR=POSTGRES \
-e DB_ADDR=$POSTGRES_NAME \
-e DB_DATABASE=$POSTGRES_DB \
-e DB_USER=$POSTGRES_USER \
-e DB_PASSWORD=$POSTGRES_PW \
-e KEYCLOAK_USER=$KC_USER \
-e KEYCLOAK_PASSWORD=$KC_PW \
-e KEYCLOAK_LOGLEVEL=DEBUG \
--network=$KC_NETWORK \
$KC_TAG "-Dkeycloak.migration.action=import -Dkeycloak.migration.provider=dir -Dkeycloak.migration.dir=/opt/jboss/keycloak/import-dir -Dkeycloak.migration.strategy=OVERWRITE_EXISTING"
if [ $? -eq 1 ]; then
exit 1
fi
# Wait until Keycloak get started
sleep 30
# Run test jetty service that is protected with Keycloak
docker run -d --rm --name $KC_TEST_SVC \
-v $(pwd)/app:/app \
--network $KC_NETWORK \
hub.databaker.io/devops/jetty-keycloak:0.1.6
if [ $? -eq 1 ]; then
exit 1
fi
# Request Tokens for credentials
KC_URL=http://$KC_SERVER/$KC_CONTEXT/realms/$KC_REALM/protocol/openid-connect/token
echo "Keycloak URL => $KC_URL"
echo "Test the client connection to Keycloak with user $KC_TEST_USER"
KC_RESPONSE=$(
docker run --rm --network=$KC_NETWORK curlimages/curl:7.71.1 -X POST \
-d "username=$KC_TEST_USER" \
-d "password=$KC_TEST_PW" \
-d "grant_type=password" \
-d "client_id=$KC_TEST_CLIENT" \
$KC_URL | docker run --rm -i stedolan/jq .
)
echo "Response from Keycloak $KC_RESPONSE"
if grep -q "error" <<< "$KC_RESPONSE"; then
echo "++++++++Error+++++++++"
exit 1
fi
KC_ACCESS_TOKEN=$(echo "$KC_RESPONSE" | docker run --rm -i stedolan/jq -r .access_token)
echo "Access token from KC => $KC_ACCESS_TOKEN"
echo "Make request to protected service"
SVC_URL=http://$KC_TEST_SVC:8080/api/health
SVC_RES=$(docker run --rm --network=$KC_NETWORK curlimages/curl:7.71.1 -v -k \
-H "Authorization: Bearer $KC_ACCESS_TOKEN" \
-H "Accept: application/json" \
$SVC_URL | docker run --rm -i stedolan/jq .status)
clean_up
echo "$SVC_RES"
if [ "$SVC_RES" != "I am healthy" ]; then
echo "Test failed."
exit 1
else
echo "Test was successful. The service response $SVC_RES."
fi
it complains:
Syntax error: redirection unexpected
due to of the following line:
if grep -q "error" <<< "$KC_RESPONSE"; then
echo "++++++++Error+++++++++"
exit 1
fi
#!/bin/sh does not support grep. How to rewrite it?
<<< is a here string and it's a bash extension. In posix shell, just pipe the data.
if printf "%s\n" "$KC_RESPONSE" | grep -q "error"; then
Do not do if [ $? -eq 1 ]; then, it's error prone. Do if ! command; then. (You might want to research set -eu).
Change #!/bin/sh to #!/bin/bash (ev. replace bash location, which bash helps to find location of bash, and to see if bash is installed), or you may need to rewrite your script in a portable way. In this case, create a temporary file, put the string in such file, redirect such file to grep (with single <), and then delete the temporary file.
With such long description, you see why there is an extension in bash, on the other hand, it is nearly a syntactic sugar: you can easily do the same.

mailgun e-mail attachment cronjob

I have a bash script with the below content, It works fine when i run it directly, If i add this to cron job it fails in attachment.
#!/bin/bash
echo "Clearing the files"
rm -f ci/assets/*.csv
sleep 10
echo "Shot report..."
node ci/export-shot.js
echo "Not shot report..."
node ci/export-not-shot.js
echo "Retouching delivered report..."
node ci/export-retouching-delivered.js
echo "Sending email"
curl -s --user 'api:key-6gdgdfg852fgggy4893g-t5cjgfseerwefgfdgdf2183' \
https://api.mailgun.net/v3/test.co/messages \
-F from='Openly Report Bot <report-bot#test.co>' \
-F to='kirthan.b#test.co' \
-F subject="`date -v-1d +%F` : Shot/Not Shot/Delivered Articles" \
-F text='Daily report of shot, not shot and delivered articles' \
--form-string html='<html>HTML version of the body</html>' \
-F attachment=#ci/assets/shot.csv \
-F attachment=#ci/assets/not-shot.csv \
-F attachment=#ci/assets/delivered.csv \
Please help me on this.

CURL Sendmail via bash

Hi i've a problem whit this code:
# up here there's more code
echo "Password changed" $(date) > lez.txt
curl -n --ssl-reqd --mail-from "me#lupetto.sh" --mail-rcpt "my mail" -T lez.txt --url smtps://smtp.gmail.com:465 --user "example#gmail.com:password"
If I run the script I get only a empty mail, but if I do this manually I get my mail.
Thanks.
It looks like it's having problems to find the file contents. What about using a here-string to avoid writing a file at all? Change your code to:
curl -n --ssl-reqd --mail-from "me#lupetto.sh" --mail-rcpt "my mail" -T - --url smtps://smtp.gmail.com:465 --user "example#gmail.com:password" <<<"Password changed $(date)"
Notice the echo statement removal, the here-string at the end of the line and the -T - to get the file from stdin.
cURL expects that the file to be sent has a empty line on the 1st row
the script below will do that for you
##!/bin/sh
# USAGE
#
# sh emailfile.sh examplefile.log
file="$1"
echo "" > temp | cat "$1" >> temp && mv temp "$1"
wait
curl --url "smtps://smtp.mail.yahoo.com:465" \
--mail-from "youremail#yahoo.com" \
--mail-rcpt "dest#domain.com" \
--user "youremail#yahoo.com:EmailAccountPassword" \
-T - --silent < "$file"
wait

Resources