Curl Works in Command Line, but Not Shell Script - bash

I'm trying to automate a post-build process to get closer to a one-click release. However, I'm having problems with a script that.
Expected: The script to work exactly like the command line.
Results: Everything works, including grabbing the versionCodes except I am getting a curl: (26) Failed to open/read local data from file/application
The script I wanted to imitate with a more automated approach.
react-native bundle \
--platform android \
--dev false \
--entry-file index.js \
--bundle-output android-release.bundle \
--sourcemap-output android-release.bundle.map &&
curl https://upload.bugsnag.com/react-native-source-map \
-F apiKey=API-KEY \
-F appVersion=10.6.7 \
-F appVersionCode=4515 \
-F dev=false \
-F platform=android \
-F sourceMap=#android-release.bundle.map \
-F bundle=#android-release.bundle
react-native bundle \
--platform ios \
--dev false \
--entry-file index.js \
--bundle-output ios-release.bundle \
--sourcemap-output ios-release.bundle.map &&
curl https://upload.bugsnag.com/react-native-source-map \
-F apiKey=API-KEY \
-F appVersion=10.6.7 \
-F appBundleVersion=4515 \
-F dev=false \
-F platform=ios \
-F sourceMap=#ios-release.bundle.map \
-F bundle=#ios-release.bundle
So I wrote a script to fix this, which is
#!/bin/sh
# Ensure this is run on the commit used to generate the published app!
# Comment this line out to enable bundling and publishing
# debug=echo
bundle () {
platform=$1
$debug yarn react-native bundle \
--platform "$platform" \
--dev false \
--entry-file index.js \
--bundle-output "$platform-release.bundle" \
--sourcemap-output "$platform-release.bundle.map"
}
upload () {
platform=$1
version=$2
version_code=$3
# See https://docs.bugsnag.com/api/rn-source-map-upload/#uploading-source-maps
curl --http1.1 https://upload.bugsnag.com/react-native-source-map \
-F apiKey="$BUGSNAG_API_KEY" \
-F appVersion="$version" \
-F appVersionCode="$version_code" \
-F dev=false \
-F platform="$platform" \
-F sourceMap=#"$platform-release-bundle.map" \
-F bundle=#"$platform-release.bundle" \
}
read_version () {
local tag=$1
local file=$2
echo "$(grep -m 1 $tag $file | grep -o '[0-9]\+\(\.[0-9]\+\)*')"
}
BUGSNAG_API_KEY=$(plutil -p ios/RVLife/Info.plist | grep BugsnagAPIKey | awk '{ print $3 }' | sed 's/"//g')
if [[ -z $BUGSNAG_API_KEY ]]; then
echo "Couldn't find bugsnag API key"
exit 1
fi
IOS_VERSION=$(read_version MARKETING_VERSION ios/RVLife.xcodeproj/project.pbxproj)
echo "iOS app version: $IOS_VERSION"
IOS_VERSION_CODE=$(read_version CURRENT_PROJECT_VERSION ios/RVLife.xcodeproj/project.pbxproj)
echo "iOS version code: $IOS_VERSION_CODE"
if [[ -z $IOS_VERSION || -z $IOS_VERSION_CODE ]]; then
echo "Couldn't get iOS app versions"
exit 1
fi
bundle ios
upload ios "$IOS_VERSION" "$IOS_VERSION_CODE"
echo ""
ANDROID_VERSION=$(read_version versionName android/app/build.gradle)
echo "Android app version: $ANDROID_VERSION"
ANDROID_VERSION_CODE=$(read_version versionCode android/app/build.gradle)
echo "Android version code: $ANDROID_VERSION_CODE"
if [[ -z $ANDROID_VERSION || -z $ANDROID_VERSION_CODE ]]; then
echo "Couldn't get Android app versions"
exit 1
fi
bundle android
upload android "$ANDROID_VERSION" "$ANDROID_VERSION_CODE"
I'm assuming I messed up some syntax or permissions or something somewhere, but can't seem to find the area I did. Any help is appreciated.
Difference between bash -x myScript and set -x on command line
Command Line
omz_termsupport_preexec:1> [[ '' == true ]]
+omz_termsupport_preexec:3> emulate -L zsh
+omz_termsupport_preexec:4> setopt extended_glob
+omz_termsupport_preexec:7> local -a cmdargs
+omz_termsupport_preexec:8> cmdargs=( curl https://upload.bugsnag.com/react-native-source-map -F -F -F -F -F - )
+omz_termsupport_preexec:10> [[ curl = fg ]]
+omz_termsupport_preexec:44> local CMD=curl
+omz_termsupport_preexec:45> local LINE='curl https://upload.bugsnag.com/react-native-source-map -F -F -F -F -F -'
+omz_termsupport_preexec:47> title '$CMD' '%100>...>$LINE%<<'
+title:1> emulate -L zsh
+title:2> setopt prompt_subst
+title:4> [[ '' == *term* ]]
+title:8> : '%100>...>$LINE%<<'
+title:10> case xterm-256color (cygwin | xterm*)
+title:12> print -Pn '\e]2;%100\>...\>\$LINE%\<\<\a'
+title:13> print -Pn '\e]1;\$CMD\a'
+-zsh:75> curl https://upload.bugsnag.com/react-native-source-map -F 'apiKey=API-KEY' -F 'appVersion=10.6.7’ -F 'appBundleVersion=4515' -F 'dev=false' -F 'platform=ios' -F 'sourceMap=#ios-release.bundle.map' -F 'bundle=#ios-release.bundle'
OK%
SH Script
+ upload ios 10.6.7 4515
+ platform=ios
+ version=10.6.7
+ version_code=4515
+ curl --http1.1 https://upload.bugsnag.com/react-native-source-map -F apiKey=API-KEY -F appVersion=10.6.7 -F appVersionCode=4515 -F dev=false -F platform=ios -F sourceMap=#ios-release-bundle.map -F bundle=#ios-release.bundle
curl: (26) Failed to open/read local data from file/application
+ echo ''

Related

How to use bash variables in Jenkins multi-line shell script

I have unsuccessfully tried to use bash variables in Jenkins pipeline.
My first attempt
sh """#!/bin/bash
for file in *.map; do
filename=`basename $file .map`
echo "##### uploading ${$filename}"
curl -X POST ${SERVER_URL}/assets/v1/sourcemaps \
-F service_name="${SERVICE_NAME}" \
-F service_version="${revision}" \
-F bundle_filepath="${main_url}${filename}" \
-F sourcemap="#${filename}.map" &
done
wait
"""
Resulted in exception: MissingPropertyException: No such property: file
The second attempt, after seeing this answer https://stackoverflow.com/a/35047530/9590251
sh """#!/bin/bash
for file in *.map; do
filename=`basename \$file .map`
echo "##### uploading \$filename"
curl -X POST ${SERVER_URL}/assets/v1/sourcemaps \
-F service_name="${SERVICE_NAME}" \
-F service_version="${revision}" \
-F bundle_filepath="${main_url}\$filename" \
-F sourcemap="#\$filename.map" &
done
wait
"""
Simply omitted bash variables. So $filename was empty.
How do I need to property encode bash variables in this scenario?
Try this:
sh """#!/bin/bash
set -x
for file in *.map; do
filename="\$(basename "\$file" .map)"
echo "Uploading \$filename"
curl -X POST "${SERVER_URL}/assets/v1/sourcemaps" \
-F service_name="${SERVICE_NAME}" \
-F service_version="${revision}" \
-F bundle_filepath="${main_url}\$filename" \
-F sourcemap="#\${filename}.map" &
done
wait
"""

How to rewrite grep in #!/bin/sh?

I have the following shell script, that does not compile:
#!/bin/sh
.....
.....
clean_up() {
echo "Clean up"
docker stop $KC_TEST_SVC
docker stop $KC_NAME
docker stop $POSTGRES_NAME
docker network rm $KC_NETWORK
}
# Check network if exists, if not then create
docker network inspect $KC_NETWORK
if [ $? -eq 1 ]; then
docker network create $KC_NETWORK
if [ $? -eq 1 ]; then
exit 1
fi
fi
docker run -d --rm --name $POSTGRES_NAME \
-e POSTGRES_DB=$POSTGRES_DB \
-e POSTGRES_USER=$POSTGRES_USER \
-e POSTGRES_PASSWORD=$POSTGRES_PW \
--network=$KC_NETWORK \
postgres:12.3
if [ $? -eq 1 ]; then
exit 1
fi
docker build --build-arg STAGE=int --tag $KC_TAG .
if [ $? -eq 1 ]; then
exit 1
fi
docker run -d --rm --name $KC_NAME \
-e DB_VENDOR=POSTGRES \
-e DB_ADDR=$POSTGRES_NAME \
-e DB_DATABASE=$POSTGRES_DB \
-e DB_USER=$POSTGRES_USER \
-e DB_PASSWORD=$POSTGRES_PW \
-e KEYCLOAK_USER=$KC_USER \
-e KEYCLOAK_PASSWORD=$KC_PW \
-e KEYCLOAK_LOGLEVEL=DEBUG \
--network=$KC_NETWORK \
$KC_TAG "-Dkeycloak.migration.action=import -Dkeycloak.migration.provider=dir -Dkeycloak.migration.dir=/opt/jboss/keycloak/import-dir -Dkeycloak.migration.strategy=OVERWRITE_EXISTING"
if [ $? -eq 1 ]; then
exit 1
fi
# Wait until Keycloak get started
sleep 30
# Run test jetty service that is protected with Keycloak
docker run -d --rm --name $KC_TEST_SVC \
-v $(pwd)/app:/app \
--network $KC_NETWORK \
hub.databaker.io/devops/jetty-keycloak:0.1.6
if [ $? -eq 1 ]; then
exit 1
fi
# Request Tokens for credentials
KC_URL=http://$KC_SERVER/$KC_CONTEXT/realms/$KC_REALM/protocol/openid-connect/token
echo "Keycloak URL => $KC_URL"
echo "Test the client connection to Keycloak with user $KC_TEST_USER"
KC_RESPONSE=$(
docker run --rm --network=$KC_NETWORK curlimages/curl:7.71.1 -X POST \
-d "username=$KC_TEST_USER" \
-d "password=$KC_TEST_PW" \
-d "grant_type=password" \
-d "client_id=$KC_TEST_CLIENT" \
$KC_URL | docker run --rm -i stedolan/jq .
)
echo "Response from Keycloak $KC_RESPONSE"
if grep -q "error" <<< "$KC_RESPONSE"; then
echo "++++++++Error+++++++++"
exit 1
fi
KC_ACCESS_TOKEN=$(echo "$KC_RESPONSE" | docker run --rm -i stedolan/jq -r .access_token)
echo "Access token from KC => $KC_ACCESS_TOKEN"
echo "Make request to protected service"
SVC_URL=http://$KC_TEST_SVC:8080/api/health
SVC_RES=$(docker run --rm --network=$KC_NETWORK curlimages/curl:7.71.1 -v -k \
-H "Authorization: Bearer $KC_ACCESS_TOKEN" \
-H "Accept: application/json" \
$SVC_URL | docker run --rm -i stedolan/jq .status)
clean_up
echo "$SVC_RES"
if [ "$SVC_RES" != "I am healthy" ]; then
echo "Test failed."
exit 1
else
echo "Test was successful. The service response $SVC_RES."
fi
it complains:
Syntax error: redirection unexpected
due to of the following line:
if grep -q "error" <<< "$KC_RESPONSE"; then
echo "++++++++Error+++++++++"
exit 1
fi
#!/bin/sh does not support grep. How to rewrite it?
<<< is a here string and it's a bash extension. In posix shell, just pipe the data.
if printf "%s\n" "$KC_RESPONSE" | grep -q "error"; then
Do not do if [ $? -eq 1 ]; then, it's error prone. Do if ! command; then. (You might want to research set -eu).
Change #!/bin/sh to #!/bin/bash (ev. replace bash location, which bash helps to find location of bash, and to see if bash is installed), or you may need to rewrite your script in a portable way. In this case, create a temporary file, put the string in such file, redirect such file to grep (with single <), and then delete the temporary file.
With such long description, you see why there is an extension in bash, on the other hand, it is nearly a syntactic sugar: you can easily do the same.

mailgun e-mail attachment cronjob

I have a bash script with the below content, It works fine when i run it directly, If i add this to cron job it fails in attachment.
#!/bin/bash
echo "Clearing the files"
rm -f ci/assets/*.csv
sleep 10
echo "Shot report..."
node ci/export-shot.js
echo "Not shot report..."
node ci/export-not-shot.js
echo "Retouching delivered report..."
node ci/export-retouching-delivered.js
echo "Sending email"
curl -s --user 'api:key-6gdgdfg852fgggy4893g-t5cjgfseerwefgfdgdf2183' \
https://api.mailgun.net/v3/test.co/messages \
-F from='Openly Report Bot <report-bot#test.co>' \
-F to='kirthan.b#test.co' \
-F subject="`date -v-1d +%F` : Shot/Not Shot/Delivered Articles" \
-F text='Daily report of shot, not shot and delivered articles' \
--form-string html='<html>HTML version of the body</html>' \
-F attachment=#ci/assets/shot.csv \
-F attachment=#ci/assets/not-shot.csv \
-F attachment=#ci/assets/delivered.csv \
Please help me on this.

Is it possible to turn off syntax highlighting in Asciidoctor?

I've used Asciidoctor to create my JHipster Mini-Book. It looks great in PDF, MOBI, EPUB, and HTML. I've also created a (PDF) version that's printable. The printable version goes though Lulu, and gets printed in black and white.
The printable PDF has its code listings in color, which causes the code listings to be hard to read when printed, especially if they're light gray (e.g., comments). Is there a way to turn off syntax-highlighting in Asciidoctor?
An easy way would be to not to say which language the code has. Just replace
[source,ruby]
----
require 'sinatra'
----
with
[source]
----
require 'sinatra'
----
Set attribute source-highlighter to html-pipeline, e.g. on CLI when calling asciidoctor:
asciidoctor -a source-highlighter=html-pipeline FILE
I figured out the answer with the help of #mojavelinux (Dan Allen) on Twitter. The key is not to pass in the source-highlighter argument.
Here's my generate-pdf.sh script after making this change.
#!/bin/bash
# Usage: `./generate-pdf.sh` to generate a printable 6x9" PDF with no syntax highlighting
# `./generate-pdf.sh screen` to generate a downloadable 8.5x11" PDF
source $HOME/.rvm/scripts/rvm
rvm use 2.3.1 --quiet
if [ ! -d .bundle/gems ]; then
rm -f Gemfile.lock
bundle config --local github.https true
bundle --path=.bundle/gems --binstubs=.bundle/.bin
fi
if [ -f "$rvm_path/scripts/rvm" ] && [ -f ".ruby-version" ] && [ -f ".ruby-gemset" ]; then
source "$rvm_path/scripts/rvm"
rvm use `cat .ruby-version`#`cat .ruby-gemset`
fi
ASCIIDOCTOR_PDF="./.bundle/.bin/asciidoctor-pdf"
OPTIMIZE_PDF="`bundle exec gem contents --show-install-dir asciidoctor-pdf`/bin/optimize-pdf"
ROOT_DIR=$(realpath $(dirname $0))
MEDIA=prepress
HIGHLIGHTING=""
if [ ! -z "$1" ]; then
MEDIA=$1
HIGHLIGHTING="-a source-highlighter=coderay"
fi
BASE_DIR="$ROOT_DIR/src/docs/asciidoc"
OUT_DIR="$ROOT_DIR/build/asciidoc/pdf-$MEDIA"
$ASCIIDOCTOR_PDF --trace -B "$BASE_DIR" \
-D "$OUT_DIR" \
-S unsafe \
-r "$ROOT_DIR/src/main/ruby/asciidoctor-pdf-extensions.rb" \
-a media=$MEDIA \
-a pdfmarks \
-a pdf-style=infoq-$MEDIA \
-a pdf-stylesdir="$BASE_DIR/styles/pdf" \
-a pdf-fontsdir="$BASE_DIR/styles/pdf/fonts" \
-a sourcedir=../../../main/webapp \
$HIGHLIGHTING \
-a imagesdir=images \
-a toc \
-a icons=font \
-a idprefix \
-a idseparator=- \
-a projectdir=../../.. \
-a rootdir=../../.. \
-a project-name=jhipster-book \
-a project-version=2.0.0-SNAPSHOT \
-a attribute-missing=warn \
"$BASE_DIR/index.adoc"
$OPTIMIZE_PDF "$OUT_DIR/index.pdf"
mv -f "$OUT_DIR/index-optimized.pdf" "$OUT_DIR/index.pdf"

Test a version of a software in a Makefile

I am trying to tell the user if the version of virtualenv he is using is not up-to-date.
But it doesn't work yet. Any idea ?
virtualenv:
if [ ! -f $(PYTHON) ]; then \
version=`$(VIRTUALENV) --version`;\
check="1.8.2";\
winner=`echo -e "$${version}\n$${check}" | sed '/^$$/d' | sort -nr | head -1`;\
if [[ "$(winner)" = "$(version)" ]]; then \
$(VIRTUALENV) $(ROOT_DIR)/lib/virtualenv; \
else \
echo "Please upgrade your virtualenv>=1.8.2";\
exit 1;\
fi \
fi
Ok it was a little Makefile trick with $
virtualenv:
if [ ! -f $(PYTHON) ]; then \
version=`$(VIRTUALENV) --version`;\
check="1.8.2";\
winner=`echo -e "$${version}\n$${check}" | sed '/^$$/d' | sort -nr | head -1`;\
if [[ "$${winner}" = "$${version}" ]]; then \
$(VIRTUALENV) $(ROOT_DIR)/lib/virtualenv; \
else \
echo "Please upgrade your $${version} version of virtualenv to >= 1.8.2";\
echo " $ sudo pip install -U distribute pip virtualenv";\
exit 1;\
fi \
fi

Resources