I want to automate installation of Redocly (API documentation generator) via the command line. A Redocly project is generated with npx (npx create-openapi-repo). The install process triggers four prompts (shown in quotes; followed by the answers I need to input). Some have processing in between.
"Do you already have an OpenAPI/Swagger 3.0 definition for your API? (y/N)" y
"Please specify the path to the OpenAPI definition (local file):" test.yml
"API Name:" My API
"The following folders will be created: openapi and docs. You can change them by running create-openapi-repo Proceed? (Y/n)" Y
The problem is that commands that seem to enable automatic completion for script files are not enabling it with the Redocly npx install, at least through my attempts.
Failed attempts, among many others, include the following (one in Windows Command Prompt and the rest in Bash). Some include answers to only three of the four prompts because I rarely got past the second one.
Attempt
cmd.cmd:
npx create-openapi-repo
Command prompt (Windows Command Prompt):
C:\Source\api-docs>(echo y && echo test.yml) | cmd.cmd
Command prompt results:
C:\Source\api-docs>npx create-openapi-repo
Welcome to the OpenAPI-Repo generator!
? Do you already have an OpenAPI/Swagger 3.0 definition for your API? Yes
test.yml
? Please specify the path to the OpenAPI definition (local file):
<ends>
Attempt
cmd.sh:
read -n 1 -p "Do you already have an OpenAPI/Swagger 3.0 definition for your API? (y/N)" Y
read -n 1 -p "Please specify the path to the OpenAPI definition (local file):" test.yml
read -n 1 -p "API Name:" My API
Command prompt (Bash):
$ ./cmd.sh
Command prompt results:
Welcome to the OpenAPI-Repo generator!
? Do you already have an OpenAPI/Swagger 3.0 definition for your API? Yes
? Please specify the path to the OpenAPI definition (local file):
Attempt
Command prompt (Bash)
printf 'y\test.yml\My API' | npx create-openapi-repo
Results same as above.
Attempt
printf '%s\n' y test.yml 'My API' | npx create-openapi-repo
Results same as above.
Attempt
cmd.txt:
y
test.yml
My API
Command prompt (Bash):
npx create-openapi-repo < cmd.txt
Results same as above.
Is anyone aware of how to accomplish this? If it matters, the goal is to execute deployment of Redocly in an Azure DevOps pipeline. The pipeline offers the ability to run commands in Bash, Windows Command Prompt, or PowerShell.
First option is to use expect as suggested by Shawn.
If you are sure of the time between two prompts (for example, less than 10 seconds), you can try things like :
#!/usr/bin/env bash
{
echo y
sleep 10
echo test.yml
sleep 10
echo My API
} | npx create-openapi-repo
Related
I use Jenkins to trigger a deployment of multiple services (using a deployment script)
There are 6 services in total and have used Jenkins Boolean Parameter to select which services to be deployed.
So, if 1st, 4th and 5th services are to be deployed, the input to the deployment script looks like below in the Jenkins Execute shell tab.
#!/bin/bash
sshpass -p <> ssh username#host "copy the deployment script to the deployment VM/server and give execute persmission...."
sshpass -p <> ssh username#host "./mydeploy.sh ${version_to_be_deployed} ${1st_service} ${4th_service} ${5th_service}"
Note: Deployment happens in a differnt server with very restricted access, so the deployment script - mydeploy.sh has to be copied from the Jenkins slave to the deployment server, then executed with the respective arguments.
How can I make this setup more robust and elegant. I dont want to pass 6 arguments, if all the 6 services are selected. What's the better way to do it ?
An array would help here.
#!/bin/bash
#hardcoded for demo purposes, but you can build dynamically from arguments
services_to_deploy=( 1 4 5 )
sshpass -p <> ssh username#host "copy the deployment script to the deployment VM/server and give execute persmission...."
sshpass -p <> ssh username#host "./mydeploy.sh ${version_to_be_deployed} ${services_to_deploy[#]}"
${services_to_deploy[#]} will expand to the list of all the services you want to deploy so that you don't have to set a unique variable for each one.
One caveat though is that running a command over ssh is similar to running a command using eval because the remote shell will reparse whatever comes through before executing it. If your services have simple names this might not matter, but if you had a hypothetical Hello World service then the remote script would treat Hello and World as two separate arguments due to Word Splitting which probably isn't what you want.
If this is a problem for you, you could fix this with either printf %q (supported by most Bash shells) or by expanding the array as "${services_to_deploy[#]#Q}" if you have Bash 4.4 or higher.
An example using printf %q might look like:
#!/bin/bash
services_to_deploy=( 1 4 5 )
remote_arguments=()
for s in "${services_to_deploy[#]}" ; do
remote_arguments+=( "$( printf '%q' "${s}" )" )
done
sshpass -p <> ssh username#host "copy the deployment script to the deployment VM/server and give execute persmission...."
sshpass -p <> ssh username#host "./mydeploy.sh ${version_to_be_deployed} ${remote_arguments[#]}"
How about you improve your script and introduce some flags.
# --all : Deploys all services
./mydeploy.sh --version 1.0 --all
# --exclude : Deploys all services other than 5th_service and 4th_service (Excludes 4th and 5th)
./mydeploy.sh --version 1.0 --exclude ${5th_service} ${4th_service}
# --include : Deploys just 4th_service and 5th_service
./mydeploy.sh --version 1.0 --include ${5th_service} ${4th_service}
I am writing an automation script to download the aws cloud-hsm client and pcks for doing a aws-cloudhsm-client init-container for a vault enterprise deployment.
The goal is to automate the config and setup of the HSM integration for vault to reference.
This is a guide that details how to do it.
https://github.com/jacobmammoliti/aws-vault-cloudhsm
My issue is that the cloud-hsm cli provided with the cloud-hsm client doesn't have a auto yes feature for when you execute the change password command. In order to automate this I have it in a EOF block for inline script execution to use the cloud-hsm cli inside of my start up script to configure it.
The issue is I'm trying to use yes | to answer the prompt but I don't think the EOF inline script method supports that and I am trying to find another way around it because the cloud-hsm cli doesn't support it which is kinda silly.
Here is a test bash script I'm running from a ubuntu:18.04 shell inside of my Kubernetes cluster to workout the automation. The HSM is on a private network so I'm using a pod to be inside the HSM network.
apt update -y
apt-get install wget -y
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh -ab
export PATH=~/miniconda3/bin:${PATH}
python --version
wget https://s3.amazonaws.com/cloudhsmv2-software/CloudHsmClient/Bionic/cloudhsm-client_latest_u18.04_amd64.deb
wget https://s3.amazonaws.com/cloudhsmv2-software/CloudHsmClient/Bionic/cloudhsm-client-pkcs11_latest_u18.04_amd64.deb
apt install -y ./cloudhsm-client_latest_u18.04_amd64.deb
apt install -y ./cloudhsm-client-pkcs11_latest_u18.04_amd64.deb
export CLOUD_HSM_IP='HSM_IP'
export CUSTOMER_CA="base64encodedca"
export VAULT_DEFAULT_ADMIN_PASSWORD='password'
export VAULT_HSM_ADMIN_PASSWORD='myadminpassword'
export VAULT_USER='vault'
export VAULT_HSM_PASSWORD='myadminpassword'
echo "Configure Cloud HSM $CLOUD_HSM_IP"
/opt/cloudhsm/bin/configure -a ${CLOUD_HSM_IP}
echo "Config File"
cat /opt/cloudhsm/etc/cloudhsm_mgmt_util.cfg
echo "Echo Customer CA"
echo "${CUSTOMER_CA}" | base64 --decode > /opt/cloudhsm/etc/customerCA.crt
echo "Cat Customer CA"
cat /opt/cloudhsm/etc/customerCA.crt
echo "Execute cloudhsm cli"
echo "
yes | /opt/cloudhsm/bin/cloudhsm_mgmt_util /opt/cloudhsm/etc/cloudhsm_mgmt_util.cfg <<'EOF'
enable_e2e
loginHSM PRECO admin ${VAULT_DEFAULT_ADMIN_PASSWORD}
changePswd PRECO admin ${VAULT_HSM_ADMIN_PASSWORD}
logoutHSM
loginHSM CO admin ${VAULT_HSM_ADMIN_PASSWORD}
createUser CU ${VAULT_USER} ${VAULT_HSM_PASSWORD}
logoutHSM
quit
EOF
" > configure_hsm
cat configure_hsm
yes | bash configure_hsm
service cloudhsm-client start
Question:
How can I get around this issue because yes | doesn't work because cloud-hsm is its own cli?
Regardless of which shell you are using, a subprocess can only receive standard input from one place.
Anything which looks like
something | binary <<eof
...
eof
is basically an ambiguous redirect. Should binary receive standard input from the pipe, or from the here document? It can't be reading both.
(Quick testing on Bash and Alpine Linux indicates that these shells prefer the here document in this situation, and ignore the pipe. I'm too lazy to check if this is defined somewhere like POSIX.)
From a brief read of the documentation I'm guessing you want to respond y to the "are you sure?" prompts after several of these commands. Just add that in the here document just like all the other input.
/opt/cloudhsm/bin/cloudhsm_mgmt_util /opt/cloudhsm/etc/cloudhsm_mgmt_util.cfg <<'EOF'
enable_e2e
loginHSM PRECO admin ${VAULT_DEFAULT_ADMIN_PASSWORD}
changePswd PRECO admin ${VAULT_HSM_ADMIN_PASSWORD}
y
logoutHSM
loginHSM CO admin ${VAULT_HSM_ADMIN_PASSWORD}
createUser CU ${VAULT_USER} ${VAULT_HSM_PASSWORD}
y
logoutHSM
quit
EOF
For what it's worth, quoting the 'EOF' marker will prevent the shell from expanding the variables in the here document; I'm guessing probably you would not want that?
I have set up the google assistant sdk on my Raspberry Pi as shown here: https://developers.google.com/assistant/sdk/prototype/getting-started-pi-python/run-sample
Now in order to re-run the assistant I have worked out the two commands are
$ source env/bin/activate
and
(env) $ google-assistant-demo
however I want to automate this process into a script that I can call from rc.local (followed by an &) in order to make the assistant boot from start up.
However if I run a simple script
#!/bin/bash
source env/bin/activate
google-assistant-demo
the assistant does not run inside the environment
my environment path is /home/pi/env/bin/activate
How can I have it so the script starts the environment and then runs the assistant inside the virtual environment?
EDIT: In the end I went with the following method:
using this as a base :
https://youtu.be/ohUszBxuQA4?t=774 – thanks to Eric Parisot
You will need to download the src file he uses and extract its contents into /home/pi/src/
However with a few changes.
I did not run gassist.sh as sudo, as it gave me the following error:
OpenAlsaHandle PcmOpen: No such file or directory
[7689:7702:ERROR:audio_input_processor.cc(756)] Input error
ON_MUTED_CHANGED:
{‘is_muted’: False}
ON_START_FINISHED
ON_ASSISTANT_ERROR:
{‘is_fatal’: True}
[7689:7704:ERROR:audio_input_processor.cc(756)] Input error
ON_ASSISTANT_ERROR:
{‘is_fatal’: True}
Fix: DO NOT run as sudo
If gassist.sh gives an error about RPi.GPIO you need to do https://youtu.be/ohUszBxuQA4?t=580:
$ cd /home/pi/env/bin
$ source activate
(env) $ pip install RPi.GPIO
(env) $ deactivate
And then I did sudo nano /etc/profile
and the appended this to the end:
#Harvs was here on 24/06/17
if pidof -x "gassist.sh" >/dev/null; then
echo ""
echo "/etc/profile says:"
echo "An instance of Google Assistant is already running, will not start again"
echo ""
else
echo "Starting Google Assistant..."
echo "If you are seeing this, perhaps you have SSH within seconds of reboot"
/home/pi/src/gassist.sh &
fi
And now it works perfectly, and inside the virtual enviroment :)
found solution here :https://raspberrypi.stackexchange.com/a/45089
Create a startup shell script in your root directory (I named mine "launch"), make it executable too :
sudo nano launch.sh
I wrote it that way :
#!/bin/bash
source /home/pi/env/bin/activate
/home/pi/env/bin/google-assistant-demo
Save the file
Edit the LXDE-pi autostart file
sudo nano /home/pi/.config/lxsession/LXDE-pi/autostart
Add this to the bottom of that file
./launch.sh
reboot
Scripts run from rc.local execute in the root directory (or possibly in the home directory of the root user, depending on the distro, I think?)
The easy fix is to code the full path to the environment.
#!/bin/bash
source /home/pi/env/bin/activate
google-assistant-demo
# or maybe /home/pi/google-assistant-demo
There is no need to explicitly background anything in rc.local
In the end I went with the following method:
using this as a base : https://youtu.be/ohUszBxuQA4?t=774 – thanks to Eric Parisot
However with a few changes.
You will need to download the src file he uses and extract its contents into /home/pi/src/
I did not run gassist.sh as sudo, as it gave me the following error:
OpenAlsaHandle PcmOpen: No such file or directory
[7689:7702:ERROR:audio_input_processor.cc(756)] Input error
ON_MUTED_CHANGED:
{‘is_muted’: False}
ON_START_FINISHED
ON_ASSISTANT_ERROR:
{‘is_fatal’: True}
[7689:7704:ERROR:audio_input_processor.cc(756)] Input error
ON_ASSISTANT_ERROR:
{‘is_fatal’: True}
Fix: DO NOT run as sudo
If gassist.sh gives an error about RPi.GPIO you need to do https://youtu.be/ohUszBxuQA4?t=580:
$ cd /home/pi/env/bin
$ source activate
(env) $ pip install RPi.GPIO
(env) $ deactivate
And then I did sudo nano /etc/profile and the appended this to the end:
#Harvs was here on 24/06/17
if pidof -x "gassist.sh" >/dev/null; then
echo ""
echo "/etc/profile says:"
echo "An instance of Google Assistant is already running, will not start again"
echo ""
else
echo "Starting Google Assistant..."
echo "If you are seeing this, perhaps you have SSH within seconds of reboot"
/home/pi/src/gassist.sh &
fi
And now it works perfectly, and inside the virtual enviroment, and in boot to CLI mode! :)
We would like Teamcity to build our solutions on every commit into subversion.
Following the documentation, we are to create a .sh script :-
SERVER=https://buildserver-url
USER=buildserver-user
PASS="<password>"
LOCATOR=$1
# The following is one-line:
(sleep 10; curl --user $USER:$PASS -X POST "$SERVER/app/rest/vcs-root-instances/commitHookNotification?locator=$LOCATOR" -o /dev/null) >/dev/null 2>&1 <&1 &
exit 0
Subversion is running on a windows environment, and so the .sh file will fail.
We are trying to convert this into a .bat file of which we have :-
set SERVER=https://buildserver-url
set USER=buildserver
set PASS=password
LOCATOR=%1%
(timeout 10; curl --user %USER%:%PASS% -X POST "%SERVER%/app/rest/vcs-root-instances/commitHookNotification?locator=%LOCATOR%" -o /dev/null) >/dev/null 2>%1% <%1% &
exit 0
However, this is still failing when trying to execute with
"The system cannot find the path specified"
It seems that perhaps we havnt converted this correctly?
Are the programs you're referencing (such as curl and timeout.exe) in locations that are present in the $PATH/%PATH% variable? How about any other files you're referencing - are you specifying full paths
Side note: Did you install curl and timeout.exe on the Windows server?
Also, /dev/null does not exist on Windows; you need to redirect to NUL. You can't just change the file extension and some of your syntax and expect a bash script to work on Windows.
Were I in your shoes, I'd skip batch altogether and write the script in something modern and sane like Powershell.
Did anyone succeed in setting up automated UIAutomation tests in Xcode?
I'm trying to set up a target in my Xcode project that should run all the UIAutomation scripts I prepared. Currently, the only Build Phase of this target is this Run Script block:
TEMPLATE="/Applications/Xcode.app/Contents/Applications/Instruments.app/Contents/PlugIns/AutomationInstrument.bundle/Contents/Resources/Automation.tracetemplate"
MY_APP="/Users/Me/Library/Application Support/iPhone Simulator/6.0/Applications/564ED15A-A435-422B-82C4-5AE7DBBC27DD/MyApp.app"
RESULTS="/Users/Me/Projects/MyApp/Tests/UI/Traces/Automation.trace"
SCRIPT="/Users/Me/Projects/MyApp/Tests/UI/SomeTest.js"
instruments -t $TEMPLATE $MY_APP -e UIASCRIPT $SCRIPT -e UIARESULTSPATH $RESULTS
When I build this target it succeeds after a few seconds, but the script didn't actually run. In the build log I get these errors:
instruments[7222:707] Failed to load Mobile Device Locator plugin
instruments[7222:707] Failed to load Simulator Local Device Locator plugin
instruments[7222:707] Automation Instrument ran into an exception while trying to run the script. UIATargetHasGoneAWOLException
+0000 Fail: An error occurred while trying to run the script.
Instruments Trace Complete (Duration : 1.077379s; Output : /Users/Me/Projects/MyApp/Tests/UI/Traces/Automation.trace)
I am pretty sure, that my javascript and my run script are both correct, because if I run the exact same instruments command in bash it works as expected.
Could this be a bug in Xcode?
I finally found a solution for this problem. It seems like Xcode is running the Run Scripts with limited rights. I'm not entirely sure, what causes the instruments command to fail, but using su to change to your user will fix it.
su $USER -l -c <instruments command>
Obviously, this will ask you for your password, but you can't enter it when running as a script. I didn't find a way to specify the password for su, however if you run it as root, you don't have to specify one. Luckily sudo can accept a password via the pipe:
echo <password> | sudo -S su $USER -l -c <instruments command>
If you don't want to hardcode your password (always a bad idea), you could use some AppleScript to ask for the password.
I posted the resulting script below. Copy that to a *.sh file in your project and run that script from a Run Script.
#!/bin/bash
# This script should run all (currently only one) tests, independently from
# where it is called from (terminal, or Xcode Run Script).
# REQUIREMENTS: This script has to be located in the same folder as all the
# UIAutomation tests. Additionally, a *.tracetemplate file has to be present
# in the same folder. This can be created with Instruments (Save as template...)
# The following variables have to be configured:
EXECUTABLE="TestApp.app"
# Optional. If not set, you will be prompted for the password.
#PASSWORD="password"
# Find the test folder (this script has to be located in the same folder).
ROOT="$( cd -P "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
# Prepare all the required args for instruments.
TEMPLATE=`find $ROOT -name '*.tracetemplate'`
EXECUTABLE=`find ~/Library/Application\ Support/iPhone\ Simulator | grep "${EXECUTABLE}$"`
SCRIPTS=`find $ROOT -name '*.js'`
# Prepare traces folder
TRACES="${ROOT}/Traces/`date +%Y-%m-%d_%H-%M-%S`"
mkdir -p "$TRACES"
# Get the name of the user we should use to run Instruments.
# Currently this is done, by getting the owner of the folder containing this script.
USERNAME=`ls -l "${ROOT}/.." | grep \`basename "$ROOT"\` | awk '{print $3}'`
# Bring simulator window to front. Depending on the localization, the name is different.
osascript -e 'try
tell application "iOS Simulator" to activate
on error
tell application "iOS-Simulator" to activate
end try'
# Prepare an Apple Script that promts for the password.
PASS_SCRIPT="tell application \"System Events\"
activate
display dialog \"Password for user $USER:\" default answer \"\" with hidden answer
text returned of the result
end tell"
# If the password is not set directly in this script, show the password prompt window.
if [ -z "$PASSWORD" ]; then
PASSWORD=`osascript -e "$PASS_SCRIPT"`
fi
# Run all the tests.
for SCRIPT in $SCRIPTS; do
echo -e "\nRunning test script $SCRIPT"
COMMAND="instruments -t \"$TEMPLATE\" \"$EXECUTABLE\" -e UIASCRIPT \"$SCRIPT\""
COMMAND="echo '$PASSWORD' | sudo -S su $USER -l -c '$COMMAND'"
echo "$COMMAND"
eval $COMMAND > results.log
SCRIPTNAME=`basename "$SCRIPT"`
TRACENAME=`echo "$SCRIPTNAME" | sed 's_\.js$_.trace_g'`
mv *.trace "${TRACES}/${TRACENAME}"
if [ `grep " Fail: " results.log | wc -l` -gt 0 ]; then
echo "Test ${SCRIPTNAME} failed. See trace for details."
open "${TRACES}/${TRACENAME}"
exit 1
break
fi
done
rm results.log
It seems as though this really might be an Xcode problem; at any rate, at least one person has filed a Radar report on it. Someone in this other thread claims you can work around this exception by disconnecting any iDevices that are currently connected to the computer, but I suspect that does not apply when you're trying to run the script as an Xcode target.
I would suggest filing a Radar report as well; you may get further details on the issue from Apple, or at least convince them that many people are having the problem and they ought to figure out what's going on.
Sorry for a not-terribly-helpful answer (should have been a comment, but comments and links/formatting do not mix very well). Please update this question with anything you find out on the issue.
Note: this is not a direct answer to the question, but it is an alternative solution to the underlying problem.
While searching for in-depth information about UIAutomation, I stumbled across a framework by Square called KIF (Keep it functional). It is a integration testing framework that allows for many of the same features as UIAutomation, but the great thing about is is that you can just write your integration tests in Objective-C.
It is very easy to setup (via CocoaPods), they have good examples too, and the best thing is that it's a breeze to set up with your CI system like Jenkins.
Have a look at: http://github.com/square/KIF
Late to the game but I have a solution that works for Xcode 5.1. Don't know if that's what broke the above solution or not. With the old solution I was still getting:
Failed to load Mobile Device Locator plugin, etc.
However, this works for the release version of Xcode 5.1.
echo <password> | sudo -S -u username xcrun instruments
Notice I removed the unneeded su command and added the xcrun command. The xcrun was the magic that was needed.
Here is my complete command:
echo <password> | sudo -S -u username xcrun instruments\
-w "iPhone Retina (3.5-inch) - Simulator - iOS 7.1"\
-D "${PROJECT_DIR}/TestResults/Traces/Traces.trace"\
-t "${DEVELOPER_DIR}/Instruments.app/Contents/PlugIns/AutomationInstrument.bundle/Contents/Resources/Automation.tracetemplate"\
"${BUILT_PRODUCTS_DIR}/MyApp.app"\
-e UIARESULTSPATH "${PROJECT_DIR}/TestResults"\
-e UIASCRIPT "${PROJECT_DIR}/UITests/main.js"
By the way if you type:
instruments -s devices
you will get a list of all the supported devices you can use for the -w option.
Edit: To make this work for different people checking out the project replace the following:
echo <password> | sudo -S -u username xcrun instruments
with
sudo -u ${USER} xcrun instruments
Since you are just doing an sudo to the same user no password is required.
Take a look at this tutorial that explains how to have Automated UI testing with Jenkins. It also uses Jasmine in the tutorial though. http://shaune.com.au/automated-ui-testing-for-ios-apps-uiautomation-jasmine-jenkins/ hope this helps. It has an example project file so you can download that as a template. Hope this helps.
In XCode - if you load up organizer (XCode->Window->Organizer)
Then select your machine under devices -> 'Enable Developer Mode'
This should remove the need for prompts with instruments.