Problems with ruby-appium-cucumber on AWS Device Farm - ruby

Question

I'm trying to run my test suite on AWS device farm using a custom environment. The test suite runs fine locally but when I run it on device farm some tests fail randomly, and some other acts as expected. Sometimes it looks like it's skipping the cucumber hooks, or just not running the steps.
Here is my custom environment configuration:
# Phases are collection of commands that get executed on Device Farm.
phases:
# The install phase includes commands that install dependencies that your tests use.
# Default dependencies for testing frameworks supported on Device Farm are already installed.
install:
commands:
# By default the ruby version installed is 2.5.1
- mkdir /tmp/tempdir
- export TMPDIR="/tmp/tempdir"
- export TMP="/tmp/tempdir"
- export TEMP="/tmp/tempdir"
- rvm install "ruby-2.6.5"
- rvm use 2.6.5
# you can switch to an alternate ruby version using below command.
#- rvm install 2.3.1 --autolibs=0
# Unpackage and install the gems
- echo "Navigate to test package directory"
- cd $DEVICEFARM_TEST_PACKAGE_PATH
# Use a pre-configured ruby environment to run your tests.
# This environment has the following gems pre-installed (appium_lib (9.16.1), test-unit (3.2.9)) along with their dependencies.
# If you are using this env, please make sure you do not upload the Gemfile.lock while packaging your tests.
# If the Gemfile.lock contains different versions for the already installed packages, it will ignore the pre-installed packages.
# Using this env can help you speed up your test set up phase as you wont have to install all the gems.
# This default env is only available for ruby 2.5.3.
- rvm gemset use default-ruby-gemset-env-version-1 --create
# Alternatively, you can create a new virtual ruby env using the command:
#- rvm gemset use env --create
# Install the gems from the local vendor/cache directory
- gem install bundler --no-document
- bundle config set path 'vendor/cache'
- gem update --system
- bundle install
# This test execution environment uses Appium version 1.9.1 by default, however we enable you to change it using the Appium version manager (avm). An
# example "avm" command below changes the version to 1.14.2.
# For your convenience, we have preinstalled the following versions: 1.9.1, 1.10.1, 1.11.1, 1.12.1, 1.13.0, 1.14.1, 1.14.2, 1.15.1 or 1.16.0.
# To use one of these Appium versions, change the version number in the "avm" command below to your desired version:
- ln -s /usr/local/avm/versions/1.9.1/node_modules/.bin/appium /usr/local/avm/versions/1.9.1/node_modules/appium/bin/appium.js
# The pre-test phase includes commands that setup your test environment.
pre_test:
commands:
# We recommend starting appium server process in the background using the command below.
# Appium server log will go to $DEVICEFARM_LOG_DIR directory.
# The environment variables below will be auto-populated during run time.
- echo "Start appium server"
- >-
appium --log-timestamp
--default-capabilities "{\"deviceName\": \"$DEVICEFARM_DEVICE_NAME\", \"platformName\":\"$DEVICEFARM_DEVICE_PLATFORM_NAME\",
\"app\":\"$DEVICEFARM_APP_PATH\", \"udid\":\"$DEVICEFARM_DEVICE_UDID\", \"platformVersion\":\"$DEVICEFARM_DEVICE_OS_VERSION\",
\"chromedriverExecutable\":\"$DEVICEFARM_CHROMEDRIVER_EXECUTABLE\"}"
>> $DEVICEFARM_LOG_DIR/appiumlog.txt 2>&1 &
- >-
start_appium_timeout=0;
while [ true ];
do
if [ $start_appium_timeout -gt 60 ];
then
echo "appium server never started in 60 seconds. Exiting";
exit 1;
fi;
grep -i "Appium REST http interface listener started on 0.0.0.0:4723" $DEVICEFARM_LOG_DIR/appiumlog.txt >> /dev/null 2>&1;
if [ $? -eq 0 ];
then
echo "Appium REST http interface listener started on 0.0.0.0:4723";
break;
else
echo "Waiting for appium server to start. Sleeping for 1 second";
sleep 1;
start_appium_timeout=$((start_appium_timeout+1));
fi;
done;
# The test phase includes commands that start your test suite execution.
test:
commands:
# Your test package is downloaded in $DEVICEFARM_TEST_PACKAGE_PATH so we first change directory to that path.
- echo "Navigate to test source code"
- cd $DEVICEFARM_TEST_PACKAGE_PATH
- echo "Start Appium Ruby test"
# Modify/Enter the command below to start the tests. The comamnd should be similar to what you use to run the tests locally.
# "bundle exec" is a Bundle command to execute a script in the context of the current bundle.
# For e.g. assuming you run your tests locally using command "ruby YOUR_TEST_FILENAME.rb.", to run your ruby tests using bundle exec command you can use:
- bundle exec rake set_environment[amazon]
- bundle exec rake test
# The post test phase includes are commands that are run after your tests are executed.
post_test:
commands:
# The artifacts phase lets you specify the location where your tests logs, device logs will be stored.
# And also let you specify the location of your test logs and artifacts which you want to be collected by Device Farm.
# These logs and artifacts will be available through ListArtifacts API in Device Farm.
artifacts:
# By default, Device Farm will collect your artifacts from following directories
- $DEVICEFARM_LOG_DIR
Here are the cucumber logs(I have tried to use backtrace to get more info but for some reason is not working in AWS)
Start Appium Ruby test
[DeviceFarm] bundle exec rake set_environment[amazon]
[DeviceFarm] bundle exec rake test
Using the test, no_bugs and pretty_progress profiles...
F------
Failing Scenarios:
cucumber -p test -p no_bugs -p pretty_progress features/regression/android/games/lightning.feature:45 # Scenario: Top up modal appears for a user without funds on lightning flow purchase attempts on all or nothing
1 scenario (1 failed)
6 steps (6 skipped)
1m1.742s
Cucumbers failed

Answer 1

I discovered that my capabilities for appium weren't correct and they lacked some capabilities, without them appium and cucumber acts up weirdly, here are the capabilities I used:
caps = {
"automationName": "UiAutomator2",
"platformName": ENV['DEVICEFARM_DEVICE_PLATFORM_NAME'],
"deviceName": ENV['DEVICEFARM_DEVICE_NAME'],
"app": ENV['DEVICEFARM_APP_PATH'],
"uuid": ENV['DEVICEFARM_DEVICE_UDID'],
"appPackage": "appPackageOfYourChoice",
"appActivity": "appActivityOfYourChoice",
"autoGrantPermissions": true
}