I'm deploying a node application in aws using aws codepipeline.
appspec.yaml
version: 0.0
os: linux
files:
- source: /
destination: /home/ubuntu/my-app
overwrite: true
hooks:
BeforeInstall:
- location: scripts/before_install.sh
timeout: 300
runas: ubuntu
AfterInstall:
- location: scripts/after_install.sh
timeout: 300
runas: ubuntu
ApplicationStart:
- location: scripts/application_start.sh
timeout: 300
runas: ubuntu
before_install.sh
#!/bin/bash
sudo apt-get update
curl -sL https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.0/install.sh -o install_nvm.sh
bash install_nvm.sh
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" # This loads nvm bash_completion
nvm install --lts
npm install --location=global yarn
yarn global add pm2
So node, npm, yarn and pm2 is installed. Since yarn is used to intall pm2 if this runs then yarn should be successfully installed.
after_install.sh
#!/bin/bash
cd /home/ubuntu/my-app
yarn install
yarn run build
After running I get following error
after_install.sh: line 7: yarn: command not found
after_install.sh: line 8: yarn: command not found
When look at build error it seems like yarn only not available in after_installation hook.
2022-06-20 18:21:49 [stdout]Installing latest LTS version.
2022-06-20 18:21:49 [stderr]v16.15.1 is already installed.
2022-06-20 18:21:50 [stdout]Now using node v16.15.1 (npm v8.11.0)
2022-06-20 18:21:51 [stdout]
2022-06-20 18:21:51 [stdout]changed 1 package, and audited 2 packages in 489ms
2022-06-20 18:21:51 [stdout]
2022-06-20 18:21:51 [stdout]found 0 vulnerabilities
2022-06-20 18:21:51 [stdout]yarn global v1.22.19
2022-06-20 18:21:51 [stdout][1/4] Resolving packages...
2022-06-20 18:21:51 [stdout][2/4] Fetching packages...
2022-06-20 18:21:51 [stdout][3/4] Linking dependencies...
2022-06-20 18:21:52 [stdout][4/4] Building fresh packages...
2022-06-20 18:21:52 [stdout]success Installed "pm2#5.2.0" with binaries:
2022-06-20 18:21:52 [stdout] - pm2
2022-06-20 18:21:52 [stdout] - pm2-dev
2022-06-20 18:21:52 [stdout] - pm2-docker
2022-06-20 18:21:52 [stdout] - pm2-runtime
2022-06-20 18:21:52 [stdout]Done in 1.27s.
2022-06-20 18:21:54 LifecycleEvent - AfterInstall
2022-06-20 18:21:54 Script - scripts/after_install.sh
2022-06-20 18:21:54 [stderr]/opt/codedeploy-agent/deployment-root/faee375f-f793-4b89-b57e-78c445c6c02b/d-WCHP7OWQH/deployment-archive/scripts/after_install.sh: line 7: yarn: command not found
2022-06-20 18:21:54 [stderr]/opt/codedeploy-agent/deployment-root/faee375f-f793-4b89-b57e-78c445c6c02b/d-WCHP7OWQH/deployment-archive/scripts/after_install.sh: line 8: yarn: command not found
Related
Here is my buildSpec.yml, node version is always pointing older version 10.9, instead of newly installed 16.X. and build is failing with error unsupported nodejs version, How can I point it to newer version installed.
version: 0.2
phases:
install:
commands:
- echo "=================Nothing in install phase =========================="
- curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
- rm -f /etc/apt/sources.list.d/sbt.list
- apt-get update
- apt dist-upgrade -y
- curl -sL https://deb.nodesource.com/setup_16.x | bash -
- apt-get install -y nodejs
- node --version
- npm cache clean -f
build:
commands:
- echo "=================Entered the build phase...=========================="
- node --version
- npm --version
- npm_config_user=root npm install -g serverless
- npm i -g fs-extra
- sls plugin install -n serverless-python-requirements
- sls deploy
post_build:
commands:
- echo "===============Entered the post_build phase...==============="
- echo Build completed on `date`
artifacts:
files:
- '**/*'
cache:
paths:
- '/node_modules/**/*'
[DEPRECATION] The trollop gem has been renamed to optimist and will no longer be supported. Please switch to optimist as soon as possible.
Webpacker requires Yarn ">=1 <2" and you are using Yarn v0.1.1 2011 Jesper Kjeldgaard
Please upgrade Yarn https://yarnpkg.com/lang/en/docs/install/
Exiting!
2021/03/03 05:06:35.767452 [INFO] Executing cleanup logic
2021/03/03 05:06:35.769905 [INFO] CommandService Response: {"status":"FAILURE","api_version":"1.0","results":[{"status":"FAILURE","msg":"Engine execution has encountered an error.","returncode":1,"events":[{"msg":"Instance deployment failed. For details, see 'eb-engine.log'.","timestamp":1614747995,"severity":"ERROR"}]}]}
2021/03/03 05:06:35.770091 [INFO] Platform Engine finished execution on command: app-deploy
[ec2-user#ip-172-31-28-237 log]$
If your are using Docker you should put these lines on your Dockerfile before the RUN apt-get update -qq && apt-get install .... part:
RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
RUN echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
I don't think this is an issue specific to lerna, I actually think it is nvm related (though I commented out nvm from my .bash_profile to test that, without success).
I'm trying to lerna bootstrap but it keeps using (and failing) an older version (1.3.2) of yarn than what I'm running:
$ yarn -v
1.22.4
$ lerna bootstrap
info cli using local version of lerna
lerna notice cli v3.22.1
lerna info versioning independent
lerna info bootstrap root only
yarn install v1.3.2
error An unexpected error occurred: "patterns.map is not a function".
info If you think this is a bug, please open a bug report with the information provided in "yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
lerna ERR! yarn install --mutex network:42424 --non-interactive exited 1 in 'root'
lerna ERR! yarn install --mutex network:42424 --non-interactive exited 1 in 'root'
$ yarn -v
1.22.4
For me, this was happening in Ubuntu 20.04, and it solved with unistalling cmdtest:
sudo apt-get purge cmdtest
I rooted around in my filesystem and removed ~/.yarn and ~/node_modules/yarn and everything works now.
Upgrade yarn. I use brew so I used brew install yarn and it worked.
In the entrypoint script of a docker -based on alpine linux-, I have the following lines:
#!/bin/sh
echo "============== START ============"
echo $#
NOLOAD=0
FILE=""
RET=1
if [ ! -f /initialized ]; then
echo "not initialized"
apk add --virtual .init-deps bash
echo "bash installed"
echo "Building from server"
apk add --virtual .init-deps git
echo "git installed"
bash load_git.sh "${GIT_SERVER}" "${GIT_USERNAME}" "${GIT_PASSWORD}" "${GIT_BRANCH}"
RET=$?
echo cloning done
fi
echo "just before purging all dependencies"
apk --purge del .init-deps
I expect bash to install, as well as git and the run the load_git.sh script using bash. I am literary logging every other line, but am getting some strange results -before the load_git.sh script is even run:
============== START ============
not initialized
(1/6) Installing ncurses-terminfo-base (6.1_p20191130-r0)
(2/6) Installing ncurses-terminfo (6.1_p20191130-r0)
(3/6) Installing ncurses-libs (6.1_p20191130-r0)
(4/6) Installing readline (8.0.1-r0)
(5/6) Installing bash (5.0.11-r1)
Executing bash-5.0.11-r1.post-install
(6/6) Installing .init-deps (20200109.202215)
Executing busybox-1.31.1-r8.trigger
OK: 18 MiB in 24 packages
bash installed
Building from server
(1/12) Installing ca-certificates (20191127-r0)
(2/12) Installing nghttp2-libs (1.40.0-r0)
(3/12) Installing libcurl (7.67.0-r0)
(4/12) Installing expat (2.2.9-r1)
(5/12) Installing pcre2 (10.34-r1)
(6/12) Installing git (2.24.1-r0)
(7/12) Upgrading .init-deps (20200109.202215 -> 20200109.202216)
(8/12) Purging bash (5.0.11-r1)
Executing bash-5.0.11-r1.pre-deinstall
(9/12) Purging readline (8.0.1-r0)
(10/12) Purging ncurses-libs (6.1_p20191130-r0)
(11/12) Purging ncurses-terminfo (6.1_p20191130-r0)
(12/12) Purging ncurses-terminfo-base (6.1_p20191130-r0)
Executing busybox-1.31.1-r8.trigger
Executing ca-certificates-20191127-r0.trigger
OK: 25 MiB in 25 packages
git installed
/init.sh: line 17: bash: not found
cloning done
outside if statement to get source
just before purging all dependencies
Build failed, starting shell
I can't wrap my head around the fact that upon installing git it also purging bash, this makes no sense at all.
EDIT: this only happens when starting the docker over an ssh connection in a synology nas, when using a local docker image and starting it locally it works perfectly fine.
It looks what's happening here is that the apk add --virtual .init-deps option creates a virtual package and makes it depend on the other packages that just got installed. When you do it a second time, it creates a new virtual package that depends (only) on the second set of packages, and upgrades the virtual package to the new version; when you do that, the first set of packages gets automatically uninstalled.
There are two easy workarounds here: either remove this --virtual .init-deps option, from both lines, or combine all of your installation into a single apk add line.
(It's usually not great practice to download packages on container startup, particularly since deleting and recreating containers is a fairly routine operation. Better practice would be to do this once in the image's Dockerfile
RUN apk add bash git
but also to consider whether you actually need either of these tools to run the application packaged in your image.)
I tried reproducing this and managed with the script below on a remote machine:
#!/bin/sh
if [ ! -f /initialized ]; then
apk add --virtual .init-deps bash
apk add --virtual .init-deps git
bash -c "bash works"
fi
But consecutive runs have varying results:
ubuntu#dev:~$ docker run c81a2d3a5f6b
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
(1/6) Installing ncurses-terminfo-base (6.1_p20191130-r0)
(2/6) Installing ncurses-terminfo (6.1_p20191130-r0)
(3/6) Installing ncurses-libs (6.1_p20191130-r0)
(4/6) Installing readline (8.0.1-r0)
(5/6) Installing bash (5.0.11-r1)
Executing bash-5.0.11-r1.post-install
(6/6) Installing .init-deps (20200109.224208)
Executing busybox-1.31.1-r8.trigger
OK: 15 MiB in 20 packages
(1/12) Installing ca-certificates (20191127-r0)
(2/12) Installing nghttp2-libs (1.40.0-r0)
(3/12) Installing libcurl (7.67.0-r0)
(4/12) Installing expat (2.2.9-r1)
(5/12) Installing pcre2 (10.34-r1)
(6/12) Installing git (2.24.1-r0)
(7/12) Upgrading .init-deps (20200109.224208 -> 20200109.224209)
(8/12) Purging bash (5.0.11-r1)
Executing bash-5.0.11-r1.pre-deinstall
(9/12) Purging readline (8.0.1-r0)
(10/12) Purging ncurses-libs (6.1_p20191130-r0)
(11/12) Purging ncurses-terminfo (6.1_p20191130-r0)
(12/12) Purging ncurses-terminfo-base (6.1_p20191130-r0)
Executing busybox-1.31.1-r8.trigger
Executing ca-certificates-20191127-r0.trigger
OK: 22 MiB in 21 packages
/test.sh: line 5: bash: not found
ubuntu#dev:~$ docker run c81a2d3a5f6b
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
(1/6) Installing ncurses-terminfo-base (6.1_p20191130-r0)
(2/6) Installing ncurses-terminfo (6.1_p20191130-r0)
(3/6) Installing ncurses-libs (6.1_p20191130-r0)
(4/6) Installing readline (8.0.1-r0)
(5/6) Installing bash (5.0.11-r1)
Executing bash-5.0.11-r1.post-install
(6/6) Installing .init-deps (20200109.224207)
Executing busybox-1.31.1-r8.trigger
OK: 15 MiB in 20 packages
OK: 15 MiB in 20 packages
bash works
So I think this is a timing problem in apk
I have a test suite which works perfect on local setup, here is the library:
https://github.com/IbrahimTanyalcin/RafX
However I cannot get it to work on Travis CI, I keep getting:
"Chromium revision is not downloaded"
Here is my yml file:
https://github.com/IbrahimTanyalcin/RafX/blob/master/.travis.yml
language: node_js
dist: trusty
addons:
apt:
packages:
# This is required to run new chrome on old trusty
- libnss3
notifications:
email: false
cache:
directories:
- node_modules
sudo: required
node_js:
- "12.13.0"
env:
#global:
# - PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=0
# allow headful tests
before_install:
# Enable user namespace cloning
- "sysctl kernel.unprivileged_userns_clone=1"
# Launch XVFB
- "export DISPLAY=:99.0"
# - export CHROME_PATH="$(pwd)/chrome-linux/chrome"
- "sh -e /etc/init.d/xvfb start"
# wait for xvfb to boot
- sleep 3
# start your web application and listen on `localhost`
# - google-chrome-stable --headless --disable-gpu --remote-debugging-port=9222 http://localhost &
install:
- npm install
- npm install -g puppeteer --unsafe-perm=true --allow-root
# - yarn add --dev jest-puppeteer
# - yarn add puppeteer
script:
- npm run testCI
And here is the failing build
https://travis-ci.org/IbrahimTanyalcin/RafX
I commented out some of the options I tried, I pretty much checked everywhere including puppeteer and jest docs, and actually took their recommended yml file.
I also tried to apply suggestions in this thread, but of no avail: https://github.com/puppeteer/puppeteer/issues/2173
Any idea how I can solve this?
For anyone struggling with this, it seems I have found a work around, in your before install:
before_install:
- node node_modules/puppeteer/install.js
If you add these lines, you are manually invoking node to install puppeteer, I do not know why npm install fails, but this seems to solve the issue.