Though I've implemented the steps described in the instructions
before_install:
# Download and unpack the stack executable
- mkdir -p ~/.local/bin
- export PATH=$HOME/.local/bin:$PATH
- travis_retry curl -L https://get.haskellstack.org/stable/linux-x86_64.tar.gz | tar xz --wildcards --strip-components=1 -C ~/.local/bin '*/stack'
in my travis.yaml these fail when I attempt to build on macOS using
jobs:
include:
...
- name: "LTS 9.6 (Haskell for macOS)"
os: osx
osx_image: xcode10
env: ...
...
with
tar: Option --wildcards is not supported
Usage:
List: tar -tf <archive-filename>
Extract: tar -xf <archive-filename>
Create: tar -cf <archive-filename> [filenames...]
Help: tar --help
curl: (23) Failed writing body (0 != 597)
The command "curl -L https://www.stackage.org/stack/linux-x86_64" failed. Retrying, 2 of 3.
curl: (23) Failed writing body (0 != 597)
The command "curl -L https://www.stackage.org/stack/linux-x86_64" failed. Retrying, 3 of 3.
curl: (23) Failed writing body (0 != 597)
The command "curl -L https://www.stackage.org/stack/linux-x86_64" failed 3 times.
How do I modify my travis.yaml jobs.include and before_install to install stack for macOS?
A couple of things.
I wouldn't use travis_retry here, since curl has the --retry flag that works better.
macOS uses BSD tar, not GNU tar. You can either:
Find a common flag that works on both
Install gnu-tar, and use that instead. (This formula requires more than just brew install gnu-tar, if you want to execute it as tar, so be sure to set it up correctly.)
Related
I am creating a lambda function from a docker image, this docker image actually runs a bash script inside of the docker container but when I tried to test that then it gives this following error. But this is successful in local. I tested with commented and uncommented entrypoint. Please help me to figure it out.
The dockerfile -
FROM amazon/aws-cli
USER root
ENV AWS_ACCESS_KEY_ID XXXXXXXXXXXXX
ENV AWS_SECRET_ACCESS_KEY XXXXXXXXXXXXX
ENV AWS_DEFAULT_REGION ap-south-1
# RUN mkdir /tmp
COPY main.sh /tmp
WORKDIR /tmp
RUN chmod +x main.sh
RUN touch file_path_final.txt
RUN touch file_path_initial.txt
RUN touch output_final.json
RUN touch output_initial.json
RUN chmod 777 file_path_final.txt
RUN chmod 777 file_path_initial.txt
RUN chmod 777 output_final.json
RUN chmod 777 output_initial.json
RUN yum install jq -y
# ENTRYPOINT ./main.sh ; /bin/bash
ENTRYPOINT ["/bin/sh", "-c" , "ls && ./tmp/main.sh"]
The error -
START RequestId: 8d689260-e500-45d7-aac8-ae260834ed96 Version: $LATEST
/bin/sh: ./tmp/main.sh: No such file or directory
/bin/sh: ./tmp/main.sh: No such file or directory
END RequestId: 8d689260-e500-45d7-aac8-ae260834ed96
REPORT RequestId: 8d689260-e500-45d7-aac8-ae260834ed96 Duration: 58.29 ms Billed Duration: 59 ms Memory Size: 128 MB Max Memory Used: 3 MB
RequestId: 8d689260-e500-45d7-aac8-ae260834ed96 Error: Runtime exited with error: exit status 127
Runtime.ExitError
Here how i did it to Run A C++ over a bash script :
#Pulling the node image from the AWS WCR PUBLIC docker hub.
FROM public.ecr.aws/lambda/provided:al2.2022.10.11.10
#Setting the working directory to /home.
WORKDIR ${LAMBDA_RUNTIME_DIR}
#Copying the contents of the current directory to the working directory.
COPY . .
#This is installing ffmpeg on the container.
RUN yum update -y
# Install sudo, wget and openssl, which is required for building CMake
RUN yum install sudo wget openssl-devel -y
# Install development tools
RUN sudo yum groupinstall "Development Tools" -y
# Download, build and install cmake
RUN yum install -y make
#RUN wget https://github.com/Kitware/CMake/releases/download/v3.22.3/cmake-3.22.3.tar.gz && tar -zxvf cmake-3.22.3.tar.gz && cd ./cmake-3.22.3 && ./bootstrap && make && sudo make install
RUN yum -y install gcc-c++ libcurl-devel cmake3 git
RUN ln -s /usr/bin/cmake3 /usr/bin/cmake
RUN ln -s /usr/bin/ctest3 /usr/bin/ctest
RUN ln -s /usr/bin/cpack3 /usr/bin/cpack
# get cmake versin
RUN cmake --version
RUN echo $(cmake --version)
#This is installing the nodejs and npm on the container.
RUN ./build.sh
RUN chmod 755 run.sh bootstrap
#This is running the nodejs application.
CMD [ "run.sh" ]
You will need a bootstrap file in the root directory : (FROM DOC)
#!/bin/sh
set -euo pipefail
# Initialization - load function handler
source $LAMBDA_RUNTIME_DIR/"$(echo $_HANDLER | cut -d. -f1).sh"
# Processing
while true
do
HEADERS="$(mktemp)"
# Get an event. The HTTP request will block until one is received
EVENT_DATA=$(curl -sS -LD "$HEADERS" -X GET "http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/next")
# Extract request ID by scraping response headers received above
REQUEST_ID=$(grep -Fi Lambda-Runtime-Aws-Request-Id "$HEADERS" | tr -d '[:space:]' | cut -d: -f2)
# Run the handler function from the script
RESPONSE=$($(echo "$_HANDLER" | cut -d. -f2) "$EVENT_DATA")
# Send the response
curl -X POST "http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/$REQUEST_ID/response" -d "$RESPONSE"
done
I need a specific version of protocol buffer which is 3.14.0 on apt its not available like this
sudo apt install -y protobuf-compiler = 3.14.0
and If I follow these steps by binary :
$ PB_REL="https://github.com/protocolbuffers/protobuf/releases"
$ curl -LO $PB_REL/download/v3.14.0/protoc-3.14.0-linux-x86_64.zip
then
unzip protoc-3.14.0-linux-x86_64.zip -d $HOME/.local
then
$ export PATH="$PATH:$HOME/.local/bin"
from this source
and it's not getting install when I do protoc --version I am getting this error
bash: /usr/bin/protoc: No such file or directory
I think Because /usr/bin/protoc doesn't exist. When you unzipped you got folder named protoc-3.14.0.
which is /usr/bin/protoc-3.14.0
try doing this
PROTOC_ZIP=protoc-3.14.0-linux-x86_64.zip
curl -OL https://github.com/protocolbuffers/protobuf/releases/download/v3.14.0/$PROTOC_ZIP
sudo unzip -o $PROTOC_ZIP -d /usr/local bin/protoc
ref: http://google.github.io/proto-lens/installing-protoc.html
Please check out the following steps to Install protoc on Ubundu, Run the commands 1 to 4 on terminal. Then 5 and 6 to finalize the settings.
Get the latest version tag of protoc release and assign it to variable:
1.
PROTOC_VERSION=$(curl -s "https://api.github.com/repos/protocolbuffers/protobuf/releases/latest" | grep -Po '"tag_name": "v\K[0-9.]+')
curl -Lo protoc.zip "https://github.com/protocolbuffers/protobuf/releases/latest/download/protoc-${PROTOC_VERSION}-linux-x86_64.zip"
Run the following command to extract executable file from a ZIP archive:
sudo unzip -q protoc.zip bin/protoc -d /usr/local
Set execute permission:
sudo chmod a+x /usr/local/bin/protoc
Now protoc command is available for all users as a system-wide command.
We can now check protoc version:
protoc --version
Remove unnecessary ZIP archive:
rm -rf protoc.zip
I need WKHTMLTOX to be installed with my AWS EB app.
I found this tutorial and it works except it supports old version.
Did anyone install the latest (0.12.3) version on AWS EB as this is a bit different folders structure?
After trying different tutorials, including this I finally got this working - I updated porteaux's answer.
I added below code to my EB *.config file in commands section:
commands:
# install WKHTML
03_command:
command: yum install xz urw-fonts libXext openssl-devel libXrender
04_command:
command: wget http://download.gna.org/wkhtmltopdf/0.12/0.12.3/wkhtmltox-0.12.3_linux-generic-amd64.tar.xz
05_command:
command: tar -xJf wkhtmltox-0.12.3_linux-generic-amd64.tar.xz
06_command:
command: cp wkhtmltox/bin/wkhtmltopdf /usr/local/bin/wkhtmltopdf
07_command:
command: cp wkhtmltox/bin/wkhtmltoimage /usr/local/bin/wkhtmltoimage
The above tutorial is for Ubuntu and AWS EB runs Amazon Linux so they use yum instead of apt-get
I had to use J switch with tar command to deal with *.xz file
Finally I had to copy both wkhtmltopdf and wkhtmltoimage files to bin folder.
Done! I hope this will help others.
UPDATE: as per dhollenbeck suggestion
04_command:
command: wget https://downloads.wkhtmltopdf.org/0.12/0.12.4/wkhtmltox-0.12.4_linux-generic-amd64.tar.xz
test: test ! -f .wkhtmltopdf
05_command:
command: tar -xJf wkhtmltox-0.12.4_linux-generic-amd64.tar.xz
test: test ! -f .wkhtmltopdf
06_command:
command: cp wkhtmltox/bin/wkhtmltopdf /usr/local/bin/wkhtmltopdf
test: test ! -f .wkhtmltopdf
07_command:
command: cp wkhtmltox/bin/wkhtmltoimage /usr/local/bin/wkhtmltoimage
test: test ! -f .wkhtmltopdf
08_command:
command: touch .wkhtmltopdf
I've updated my script already and can confirm that this work.
Thanks dhollenbeck
None of the other answers appear to be fully function at the moment (some because the links are no longer valid and some because of new dependency issues). I don't have enough reputation to comment on Lucas D'Avila's answer so here is the solution that is working for me today with Amazon-Linux/2.9.7:
Create an .ebextensions file named something like .ebextensions/wkhtmltopdf.config
container_commands:
1_install_wkhtmltopdf:
command: yum -y install fontconfig libXrender libXext xorg-x11-fonts-Type1 xorg-x11-fonts-75dpi freetype libpng zlib libjpeg-turbo openssl icu
ignoreErrors: true
2_install_wkhtmltopdf:
# see: https://wkhtmltopdf.org/downloads.html for updates
command: wget https://github.com/wkhtmltopdf/wkhtmltopdf/releases/download/0.12.4/wkhtmltox-0.12.4_linux-generic-amd64.tar.xz --dns-timeout=5 --connect-timeout=5 --no-check-certificate
test: test ! -f .wkhtmltopdf
3_install_wkhtmltopdf:
command: tar -xJf wkhtmltox-0.12.4_linux-generic-amd64.tar.xz
test: test ! -f .wkhtmltopdf
4_install_wkhtmltopdf:
command: cp wkhtmltox/bin/wkhtmltopdf /usr/local/bin/wkhtmltopdf
test: test ! -f .wkhtmltopdf
5_install_wkhtmltopdf:
command: cp wkhtmltox/bin/wkhtmltoimage /usr/local/bin/wkhtmltoimage
test: test ! -f .wkhtmltopdf
6_install_wkhtmltopdf:
command: touch .wkhtmltopdf
I have insufficient reputation to comment everywhere, so my apologies that this is another answer, rather than just a comment on #dhollenbeck answer. Happy to delete this if that is updated.
gna.org has shut down thus 04_command will fail. A working list of downloads is on wkhtmltopdf.org.
The updated YAML scripts would therefore be.
Create yaml file .ebextensions/wkhtmltopdf.config:
commands:
03_command:
command: yum install --assumeyes zlib fontconfig freetype X11
04_command:
command: wget https://downloads.wkhtmltopdf.org/0.12/0.12.4/wkhtmltox-0.12.4_linux-generic-amd64.tar.xz
05_command:
command: tar -xJf wkhtmltox-0.12.4_linux-generic-amd64.tar.xz
06_command:
command: cp wkhtmltox/bin/wkhtmltopdf /usr/local/bin/wkhtmltopdf
07_command:
command: cp wkhtmltox/bin/wkhtmltoimage /usr/local/bin/wkhtmltoimage
and if you only want to install wkhtmltopdf once to speed up subsequent deployments:
commands:
03_command:
command: yum install --assumeyes zlib fontconfig freetype X11
test: test ! -f .wkhtmltopdf
04_command:
command: wget https://downloads.wkhtmltopdf.org/0.12/0.12.4/wkhtmltox-0.12.4_linux-generic-amd64.tar.xz
test: test ! -f .wkhtmltopdf
05_command:
command: tar -xJf wkhtmltox-0.12.4_linux-generic-amd64.tar.xz
test: test ! -f .wkhtmltopdf
06_command:
command: cp wkhtmltox/bin/wkhtmltopdf /usr/local/bin/wkhtmltopdf
test: test ! -f .wkhtmltopdf
07_command:
command: cp wkhtmltox/bin/wkhtmltoimage /usr/local/bin/wkhtmltoimage
test: test ! -f .wkhtmltopdf
08_command:
command: touch .wkhtmltopdf
An updated answer for wkhtmltopdf 0.12.4 installed on 64bit Amazon Linux 2016.09 v3.3.0.
Create yaml file .ebextensions/wkhtmltopdf.config
commands:
03_command:
command: yum install --assumeyes zlib fontconfig freetype X11
04_command:
command: wget http://download.gna.org/wkhtmltopdf/0.12/0.12.4/wkhtmltox-0.12.4_linux-generic-amd64.tar.xz
05_command:
command: tar -xJf wkhtmltox-0.12.4_linux-generic-amd64.tar.xz
06_command:
command: cp wkhtmltox/bin/wkhtmltopdf /usr/local/bin/wkhtmltopdf
07_command:
command: cp wkhtmltox/bin/wkhtmltoimage /usr/local/bin/wkhtmltoimage
If you only want to install wkhtmltopdf once to speed up subsequent deployments:
commands:
03_command:
command: yum install --assumeyes zlib fontconfig freetype X11
test: test ! -f .wkhtmltopdf
04_command:
command: wget http://download.gna.org/wkhtmltopdf/0.12/0.12.4/wkhtmltox-0.12.4_linux-generic-amd64.tar.xz
test: test ! -f .wkhtmltopdf
05_command:
command: tar -xJf wkhtmltox-0.12.4_linux-generic-amd64.tar.xz
test: test ! -f .wkhtmltopdf
06_command:
command: cp wkhtmltox/bin/wkhtmltopdf /usr/local/bin/wkhtmltopdf
test: test ! -f .wkhtmltopdf
07_command:
command: cp wkhtmltox/bin/wkhtmltoimage /usr/local/bin/wkhtmltoimage
test: test ! -f .wkhtmltopdf
08_command:
command: touch .wkhtmltopdf
Complementing Elric Pedder anwser.
If wkhtmltopdf is already installed, it will give and error because yum will give a "Error: Nothing to do" and exit with -1 state. This will occur when you re deploy your elastic beanstalk.
In order to install wkhtmltopdf without failing, i check if is already installed.
wkhtmltopdf.config
container_commands:
01_install_wkhtmltopdf:
command: sudo yum -q list installed wkhtmltox.x86_64 &>/dev/null && sudo yum -y reinstall https://github.com/wkhtmltopdf/packaging/releases/download/0.12.6-1/wkhtmltox-0.12.6-1.amazonlinux2.x86_64.rpm || sudo yum -y install https://github.com/wkhtmltopdf/packaging/releases/download/0.12.6-1/wkhtmltox-0.12.6-1.amazonlinux2.x86_64.rpm
You can also install wkhtmltopdf by creating a file like .ebextensions/packages.config with this content:
packages:
rpm:
# find more versions on https://wkhtmltopdf.org/downloads.html
wkhtmltopdf: https://downloads.wkhtmltopdf.org/0.12/0.12.5/wkhtmltox-0.12.5-1.centos7.x86_64.rpm
wget https://downloads.wkhtmltopdf.org/0.12/0.12.5/wkhtmltox-0.12.5-1.centos6.x86_64.rpm
sudo yum install wkhtmltox-0.12.5-1.centos6.x86_64.rpm
I installed pigz via homebrew on my Macbook Air (OS X 10.10.5) to get better performance for compress/decompress.
To compress, I use tar --use-compress-program=pigz -cf test.tgz test and it's ok.
But the command to uncompress, tar --use-compress-program=pigz -xf test.tgz output error:
tar: Unrecognized archive format
tar: Error exit delayed from previous errors.
Or sometimes it output:
tar: Unrecognized archive format
pigz: abort: write error on <stdout> (Broken pipe)
tar: Child process exited with status 32
tar: Error exit delayed from previous errors.
I read the manual of tar, and have no clue why it doesn't work.
I noticed that even tar --use-compress-program=gzip -xf test.tgz generate the same error. So is this a bug of OSX's tar implementation?
Note: I know pipe style pigz -d test.tgz | tar -xf works and in this case I could also just use tar -xf test.tgz which call built-in gzip. But I just want to confirm whether it is a bug.
The program works as designed: there is no provision in its command-line to pass along the options needed to use gzip for decompressing. Instead of "gzip" for decompressing, you should use the wrapped gzcat, e.g.,
tar --use-compress-program gzip -cf foo.compressed foo
tar --use-compress-program gzcat -tf foo.compressed
A quick check shows that this does not work:
tar --use-compress-program 'gzip -d' -tf foo.compress
although that could change some time (it is doable, but not done).
According to pigz's manual page, it has unpigz, which is what you can use for that program.
This is an issue between BSD tar and GNU tar.
I fixed this by installing gnu-tar from homebrew and placing that on the path.
See this similar question on superuser: https://superuser.com/questions/318809/linux-os-x-tar-incompatibility-tarballs-created-on-os-x-give-errors-when-unt
on linux I renamed gzip to gzip.sav and made a soft link to pigz so tar, dolphin, yum, and any other program that calls gzip actually calls pigz
I am trying to run a container which runs an automated build. Here is the dockerfile:
FROM ubuntu:14.04
MAINTAINER pmandayam
# update dpkg repositories
RUN apt-get update
# install wget
RUN apt-get install -y wget
# get maven 3.2.2
RUN wget --no-verbose -O /tmp/apache-maven-3.2.2.tar.gz http://archive.apache.or
g/dist/maven/maven-3/3.2.2/binaries/apache-maven-3.2.2-bin.tar.gz
# verify checksum
RUN echo "87e5cc81bc4ab9b83986b3e77e6b3095 /tmp/apache-maven-3.2.2.tar.gz" | md5
sum -c
# install maven
RUN tar xzf /tmp/apache-maven-3.2.2.tar.gz -C /opt/
RUN ln -s /opt/apache-maven-3.2.2 /opt/maven
RUN ln -s /opt/maven/bin/mvn /usr/local/bin
RUN rm -f /tmp/apache-maven-3.2.2.tar.gz
ENV MAVEN_HOME /opt/maven
# remove download archive files
RUN apt-get clean
# set shell variables for java installation
ENV java_version 1.8.0_11
ENV filename jdk-8u11-linux-x64.tar.gz
ENV downloadlink http://download.oracle.com/otn-pub/java/jdk/8u11-b12/$filename
# download java, accepting the license agreement
RUN wget --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie
" -O /tmp/$filename $downloadlink
# unpack java
RUN mkdir /opt/java-oracle && tar -zxf /tmp/$filename -C /opt/java-oracle/
ENV JAVA_HOME /opt/java-oracle/jdk$java_version
ENV PATH $JAVA_HOME/bin:$PATH
# configure symbolic links for the java and javac executables
RUN update-alternatives --install /usr/bin/java java $JAVA_HOME/bin/java 20000 &
& update-alternatives --install /usr/bin/javac javac $JAVA_HOME/bin/javac 20000
# install mongodb
RUN echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen
' | sudo tee /etc/apt/sources.list.d/mongodb.list && \
apt-get update && \
apt-get --allow-unauthenticated install -y mongodb-org mongodb-org-s
erver mongodb-org-shell mongodb-org-mongos mongodb-org-tools && \
echo "mongodb-org hold" | dpkg --set-selections && \
echo "mongodb-org-server hold" | dpkg --set-selections && \
echo "mongodb-org-shell hold" | dpkg --set-selections &&
\
echo "mongodb-org-mongos hold" | dpkg --set-selectio
ns && \
echo "mongodb-org-tools hold" | dpkg --set-selec
tions
RUN mkdir -p /data/db
VOLUME /data/db
EXPOSE 27017
COPY build-script /build-script
CMD ["/build-script"]
I can build the image successfully but when I try to run the container I get this error:
$ docker run mybuild
no such file or directory
Error response from daemon: Cannot start container 3e8aa828909afcd8fb82b5a5ac894
97a537bef2b930b71a5d20a1b98d6cc1dd6: [8] System error: no such file or directory
what does it mean 'no such file or directory'?
Here is my simple script:
#!/bin/bash
sudo service mongod start
mvn clean verify
sudo service mongod stop
I copy it like this: COPY build-script /build-script
and run it like this: CMD ["/build-script"] not sure why its not working
Using service isn't going to fly - the Docker base images are minimal and don't support this. If you want to run multiple processes, you can use supervisor or runit etc.
In this case, it would be simplest just to start mongo manually in the script e.g. /usr/bin/mongod & or whatever the correct incantation is.
BTW the lines where you try to clean up don't have much effect:
RUN rm -f /tmp/apache-maven-3.2.2.tar.gz
...
# remove download archive files
RUN apt-get clean
These files have already been committed to a previous image layer, so doing this doesn't save any disk-space. Instead you have to delete the files in the same Dockerfile instruction in which they're added.
Also, I would consider changing the base image to a Java one, which would save a lot of work. However, you may have trouble finding one which bundles the official Oracle JDK rather than OpenJDK if that's a problem.