I am newbie to Docker. I can create a docker image for Java and Maven from https://github.com/carlossg/docker-maven/blob/master/jdk-13/Dockerfile . I can understand most of the commands there inside dockerfile, there are some that I could not find sufficient info on net. Can someone please help me ?
(1) What does below ARG SHA do. If I understand it right, SHA is immutable identifier that is associated with image, so I am downloading image with that identifier, I mean specific image with changes I want and stored with that SHA, is this right?
ARG SHA=c35a1803a6e70a126e80b2b3ae33eed961f83ed74d18fcd16909b2d44d7dada3203f1ffe726c17ef8dcca2dcaa9fca676987befeadc9b9f759967a8cb77181c0
(2) I know what RUN, echo does and how the variable works. But not sure what is happening below with curl command . No idea what below lines of code does for sure.
RUN mkdir -p /usr/share/maven /usr/share/maven/ref \
&& curl -fsSL -o /tmp/apache-maven.tar.gz ${BASE_URL}/apache-maven-${MAVEN_VERSION}-bin.tar.gz \
&& echo "${SHA} /tmp/apache-maven.tar.gz" | sha512sum -c - \
&& tar -xzf /tmp/apache-maven.tar.gz -C /usr/share/maven --strip-components=1 \
&& rm -f /tmp/apache-maven.tar.gz \
&& ln -s /usr/share/maven/bin/mvn /usr/bin/mvn```
You have to read it like a shell script.
1.
SHA is SHA512 hash
function used in line 10 to
check if downloaded /tmp/apache-maven.tar.gz is what we expect. It
has nothing to do with Docker image ID, if you mean that. You can
reproduce the check locally on your system:
$ SHA=c35a1803a6e70a126e80b2b3ae33eed961f83ed74d18fcd16909b2d44d7dada3203f1ffe726c17ef8dcca2dcaa9fca676987befeadc9b9f759967a8cb77181c0
$ BASE_URL=https://apache.osuosl.org/maven/maven-3/${MAVEN_VERSION}/binaries
$ curl -fsSL -o /tmp/apache-maven.tar.gz ${BASE_URL}/apache-maven-${MAVEN_VERSION}-bin.tar.gz
$ echo "${SHA} /tmp/apache-maven.tar.gz" | sha512sum -c -
/tmp/apache-maven.tar.gz: OK
(Notice that $ here is a command line
prompt
used to indicate start of a new line, not a part of the
command).
curl here downloads
https://apache.osuosl.org/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz
to /tmp/apache-maven.tar.gz.
2.
Again, read it like a shell script. && is used for chaining commands and \ is used to concatenate lines.
RUN mkdir -p /usr/share/maven /usr/share/maven/ref
Create /usr/share/maven and /usr/share/maven/ref directories.
curl -fsSL -o /tmp/apache-maven.tar.gz ${BASE_URL}/apache-maven-${MAVEN_VERSION}-bin.tar.gz
Download temporary apache-maven tarball to /tmp/apache-maven.tar.gz.
echo "${SHA} /tmp/apache-maven.tar.gz" | sha512sum -c -
Check if the downloaded tarball has the correct checksum.
tar -xzf /tmp/apache-maven.tar.gz -C /usr/share/maven --strip-components=1
Extract /tmp/apache-maven.tar.gz to /usr/share/maven.
rm -f /tmp/apache-maven.tar.gz
Remove temporary tarball after extracting it.
ln -s /usr/share/maven/bin/mvn /usr/bin/mvn
Create /usr/bin/mvn that points to /usr/share/maven/bin/mvn. This
is done because /usr/bin directory is typically in $PATH so that
mvn can be run without providing a full path to it.
Say I have a file at the URL http://mywebsite.example/myscript.txt that contains a script:
#!/bin/bash
echo "Hello, world!"
read -p "What is your name? " name
echo "Hello, ${name}!"
And I'd like to run this script without first saving it to a file. How do I do this?
Now, I've seen the syntax:
bash < <(curl -s http://mywebsite.example/myscript.txt)
But this doesn't seem to work like it would if I saved to a file and then executed. For example readline doesn't work, and the output is just:
$ bash < <(curl -s http://mywebsite.example/myscript.txt)
Hello, world!
Similarly, I've tried:
curl -s http://mywebsite.example/myscript.txt | bash -s --
With the same results.
Originally I had a solution like:
timestamp=`date +%Y%m%d%H%M%S`
curl -s http://mywebsite.example/myscript.txt -o /tmp/.myscript.${timestamp}.tmp
bash /tmp/.myscript.${timestamp}.tmp
rm -f /tmp/.myscript.${timestamp}.tmp
But this seems sloppy, and I'd like a more elegant solution.
I'm aware of the security issues regarding running a shell script from a URL, but let's ignore all of that for right now.
source <(curl -s http://mywebsite.example/myscript.txt)
ought to do it. Alternately, leave off the initial redirection on yours, which is redirecting standard input; bash takes a filename to execute just fine without redirection, and <(command) syntax provides a path.
bash <(curl -s http://mywebsite.example/myscript.txt)
It may be clearer if you look at the output of echo <(cat /dev/null)
This is the way to execute remote script with passing to it some arguments (arg1 arg2):
curl -s http://server/path/script.sh | bash /dev/stdin arg1 arg2
For bash, Bourne shell and fish:
curl -s http://server/path/script.sh | bash -s arg1 arg2
Flag "-s" makes shell read from stdin.
Use:
curl -s -L URL_TO_SCRIPT_HERE | bash
For example:
curl -s -L http://bitly/10hA8iC | bash
Using wget, which is usually part of default system installation:
bash <(wget -qO- http://mywebsite.example/myscript.txt)
You can also do this:
wget -O - https://raw.github.com/luismartingil/commands/master/101_remote2local_wireshark.sh | bash
The best way to do it is
curl http://domain/path/to/script.sh | bash -s arg1 arg2
which is a slight change of answer by #user77115
You can use curl and send it to bash like this:
bash <(curl -s http://mywebsite.example/myscript.txt)
I often using the following is enough
curl -s http://mywebsite.example/myscript.txt | sh
But in a old system( kernel2.4 ), it encounter problems, and do the following can solve it, I tried many others, only the following works
curl -s http://mywebsite.example/myscript.txt -o a.sh && sh a.sh && rm -f a.sh
Examples
$ curl -s someurl | sh
Starting to insert crontab
sh: _name}.sh: command not found
sh: line 208: syntax error near unexpected token `then'
sh: line 208: ` -eq 0 ]]; then'
$
The problem may cause by network slow, or bash version too old that can't handle network slow gracefully
However, the following solves the problem
$ curl -s someurl -o a.sh && sh a.sh && rm -f a.sh
Starting to insert crontab
Insert crontab entry is ok.
Insert crontab is done.
okay
$
Also:
curl -sL https://.... | sudo bash -
Just combining amra and user77115's answers:
wget -qO- https://raw.githubusercontent.com/lingtalfi/TheScientist/master/_bb_autoload/bbstart.sh | bash -s -- -v -v
It executes the bbstart.sh distant script passing it the -v -v options.
Is some unattended scripts I use the following command:
sh -c "$(curl -fsSL <URL>)"
I recommend to avoid executing scripts directly from URLs. You should be sure the URL is safe and check the content of the script before executing, you can use a SHA256 checksum to validate the file before executing.
instead of executing the script directly, first download it and then execute
SOURCE='https://gist.githubusercontent.com/cci-emciftci/123123/raw/123123/sample.sh'
curl $SOURCE -o ./my_sample.sh
chmod +x my_sample.sh
./my_sample.sh
This way is good and conventional:
17:04:59#itqx|~
qx>source <(curl -Ls http://192.168.80.154/cent74/just4Test) Lord Jesus Loves YOU
Remote script test...
Param size: 4
---------
17:19:31#node7|/var/www/html/cent74
arch>cat just4Test
echo Remote script test...
echo Param size: $#
If you want the script run using the current shell, regardless of what it is, use:
${SHELL:-sh} -c "$(wget -qO - http://mywebsite.example/myscript.txt)"
if you have wget, or:
${SHELL:-sh} -c "$(curl -Ls http://mywebsite.example/myscript.txt)"
if you have curl.
This command will still work if the script is interactive, i.e., it asks the user for input.
Note: OpenWRT has a wget clone but not curl, by default.
bash | curl http://your.url.here/script.txt
actual example:
juan#juan-MS-7808:~$ bash | curl https://raw.githubusercontent.com/JPHACKER2k18/markwe/master/testapp.sh
Oh, wow im alive
juan#juan-MS-7808:~$
I want to get and install in bash one line, like:
curl XXX.deb | dpkg -i
but dpkg report argument missing
how to get it work?
I suggest to add -o to curl to avoid to redirect to stdout the binary file like:
curl http://security.ubuntu.com/ubuntu/pool/universe/e/eigen3/libeigen3-dev_3.3.2-1_all.deb -o libeigen3-dev_3.3.2-1_all.deb && dpkg -i libeigen3-dev_3.3.2-1_all.deb
You can't pipe information into dpkg like that. One possibility is combining them with &&. Meaning the first command must succeed for the next command to be executed.
curl XXX.deb && dpkg -i XXX.deb
Assuming you know the filename beforehand and can pass it to both statements.
You can use wget in a similar way.
wget https://example.com/path/someapp.deb -O app.deb && sudo dpkg -i app.deb && rm -f app.deb
Plus wget shows progress bar and the local filename is forced (maybe you can't predict from url).
#!/bin/bash
mkdir /tmp
curl -O http://www.mucommander.com/download/nightly/mucommander-current.app.tar.gz /tmp/mucommander.tgz
tar -xvzf /tmp/mucommander.tgz */mucommander.app/*
cp -r /tmp/mucommander.app /Applications
rm -r /tmp
I'm trying to create a shell script to download and extract muCommander to my applications directory on a Mac.
I tried cd into the tmp dir, but then the script stops when I do that.
I can extract all using the -C argument, but the current tgz path is muCommander-0_9_0/mucommander.app, which could change on later builds, so I'm trying to keep it generic.
Can anyone give me pointers where I'm going wrong?
Thanks in advance.
Strip the first path component when you untar the archive, from tar(1):
--strip-components count
(x mode only) Remove the specified number of leading path ele-
ments. Pathnames with fewer elements will be silently skipped.
Note that the pathname is edited after checking inclusion/exclu-
sion patterns but before security checks.
Update
Here is a working bash example of how to, fairly generically, copy the contents of the tgz file to /Applications.
shopt -s nocaseglob
TMPDIR=/tmp
APP=mucommander
TMPAPPDIR=$TMPDIR/$APP
mkdir -p $TMPAPPDIR
curl -o $TMPDIR/$APP.tgz http://www.mucommander.com/download/nightly/mucommander-current.app.tar.gz
tar --strip-components=1 -xvzf $APP.tgz -C $TMPAPPDIR
mv $TMPAPPDIR/${APP}* /Applications
# rm -rf $TMPAPPDIR $TMPDIR/$APP
The rm command is commented out for now, verify that it does no harm before you use it.
The following will update your muCommander.
#for the safety, remove old temporary extraction from the /tmp
rm -rf /tmp/muCommander.app
#kill the running mucommander - you dont want replace the runnung app
ps -ef | grep ' /Applications/muCommander.app/' | grep -v grep | awk '{print $2}' | xargs kill
#download, extract, remove old, move new, open
#each command run only when the previous ended with success
curl http://www.mucommander.com/download/nightly/mucommander-current.app.tar.gz |\
tar -xzf - -C /tmp --strip-components=1 '*/muCommander.app' && \
rm -rf /Applications/muCommander.app && \
mv /tmp/muCommander.app /Applications && \
open /Applications/muCommander.app
Beware, after the '\' must following new line, and not any spaces...
I have this simple bash script that gets a copy from my dev server:
#!/bin/sh
DATE=`date +%Y-%m-%d_%H%M.%S`
BASEDIR="/var/www/db"
RELEASEDIR="$DATE";
RELEASEDIRFULL="$BASEDIR/releases/$RELEASEDIR"
mkdir -p "$RELEASEDIRFULL"
echo "Chdir to \"$RELEASEDIRFULL\""
cd "$RELEASEDIRFULL"
echo "Getting copy from dev"
ssh dev.example.tld "cd /tmp; cd /sites/db; tar -zcvp --exclude data --exclude scripts -f - *" | tar zxvpf -
ln -s /var/www/db/data data
ln -s /var/www/db/scripts scripts
cd $BASEDIR
rm htdocs; ln -s releases/$RELEASEDIR htdocs
Recently it stopped working properly with no apparent reason. It gets to the ssh line, executes it fine (files appear on live server) but does not proceed with ln commands. If I comment the ssh line out, ln lines will get executed properly.
UPDATE: I noticed that when I'm logged on as www-data and start the script, it completes as expected, without errors.
No time to check up the man page, but looks like your tar input is - * - all files + stdin? Are you meaning -- for suspension of further argument processing (if tar supports that)