Dockerfile:
FROM ubuntu:latest
RUN apt install -y bash
CMD []
build and run:
docker build -t test .
docker run -it test bash
minimal reproduction:
root#8807902e27b4:/# mkdir parent
root#8807902e27b4:/# cd parent
root#8807902e27b4:/parent# mkdir example
root#8807902e27b4:/parent# chmod 000 example
root#8807902e27b4:/parent# ls -la
total 12
drwxr-xr-x 3 root root 4096 Apr 28 19:33 .
drwxr-xr-x 1 root root 4096 Apr 28 19:32 ..
d--------- 2 root root 4096 Apr 28 19:33 example
root#8807902e27b4:/parent# cd example
root#8807902e27b4:/parent/example# echo "test" > test.txt
root#8807902e27b4:/parent/example# chmod 100 test.txt
root#8807902e27b4:/parent/example# cat test.txt
test
root#8807902e27b4:/parent/example# ls -la
total 12
d--------- 2 root root 4096 Apr 28 19:33 .
drwxr-xr-x 3 root root 4096 Apr 28 19:33 ..
---x------ 1 root root 5 Apr 28 19:33 test.txt
In the above example, the cd example command should fail, and even if it doesn't, running cat test.txt should fail. Anyone know what's up?
Here are the same (working) commands run in osx:
beaushinkle#Beaus-MBP ~/p/example-docker> mkdir parent
beaushinkle#Beaus-MBP ~/p/example-docker> cd parent
beaushinkle#Beaus-MBP ~/p/e/parent> mkdir example
beaushinkle#Beaus-MBP ~/p/e/parent> chmod 000 example
beaushinkle#Beaus-MBP ~/p/e/parent> cd example
cd: Permission denied: 'example'
beaushinkle#Beaus-MBP ~/p/e/parent [1]> chmod 777 example
beaushinkle#Beaus-MBP ~/p/e/parent> cd example
beaushinkle#Beaus-MBP ~/p/e/p/example> echo "test" > test.txt
beaushinkle#Beaus-MBP ~/p/e/p/example> chmod 100 test.txt
beaushinkle#Beaus-MBP ~/p/e/p/example> cat test.txt
cat: test.txt: Permission denied
If the prompt is anything to go by, we are logged in as root in the minimal reproduction. Thus, we have root privileges and can read and write all files (external link).
Related
I'm building a gitlab ci pipeline, and try to create a directory if it not exists.
Can anybody tell me what I'm doing wrong here?
$ if [ ! -d aws ]
$ then
$ mkdir aws
mkdir: cannot create directory ‘aws’: File exists
ERROR: Job failed: exit code 1
the relevant part of the gitlab-ci.yml
script:
- export
- ls -al
- if [ ! -d aws ]
- then
- mkdir aws
- fi
$ ls -al
total 128
drwxrwxrwx 16 root root 4096 Sep 17 12:07 .
drwxrwxrwx 6 root root 4096 Sep 17 12:07 ..
drwxrwxrwx 2 root root 4096 Sep 17 12:07 aws
I now just used mkdir -p and removed the test
you have something with aws name, which might be symbolic link, hard link, regular file, vs.
first delete or move that file to somewhere else then try again
you can try -e (returns true if file exists regardless of type).
My understanding of tar command that it will override the content of the file if file exist. Otherwise it would keep as existing.
[root#something~]# ls -al /etc/init.d/
total XX
drwxr-xr-x. 2 root root 83 Jun 14 2018 .
drwxr-xr-x. 10 root root 127 Jun 6 2017 ..
-rwxr-xr-x. 1 root root 7293 Jan 2 2018 network
-rw-r--r--. 1 root root 1160 Feb 20 2018 README
[root#something~]# tar tvf /tmp/env_pkg_1.tar
drwxr-xr-x staff 0 2020-05-29 19:42 etc/
drwxr-xr-x user/staff 0 2020-05-29 18:04 etc/init.d/
-rw-r--r-- user/staff 3383 2020-05-29 18:04 etc/init.d/sshd
[root#something~]# cd /
[root#something /]# tar xf /tmp/env_pkg_1.tar
[root#something/]# ls -al /etc/init.d/
total 16
drwxr-xr-x 2 XXXXXX XXXXXX 18 May 29 18:04 .
drwxr-xr-x. 85 XXXXXX XXXXXX 8192 May 29 19:42 ..
-rw-r--r-- 1 XXXXXX XXXXXX 3383 May 29 18:04 sshd
I am not understand why tar is replacing the entire contents of /etc/init.d
Any inputs would be helpful ?
I belive that /etc/init.d is a link to /etc/rc.d/init.d.
When you untarred that file, it overwrote the link with a directory. All of your files are still in /etc/rc.d/init.d.
To fix your situation, remove /etc/init.d, relink it, and add a h to the tar command:
rm -rf /etc/init.d
cd /etc
ln -s ./rc.d/init.d
cd /
tar xhf /tmp/env_pkg_1.tar
You can use -k or --keep-old-files, so it does not touch any files that are already within the destination. Judging by your output in /etc/init.d/ you want to keep network and README and next to them extract sshd, so in your case, they do not overlap.
Alternatively --keep-newer-files will have tar replace files that are newer from the tar archive, than what's on the destination..
I'm trying to iterate over url entries in a file and use each file as an input for a crawler tool. It's result should be written to a file.
here is the gitlab-ci.yml file:
stages:
- test
test:
stage: test
tags:
- shell-docker
script:
- wget https://github.com/FaKeller/sireg/releases/download/v0.3.1/sireg-linux
- chmod 775 sireg-linux
- mkdir output
- ls -alF
- while read line; do
echo $line;
./sireg-linux exec --loader-sitemap-sitemap \"$line\" >> ./output/${line##*/}_out.txt;
done < sitemap-index
- ls -alF output
artifacts:
paths:
- output/*
expire_in: 1 hrs
and here is the sitemap-index file (only one entry):
http://example.com/sitemap.xml
both files are in the same directory. I expect a file sitemap.xml_out.txt to be written into the output folder(also the same directory). I am pretty sure the ./sireg-linux script does not execute because it usually takes few minutes to complete (tested locally).
the output of the stage looks like this:
2020-04-02 18:22:21 (4,26 MB/s) - »sireg-linux« saved [62566347/62566347]
$ chmod 775 sireg-linux
$ mkdir output
$ ls -alF
total 61128
drwxrwxr-x 4 gitlab-runner gitlab-runner 4096 Apr 2 18:22 ./
drwxrwxr-x 10 gitlab-runner gitlab-runner 4096 Apr 2 15:46 ../
drwxrwxr-x 5 gitlab-runner gitlab-runner 4096 Apr 2 18:22 .git/
-rw-rw-r-- 1 gitlab-runner gitlab-runner 512 Apr 2 18:22 .gitlab-ci.yml
drwxrwxr-x 2 gitlab-runner gitlab-runner 4096 Apr 2 18:22 output/
-rw-rw-r-- 1 gitlab-runner gitlab-runner 30 Apr 2 15:46 README.md
-rwxrwxr-x 1 gitlab-runner gitlab-runner 62566347 Nov 11 2017 sireg-linux*
-rw-rw-r-- 1 gitlab-runner gitlab-runner 55 Apr 2 18:08 sitemap-index
$ while read line; do echo $line; ./sireg-linux **exec** --loader-sitemap-sitemap \"$line\" >>
./output/${line##*/}_out.txt; done < sitemap-index
$ ls -alF output
total 8
drwxrwxr-x 2 gitlab-runner gitlab-runner 4096 Apr 2 18:22 ./
drwxrwxr-x 4 gitlab-runner gitlab-runner 4096 Apr 2 18:22 ../
Uploading artifacts...
Runtime platform arch=amd64 os=linux pid=23813 revision=1f513601 version=11.10.1
WARNING: output/*: no matching files
ERROR: No files to upload
Job succeeded
update
tried to move all steps into a separate script but that did not work either.
update 2
forgot to add exec in the command:
./sireg-linux exec --loader-sitemap-sitemap \"$line\" >>
./output/${line##*/}_out.txt;
unfortunately it didn't help.
what can I do to make it working?
Try changing ./sireg-linux --loader-sitemap-sitemap \"$line\" to ./sireg-linux exec --loader-sitemap-sitemap "$line". Hope this helps!
EDIT: Also, it looks like the script doesn't enter the while loop at all. Maybe the file sitemap-index is empty or it has only one line without a newline at the end?
EDIT 2: The back-slashes in the command line are wrong. corrected my answer
You can of course painfully debug multi-line commands in YAML.
You can even use YAML multi-line strings:
How do I break a string over multiple lines?
https://gitlab.com/snippets/1717579
But I would just wrap code into a shell script, store it in the same GitLab repo, and just call it in .gitlab-ci.yml.
This way you can run this script exactly the same way both locally and in CI, which is a best practice in Continuous Delivery.
- ./script.sh
I´m using lemonlatte / docker-webvirtmgr as base file, but the problem is that there are no ssh keys configured for the user www-data, so I wrote the following shell script:
#!/bin/sh
if [ ! -d "/var/local/webvirtmgr/nginxhome" ]; then
mkdir /var/local/webvirtmgr/nginxhome
chown -R www-data:www-data /var/local/webvirtmgr/nginxhome
usermod -d /var/local/webvirtmgr/nginxhome www-data
su - www-data -s /bin/bash -c "ssh-keygen -b 2048 -t rsa -f ~/.ssh/id_rsa -q -N ''"
su - www-data -s /bin/bash -c "touch /var/local/webvirtmgr/nginxhome/.ssh/config && echo -e 'StrictHostKeyChecking=no\nUserKnownHostsFile=/dev/null' >> /var/local/webvirtmgr/nginxhome/.ssh/config"
su - www-data -s /bin/bash -c "chmod 0600 ~/.ssh/config"
fi
After that I added the two statements to the dockerfile:
ADD setupssh.sh /webvirtmgr/setupssh.sh
RUN /bin/sh -c "/webvirtmgr/setupssh.sh"
I already tried CMD /webvirtmgr/setupssh.sh, RUN /webvirtmgr/setupssh.sh but with no success...
When I run the script inside the container by hand it is working fine.
What is wrong here?
greetings
UPDATE:
Here is the link to the repo of the maintainer: link
UPDATE 2:
The build of the dockerfile was successful and I put the statement between:
RUN apt-get -ys clean
<statements were here>
WORKDIR /
The directory /var/local/webvirtmgr is defined as a volume.
VOLUME /var/local/webvirtmgr
Therefore this directory is a mountpoint in the running container and what you have added to it gets overwritten.
You will have to use a different directory, then your script will work.
Here´s a Dockerfile to test it:
FROM lemonlatte/docker-webvirtmgr
RUN mkdir /var/local/webvirtmgr2
RUN touch /var/local/webvirtmgr2/t && touch /var/local/webvirtmgr/t
RUN ls -la //var/local/webvirtmgr
RUN ls -la /var/local/webvirtmgr2
Output:
Sending build context to Docker daemon 4.608 kB
Sending build context to Docker daemon
Step 0 : FROM lemonlatte/docker-webvirtmgr
---> 18e2839dffea
Step 1 : RUN mkdir /var/local/webvirtmgr2
---> Running in d7a1e897108e
---> cc029293525e
Removing intermediate container d7a1e897108e
Step 2 : RUN touch /var/local/webvirtmgr2/t && touch /var/local/webvirtmgr/t
---> Running in 1a1375651fa7
---> e314c2529d90
Removing intermediate container 1a1375651fa7
Step 3 : RUN ls -la //var/local/webvirtmgr
---> Running in 5228691c84f5
total 8
drwxr-xr-x 2 www-data www-data 4096 Jun 6 09:22 .
drwxr-xr-x 6 root root 4096 Jun 6 09:22 ..
---> ec4113936961
Removing intermediate container 5228691c84f5
Step 4 : RUN ls -la /var/local/webvirtmgr2
---> Running in a6d2a683391a
total 8
drwxr-xr-x 2 root root 4096 Jun 6 09:22 .
drwxr-xr-x 6 root root 4096 Jun 6 09:22 ..
-rw-r--r-- 1 root root 0 Jun 6 09:22 t
---> 3cb98c5c1baf
Removing intermediate container a6d2a683391a
Successfully built 3cb98c5c1baf
I am using the Bash FTP command to ftp files, however i have a problem where i try to create a directory that is more than 2 folders deep. It works if i use two folders deep but if i go to three folders deep then it fails. For example:
mkdir foo/bar - this works
mkdir foo/bar/baz - this fails
I have also tried this:
mkdir -p foo/bar/baz - which didn't work, it ended up creating a '-p' directory
The shell script i am trying to run is actually quite simple but as you can see the directory structure is 3 folders deep and it fails to create the required folders:
#!/bin/bash
DIRECTORY="foo/bar/baz"
FILE="test.pdf"
HOST="testserver"
USER="test"
PASS="test"
ftp -n $HOST <<END_SCRIPT
quote USER $USER
quote PASS $PASS
mkdir $DIRECTORY
cd $DIRECTORY
binary
put $FILE
quit
END_SCRIPT
mkdir under ftp is implemented by the ftp server, not by calling /bin/mkdir no such options as -p,
what you should do is
mkdir foo
cd foo
mkdir bar
cd bar
mkdir baz
cd baz
If you still want your original construct, you can also do it like this:
#!/bin/bash
foo() {
local r
local a
r="$#"
while [[ "$r" != "$a" ]] ; do
a=${r%%/*}
echo "mkdir $a"
echo "cd $a"
r=${r#*/}
done
}
DIRECTORY="foo/bar/baz"
FILE="test.pdf"
HOST="testserver"
USER="test"
PASS="test"
ftp -n $HOST <<END_SCRIPT
quote USER $USER
quote PASS $PASS
$(foo "$DIRECTORY")
binary
put $FILE
quit
END_SCRIPT
Try lftp instead:
[dong#idc1-server1 ~]$ lftp sftp://idc1-server2
lftp idc1-server2:~> ls
drwxr-xr-x 3 dong dong 4096 Jun 16 09:11 .
drwxr-xr-x 18 root root 4096 Apr 1 22:25 ..
-rw------- 1 dong dong 116 Jun 16 09:28 .bash_history
-rw-r--r-- 1 dong dong 18 Oct 16 2013 .bash_logout
-rw-r--r-- 1 dong dong 176 Oct 16 2013 .bash_profile
-rw-r--r-- 1 dong dong 124 Oct 16 2013 .bashrc
drwx------ 2 dong dong 4096 Jul 24 2014 .ssh
lftp idc1-server2:~> mkdir a/b/c/d
mkdir: Access failed: No such file (a/b/c/d)
lftp idc1-server2:~> mkdir -p a/b/c/d
mkdir ok, `a/b/c/d' created