I'm trying to run the unzip shell command from within Groovy.
I'm running the command
"unzip ~/Documents/myFile.txt.zip -d ~/Documents/".execute()
but it's not working. When I copy the exact command into my terminal it works. How can I do this from groovy?
There's no ~ as far as Groovy is concerned; use an actual path.
groovy:000> p = "ls -CF /Users/Dave".execute()
===> java.lang.UNIXProcess#2603826d
groovy:000> p.waitFor()
===> 0
groovy:000> p.in.text
===> Desktop/ Movies/ bin/
Documents/ Music/ node_modules/
Downloads/ Pictures/
Dropbox/ Public/
Library/ ScreenshotOnFail/
You can always use System.getProperty("user.home"), e.g.,
p = "ls -CF ${System.getProperty('user.home')}".execute()
Related
Bash script:
clonePath=/data/config/
git branch -r | fgrep -v 'origin/HEAD' | sed 's| origin/|git checkout |' > checkoutAllBranches.sh
chmod +x checkoutAllBranches.sh
echo "Fetch branch: `cat checkoutAllBranches.sh`"
./checkoutAllBranches.sh
git checkout master
git remote rm origin
rm checkoutAllBranches.sh
for config_dir in `ls -a`; do
cp -r $config_dir $clonePath/;
done
echo "API Config update complete..."
Dockerfile which issues this script execution
ENTRYPOINT ["sh","config-update-force.sh","|| true"]
The error below causes the container startup failure despite setting the command status to 0 manually using || true
ERROR:
Error:
cp: cannot create regular file '/data/./.git/objects/pack/pack-27a9d...fb5e368e4cf.pack': Permission denied
cp: cannot create regular file '/data/./.git/objects/pack/pack-27a9d...fbae25e368e4cf.idx': Permission denied
I am looking for 2 options here:
Change these file permissions and then store them in the remote with rwx permissions
Do something to the docker file to ignore this script failure error and start the container.
DOCKERFILE:
FROM docker.hub.com/java11-temurin:latest
USER root
RUN apt-get update
RUN apt-get -y upgrade
RUN apt-get install -y rsync telnet vim wget git
RUN mkdir -p /opt/config/clone/data
RUN chown -R 1001:1001 /opt/config
USER 1001
ADD build/libs/my-api-config-server.jar .
ADD config-update-force.sh .
USER root
RUN chmod +x config-update-force.sh
USER 1001
EXPOSE 8080
CMD java $BASE_JAVA_OPTS $JAVA_OPTS -jar my-api-config-server.jar
ENTRYPOINT ["sh","config-update-force.sh","|| true"]
BASH SCRIPT:
#!/bin/bash
set +e
set +x
clonePath=/opt/clone/data/data
#source Optumfile.properties
echo "properties loaded: example ${git_host}"
if [ -d my-api-config ]; then
rm -rf my-api-config;
echo "existing my-api-config dir deleted..."
fi
git_url=https://github.com/my-api-config-server
git clone https://github.com/my-api-config-server
cd my-api-config-server
git branch -r | fgrep -v 'origin/HEAD' | sed 's| origin/|git checkout |' > checkoutAllBranches.sh
chmod +x checkoutAllBranches.sh
echo "Fetch branch: `cat checkoutAllBranches.sh`"
./checkoutAllBranches.sh
git checkout master
git remote rm origin
rm checkoutAllBranches.sh
for config_dir in `ls -a`; do
cp -r $config_dir $clonePath/;
done
echo "My API Config update complete..."
When you do in the script...
chmod +x checkoutAllBranches.sh
...than why not before cp
chmod -R +rwx ${clonePath}
...or if the stderr message 'wont impact anything'...
cp -r $config_dir $clonePath/ 2>/dev/null;
...even cp dont copy -verbosly.
?
When your Dockerfile declares an ENTRYPOINT, that command is the only thing the container does. If it also declares a CMD, the CMD is passed as additional arguments to the ENTRYPOINT; it is not run on its own unless the ENTRYPOINT makes sure to execute it.
Shell errors are not normally fatal, and especially if you explicitly set +e, even if a shell command fails the shell script will keep running. You see this in your output where you get multiple cp errors; the first error does not terminate the script.
You need to do two things here. The first is to set the ENTRYPOINT to actually run the CMD; the simplest and most common way to do this is to end the script with
exec "$#"
The second is to remove the || true from the Dockerfile. As you have it written out currently, this is passed as the first argument to the entrypoint wrapper – it is not run through a shell and it is not interpreted as a "or" operator. If your script begins with a "shebang" line and is marked executable (both of these are correct in the question) the you do not explicitly need the sh interpreter.
# must be a JSON array; no additional "|| true" argument; no sh -c wrapper
ENTRYPOINT ["./config-update-force.sh"]
# any valid CMD will work with `exec "$#"
CMD java $BASE_JAVA_OPTS $JAVA_OPTS -jar my-api-config-server.jar
I am writing bash script for Windows 7 and have limitation to bash.exe
mkdir -p does not work (command not found)
I am looking for any replacement/cheat with bash builtins. (the -p flag is not mandatory, could do this with a loop instead)
There is some idea to Transform a file into directory or some other nice hack only a expert will know. This question is not about how to fix a missing mkdir binary, but about reducing dependencies of (broken) core utils.
this are the available bash builtins:
& (( . : [ [[ alias bg bind break builtin caller case cd command compgen complete compopt continue coproc declare dirs disown echo enable eval exec exit export false fc fg for function getopts hash help history if jobs kill let local logout mapfile popd printf pushd pwd read readarray readonly return select set shift shopt source suspend test time times trap true type typeset ulimit umask unalias unset until wait while {
if not possible with bash builtins there are some external utils available. There is already one example which depends on cp
Create a new folder using bash without mkdir command
this is all i have:
bash blobpack blobunpack bzip2 cat chmod clear cpio cut dd dhtbsign dos2unix dumpimage elftool expr file find futility grep gzip hexdump kernel_dump loki_tool ls lz4 lzop mac2unix mboot md5sum mkbootimg mkimage mkmtkhdr mv printf pxa-mkbootimg pxa-unpackbootimg rkcrc rm sed stat sudo tail tar touch unix2dos unix2mac unpackbootimg unpackelf xz
Note: I have edited the question to clarify the needs.
If I'm reading the contents of that pastebin.com link correctly, this is not a true/complete cygwin installation.
That link shows the following (in a windows environment):
PATH = ... C:\Android\bash ... # this is not where cygwin is typically installed though, yeah, you could override the default installation directory
C:\Users\mint>bash # this is not how `cygwin/bash` is invoked
bash-4.1$
bash-4.1$ ls /cygdrive/c/Android/bash # full cygwin install does not throw everything under a single directory like this:
bash.exe ... snip ... mv.exe
At this point this doesn't appear to be an actual cygwin installation but rather some sort of reduced/incomplete/bastardized bash installation under the C:\Android\bash directory.
Where/How did OP 'install' the contents under the C:\Android\bash directory?
For comparison purposes (my windows machine):
# cygwin installation directory:
C:\cygwin64>dir
Volume in drive C is Windows7
Volume Serial Number is xxxx-yyyy
Directory of C:\cygwin64
11/06/2020 12:29 <DIR> .
11/06/2020 12:29 <DIR> ..
03/30/2021 16:08 <DIR> bin
05/25/2019 17:15 53,342 Cygwin-Terminal.ico
05/25/2019 18:46 95 Cygwin.2.bat
05/25/2019 17:15 88 Cygwin.bat
05/25/2019 17:15 157,097 Cygwin.ico
02/08/2021 13:01 <DIR> dev
02/04/2021 12:06 <DIR> etc
02/26/2021 16:35 <DIR> home
02/02/2021 11:34 <DIR> lib
07/12/2020 17:28 <DIR> sbin
11/06/2020 12:29 <DIR> srv
05/07/2021 07:46 <DIR> tmp
02/02/2021 11:34 <DIR> usr
05/25/2019 17:15 <DIR> var
6 File(s) 210,622 bytes
12 Dir(s) 20,305,154,048 bytes free
C:\cygwin64\bin>dir
Volume in drive C is Windows7
Volume Serial Number is xxxx-yyyy
Directory of C:\cygwin64\bin
03/30/2021 16:08 <DIR> .
03/30/2021 16:08 <DIR> ..
... snip ...
02/03/2017 14:40 37,395 base64.exe
02/03/2017 14:40 29,715 basename.exe
01/27/2017 14:13 739,859 bash.exe # bash binary
01/27/2017 14:13 7,291 bashbug
10/17/2014 17:00 81,949 bc.exe
... snip ...
12/20/2020 17:01 11,564 mintheme
02/03/2017 14:40 62,995 mkdir.exe # mkdir binary
02/03/2017 14:40 29,715 mkfifo.exe
08/22/2020 14:00 21,523 mkgroup.exe
... snip ...
2/19/2020 11:37 30 zstdless
2/03/2017 14:41 64,019 [.exe
1130 File(s) 541,740,761 bytes # 1100+ binaries in this directory
2 Dir(s) 20,305,154,048 bytes free
# example cygwin session startup
C:\cygwin64\bin\mintty.exe -i /Cygwin-Terminal.ico -
# user has option to startup a few different tty's;
# actual OS is determined from /etc/passwd entry (/usr/bin/bash in my case)
At this point if OP wants to run cygwin/bash then I'd suggest installing an actual cygwin environment (see cygwin.org ), making sure to also install the bash package.
As for how to simulate mkdir with ... whatever is installed under C:\Android\bash ... shrug ... have the originator pull more of the binaries (eg, mkdir.exe) from a complete cygwin/bash installation?
----------------------- previous answer (before reviewing the contents of the pastebin.com link)
I'm wondering if this could be an issue of an incomplete/corrupted cygwin/bash installation or an invalid $PATH ... ?
From my cygwin environment:
$ bash --version
GNU bash, version 4.4.12(3)-release (x86_64-unknown-cygwin)
$ which mkdir
/usr/bin/mkdir
$ command -v mkdir
/usr/bin/mkdir
$ mkdir --version
mkdir (GNU coreutils) 8.26
Packaged by Cygwin (8.26-2)
Does /usr/bin/mkdir exist?
What is returned when running find / -name mkdir*?
If you can find mkdir(.exe) then the next check would be for the location/path also being defined in $PATH ...
This trick can be helpful to run Windows md shell builtin from within bash script
echo -e 'md %*\r' > mkdir.bat
./mkdir.bat test
I am trying to create tar from a file, which contains list of other files and saving it to stdout.
let suppose there is a file called "files-to-create" which has path of other files like /home/abc.txt /home/def.txt and I want to create tar of abc.txt,def.txt.
my script contains:
exec 100>&1
tar cf - -T files-to-sync >&100
and I am calling the script and saving it to some other file like:
/script.sh > final_tar.tar
But while creating the tar I am getting error, can somebody help me out?
You can use the following script to reach your goal, let me know if something is unclear:
Prototype 1:
$ cat scriptTar.sh
#!/bin/bash
readonly HELP="$(basename "$0") <list_of_files> <output_tar>
this script will generate a tar file composed of all files present in <list_of_files> input file
the output tar file will be saved as <output_tar>
to run the script provide the input and output filenames"
readonly INPUT_LIST_FILE=$1
readonly OUTPUT_TAR_FILE=$2
if [ -z "$INPUT_LIST_FILE" -o -z "$OUTPUT_TAR_FILE" ]
then
echo $HELP;
exit 1;
fi
tar cf - -T $INPUT_LIST_FILE > $OUTPUT_TAR_FILE
exit $?
Folder content:
$ tree .
.
├── a
│ └── abc.txt
├── b
│ └── def.txt
├── c
│ └── ghj.txt
├── files-to-sync.in
└── scriptTar.sh
3 directories, 5 files
List file content:
$ cat files-to-sync.in
./a/abc.txt
./b/def.txt
./c/ghj.txt
Execution:
$ ./scriptTar.sh files-to-sync.in output.tar
tar file content:
$ tar -tvf output.tar
-rw-rw-r-- arobert/arobert 4 2018-02-22 16:50 ./a/abc.txt
-rw-rw-r-- arobert/arobert 4 2018-02-22 16:50 ./b/def.txt
-rw-rw-r-- arobert/arobert 4 2018-02-22 16:50 ./c/ghj.txt
Or use the following script if you really want to display it on stdout:
Prototype 2 via ssh:
#!/bin/bash
readonly HELP="ERROR: $(basename "$0") <list_of_files>
this script will generate to stdout a tar file composed of all files present in <list_of_files> input file
to run the script provide the input file and redirect the output to a file"
readonly INPUT_LIST_FILE=$1
if [ -z "$INPUT_LIST_FILE" ]
then
echo $HELP;
exit 1;
fi
tar cf - -T $INPUT_LIST_FILE
Execution via ssh:
$ ssh user#localhost "cd /home/user/test_tar/; ./scriptTar.sh files-to-sync.in" > output.tar
user#localhost's password:
Content of the tar generated:
tar -tf output.tar
./a/abc.txt
./b/def.txt
./c/ghj.txt
extracting the content:
tar xvf output.tar
./a/abc.txt
./b/def.txt
./c/ghj.txt
checking the files:
more ?/*.txt
::::::::::::::
a/abc.txt
::::::::::::::
abc
::::::::::::::
b/def.txt
::::::::::::::
abc
::::::::::::::
c/ghj.txt
::::::
However if I were you, I would not only generate a tar file but add some compression (tar.gz) and transfer the file with rsync to be able to restart the download from the point where it stopped in case of transfer error.
So the proper solution is
Case1: If you are passing the list of file as an argument
you can use this:
files-to-sync=$1
tar cf - -T files-to-sync
Case2: If you want to use absolute path for the list of file
you can use this:
tar cfP - -T /path/to/the/file
use -P in case of absolute path.
This problem is killing me and I feel like I've tried everything.
First off, the problem started happening when upgrading to Capistrano 3. Capistrano now utilizes /usr/bin/env before every command when deploying, to make sure the environment setup is correct.
When Capistrano goes to create symlinks to the necessary shared directory and respective files, it attempts commands like:
/usr/bin/env ln -s /full/path /different/full/path
...and then it errors out:
/usr/bin/env: ln: Too many levels of symbolic links
I realize it's not Capistrano's fault, so I began troubleshooting by ssh'ing to my server and trying the same command, and I receive the same error (which at least is good for consistency). I then try the same command without /usr/bin/env:
ln -s /full/path /different/full/path
And it works!!!! Maybe you can see the real solution that I can't?
here is the output of just the /usr/bin/env command:
rvm_bin_path=/home/deployer/.rvm/bin
GEM_HOME=/home/deployer/.rvm/gems/ruby-1.9.3-p392
TERM=xterm-256color
SHELL=/bin/bash
IRBRC=/home/deployer/.rvm/rubies/ruby-1.9.3-p392/.irbrc
SSH_CLIENT=...
OLDPWD=/home/deployer/Sites/example.com
MY_RUBY_HOME=/home/deployer/.rvm/rubies/ruby-1.9.3-p392
SSH_TTY=/dev/pts/0
USER=deployer
LS_COLORS= .....
_system_type=Linux
rvm_path=/home/deployer/.rvm
SSH_AUTH_SOCK=....
rvm_prefix=/home/deployer
MAIL=/var/mail/deployer
PATH=/home/deployer/.rvm/gems/ruby-1.9.3-p392/bin:/home/deployer/.rvm/gems/ruby-1.9.3-p392#global/bin:/home/deployer/.rvm/rubies/ruby-1.9.3-p392/bin:/home/deployer/.rvm/bin:/opt/rubyee/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/home/deployer/.rvm/bin
PWD=/home/deployer/Sites
LANG=en_US.UTF-8
_system_arch=i386
_system_version=12.04
rvm_version=1.26.4 (latest)
SHLVL=1
HOME=/home/deployer
LOGNAME=deployer
GEM_PATH=/home/deployer/.rvm/gems/ruby-1.9.3-p392:/home/deployer/.rvm/gems/ruby-1.9.3-p392#global
SSH_CONNECTION=....
LESSOPEN=| /usr/bin/lesspipe %s
LESSCLOSE=/usr/bin/lesspipe %s %s
RUBY_VERSION=ruby-1.9.3-p392
_system_name=Ubuntu
_=/usr/bin/env
I have also tried commands like the following, to find potential symlink loops:
find . -maxdepth 20 -type l -exec ls -ld {} +
But is not producing correct results:
lrwxrwxrwx 1 deployer deployer ...
You might not being using the same ln utility.
When invoking it directly from the interactive shell, ln might be overridden e.g. by an alias or by some shell function ln() {...;}.
This does not happen when /usr/bin/env tries to do that (AFAIK it looks for ln in PATH). I suspect that the ln it finds has issues, so you are getting this error.
This is an example scenario that might be similar to your case:
# start from an empty directory
$ ls -l
total 0
# create a problematic `ln` in the current directory
$ ln -s ln ln
$ ls -l
total 0
lrwxrwxrwx 1 me me 2 Jan 7 20:28 ln -> ln
# have an alias for the "real" ln
$ alias ln=/bin/ln
# mess up PATH
$ PATH="$PWD"
Now let's try the two alternatives, /usr/bin/env goes first:
$ /usr/bin/env ln -s /some/path /tmp/path
/usr/bin/env: ln: Too many levels of symbolic links
Then plain ln (remember that we aliased it):
$ ln -s /some/path /tmp/path
$ echo $?
0
$ /bin/ls -l /tmp/path
lrwxrwxrwx 1 me me 10 Jan 7 20:31 /tmp/path -> /some/path
So my suggestion is: look at issues with ln, e.g. by finding all different alternatives that might be visible. In bash you might run this:
$ type -a ln
Try this to find symlink loops:
find . -follow -printf ""
if you are using docker so you should install ruby in your case
docker run ruby
source
https://github.com/docker/for-win/issues/5763#issuecomment-585749243
I am trying to get alias's setup so that they print out the command, then run the command.
Ex:
> alias ls='ls -alh'
> ls
Running "ls -alh"
total 1.8G
drwxr-x--- 36 root root 4.0K Apr 23 09:44 ./
drwxr-xr-x 28 root root 4.0K Mar 6 17:24 ../
Is this possible? I was thinking of using a wrapper function, but I am unsure as to how one would acomplish this.
Thanks!
Just add an echo command in your alias before the actual command:
alias ls='echo "Running ls -alh"; ls -alh'
alias ls='echo "Running ls -alh" && ls -alh'
This runs two commands one after the other. The first command is echo "Running ls -alh", the && checks the return value of the echo command, if that's 0, then the command ls -alh is run. However, if for some reason there is a problem with the echo command and its return value is not 0 then the ls command won't be run.
The && command can come in very handy when writing scripts to run one command only when another is successful.