I have a zip file with name ***.zip. I used below command to unzip it. Once I unzip, the files inside them are also "Zip" files(More than 3 zip files). Could you please let me know how can I unzip even these files as well.
unzip zipFile: "$project_version",dir:"D:\\jenkins\\DEV\\extract\\project", quiet: true
Trying to do-
unzip dir: 'D:\\jenkins\\DEV\\extract\\project', glob: '', zipFile: 'D:\\jenkins\\DEV\\extract\\project\\project_*.zip'
Error Logs
java.io.IOException: D:\jenkins\DEV\extract\project\project_*.zip does not exist.
at org.jenkinsci.plugins.pipeline.utility.steps.zip.UnZipStepExecution.run(UnZipStepExecution.java:77)
at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
at hudson.security.ACL.impersonate(ACL.java:260)
at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Finished: FAILURE
Files under my directory after I unzip the main zip file.
05/16/2018 04:31 PM <DIR> .
05/16/2018 04:31 PM <DIR> ..
05/15/2018 12:51 PM 265,637 project-project1_1.0.0.24_bdd86e0c.zip
05/15/2018 12:51 PM 7,924,188 project-project2_1.4.0.130_43dce5e4.zip
05/15/2018 12:51 PM 6,862,842 project-project3_1.0.0.207_c7d5d471.zip
3 File(s) 15,052,667 bytes
2 Dir(s) 432,451,330,048 bytes free
I need similar command in windows-
for file in `ls 123_*.zip'; do unzip $file -d `echo $file | cut -d "." -f 1`; done
You can use pipeline utility step in the steps block
should work with both scripted and declarative styles of pipeline
steps {
unzip zipFile: 'file.zip', dir: '<directory>'
}
Add-Type -AssemblyName System.IO.Compression.FileSystem
function Unzip
{
param([string]$zipfile, [string]$outpath)
[System.IO.Compression.ZipFile]::ExtractToDirectory($zipfile, $outpath)
}
Unzip "C:\Temp\Powershell\Test.zip" "C:\Temp\Powershell\"
$DirName = Get-ChildItem "C:\Temp\Powershell\Test"
foreach ( $item in $DirName){
Unzip "C:\Temp\Powershell\Test\$item" "C:\Temp\Powershell\Test\"
}
bat 'for /R . %I in ("*.zip") do ( "C:\\Program Files\\7-Zip\\7z.exe" x -y -o"%~dpnI" "%~fI" )'
We can use this under Jenkins pipeline script to achieve above requirement
Related
I am writing bash script for Windows 7 and have limitation to bash.exe
mkdir -p does not work (command not found)
I am looking for any replacement/cheat with bash builtins. (the -p flag is not mandatory, could do this with a loop instead)
There is some idea to Transform a file into directory or some other nice hack only a expert will know. This question is not about how to fix a missing mkdir binary, but about reducing dependencies of (broken) core utils.
this are the available bash builtins:
& (( . : [ [[ alias bg bind break builtin caller case cd command compgen complete compopt continue coproc declare dirs disown echo enable eval exec exit export false fc fg for function getopts hash help history if jobs kill let local logout mapfile popd printf pushd pwd read readarray readonly return select set shift shopt source suspend test time times trap true type typeset ulimit umask unalias unset until wait while {
if not possible with bash builtins there are some external utils available. There is already one example which depends on cp
Create a new folder using bash without mkdir command
this is all i have:
bash blobpack blobunpack bzip2 cat chmod clear cpio cut dd dhtbsign dos2unix dumpimage elftool expr file find futility grep gzip hexdump kernel_dump loki_tool ls lz4 lzop mac2unix mboot md5sum mkbootimg mkimage mkmtkhdr mv printf pxa-mkbootimg pxa-unpackbootimg rkcrc rm sed stat sudo tail tar touch unix2dos unix2mac unpackbootimg unpackelf xz
Note: I have edited the question to clarify the needs.
If I'm reading the contents of that pastebin.com link correctly, this is not a true/complete cygwin installation.
That link shows the following (in a windows environment):
PATH = ... C:\Android\bash ... # this is not where cygwin is typically installed though, yeah, you could override the default installation directory
C:\Users\mint>bash # this is not how `cygwin/bash` is invoked
bash-4.1$
bash-4.1$ ls /cygdrive/c/Android/bash # full cygwin install does not throw everything under a single directory like this:
bash.exe ... snip ... mv.exe
At this point this doesn't appear to be an actual cygwin installation but rather some sort of reduced/incomplete/bastardized bash installation under the C:\Android\bash directory.
Where/How did OP 'install' the contents under the C:\Android\bash directory?
For comparison purposes (my windows machine):
# cygwin installation directory:
C:\cygwin64>dir
Volume in drive C is Windows7
Volume Serial Number is xxxx-yyyy
Directory of C:\cygwin64
11/06/2020 12:29 <DIR> .
11/06/2020 12:29 <DIR> ..
03/30/2021 16:08 <DIR> bin
05/25/2019 17:15 53,342 Cygwin-Terminal.ico
05/25/2019 18:46 95 Cygwin.2.bat
05/25/2019 17:15 88 Cygwin.bat
05/25/2019 17:15 157,097 Cygwin.ico
02/08/2021 13:01 <DIR> dev
02/04/2021 12:06 <DIR> etc
02/26/2021 16:35 <DIR> home
02/02/2021 11:34 <DIR> lib
07/12/2020 17:28 <DIR> sbin
11/06/2020 12:29 <DIR> srv
05/07/2021 07:46 <DIR> tmp
02/02/2021 11:34 <DIR> usr
05/25/2019 17:15 <DIR> var
6 File(s) 210,622 bytes
12 Dir(s) 20,305,154,048 bytes free
C:\cygwin64\bin>dir
Volume in drive C is Windows7
Volume Serial Number is xxxx-yyyy
Directory of C:\cygwin64\bin
03/30/2021 16:08 <DIR> .
03/30/2021 16:08 <DIR> ..
... snip ...
02/03/2017 14:40 37,395 base64.exe
02/03/2017 14:40 29,715 basename.exe
01/27/2017 14:13 739,859 bash.exe # bash binary
01/27/2017 14:13 7,291 bashbug
10/17/2014 17:00 81,949 bc.exe
... snip ...
12/20/2020 17:01 11,564 mintheme
02/03/2017 14:40 62,995 mkdir.exe # mkdir binary
02/03/2017 14:40 29,715 mkfifo.exe
08/22/2020 14:00 21,523 mkgroup.exe
... snip ...
2/19/2020 11:37 30 zstdless
2/03/2017 14:41 64,019 [.exe
1130 File(s) 541,740,761 bytes # 1100+ binaries in this directory
2 Dir(s) 20,305,154,048 bytes free
# example cygwin session startup
C:\cygwin64\bin\mintty.exe -i /Cygwin-Terminal.ico -
# user has option to startup a few different tty's;
# actual OS is determined from /etc/passwd entry (/usr/bin/bash in my case)
At this point if OP wants to run cygwin/bash then I'd suggest installing an actual cygwin environment (see cygwin.org ), making sure to also install the bash package.
As for how to simulate mkdir with ... whatever is installed under C:\Android\bash ... shrug ... have the originator pull more of the binaries (eg, mkdir.exe) from a complete cygwin/bash installation?
----------------------- previous answer (before reviewing the contents of the pastebin.com link)
I'm wondering if this could be an issue of an incomplete/corrupted cygwin/bash installation or an invalid $PATH ... ?
From my cygwin environment:
$ bash --version
GNU bash, version 4.4.12(3)-release (x86_64-unknown-cygwin)
$ which mkdir
/usr/bin/mkdir
$ command -v mkdir
/usr/bin/mkdir
$ mkdir --version
mkdir (GNU coreutils) 8.26
Packaged by Cygwin (8.26-2)
Does /usr/bin/mkdir exist?
What is returned when running find / -name mkdir*?
If you can find mkdir(.exe) then the next check would be for the location/path also being defined in $PATH ...
This trick can be helpful to run Windows md shell builtin from within bash script
echo -e 'md %*\r' > mkdir.bat
./mkdir.bat test
I am experiencing an issue where MSYS2 running in docker can create and modify files in a mounted volume, but not delete them. This can be observed with commands where the deletion is explicit, such as rm a.txt, or with commands where the deletion is implicit, such as sed -i 's/foo/bar/' b.txt (b.txt should be edited in-place, and appears to be read, removed and recreated from the output (1)). The same files can be deleted with the equivalent DOS commands, such as del a.txt.
Using the following Dockerfile:
# Altered from https://github.com/StefanScherer/dockerfiles-windows/blob/main/msys2/Dockerfile
FROM mcr.microsoft.com/windows/servercore:20H2
RUN powershell -Command \
$ProgressPreference = 'SilentlyContinue' ; \
Write-Output 'Downloading msys2' ; \
curl.exe -o msys2.tar.xy http://repo.msys2.org/distrib/msys2-x86_64-latest.tar.xz ; \
Install-Package -Scope CurrentUser -Force 7Zip4Powershell ; \
Write-Output 'Extracting tar.xz' ; \
Expand-7zip msys2.tar.xy . ; \
Write-Output 'Extracting tar' ; \
Expand-7zip msys2.tar C:/ ; \
Write-Output 'Done'
RUN powershell -Command \
setx /M PATH $('C:\msys64\mingw64\bin;C:\msys64\usr\bin;C:\msys64\mingw32\bin;' + $Env:PATH)
Build and launch the container with:
docker pull mcr.microsoft.com/windows/servercore:20H2
docker build -t docker-msys .
mkdir local
docker run -ti -v "$(pwd)/local:C:/local" docker-msys cmd.exe
Using MSYS2 commands from within the container, lets manipulate files with:
touch a.txt
cp a.txt b.txt
rm a.txt
rm -f b.txt
ls *.txt
If launched from a non-mounted directory, the following is observed:
C:\>touch a.txt
C:\>cp a.txt b.txt
C:\>rm a.txt
C:\>rm -f b.txt
C:\>ls *.txt
ls: cannot access '*.txt': No such file or directory
Both files have been deleted as expected.
If launched from a mounted directory, here C:\local, the following is observed:
cd C:\local
C:\local>touch a.txt
C:\local>cp a.txt b.txt
C:\local>rm a.txt
rm: cannot remove 'a.txt': Invalid argument
C:\local>rm -f b.txt
C:\local>ls *.txt
a.txt b.txt
Both files still remain.
Other MSYS commands will have unexpected behavior in the mounted directory:
C:\local>sed -i 's/foo/bar/' b.txt
sed: cannot remove ./sed7E6QmG: Invalid argument
The files can be removed from the mounted directory with the equivalent dos command:
C:\local>del /Q *
C:\local>dir
Volume in drive C has no label.
Volume Serial Number is 541F-A7C7
Directory of C:\local
11/26/2020 04:05 PM <DIR> .
11/26/2020 04:05 PM <DIR> ..
0 File(s) 0 bytes
2 Dir(s) 48,827,674,624 bytes free
Changing from a mounted directory to a named volume affect in nothing the situation. The previous output is the same when the container is launched with:
docker volume create local-volume
docker run -ti -v local-volum:C:/local docker-msys cmd
The problem doesn't appear related to my Dockerfile. I was able to reproduce it using a container from https://hub.docker.com/r/mizarjp/winci-msys2 .
docker pull mizarjp/winci-msys2:1909
docker run -ti --rm -v "$(pwd)/local:C:/local" mizarjp/winci-msys2:1909 cmd.exe
Once in the container, perform set PATH=C:\msys64\mingw64\bin;C:\msys64\usr\bin;C:\msys64\mingw32\bin;%PATH%; before repeating the previous tests to obtain the same results.
Finally, rather than installing MSYS2 inside docker, mounting it's directory inside a container produce the same situation.
Is there anything I am missing?
Details:
Windows 10 Pro, 20H2, 19042.746
Docker Desktop 3.1.0 (51484)
MSYS2 (http://repo.msys2.org/distrib/msys2-x86_64-latest.tar.xz) 2020-07-03 19:34
(1) My humble interpretation!
Adding --isolation=process fixed it. rm no longer fails for me!
docker pull mcr.microsoft.com/windows/servercore:20H2
docker build -t docker-msys .
mkdir local
docker run --rm -ti --isolation=process -v "$(pwd)/local:C:/local" docker-msys cmd.exe
Re-running the previous commands produces:
C:\>cd local
C:\local>touch a.txt
C:\local>cp a.txt b.txt
C:\local>rm a.txt
C:\local>rm -f b.txt
C:\local>ls *.txt
ls: cannot access '*.txt': No such file or directory
I'm building a gitlab ci pipeline, and try to create a directory if it not exists.
Can anybody tell me what I'm doing wrong here?
$ if [ ! -d aws ]
$ then
$ mkdir aws
mkdir: cannot create directory ‘aws’: File exists
ERROR: Job failed: exit code 1
the relevant part of the gitlab-ci.yml
script:
- export
- ls -al
- if [ ! -d aws ]
- then
- mkdir aws
- fi
$ ls -al
total 128
drwxrwxrwx 16 root root 4096 Sep 17 12:07 .
drwxrwxrwx 6 root root 4096 Sep 17 12:07 ..
drwxrwxrwx 2 root root 4096 Sep 17 12:07 aws
I now just used mkdir -p and removed the test
you have something with aws name, which might be symbolic link, hard link, regular file, vs.
first delete or move that file to somewhere else then try again
you can try -e (returns true if file exists regardless of type).
I'm trying to run the unzip shell command from within Groovy.
I'm running the command
"unzip ~/Documents/myFile.txt.zip -d ~/Documents/".execute()
but it's not working. When I copy the exact command into my terminal it works. How can I do this from groovy?
There's no ~ as far as Groovy is concerned; use an actual path.
groovy:000> p = "ls -CF /Users/Dave".execute()
===> java.lang.UNIXProcess#2603826d
groovy:000> p.waitFor()
===> 0
groovy:000> p.in.text
===> Desktop/ Movies/ bin/
Documents/ Music/ node_modules/
Downloads/ Pictures/
Dropbox/ Public/
Library/ ScreenshotOnFail/
You can always use System.getProperty("user.home"), e.g.,
p = "ls -CF ${System.getProperty('user.home')}".execute()
Basically my question is how to use bash shell command to do following automatically, so I can track modified files easily.
list svn check-out files
create link files to above files in an directory called "change"
laptop$ svn status -q
M rcms/src/config/ta_show.c
M rcms/src/config/ta_config.c
laptop$ cd change
laptop$ link -s ../rcms/src/config/ta_show.c ta_show.c
laptop$ link -s ../rcms/src/config/ta_config.c ta_config.c
laptop$ ls
lrwxrwxrwx 1 root root 59 Nov 27 12:24 ta_show.c -> ../rcms/src/config/ta_show.c
lrwxrwxrwx 1 root root 59 Nov 27 12:24 ta_config.c -> ../rcms/src/config/ta_config.c
I am thinking to use shell command like below:
$ svn status -q | sed 's/M //' | xargs -I xxx ln -s ***BETWEEN REAL FILE AND BASE FILENAME***
you have two things need to be concerned:
the empty line between each file with svn status 'M'
extract the file name
the awk one liner could do it:
awk '$0{x=$2;gsub(".*/","",x);print "ln -s ../"$2" "x}'
so if you pipe your svn status output to the line above, it print the ln -s command lines for you.
if you want the ln -s lines to get executed, you could either pipe the output to sh (svn status|awk ...|sh) or replace the print with system
at the end i would like to show the output below as an exmple:
kent$ echo "M rcms/src/config/ta_show.c
M rcms/src/config/ta_config.c"|awk '$0{x=$2;gsub(".*/","",x);print "ln -s .."$2" "x}'
ln -s ../rcms/src/config/ta_show.c ta_show.c
ln -s ../rcms/src/config/ta_config.c ta_config.c