Hi I have 2 parent directories with these contents, under /tmp:
Note parent directory names have ";" in it- not recommended in Unix like systems, but those directories are pushed by an external application, and that's the way we have to deal with it.
I need to move these parent directories (along with their contents) to /tmp/archive - on a RHEL 7.9 (Maipo) machine
My simple code:
ARCHIVE="/tmp/archive"
[ -d "${ARCHIVE}" ] && mkdir -p "${ARCHIVE}"
ls -lrth /tmp | awk "\$NF ~ /2021-.*/{print \$NF}" | xargs -I "{}" mv "{}" ${ARCHIVE}/
But when I run this script, mv copies one of the parent directory as it is, but for the other one, it just moves the contents of the parent directory, not the directory itself:
I tried the same script with cp -pvr command in place of mv, and its the same behavior
When I run the same script in a Ubuntu 18 system, the behavior is as expected i.e - the parent directories get moved to archive folder.
Why is there this difference in behavior between a Ubuntu and a RHEL system, for the same script
Try a simpler approach:
mkdir -p /tmp/archive
mv -v /tmp/2021-*\;*\;*/ /tmp/archive
Related
I have the following executable bash script
#!/usr/bin/env bash
function testRpm(){
local rpm=$1
local tempDir=$(mktemp -d)
pushd $tempDir #>/dev/null
rpm2cpio $rpm | cpio -idmuv
find store -name "*.jar"
}
testRpm $1
It seems pretty straight forward to me, extract the RPM, show the files. The problem is when I run it, the find doesn't show the files, it shows the directories though. If I manually enter the commands it works great.
eg.
bash -x ./test.sh myrpm.rpm
+ testRpm myrpm.rpm
+ local rpm=myrpm.rpm
++ mktemp -d
+ local tempDir=/var/folders/z4/7cl6z4_x5vq1dllx8l6vf73r0000gn/T/tmp.YWdEnKUG
+ pushd /var/folders/z4/7cl6z4_x5vq1dllx8l6vf73r0000gn/T/tmp.YWdEnKUG
/var/folders/z4/7cl6z4_x5vq1dllx8l6vf73r0000gn/T/tmp.YWdEnKUG
~/IdeaProjects
+ rpm2cpio myrpm.rpm
+ cpio -idmuv
./store/tmp/myfile1
./store/tmp/myfile2
33279 blocks
+ find store
store
store/tmp
The above script appears to work perfectly on Redhat, but not macos. If anyone has any suggestions, tips or solutions, I'd appreciate it.
What may be happening is that when you pushd, you're losing access to the RPM file, e.g. when you're in /home/me, "foo.rpm" refers to /home/me/foo.rpm but when you change directory to /tmp, "foo.rpm" now refers to /tmp/foo.rpm.
Solve this by using the absolute path to the RPM when extracting using realpath:
#!/usr/bin/env bash
function testRpm(){
local rpm=$(realpath -- "$1")
local tempDir=$(mktemp -d)
pushd $tempDir #>/dev/null
rpm2cpio "$rpm" | cpio -idmuv
find store -name "*.jar"
}
testRpm "$1"
I am working on bash to create a back up system. My code is
#!/bin/bash
if [ ! -d "BackUp" ]
then
mkdir BackUp
fi
echo "enter number of access days you want to take for back up."
read days
bak="$(find . -mtime +$days)"
for file in $bak
do
mv $file BackUp
done
tar -cvf BackUp.tgz BackUp >> backUp.log
So, currently I am only taking log file from tar. so it does not prints the full path it only takes current working directory for text in log file.My last line of code takes up input for log file.
But the path stored is
.BackUp/foo1
.BackUp/foo2
.BackUp/foo3
instead i want it to be
home/ubuntu/Downloads/BackUp/foo1
home/ubuntu/Downloads/BackUp/foo2
home/ubuntu/Downloads/BackUp/foo3
You could store the absolute path in a variable and use it in the tar command:
BackUpDirFullPath=$(cd BackUp && pwd)
As command substitution invokes a subshell you are not leaving the current directory by executing cd.
Update:
In order to make -v output absolute paths (on Mac OS) I had to change to the root directory in a subshell and execute it from there ... something like that:
(cd / && tar -cvf /$OLDPWD/BackUp.tgz $BackUpDirFullPath)
This does output absolute paths ... in order to preserve the leading / you might try -P which preserves path names.
I'm trying to make this bash script but get this: Error reading *.docx. The file doesn’t exist
Here's the script:
#!/bin/bash
textutil -convert txt *.docx
cat *.txt | wc -w
I'm currently running it from the folder but I'd like to make it a global script I can just call from any current folder.
If you want to make it available on your whole system you need to move it to a bin location like so
chmod a+rx yourscript.sh && sudo mv yourscript.sh /usr/local/bin/yourscript
then you can use it like a normal script in any folder
I have seen several questions about this topic, but I lack the ability to translate this to my specific problem. I have a for loop that loops through sub directories and then executes a .sh script on a compressed text file inside each directory. I want to parallelize this process, but I'm struggling to apply gnu parallel.
Here is my loop:
for d in ./*/ ; do (cd "$d" && script.sh); done
I understand I need to input a list into parallel, so i have been trying this:
ls -d */ | parallel cd && script.sh
While this appears to get started, I get an error when gzip tries to unzip one of the txt files inside the directory, saying the file does not exist:
gzip: *.txt.gz: No such file or directory
However, when I run the original for loop, I have no issues aside from it taking a century to finish. Also, I only get the gzip error once when using parallel, which is so weird considering I have over 1000 sub-directories.
My questions are:
How do I get Parallel to work in my case? How do I get parallel to parallelize the application of a .sh script to 1000s of files in their own sub-directories? ie- what is the solution to my problem? I gotta make progress.
What am I missing? Syntax, loop, bad script? I want to learn.
Is Parallel actually attempting to run all these .sh scripts in parallel? Why dont I get an error for every .txt.gz file?
Is parallel the best option for the application? Is there another option that is better suited to my needs?
Two problems:
In:
ls -d */ | parallel cd && script.sh
what is paralleled is just cd, not script.sh. script.sh is only executed once, after all parallel cd jobs have run, if there was no error. It is the same as:
ls -d */ | parallel cd
if [ $? -eq 0 ]; then script.sh; fi
You do not pass the target directory to cd. So, what is executed by parallel is just cd, which just changes the current directory to your home directory. The final script.sh is executed in the current directory (from where you invoked the command) where there are probably no *.txt.gz files, thus the error.
You can check yourself the effect of the first problem with:
$ mkdir /tmp/foobar && cd /tmp/foobar && mkdir a b c
$ ls -d */ | parallel cd && pwd
/tmp/foobar
The output of pwd is printed only once, even if you have more than one input directory. You can fix it by quoting the command and then check the second problem with:
$ ls -d */ | parallel 'cd && pwd'
/homes/myself
/homes/myself
/homes/myself
You should see as many pwd outputs as there are input directories but it is always the same output: your home directory. You can fix the second problem by using the {} replacement string that is substituted with the current input. Check it with:
$ ls -d */ | parallel 'cd {} && pwd'
/tmp/foobar/a
/tmp/foobar/b
/tmp/foobar/c
Now, you should have all input directories properly listed in the output.
For your specific problem this should work:
ls -d */ | parallel 'cd {} && script.sh'
i need to just find one section of the path to be used as the parent path in a bash script function.
for example
cd = /var/www/
pwd = /var/www/alpha/something/somewhere/somewhere
pwd = /var/www/1042/someplace/somehow
now i need to find the third section of this path
eg: migration, or 1042
to be the new root so i can tell bash to look for some other child folders no matter where and how deep i'm currently at.
so that if for example im at /8 of alpha/
pwd = /var/www/alpha/1/2/3/4/5/6/7/8
/a/b/c/d/
i can issue commands to /d quickly
Quick one liner to print the directory:
pwd | cut -d / -f 4
To actually run a command from that directory, use:
(cd `pwd | cut -d / -f 1-4`; your-command-here)