Modify a path stored in a bash script variable - bash

I have a variable f in a bash script
f=/path/to/a/file.jpg
I'm using the variable as an input argument to a program that requires and input and an output path.
For example the program's usage would look like this
./myprogram -i inputFilePath -o outputFilePath
using my variable, I'm trying to maintain the same basename, change the extension, and put the output file into a sub directory. For example
./myprogram -i /path/to/a/file.jpg -o /path/to/a/new/file.tiff
I'm trying to do that by doing this
./myprogram -i "$f" -o "${f%.jpg}.tiff"
of course this keeps the basename, changes the extension, but doesn't put the file into the new subdirectory.
How can I modify f to to change /path/to/a/file.jpg into /path/to/a/new/file.tiff?

Actually you can do this in several ways:
Using sed as pointed out by #anubhava
Using dirname and basename:
./myprogram -i "$f" -o "$(dirname -- "$f")/new/$(basename -- "$f" .jpg).tiff"
Using only Bash:
./myprogram -i "$f" -o "${f%/*}/new/$(b=${f##*/}; echo -n ${b%.jpg}.tiff)"
Note that unlike the second solution (using dirname/basename) that is more robust, the third solution (in pure Bash) won't work if "$f" does not contain any slash:
$ dirname "file.jpg"
.
$ f="file.jpg"; echo "${f%/*}"
file.jpg

You may use this sed:
s='/path/to/a/file.jpg'
sed -E 's~(.*/)([^.]+)\.jpg$~\1new/\2.tiff~' <<< "$s"
/path/to/a/new/file.tiff

If you're on a system that supports the basename and dirnamecommands you could use a simple wrapper function eg:
$ type newSubDir
newSubDir is a function
newSubDir ()
{
oldPath=$(dirname "${1}");
fileName=$(basename "${1}");
newPath="${oldPath}/${2}/${fileName}";
echo "${newPath}"
}
$ newSubDir /path/to/a/file.jpg new
/path/to/a/new/file.jpg
If your system doesn't have those, you can accomplish the same thing using string manipulation:
$ file="/path/to/a/file.jpg"
$ echo "${file%/*}"
/path/to/a
$ echo "${file##*/}"
file.jpg

Related

How to write a Bash script to edit many text files using the same commands? [duplicate]

This question already has answers here:
Run script on multiple files
(3 answers)
Closed 3 years ago.
I'm very new to bash. I have ten text files that I want to edit with the same line of code.
#!/bin/bash
sed -i -e 's/.\{6\}/&\n/g' -e 's/edit/edit2/g' | tr -d "\n" | sed 's/edit2/edit/g'| grep -o "here.*there" | sed -r '/^.{,100}$/d'
< files 1-10
I know I could use sed -f sed.sh <file1 >file1 but that only works with sed commands and it only works one file at a time?
Do I have to run a loop?
There's some great existing answers on the Unix stack exchange that help deal with your problem. Specifically, from this post, they use a loop to recursively loop through all the files in a particular directory, as follows:
( shopt -s globstar dotglob;
for file in **; do
if [[ -f $file ]] && [[ -w $file ]]; then
sed -i -- 's/foo/bar/g' "$file"
fi
done
)
Note the line, shopt -s globstar dotglob;, which allows us to use globbing patterns in the for loop. We also enclose the code in brackets, to prevent the shopt -s globstar dotglob; line option from becoming a global setting.
If you would like to apply this example to your file, you can just place your files in the current directory, and the code would probably look something like this:
( shopt -s globstar dotglob;
for file in **; do
if [[ -f $file ]] && [[ -w $file ]]; then
sed -i -e 's/.\{6\}/&\n/g' -e 's/edit/edit2/g' | tr -d "\n" | sed 's/edit2/edit/g' | grep -o "here.*there" | sed -r '/^.{,100}$/d' "$file"
fi
done
)
Note that we have placed a "$file" variable beside each of the seds that you used in your code, this replaces the name of the file for each command.
There is another example given in the code that allows you to pick which files to run on, rather than all the files in a directory, which you can also re-purpose for your code, as given here:
( shopt -s globstar dotglob
sed -i -- 's/foo/bar/g' **baz*
sed -i -- 's/foo/bar/g' **.baz
)
To answer your question of doing a loop on each line, you will need to put a loop for each line inside your for loop, like so:
while read line ; do
: sed -i -e 's/.\{6\}/&\n/g' -e 's/edit/edit2/g' | tr -d "\n" | sed 's/edit2/edit/g' | grep -o "here.*there" | sed -r '/^.{,100}$/d' "$line”
done
)
Although the for loop can be useful for dealing with files in recursive directories, I would recommend against also using another loop to grab lines, since it muddies your code, and it’s possible there is a better way to do it without parsing line by line.
The linked question is a fairly complete guide to many of the cases you may come across, and is also worth a read if you want to learn more.
Hope that helps!
You could use a for loop.
You could use the tool parallel.
Example
Create a set of test files using a for-loop
mkdir -p /tmp/so58333536
cd /tmp/so58333536
for i in 1.txt 2.txt 3.txt 4.txt 5.txt;do echo "The answer is 41" > $i;done
cat /tmp/so58333536/*
Now correct your mistake using parallel [1].
mkdir /tmp/so58333536.new
ls /tmp/so58333536/* |parallel "sed 's/41/42/' {} > /tmp/so58333536.new/{/}"
cat /tmp/so58333536.new/*
{}:: refers to the current file
{/}:: refers to name of the current file (path is removed)
Reads: List all files in so58333536 and apply the following sed command to each file and write the output to so58333536.new.
[1] Another option is to use sed -i for in-place editing.
Be very carefull with this!! Mistakes can cause serious damages!
# !! Do not use -i option regularly !!
ls /tmp/so58333536/* |parallel "sed -i 's/41/42/'"

Running a shell script on several files as inputs

I have a shell command with the following format:
my_cmd -I file1.inp -O file1.out
Where some processing is done on file1.inp and the results are stored in file1.out
In my main directory, I have many files with the format: *.inp and I would like to run this command for all of them and the store the results to *.out. Can I only use shell script to achieve this?
You can use a simple loop:
for file in *.inp ; do
my_cmd -I "${file}" -O "${file%%.inp}.out"
done
${file%%.inp} is a so called parameter expansion. It will effectively remove the extension .inp from the input filename.
One thing (thanks Jean-François Fabre). If the folder does not contain any .inp files the above loop would run once with $file having the literal value *.inp. To avoid that you need to set the nullglob option:
shopt -s nullglob # set the nullglob option
for file in *.inp ; do
my_cmd -I "${file}" -O "${file%%.inp}.out"
done
shopt -u nullglob # unset the nullglob option
Using GNU parallel
parallel my_cmd -I {} -O {.}.out ::: *.inp
By default, this will jobs in parallel, one job per core. {} is an unchanged argument, {.} is the same argument minus its extension. The arguments are taken from the words that follow :::.
ls *.inp| xargs -l1 -I % my_cmd -I % -O %.out

Readlink - How to crop full path?

I use readlink to find a file's full path:
cek=$(readlink -f "$1")
mkdir -p "$ydk$cek"
mv "$1" "$ydk/$cek/$ydkfile"
But readlink -f "$1" gives me the full path. How can I crop the full path?
For example:
/home/test/test/2014/10/13/log.file
But I need just
/test/2014/10/13/
How can I do it?
Judging from multiple comments:
The output should be the last four directory components of the full path returned by readlink.
Given:
full_path=/home/some/where/hidden/test/2014/08/29/sparefile.log
the output should be:
test/2014/08/29
(Don't build any assumption about today's date into the path trimming code.)
If you need the last four directory components of the full path, and if you don't have newlines in the full path, and if you have GNU grep or BSD (Mac OS X) grep with support for -o (output only the matched material) then this gives the required result:
$ cek="/home/test/test/2014/10/13/log.file"
$ echo "${cek%/*}"
/home/test/test/2014/10/13
$ echo "${cek%/*}" | grep -o -E -e '(/[^/]+){4}$'
/test/2014/10/13
$ full_path=/home/some/where/hidden/test/2014/08/29/sparefile.log
$ echo "${full_path%/*}" | grep -o -E -e '(/[^/]+){4}$'
/test/2014/08/29
$
I need path starting /201[0-9]:
/home/bla/bla2/bla3/2014/01/13/13… ⟶ /2014/01/13/13….
So, you need to use grep -o again, starting with the year pattern:
echo "${fullpath%/*}" | grep -o -e '/201[0-9]/.*$'
This is much simpler; you don't even need extended regular expressions for this!
If you need the path element before the year too, then you need:
echo "{fullpath%/*}" | grep -o -e '/[^/][^/]*/201[0-9]/.*$'
Do you really need to remove "/home" ?
cek="/home/test/test/2014/10/13/log.file"
dir=$(dirname "$cek")
echo "${dir#/home}"
/test/test/2014/10/13
Just last 4 directory components:
last4dirs() {
local IFS=/
local -a components=($1)
local l=${#components[#]}
echo "${components[*]:l-5:4}"
}
last4dirs /home/some/where/hidden/test/2014/08/29/sparefile.log
test/2014/08/29

understanding a shell script code using expand function

I am working on shell script.
EXEC $CXCHOME+"/etc/expand_in_place" $MMSHOME+"/PDM/bin/pmr_pdm_aos"**
CXCHOME "/opt/ericsson/aos/PDM"
MMSHOME "/opt/ericsson/aos"
the code of expand_in_place is as below -
#!/bin/bash
. $INST_DATADIR/$PKG/install/aosbootcommon.sh
filename=`basename $1`
tmpfile="/tmp/$filename.$$"
rm -f "$tmpfile"
cp -p "$1" "$tmpfile"
echoLog "Expanding $1..."
expand "$tmpfile" "$1"
rm -f "$tmpfile"
cleanExit 0
I wanted to know the working of "expand_in_place".
Long answer:
#!/bin/bash
Use /bin/bash program to process the code that follows.
. $INST_DATADIR/$PKG/install/aosbootcommon.sh
Read and execute the code in this file in the same process
filename=`basename $1`
Run the basename program, passing-in the first command-line argument (use man basename to find out what that does). The back-ticks are a deprecated way to capture the output from a program. In this case the output from basename is placed into the filename variable.
tmpfile="/tmp/$filename.$$"
Set the variable tmpfile to be /tmp, followed by the values of filename, followed by out current process id. The $ is an operator which gives us the value of a variable. $$ gives us the value of our current PID.
rm -f "$tmpfile"
Run the rm program passing these parameters : use man rm to find out what that does.
cp -p "$1" "$tmpfile"
Run the cp program passing those parameters: use man cp to find out what that does.
echoLog "Expanding $1..."
I have no idea what echoLog does, it is probably a local function defined in $INST_DATADIR/$PKG/install/aosbootcommon.sh
expand "$tmpfile" "$1"
Run the expand program using those parameters, use man expand to find out what that does.
rm -f "$tmpfile"
Run the rm program passing these parameters : use man rm to find out what that does.
cleanExit 0
I have no idea what cleanExit does, it is probably a local function defined in $INST_DATADIR/$PKG/install/aosbootcommon.sh

grep spacing error

Hi guys i've a problem with grep . I don't know if there is another search code in shell script.
I'm trying to backup a folder AhmetsFiles which is stored in my Flash Disk , but at the same time I've to group them by their extensions and save them into [extensionName] Folder.
AhmetsFiles
An example : /media/FlashDisk/AhmetsFiles/lecture.pdf must be stored in /home/$(whoami)/Desktop/backups/pdf
Problem is i cant copy a file which name contains spaces.(lecture 2.pptx)
After this introduction here my code.
filename="/media/FlashDisk/extensions"
count=0
exec 3<&0
exec 0< $filename
mkdir "/home/$(whoami)/Desktop/backups"
while read extension
do
cd "/home/$(whoami)/Desktop/backups"
rm -rf "$extension"
mkdir "$extension"
cd "/media/FlashDisk/AhmetsFiles"
files=( `ls | grep -i "$extension"` )
fCount=( `ls | grep -c -i "$extension"` )
for (( i=0 ; $i<$fCount ; i++ ))
do
cp -f "/media/FlashDisk/AhmetsFiles/${files[$i]}" "/home/$(whoami)/Desktop/backups/$extension"
done
let count++
done
exec 0<&3
exit 0
Your looping is way more complicated than it needs to be, no need for either ls or grep or the files and fCount variables:
for file in *.$extension
do
cp -f "/media/FlashDisk/AhmetsFiles/$file" "$HOME/Desktop/backups/$extension"
done
This works correctly with spaces.
I'm assuming that you actually wanted to interpret $extension as a file extension, not some random string in the middle of the filename like your original code does.
Why don't you
grep -i "$extension" | while IFS=: read x ; do
cp ..
done
instead?
Also, I believe you may prefer something like grep -i ".$extension$" instead (anchor it to the end of line).
On the other hand, the most optimal way is probably
cp -f /media/FlashDisk/AhmetsFiles/*.$extension "$HOME/Desktop/backups/$extension/"

Resources