I have many files with the extension .com, so the files are named 001.com, 002.com, 003.com, and so on.
And I have another file called headname which contains the following information:
abc=chd
dha=djj
cjas=FILENAME.chk
dhdh=hsd
I need to put the information of the file headname inside (and at the begin of) the files 001.com, 002.com, 003.com and so on... But FILENAME needs to be the filename of the file that will receive the headname information (without the .com extension).
So the output need to be:
For the 001.com:
abc=chd
dha=djj
cjas=001.chk
dhdh=hsd
For the 002.com:
abc=chd
dha=djj
cjas=002.chk
dhdh=hsd
For the 003.com:
abc=chd
dha=djj
cjas=003.chk
dhdh=hsd
And so on...
set -e
for f in *.com
do
cat <(sed "s#^cjas=FILENAME.chk\$#cjas=${f%.com}.chk#" headname) "$f" > "$f.new"
mv -f "$f.new" "$f"
done
Explanation:
for f in *.com -- this loops over all file names ending with .com.
sed is a program that can be used to replace text.
s#...#...# is the substitute command.
${f%.com} is the file name without the .com suffix.
cat <(...) "$f" -- this merges the new head with the body of the .com file.
The output of cat is stored into a file named 123.com.new -- mv -f "$f.new" "$f" is used to rename 123.com.new to 123.com.
Something like this should work:
head=$(<headname) # read head file into variable
head=${head//$'\n'/\\n} # replace literal newlines with "\n" for sed
for f in *.com; do # loop over all *.com files
# make a backup copy of the file (named 001.com.bak etc).
# insert the contents of $head with FILENAME replaced by the
# part of the filename before ".com" at the beginning of the file
sed -i.bak "1i${head/FILENAME/${f%.com}}" "$f"
done
Related
I have a directory containing hundreds of image files, each named differently, such as:
abdogf.png
abcatf.png
abhorsef.png
I have created a changenames.csv file containing two columns with oldstring in the first column and newstring in the second such as:
"dog","woof"
"cat","miaow"
"horse","neigh"
These strings are currently in quotation marks, as shown.
I would like to invoke a bash command or .sh script from the command line to replace each oldstring substring with each newstring substring in the directory's filenames (not within file contents), such that the directory newly contains files called:
abwooff.png
abmiaowf.png
abneighf.png
instead of the original files.
I have tried various solutions such as https://superuser.com/questions/508731/find-and-replace-string-in-filenames/508758#508758 and How to find and replace part of filenames from list without success.
For example, I have tried invoking the following within the directory with the files:
#!/bin/bash
inputfile=${1}
while read line
do
IFS=',' read -a names <<< "${line}"
for file in `find . -maxdepth 1 -type f -name "*${names[0]}*"`; do
rename "s/${names[0]}/${names[1]}/" *
done
done < ${inputfile}
using the command line command test.sh changenames.csv.
This produces no error but makes no changes to the filenames.
I have also tried this solution https://stackoverflow.com/a/55866613/10456769 which generated an error in which #echo was not a recognised command.
Thank you in advance for any help.
You need to strip the double quotes off at first. The code tries to find
files such as *"cat"* which do not exit.
Moreover you do not need to execute the find command. You are not
using the variable file at all.
Would you please try the following:
while IFS=',' read -r old new; do
old=${old//\"/} # remove leading and trailing double-quotes
new=${new//\"/} # same as above
rename "s/$old/$new/" *
done < "$1"
The IFS=',' read -a names <<< "${line}" does not remove " from the input. Your filenames do not have " in them, so you have to remove them too.
Backticks ` are discouraged. Don't use them. Use $(....) instead.
"for file in `" is as bad as for file in $(cat) - it's a common bash antipattern. Don't use it - you will have problems with elements with spaces or tabs. Use while IFS= read -r line to read something like by line.
There is a problem with rename, there are two common versions of rename - GNU rename and perl rename. Your script seems to aim the perl version - make sure it is the one installed.
Let rename do the rename - there is no need for for file in find here.
If you do while read line and then IFS=, read <<<"$line" is duplicating the work, just do while IFS=, read -a names; do from the beginning.
So you could do:
# split the input on ','
while IFS=',' read -r pre post; do
# remove quotes
pre=${pre//\"/}
post=${post//\"/}
# do the rename
rename "s/${pre}/${post}/" *
done < ${inputfile}
I think I would do the following script that uses sed:
# find all files in a directory
find . -maxdepth 1 -type f |
# convert filenames into pairs (filename,new_filename) separated by newline
sed "
# hold the line in hold space
h
# replace the characters as in the other file
$(
# generate from "abc","def" -> s"abc"def"g
sed 's#"\([^"]*\)","\([^"]*\)"#s"\1"\2"g#' changenames.csv
)
# switch pattern and hold space
x
# append the line
G
# remove the line if substitute is the same
/^\(.*\)\n\1$/d
" |
# outputs two lines per each filename:
# one line with old filename and one line with new filename
# so just pass that to mv
xargs -l2 echo mv -v
and a one liner:
find . -maxdepth 1 -type f | sed "h;$(sed 's#"\([^"]*\)","\([^"]*\)"#s"\1"\2"g#' changenames.csv);x;G; /^\(.*\)\n\1$/d" | xargs -l2 echo mv -v
With the following recreation of files structure:
touch abdogf.png abcatf.png abhorsef.png
cat <<EOF >changenames.csv
"dog","woof"
"cat","miaow"
"horse","neigh"
EOF
The script outputs on repl:
mv -v ./abdogf.png ./abwooff.png
mv -v ./abcatf.png ./abmiaowf.png
mv -v ./abhorsef.png ./abneighf.png
I am trying to use bash to rename/update the filename of a text file in /home/cmccabe/Desktop/percent based on a partial match of digits with another text file in /home/cmccabe/Desktop/analysis.txt. The match will always be in either lines 3,4,or 5 of this file. I am not able to do this but hopefully the 'bash` below is a start. Thank you :).
text file in /home/cmccabe/Desktop/percent - there could be a maximum of 3 files in this directory
00-0000_fbn1_20xcoverage.txt
text file in /home/cmccabe/Desktop/analysis.txt
status: complete
id names:
00-0000_Last-First
01-0101_LastN-FirstN
02-0202_La-Fi
desired result in /home/cmccabe/Desktop/percent
00-0000_Last-First_fbn1_20xcoverage.txt
bash
for filename in /home/cmccabe/Desktop/percent/*.txt; do echo mv \"$filename\" \"${filename//[0-9]-[0-9]/}\"; done < /home/cmccabe/Desktop/analysis.txt
Using a proper Process-Substitution syntax with a while-loop,
You can run the script under /home/cmccabe/Desktop/percent
#!/bin/bash
# ^^^^ needed for associative array
# declare the associative array
declare -A mapArray
# Read the file from the 3rd line of the file and create a hash-map
# as mapArray[00-0000]=00-0000_Last-First and so on.
while IFS= read -r line; do
mapArray["${line%_*}"]="$line"
done < <(tail -n +3 /home/cmccabe/Desktop/analysis.txt)
# Once the hash-map is constructed, rename the text file accordingly.
# echo the file and the name to be renamed before invoking the 'mv'
# command
for file in *.txt; do
echo "$file" ${mapArray["${file%%_*}"]}"_${file#*_}"
# mv "$file" ${mapArray["${file%%_*}"]}"_${file#*_}"
done
This is another similar bash approach:
while IFS="_" read -r id newname;do
#echo "id=$newid - newname=$newname" #for cross check
oldfilename=$(find . -name "${id}*.txt" -printf %f)
[ -n "$oldfilename" ] && echo mv \"$oldfilename\" \"${id}_${newname}_${oldfilename#*_}\";
done < <(tail -n+3 analysis)
We read the analysis file and we split each line (i.e 00-0000_Last-First) to two fields using _ as delimiter:
id=00-000
newname=Last-First
Then using this file id we read from file "analysis" we check (using find) to see if a file exists starting with the same id.
If such a file exists, it's filename is returned in variable $oldfilename.
If this variable is not empty then we do the mv.
tail -n+3 is used to ignore the first three lines of the file results.txt
Test this solution online here
I have on a folder multiple txt file containing one or several lines. Each file name is an email address containing different email(s) address(es) inside.
For example, I have 3 files on my folder :
distribution-list1#example.com.txt
distribution-list2#example.com.txt
distribution-list3#example.com.txt
Content of each files:
cat distribution-list1#example.com.txt
john#example.com
aurel#example.com
cat distribution-list2#example.com.txt
doe#example.com
cat distribution-list3#example.com.txt
jack#example.com
gilbert#example.com
jane#example.com
I would like to build only one file containing those data:
distribution-list1#example.com;john#example.com
distribution-list1#example.com;aurel#example.com
distribution-list2#example.com;doe#example.com
distribution-list3#example.com;jack#example.com
distribution-list3#example.com;gilbert#example.com
distribution-list3#example.com;jane#example.com
lists_merge.sh
#!/usr/bin/env bash
shopt -s nullglob;
for fname in *.txt;
do
while read line;
do
printf "%s;%s\n" "$fname" "$line";
done <"$fname";
done;
output
$ ./lists_merge.sh
distribution-list1#example.com.txt;john#example.com
distribution-list1#example.com.txt;aurel#example.com
distribution-list2#example.com.txt;doe#example.com
distribution-list3#example.com.txt;jack#example.com
distribution-list3#example.com.txt;gilbert#example.com
distribution-list3#example.com.txt;jane#example.com
note: script assumed to be in same directory as distribution list text
files. assumed no other text files are in this directory
reference
nullglob info
You can use sed:
for emailfile in *.txt; do
email=${emailfile%.txt}
sed "s:^:$email;:" "$emailfile"
done
This will fail if an email ID has a colon (:), but I doubt you'd have such an example.
I have multiple files named as such --> 100.txt, 101.txt, 102.txt, etc.
The files are located within a directory. For every one of these files, I need to append the number before the extension in the file name to every line in the file.
So if the file content of 100.txt is:
blahblahblah
blahblahblah
...
I need the output to be:
blahblahblah 100
blahblahblah 100
...
I need to do this using sed.
My current code looks like this, but it is ugly and not very concise:
dir=$1
for file in $dir/*
do
base=$(basename $file)
filename="${base%.*}"
sed "s/$/ $filename/" $file
done
Is it possible to do this in such a way?
find $dir/* -exec sed ... {} \;
The code you already have is essentially the simplest, shortest way of performing the task in bash. The only changes I would make are to pass -i to sed, assuming you are using GNU sed (otherwise you will need to redirect the output to a temporary file, remove the old file, and move the new file into its place), and to provide a default value in case $1 is empty.
dir="${1:-.}"
the following command line will find all files that that has filename with only numbers with an extension and append the filename (numbers) at the end of each line in that file..(I tested with a couple of files)
find <directory path> -type f -name '[0-9]*' -exec bash -c 'num=`basename "{}"|sed "s/^\([0-9]\{1,\}\)\..*/\1/"`;sed -i.bak "s/.$/& $num/" "{}"' \;
Note: command line using sed not tested in OS X
replace <directory path> with the path of your directory
I have a template file I want to copy and then edit from a script, inserting content at specific template points. For example, my template file might be something like,
...
rm -rf SomeDirectory
make install
#{INSERT-CONTENT-HERE}
do-something-else
...
In another script, I want to add content at "#{INSERT-CONTENT-HERE}" within a loop, i.e.
for i in c; do
# Write content to the template file copy at the correct point.
done
I think sed is the right tool, but I'm not familiar enough to know the syntax, and the man page isn't helping.
An example:
echo "Line #{INSERT-CONTENT-HERE}" | sed 's/#{INSERT-CONTENT-HERE}/---/'
To modify a file:
sed -i 's/#{INSERT-CONTENT-HERE}/---#{INSERT-CONTENT-HERE}/' filename
where -i means in-place edit so be warned
if you do:
sed -i.bak 's/#{INSERT-CONTENT-HERE}/---/' filename
it should back up original as filename.bak
also to make multiple substitutions at each line use the g flag:
sed -i.bak 's/#{INSERT-CONTENT-HERE}/---/g' filename
You can copy the output of all the commands into a temporary file and then copy the contents of that entire file into the template file:
TEMPFILE=`mktemp` && (
for i in c
echo "SomeTextBasedOn $i" >> $TEMPFILE
done
sed -i '/{INSERT-CONTENT-HERE}/r '$TEMPFILE targetfile
rm $TEMPFILE
)