I'm new in bash, and I'm trying to delete a line in a file I'm creating.
So without further ado :
if [[ $(ls -1 | grep 'fichiers.toCheck' | wc -l) -eq 0 ]]; then
touch fichiers.toCheck
fi
find . -name '*.mp4' > fichiers.toCheck
while read p; do
echo $p
sed -i "$p/d" ./fichiers.toCheck
done <fichiers.toCheck
Console is giving me this:
sed: 1: "./fichiers.toCheck": invalid command code .
I'm suspecting sed interprets the "/" in the line as an argument (the line is something like "./nosound.mp4".
What's your guess?
edit 2 = the correct syntax was with -i.bak
sed -i.bak "s#$p##" fichiers.toCheck
edit = so here's my experiments :
1
while read p; do
echo $p
sed -i "/$p/d" fichiers.toCheck
done <fichiers.toCheck
And I get :
sed: 1: "fichiers.toCheck": invalid command code f
2
sed -i "#$p#d" fichiers.toCheck
and same error :
sed: 1: "fichiers.toCheck": invalid command code f
Your sed syntax is wrong, to delete a line containing a pattern from a bash variable. Also being using FreeBSD native sed in OS X use the -i.bak for in-place edits.
sed -i.bak "/$p/d" fichiers.toCheck
If you suspect your variable contains / change the sed separator to # and use the traditional pattern s/<pattern>/<replacement>/ style with the replacement part set to empty, i.e.
sed -i.bak "s#$p##" fichiers.toCheck
Related
I have an input file containing the following numbers
-45.0005
-43.0022
-41.002
.
.
.
I have a target txt file
line:12 Angle=30
line:42 Angle=60
line:72 Angle=90
.
.
.
Using sed I want to replace the first instance of Angle entry in the target file with the first entry from the input file, the second entry of Angle with the second entry of the input file so and so forth...
Expected output:
line:12 Angle=-45.005
line:42 Angle=-43.002
line:72 Angle=-41.002
.
.
.
This is what I have managed to write but I am not getting the expected output
a=`head -1 temp.txt`
#echo $a
sed -i "12s/Angle = .*/Angle = $a/g" $procfile
for i in {2..41..1}; do
for j in {42..1212..30}; do
c=$(( $i - 1 ))
#echo "this is the value of c: $c"
b=`head -$i temp.txt | tail -$c`
#echo "This is the value of b: $b"
sed -i "$js/Angle = .*/Angle = $b/g" $procfile 2> /dev/null
done
done
Could you help me improve the script?
Thanks!
You may create an iterator i and then use it in sed to perform substitution in each line.
i=0;
while read -r line; do
i=$((i+1));
sed -i "${i}s/Angle=.*/Angle=${line}/g" $procfile;
done < temp.txt
So I guess you want to paste files - marge files line by line. Then replace the field with a regex for example.
paste target_file input_file | sed 's/\(Angle=\)[^\t]*\t/\1/'
This might work for you (GNU sed):
sed '/Angle=/R inputFile' targetFile | sed '/Angle=/{N;s/=.*\n/=/}'
In the first sed invocation append the input line.
In the second sed invocation remove the original angle and the newline delimiter.
pr might help here, please try this:
pr -m -t target input | sed -r 's/(Angle=)[^\s]+\s+/\1/'
Please note - this works for your first two showed files, your code assumes some different input - e.g. spaces around "=".
So I was able to come up with this solution
#!/bin/bash
infile=$1
cp $infile ORIG_${infile}
grep "Angle = " $infile | sed 's/Angle = //g' | sort -n > temp.txt
iMax=`cat temp.txt | wc -l`
jMax=`grep -n "Angle = " $infile | tail -1 | sed 's/:.*//g'`
for ((i=1,j=12; i<=${iMax} && j<=${jMax};i+=1,j+=30));do
a=`head -$i temp.txt | tail -1`
sed -i "${j}s/Angle = .*/Angle = $a/g" $infile
done
rm temp.txt
Many thanks to william pursell for clarifying the syntax for incrementing var counts in bash.
I am using the sed command on Ubuntu to replace content.
This initial command comes from here.
sed -i '$ s/$/ /replacement/' "$DIR./result/doc.md"
However, as you can see, I have a slash in the replacement. The slash causes the command to throw:
sed: -e expression #1, char 9: unknown option to `s'
Moreover, my replacement is stored in a variable.
So the following will not work because of the slash:
sed -i "$ s/$/ $1/" "$DIR./result/doc.md"
As stated here and in duplicate, I should use another delimiter. If I try with #:
sed -i "$ s#$# $1#" "$DIR./result/doc.md"
It gives the error:
sed: -e expression #1, char 42: unterminated `s' command
My question is:
How can I use a variable in this command as well as other delimiter than / ?
Don't use sed here; perl and awk allow more robust approaches.
sed doesn't allow variables to be passed out-of-band from code, so they always need to be escaped. Use a language without that limitation, and you have code that always works, no matter what characters your data contains.
The Short Answer: Using perl
The below is taken from BashFAQ #21:
inplace_replace() {
local search=$1; shift; local replace=$1; shift
in="$search" out="$replace" perl -pi -e 's/\Q$ENV{"in"}/$ENV{"out"}/g' "$#"
}
inplace_replace '#' "replacement" "$DIR/result/doc.md"
The Longer Answer: Using awk
...or, using awk to do a streaming replacement, and a shell function to make that file replacement instead:
# usage as in: echo "in should instead be out" | gsub_literal "in" "out"
gsub_literal() {
local search=$1 replace=$2
awk -v s="${search//\\/\\\\}" -v r="${rep//\\/\\\\}" 'BEGIN {l=length(s)} {o="";while (i=index($0, s)) {o=o substr($0,1,i-1) r; $0=substr($0,i+l)} print o $0}'
}
# usage as in: inplace_replace "in" "out" /path/to/file1 /path/to/file2 ...
inplace_replace() {
local search=$1 replace=$2 retval=0; shift; shift
for file; do
tempfile=$(mktemp "$file.XXXXXX") || { retval |= $?; continue; }
if gsub_literal "$search" "$replace" <"$file" >"$tempfile"; then
mv -- "$tempfile" "$file" || (( retval |= $? ))
else
rm -f -- "$tempfile" || (( retval |= $? ))
fi
done
}
TL;DR:
Try:
sed -i '$ s#$# '"$1"'#' "$DIR./result/doc.md"
Long version:
Let's start with your original code:
sed -i '$ s/$/ /replacement/' "$DIR./result/doc.md"
And let's compare it to the code you referenced:
sed -i '$ s/$/abc/' file.txt
We can see that they don't exactly match up. I see that you've correctly made this substitution:
file.txt --> "$DIR./result/doc.md"
That looks fine (although I do have my doubts about the . after $DIR ). However, the other substitution doesn't look great:
abc --> /replacement
You actually introduced another delimeter /. However, if we replace the delimiters with '#' we get this:
sed -i '$ s#$# /replacement#' "$DIR./result/doc.md"
I think that the above is perfectly valid in sed/bash. The $# will not be replaced by the shell because it is single quoted. The $DIR variable will be interpolated by the shell because it is double quoted.
Looking at one of your attempts:
sed -i "$ s#$# $1#" "$DIR./result/doc.md"
You will have problems due to the shell interpolation of $# in the double quotes. Let's correct that by replacing with single quotes (but leaving $1 unquoted):
sed -i '$ s#$# '"$1"'#' "$DIR./result/doc.md"
Notice the '"$1"'. I had to surround $1 with '' to basically unescape the surrounding single quotes. But then I surrounded the $1 with double quotes so we could protect the string from white spaces.
Use shell parameter expansion to add escapes to the slashes in the variable:
$ cat file
foo
bar
baz
$ set -- ' /repl'
$ sed "s/$/$1/" file
sed: 1: "s/$/ /repl/": bad flag in substitute command: 'r'
$ sed "s/$/${1//\//\\\/}/" file
foo /repl
bar /repl
baz /repl
That is a monstrosity of leaning toothpicks, but it serves to transform this:
sed "s/$/ /repl/"
into
sed "s/$/ \/repl/"
The same technique can be used for whatever you choose as the sed s/// delimiter.
I would like to find and replace a string in multiple files using bash command. I am using sed which I am not really familiar with.
My variables:
$FILE = (/home/user/file1.txt, /home/user/file2.txt)
$REL = 5.0
My code:
for f in ${FILES[#]}; do sed -i "$f" "s/__ver__ =*/__ver__=$REL/g";
output:
sed: -e expression #1, char 2: unknown command: `/'
sed: -e expression #1, char 2: unknown command: `/'
What is wrong with my expression?
1) The filename should be specified as the last argument for sed expression:
2) bash's for loop should ended with done keyword
for f in ${FILES[#]}; do sed -i "s/__ver__ =*/__ver__=$REL/" "$f"; done
If your files have similar naming format you can avoid for loop:
sed -i 's/__ver__ =*/__ver__=$REL/' /home/user/file[2].txt
I have a file called file_names_list.txt which contains absolute file names, for example, the first line is:
~/Projects/project/src/files/file.mm
I run a script to grep each of these files,
for file in $(cat file_names_list.txt); do
echo "doing file: $file"
grep '[ \t]*if (.* = .*) {' $file | while read -r line ; do ...
and I get the output:
doing file: ~/Projects/project/src/files/file.mm
grep: ~/Projects/project/src/files/file.mm: No such file or directory
But if I go to the terminal and enter
grep '[ \t]*if (.* = .*) {' ~/Projects/project/src/files/file.mm
I get the proper grep output
What's the problem here? I'm out of ideas
The problem is with the ~ character. That character gets expanded to your home directory when you use it in bash but in this case, it is just another character stored in the variable $file. To see the difference, try this:
file='~'
echo $file
echo ~
So now you have to either recreate the file file_names_list.txt or try to fix it, e.g. with sed:
sed -i -e "s|^~/|$HOME/|" file_names_list.txt
Also note that it would be prefereable to use a while loop instead of a for loop:
while IFS= read -r file; do
# write your code here
done < file_names_list.txt
You can use your script like this:
while IFS= read -r f; do
grep '[ \t]*if .* = .* {' "${f/#\~/\$HOME}"
done < file_names_list.txt
Since ~ cannot be stored in a variable and expanded we are replacing starting ~ by $HOME in each line in this BASH expression: "${f/#\~/\$HOME}"
my question seems to be general, but i can't find any answers.
In sed command, how can you replace the substitution pattern by a value returned by a simple bash function.
For instance, I created the following function :
function parseDates(){
#Some process here with $1 (the pattern found)
return "dateParsed;
}
and the folowing sed command :
myCatFile=`sed -e "s/[0-3][0-9]\/[0-1][0-9]\/[0-9][0-9]/& parseDates &\}/p" myfile`
I found that the caracter '&' represents the current pattern found, i'd like it to be passed to my bash function and the whole pattern to be substituted by the pattern found +dateParsed.
Does anybody have an idea ?
Thanks
you can use the "e" option in sed command like this:
cat t.sh
myecho() {
echo ">>hello,$1<<"
}
export -f myecho
sed -e "s/.*/myecho &/e" <<END
ni
END
you can see the result without "e":
cat t.sh
myecho() {
echo ">>hello,$1<<"
}
export -f myecho
sed -e "s/.*/myecho &/" <<END
ni
END
Agree with Glenn Jackman.
If you want to use bash function in sed, something like this :
sed -rn 's/^([[:digit:].]+)/`date -d #&`/p' file |
while read -r line; do
eval echo "$line"
done
My file here begins with a unix timestamp (e.g. 1362407133.936).
Bash function inside sed (maybe for other purposes):
multi_stdin(){ #Makes function accepet variable or stdin (via pipe)
[[ -n "$1" ]] && echo "$*" || cat -
}
sans_accent(){
multi_stdin "$#" | sed '
y/àáâãäåèéêëìíîïòóôõöùúûü/aaaaaaeeeeiiiiooooouuuu/
y/ÀÁÂÃÄÅÈÉÊËÌÍÎÏÒÓÔÕÖÙÚÛÜ/AAAAAAEEEEIIIIOOOOOUUUU/
y/çÇñÑߢÐð£Øø§µÝý¥¹²³ªº/cCnNBcDdLOoSuYyY123ao/
'
}
eval $(echo "Rogério Madureira" | sed -n 's#.*#echo & | sans_accent#p')
or
eval $(echo "Rogério Madureira" | sed -n 's#.*#sans_accent &#p')
Rogerio
And if you need to keep the output into a variable:
VAR=$( eval $(echo "Rogério Madureira" | sed -n 's#.*#echo & | desacentua#p') )
echo "$VAR"
do it step by step. (also you could use an alternate delimiter , such as "|" instead of "/"
function parseDates(){
#Some process here with $1 (the pattern found)
return "dateParsed;
}
value=$(parseDates)
sed -n "s|[0-3][0-9]/[0-1][0-9]/[0-9][0-9]|& $value &|p" myfile
Note the use of double quotes instead of single quotes, so that $value can be interpolated
I'd like to know if there's a way to do this too. However, for this particular problem you don't need it. If you surround the different components of the date with ()s, you can back reference them with \1 \2 etc and reformat however you want.
For instance, let's reverse 03/04/1973:
echo 03/04/1973 | sed -e 's/\([0-9][0-9]\)\/\([0-9][0-9]\)\/\([0-9][0-9][0-9][0-9]\)/\3\/\2\/\1/g'
sed -e 's#[0-3][0-9]/[0-1][0-9]/[0-9][0-9]#& $(parseDates &)#' myfile |
while read -r line; do
eval echo "$line"
done
You can glue together a sed-command by ending a single-quoted section, and reopening it again.
sed -n 's|[0-3][0-9]/[0-1][0-9]/[0-9][0-9]|& '$(parseDates)' &|p' datefile
However, in contrast to other examples, a function in bash can't return strings, only put them out:
function parseDates(){
# Some process here with $1 (the pattern found)
echo dateParsed
}