sed not working while extracting lines from a log file? - bash

When I do echo /$date $hh:$mq[$mr-9]/,/$date 23:59/p, I get this in return /2017-12-31 19:1[4-9]/,/2017-12-31 23:59/p but when I do this sed -n '/$date $hh:$mq[$mr-9]/,/$date 23:59/p' $error_log_file_name I get this
gesed: -e expression #1, char 22: Invalid range end
Any suggestion where I'm going wrong? $error_log_file_name is correct.

because without quotes $date variable is expanded,and within single quotes there is no expansion, compare
echo /$date $hh:$mq[$mr-9]/,/$date 23:59/p
echo "/$date $hh:$mq[$mr-9]/,/$date 23:59/p"
echo '/$date $hh:$mq[$mr-9]/,/$date 23:59/p'
also without quotes, arguments are split on space or tabs and file globbing can occur.

Related

Sed insert blank line below search string then insert string on the next line

I have a sed expression in a function that I pass parameters to.
insert_after_new_line() {
if ! grep -q "${2}" "${3}"; then
sed -i "/${1}/{N;N;a ${2}}" "${3}"
fi
}
insert_after_new_line search insert textfile.txt
I am trying to have a blank line inserted below the search string and the insert string inserted after.
so
text
text
search
text
becomes
text
text
search
insert
text
but I keep getting the error
sed: -e expression #1, char 0: unmatched `{'
I tested this. works in command line
sed -i '/search/a \\ninsert' file
sed really delimiters commands by a newline. There is a ; but it does not work for all commands, mostly these which take file name as an argument. ; does not work for r R or for example a. Sed will read everything after a command, so sed interprets is like a ${2}} as a single command, in result it does not find enclosing }, cause it's been eaten by a command. You need a newline:
sed -i "/${1}/{N;N;a ${2}
}" "${3}"
or
sed -i "/${1}/{N;N;a ${2}"$'\n'"}" "${3}"
this should work:
sed -i '/search/{G;ainsert
}' file
You can replace the text by shell variable, but replace the single quotes by double quotes too.

Concatenation String in shell

I have this shell line to concatenate 2 string:
new_group="second result is: ${result},\"${policyActivite}_${policyApplication}\""
echo "result is: ${new_group}"
The result:
result is: "Team","Application_Team"
How can change the result to: result is: "Team, Application_Team"
Use sed:
echo "$new_group" | sed 's/"//g;s/^\(.*\)$/"\1"/'
The first statement is removing all double quotes. The second one add double at the start and the end of the line.
Alternatively, if you want to replace "," with ,, use this sed command: sed 's/","/, /g'

Adding a comma after $variable

I'm writing a for loop in bash to run a command and I need to add a comma after one of my variables. I can't seem to do this without an extra space added. When I move "," right next to $bams then it outputs *.sorted,
#!/bin/bash
bams=*.sorted
for i in $bams
do echo $bams ","
done;
Output should be this:
'file1.sorted','file2.sorted','file3.sorted'
The eventual end goal is to be able to insert a list of files into a --flag in the format above. Not sure how to do that either.
First, a literal answer (if your goal were to generate a string of the form 'foo','bar','baz', rather than to run a program with a command line equivalent to somecommand --flag='foo','bar','baz', which is quite different):
shopt -s nullglob # generate a null result if no matches exist
printf -v var "'%s'," *.sorted # put list of files, each w/ a comma, in var
echo "${var%,}" # echo contents of var, with last comma removed
Or, if you don't need the literal single quotes (and if you're passing your result to another program on its command line with the single quotes being syntactic rather than literal, you absolutely don't want them):
files=( *.sorted ) # put *.sorted in an array
IFS=, # set the comma character as the field separator
somecommand --flag "${files[*]}" # run your program with the comma-separated list
try this -
lst=$( echo *.sorted | sed 's/ /,/g' ) # stack filenames with commas
echo $lst
if you really need the single-ticks around each filename, then
lst="'$( echo *.sorted | sed "s/ /','/g" )'" # commas AND quotes
#!/bin/bash
bams=*.sorted
for i in $bams
do flag+="${flag:+,}'$i'"
done
echo $flag

Why is bash ignoring the ending double quote (")

I have this file:
http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/I_Cuestionario_general_estimaciones_endireh2016.xlsx
http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/IV_Ingresos_y_recursos_estimaciones_endireh2016.xlsx
http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/VI_ambito_escolar_estimaciones_endireh2016.xlsx
http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/VII_ambito_laboral_estimaciones_endireh2016.xlsx
http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/VIII_ambito_comunitario_estimaciones_endireh2016.xlsx
http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/IX_Atencion_Obstetrica_estimaciones_endireh2016.xlsx
http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/X_ambito_familiar_estimaciones_endireh2016.xlsx
And this bash script:
while read p; do
echo "\"$p\""
done < file.txt
I would expect the same file but with double quotes around each line, but this is what bash is outputting:
"http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/I_Cuestionario_general_estimaciones_endireh2016.xlsx
"http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/IV_Ingresos_y_recursos_estimaciones_endireh2016.xlsx
"http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/VI_ambito_escolar_estimaciones_endireh2016.xlsx
"http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/VII_ambito_laboral_estimaciones_endireh2016.xlsx
"http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/VIII_ambito_comunitario_estimaciones_endireh2016.xlsx
"http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/IX_Atencion_Obstetrica_estimaciones_endireh2016.xlsx
"http://www.beta.inegi.org.mx/contenidos/proyectos/enchogares/especiales/endireh/2016/tabulados/X_ambito_familiar_estimaciones_endireh2016.xlsx
Anyone know why bash is behaving this way? And how to output both " double quotes? (beginning and end)
I'm near certain that the line endings on your input file are CR/LF rather than just LF. This would output:
";
the web address;
a CR returning the cursor to the beginning of the line;
"; and, finally,
moving to a new line.
Capture the output to a file and pass it through a dump utility like od -xcb, that should show you the raw bytes being output.
As a test, creating a file consisting of the two lines 123<CR> and 456, I see:
pax> while read p; do echo "\"$p\""; done <testfile
"123
"456"
which seems to indicate the problem is as described.
If you're having trouble escaping the leading and trailing double quotes, you can just use single quotes around your echo statement. Any double quotes inside of single quotes have no significance in terms of defining a string literal, and vice versa:
while read p; do
echo '"$p"'
done < file.txt

unexpected EOF while looking for matching `'' while using sed

Yes this question has been asked many times, and in the answer it is said to us \ escape character before the single quote.
In the below code it isn't working:
LIST="(96634,IV14075295,TR14075685')"
LIST=`echo $LIST | sed 's/,/AAA/g' `
echo $LIST # Output: (96634AAAIV14075295AAATR14075685')
# Now i want to quote the list elements
LIST=`echo $LIST | sed 's/,/\',\'/g' ` # Giving error
# exit 0
Error :
line 7: unexpected EOF while looking for matching `''
line 8: syntax error: unexpected end of file
Instead of single quotes, use double quotes in sed command, and also remove the space before last backtick. If there is single quote present in the sed pattern then use an alternative enclosing quotes(ie, double quotes),
sed "s/,/\',\'/g"
And the line would be,
LIST=$(echo $LIST | sed "s/,/\',\'/g")
Don't use backticks inside the sripts instead of bacticks, use $()
You can use awk
echo $LIST
(96634,IV14075295,TR14075685')
LIST=$(awk '{gsub(/,/,q"&"q)};gsub(/\(/,"&"q)1' q="'" <<< $LIST)
echo $LIST
('96634','IV14075295','TR14075685')
To prevent problems with the single quote, I just set it to an awk variable.
consider
LIST="$(sed "s/,/\',\'/g" <<< "$LIST")"
but first and last elements probably won't get quoted completely, because of missing leading and trailing comma
btw, you don't need to subshell to sed - string matching and substitution is entirely within the capabilities of bash:
LIST="${LIST//,/\',\'}"

Resources