Using sed to replace single line in while read loop - bash

I'm needing help setting up a bash script for initializing some BC's in a file. Ideally, my program would iterate through each line and:
1) Read in BC type - (wall, outlet, inlet).
2) Change "type" field based on appropriate BC type.
Unfortunately, my program seems to replace all type fields in Step 2 instead of only the type field associated with the correct BC.
I think this has something to do with the sed command operating over the whole file instead of just the $line variable.
while IFS= read -r line || [[ -n "$line" ]]; do #go through each line of T2 file
if [[ $line == *wall_* ]] #if wall_*
then
echo "attempted to assign wall_*"
var=1 #wall #Go down 2 lines
elif [[ $line == *velocity-inlet* ]]
then
echo "attempted to assign outflow"
var=2 #inlet
elif [[ $line == *outflow* ]]
then
var=3 #outlet
fi
echo $var
if [[ $line == *type* && $var == 1 ]]
then
sed -i 's/.*type.*/type zeroGradient/' 0/T3
echo "Attempted wall zeroGradient"
elif [[ $line == *type* && $var == 2 ]]
then
sed -i 's/.*type.*/type fixedValue\nvalue uniform (3 0 0)/' 0/T3
elif [[ $line == type* && $var == 3 ]]
then
sed -i 's/.*type.*/type zeroGradient/' 0/T3
fi
sed -i '/nFaces*/d' 0/T3 #Deletes irrelevant stuff from boundary file copy
sed -i '/startFace*/d' 0/T3
done <0/T3.
For example, it is supposed to change:
velocity-inlet_1
{
type patch;
nFaces 336;
startFace 75515;
}
outflow_2
{
type patch;
nFaces 136;
startFace 75851;
}
To:
velocity-inlet_1
{
type fixedValue;
value uniform (3 0 0);
}
outflow_2
{
type zeroGradient;
}
But instead changes it wrongly changes it to:
velocity-inlet_1
{
type fixedValue;
value uniform (3 0 0);
}
outflow_2
{
type fixedValue;
value uniform (3 0 0);
}
Help me stack overflow, you're my only hope.

You have a few issues. sed will effect a whole line by default, and you're not telling it which line to modify in the first place. You're also modifying a file as you read it. I might go with something like this:
var="none"
while IFS= read -r line; do
if [[ "$line" =~ "wall" ]]; then
var=wall
elif [[ "$line" =~ "velocity-inlet" ]]; then
var=inlet
fi
if [[ "$line" =~ "type" && "$var" == "wall" ]]; then
echo "$line" | sed 's|\(type *\).*|\1zeroGradient|'
elif [[ "$line" =~ "type" && "$var" == "inlet" ]]; then
echo "$line" | sed 's|\(type *\).*|\1uniform (3 0 0)|'
else
echo "$line"
fi
done
And then do
script.sh < 0/T3 > 0/T3.mod
You can of course modify this to read/write from particular files as well, and you can avoid sed (see here...)

Related

Bash sed command gives me "invalid command code ."

I'm trying to automate a build process by replacing .js chunks for particular lines in my main.config.php file. When I run the following code:
declare -a js_strings=("footer." "footerJQuery." "headerCSS." "headerJQuery.")
build_path="./build/build"
config_path="./system/Config/main.config.php"
while read -r line;
do
for js_string in ${js_strings[#]}
do
if [[ $line == *$js_string* ]]
then
for js_file in "$build_path"/*
do
result="${js_file//[^.]}"
if [[ $js_file == *$js_string* ]] && [[ ${#result} -eq 3 ]]
then
sed -i "s/$line/$line$(basename $js_file)\";/g" $config_path
fi
done
fi
done
done < "$config_path"
I get this message back, and file has not been updated/edited:
sed: 1: "./system/Config/main.co ...": invalid command code .
I haven't been able to find anything in my searches that pertain to this specific message. Does anyone know what I need to change/try to get the specific lines replaced in my .php file?
Updated script with same message:
declare -a js_strings=("footer." "footerJQuery." "headerCSS." "headerJQuery.")
build_path="./build/build"
config_path="./system/Config/main.config.php"
while read -r line;
do
for js_string in ${js_strings[#]}
do
if [[ $line == *$js_string* ]]
then
for js_file in "$build_path"/*
do
result="${js_file//[^.]}"
if [[ $js_file == *$js_string* ]] && [[ ${#result} -eq 3 ]]
then
filename=$(basename $js_file)
newline="${line//$js_string*/$filename\";}"
echo $line
echo $newline
sed -i "s\\$line\\$newline\\g" $config_path
echo ""
fi
done
fi
done
done < "$config_path"
Example $line:
$config['public_build_header_css_url'] = "http://localhost:8080/build/headerCSS.js";
Example $newline:
$config['public_build_header_css_url'] = "http://localhost:8080/build/headerCSS.7529a73071877d127676.js";
Updated script with changes suggested by #Vercingatorix:
declare -a js_strings=("footer." "footerJQuery." "headerCSS." "headerJQuery.")
build_path="./build/build"
config_path="./system/Config/main.config.php"
while read -r line;
do
for js_string in ${js_strings[#]}
do
if [[ $line == *$js_string* ]]
then
for js_file in "$build_path"/*
do
result="${js_file//[^.]}"
if [[ $js_file == *$js_string* ]] && [[ ${#result} -eq 3 ]]
then
filename=$(basename $js_file)
newline="${line//$js_string*/$filename\";}"
echo $line
echo $newline
linenum=$(grep -n "^${line}\$" ${config_path} | cut -d':' -f 1 )
echo $linenum
[[ -n "${linenum}" ]] && sed -i "${linenum}a\\
${newline}
;${linenum}d" ${config_path}
echo ""
fi
done
fi
done
done < "$config_path"
Using sed's s command to replace a line of that complexity is a losing proposition, because whatever delimiter you choose may appear in the line and mess things up. If these are in fact entire lines, it is better to delete them and insert a new one:
linenum=$(fgrep -nx -f "${line}" "${config_path}" | awk -F : "{print \$1}" )
[[ -n "${linenum}" ]] && sed -i "" "${linenum}a\\
${newline}
;${linenum}d" "${config_path}"
What this does is search for the line number of the line that matches $line in its entirety, then extracts the line number portion. fgrep is necessary otherwise the symbols in your file are interpreted as regular expressions. If there was a match, then it runs sed, appending the new line (a) and deleting the old one (d).

checking a string in bash to see if its a domain

I am working on a project in Bash that takes a live xlsx file, converts it into a csv file, and checks the file to make sure that the data inside it are urls. This is part of a larger progragam that will eventually test each url for domain squatting.
I am having problems with the verification of the string data. I am having to teach myself bash as i go along since this is a self study class. Thanks for the Help!
INPUT=domain3.csv
while IFS= read -r line
do
if [[ "$line" == *".com"*] || [ "$line" == *".net"*] || [ "$line" == *".org"*] || [ "$line" == *".biz"*]];
then echo "$line"
else echo "$line is not an URL"
fi
echo "Finished!"
done
Use the =~ to perform regular expression match:
if [[ $INPUT =~ \.(com|net|org)$ ]]
then
echo $INPUT is a domain
else
echo $INPUT is not a domain
fi
The expression reads that if $INPUT matches a dot (\.), then one of "com", "net", or "org", then end of string ($), then it is a domain.
[[ ... ]] (since bash 4.1) temporarily enables the extglob option, so you can write
if [[ "$line" == *.#(com|net|org|biz)* ]];
You probably don't actually want the trailing *, which would let you match things like foo.comzzz.
A case statement.
#!/bin/sh
while IFS= read -r line; do
case $line in
*.com|*.net|*.org|*.biz)
echo "$line";;
*) printf >&2 '%s is not a url!\n' "$line" ;;
esac
done
Please execute the below code once and then compare it with your's to find out the error.
while IFS= read -r line
do
if [[ "$line" == *".com"* ]] || [[ "$line" == *".net"* ]] || [[ "$line" == *".org"* ]] || [[ "$line" == *".biz"* ]]
then
echo "$line"
else
echo "$line is not an URL"
fi
done < $INPUT
echo "Finished!"

Losing newlines when reading from stdin

I have been beating my head up about this.
I wanted to loop over a multiline string character by character in bash but was loosing all newlines. First thing I did when i didn't find any obvious error was to run shellcheck on it, it seemed fine with the program.
script.sh:
#!/usr/bin/env bash
transform_single() {
if [[ $# -ne 1 ]]; then
echo 'Error: illegal number of args' 1>&2
fi
equation=''
delim0=0
delim="$1"
while IFS= read -rn1 c; do
if [[ $delim0 -eq 0 ]] && [[ "$c" == "$delim" ]]; then
delim0=1
equation=''
elif [[ $delim0 -ne 0 ]] && [[ "$c" == "$delim" ]]; then
delim0=0
echo -n "$equation" | texmath
elif [[ $delim0 -ne 0 ]]; then
equation="$equation$c"
else
echo -n "$c"
fi
done
}
transform_single '$'
input.txt:
<newlines>
<newlines>
# Hello world!
<newlines>
This is a test string.
<newlines>
invocation:
bash script.sh < input.txt
output:
# Hello world!This is a test string.
excepted output:
The same as in the input file.
Working script
#!/bin/bash
transform_single() {
if (($# != 1)); then echo 'Error: illegal number of args' 1>&2; fi
equation=''
delim0=0
delim="$1"
while IFS= read -r -d $'\0' -n 1 c; do
if ((delim0 == 0)) && [[ "$c" == "$delim" ]]; then
delim0=1
equation=''
elif ((delim0 != 0)) && [[ "$c" == "$delim" ]]; then
delim0=0
echo -n "$equation" | texmath
elif ((delim0 != 0)); then
equation="$equation$c"
else
echo -n "$c"
fi
done
}
transform_single '$'
The issue is that you must set a delimiter to read, a null character, to preserve line feed.

Increment value in for loop while reading a file

My code looks something like this:
for line in `cat fileName`
do
if [[ $line == "Marker 1" ]]
then
while [[ $line != "---" ]]
do
#basically I want to read all the data below "Marker 1" till "---"
echo $line
((line++)) #this is wrong
done
elif [[ $line == "Marker 2" ]]
then
while [[ $line != "---" ]]
do
echo $line
((line++))
done
fi
done
How do I increment the value of $line when in the while loop? ((line++)) doesn't work
Using sed
If the goal is to echo all the lines from a line with Marker 1 or Marker 2 to a line with ---, then that entire shell loop can be replaced with this simple sed command:
sed -n '/^Marker [12]$/,/^---$/p' File
Example
Consider this test file:
$ cat File
beginning
Marker 1
one
---
more
Marker 2
two
Two
---
end
Now, let's run our command:
$ sed -n '/^Marker [12]$/,/^---$/p' File
Marker 1
one
---
Marker 2
two
Two
---
Using awk
With the same test file:
$ awk '/^Marker [12]$/,/^---$/' File
Marker 1
one
---
Marker 2
two
Two
---
((line++)) is for incrementing an integer value.
But the value of line in your example is a string.
In fact what you seem to want is get the next line from the file.
You need a different approach for that, and use while read instead of a for-loop.
#!/usr/bin/env bash
read_and_print_until_dashes() {
while read -r line; do
[[ $line = '---' ]] && break
echo "$line"
done
}
while read -r line; do
if [[ $line = "Marker 1" ]]; then
echo "$line"
read_and_print_until_dashes
elif [[ $line = "Marker 2" ]]; then
echo "$line"
read_and_print_until_dashes
fi
done < file.txt

Remove Newline Character between two specific strings

I am using Unix Shell. How to remove newline character between two specific strings.
For example, input is:
CASE when a in ('abcd','bdcdf') then
Shng,
END as xyz
Output should be:
CASE when a in ('abcd','bdcdf') then Shng END as xyz,
Parse the file line for line and remember when you see a CASE or END.
The code beneath uses a short syntax for an if-statement and an echo that suppresses \n by the -n parameter.
incase=0
cat x.sql | while read -r line; do
[[ ${line} = CASE* ]] && incase=1;
[[ ${line} = END* ]] && incase=0
[[ ${incase} = 0 ]] && echo "${line}"
[[ ${incase} = 1 ]] && echo -n "${line} "
done
EDIT:
When you have nested CASEs (like CASE ... CASE ... END ... END) and all
CASEs start on different lines you can count how deep your nested.
incase=0
cat x.sql | while read -r line; do
[[ ${line} = CASE* ]] && (( incase = incase + 1)) ;
[[ ${line} = END* ]] && (( incase = incase - 1))
[[ ${incase} = 0 ]] && echo "${line}"
[[ ${incase} > 0 ]] && echo -n "${line} "
done
# You might want an extra echo here so your last line will finish with a \n
echo
EDIT 2: Often you can avoid cat (look for UUOC). Here the code is better as
incase=0
cat x.sql | while read -r line; do
[[ ${line} = CASE* ]] && incase=1;
[[ ${line} = END* ]] && incase=0
[[ ${incase} = 0 ]] && echo "${line}"
[[ ${incase} = 1 ]] && echo -n "${line} "
done
EDIT:
When you have nested CASEs (like CASE ... CASE ... END ... END) and all
CASEs start on different lines you can count how deep your nested.
incase=0
while read -r line; do
[[ ${line} = CASE* ]] && (( incase = incase + 1)) ;
[[ ${line} = END* ]] && (( incase = incase - 1))
[[ ${incase} = 0 ]] && echo "${line}"
[[ ${incase} > 0 ]] && echo -n "${line} "
done < x.sql
# You might want an extra echo here so your last line will finish with a \n
echo

Resources