Increment value in for loop while reading a file - shell

My code looks something like this:
for line in `cat fileName`
do
if [[ $line == "Marker 1" ]]
then
while [[ $line != "---" ]]
do
#basically I want to read all the data below "Marker 1" till "---"
echo $line
((line++)) #this is wrong
done
elif [[ $line == "Marker 2" ]]
then
while [[ $line != "---" ]]
do
echo $line
((line++))
done
fi
done
How do I increment the value of $line when in the while loop? ((line++)) doesn't work

Using sed
If the goal is to echo all the lines from a line with Marker 1 or Marker 2 to a line with ---, then that entire shell loop can be replaced with this simple sed command:
sed -n '/^Marker [12]$/,/^---$/p' File
Example
Consider this test file:
$ cat File
beginning
Marker 1
one
---
more
Marker 2
two
Two
---
end
Now, let's run our command:
$ sed -n '/^Marker [12]$/,/^---$/p' File
Marker 1
one
---
Marker 2
two
Two
---
Using awk
With the same test file:
$ awk '/^Marker [12]$/,/^---$/' File
Marker 1
one
---
Marker 2
two
Two
---

((line++)) is for incrementing an integer value.
But the value of line in your example is a string.
In fact what you seem to want is get the next line from the file.
You need a different approach for that, and use while read instead of a for-loop.
#!/usr/bin/env bash
read_and_print_until_dashes() {
while read -r line; do
[[ $line = '---' ]] && break
echo "$line"
done
}
while read -r line; do
if [[ $line = "Marker 1" ]]; then
echo "$line"
read_and_print_until_dashes
elif [[ $line = "Marker 2" ]]; then
echo "$line"
read_and_print_until_dashes
fi
done < file.txt

Related

Bash sed command gives me "invalid command code ."

I'm trying to automate a build process by replacing .js chunks for particular lines in my main.config.php file. When I run the following code:
declare -a js_strings=("footer." "footerJQuery." "headerCSS." "headerJQuery.")
build_path="./build/build"
config_path="./system/Config/main.config.php"
while read -r line;
do
for js_string in ${js_strings[#]}
do
if [[ $line == *$js_string* ]]
then
for js_file in "$build_path"/*
do
result="${js_file//[^.]}"
if [[ $js_file == *$js_string* ]] && [[ ${#result} -eq 3 ]]
then
sed -i "s/$line/$line$(basename $js_file)\";/g" $config_path
fi
done
fi
done
done < "$config_path"
I get this message back, and file has not been updated/edited:
sed: 1: "./system/Config/main.co ...": invalid command code .
I haven't been able to find anything in my searches that pertain to this specific message. Does anyone know what I need to change/try to get the specific lines replaced in my .php file?
Updated script with same message:
declare -a js_strings=("footer." "footerJQuery." "headerCSS." "headerJQuery.")
build_path="./build/build"
config_path="./system/Config/main.config.php"
while read -r line;
do
for js_string in ${js_strings[#]}
do
if [[ $line == *$js_string* ]]
then
for js_file in "$build_path"/*
do
result="${js_file//[^.]}"
if [[ $js_file == *$js_string* ]] && [[ ${#result} -eq 3 ]]
then
filename=$(basename $js_file)
newline="${line//$js_string*/$filename\";}"
echo $line
echo $newline
sed -i "s\\$line\\$newline\\g" $config_path
echo ""
fi
done
fi
done
done < "$config_path"
Example $line:
$config['public_build_header_css_url'] = "http://localhost:8080/build/headerCSS.js";
Example $newline:
$config['public_build_header_css_url'] = "http://localhost:8080/build/headerCSS.7529a73071877d127676.js";
Updated script with changes suggested by #Vercingatorix:
declare -a js_strings=("footer." "footerJQuery." "headerCSS." "headerJQuery.")
build_path="./build/build"
config_path="./system/Config/main.config.php"
while read -r line;
do
for js_string in ${js_strings[#]}
do
if [[ $line == *$js_string* ]]
then
for js_file in "$build_path"/*
do
result="${js_file//[^.]}"
if [[ $js_file == *$js_string* ]] && [[ ${#result} -eq 3 ]]
then
filename=$(basename $js_file)
newline="${line//$js_string*/$filename\";}"
echo $line
echo $newline
linenum=$(grep -n "^${line}\$" ${config_path} | cut -d':' -f 1 )
echo $linenum
[[ -n "${linenum}" ]] && sed -i "${linenum}a\\
${newline}
;${linenum}d" ${config_path}
echo ""
fi
done
fi
done
done < "$config_path"
Using sed's s command to replace a line of that complexity is a losing proposition, because whatever delimiter you choose may appear in the line and mess things up. If these are in fact entire lines, it is better to delete them and insert a new one:
linenum=$(fgrep -nx -f "${line}" "${config_path}" | awk -F : "{print \$1}" )
[[ -n "${linenum}" ]] && sed -i "" "${linenum}a\\
${newline}
;${linenum}d" "${config_path}"
What this does is search for the line number of the line that matches $line in its entirety, then extracts the line number portion. fgrep is necessary otherwise the symbols in your file are interpreted as regular expressions. If there was a match, then it runs sed, appending the new line (a) and deleting the old one (d).

Shell - Skip matching lines while read

I'm looking for help with my shell... Hope I'll find it here...
Here's my code :
#!/bin/sh
while IFS= read -r line || [ -n "$line" ]
do
[[ "$line" =~ ^[[:space:]]*\# ]] && continue # This line must stay
[[ "$line" =~ *read[[:space:]]-r[[:space:]]line* ]] && continue
echo "${line%$NL}"
done < $0
First test will suppress "only comment lines".
Second test purpose is to suppress the "while IFS= read..." lines - no matter what, it's just a test :-)
"done < $0" has been here written intentionaly... for the test !
Running the shell outputs this :
while IFS= read -r line || [ -n "$line" ]
do
[[ "$line" =~ ^[[:space:]]*\# ]] && continue # This line must stay
[[ "$line" =~ *read[[:space:]]-r[[:space:]]line* ]] && continue
echo "${line%$NL}"
done < $0
as I thought the first line will be gone because of matching then 2nd test.
What's my mistake ?
For the record, I don't want to use extra sed or awk sentence.
Actually, the input data (here $0 file) has to be standard input (eg extract from tee command). I read lot of stackOverflow subject about this, with sed or awk responses that didn't match my purpose.
The regex is invalid. A short test shows:
> [[ 'while IFS= read -r line || [ -n "$line" ]' =~ *read[[:space:]]-r[[:space:]]line* ]]
> echo $?
2
The * can't be "alone" - it can't be the first character in a POSIX extended regular expression. It has to "bind" to something, ex. a dot .. A dot represents any character. You want:
[[ $line =~ .*read[[:space:]]-r[[:space:]]line.* ]]
Problem is presence of starting quantifier * in the regex in second continue line. You may use:
[[ "$line" =~ read[[:space:]]+-r[[:space:]]+line ]] && continue
There is no need to match anything before read or after line in this regex.
Also it is better to use quantifier + after [[:space:]] to make it match 1 or more white spaces.
You can do more refactoring and combine both regex into one by using alternation as in this code:
while IFS= read -r line || [[ -n $line ]]
do
[[ $line =~ ^[[:space:]]*#|read[[:space:]]+-r[[:space:]]+line ]] && continue
echo "${line%$NL}"
done < $0

Using sed to replace single line in while read loop

I'm needing help setting up a bash script for initializing some BC's in a file. Ideally, my program would iterate through each line and:
1) Read in BC type - (wall, outlet, inlet).
2) Change "type" field based on appropriate BC type.
Unfortunately, my program seems to replace all type fields in Step 2 instead of only the type field associated with the correct BC.
I think this has something to do with the sed command operating over the whole file instead of just the $line variable.
while IFS= read -r line || [[ -n "$line" ]]; do #go through each line of T2 file
if [[ $line == *wall_* ]] #if wall_*
then
echo "attempted to assign wall_*"
var=1 #wall #Go down 2 lines
elif [[ $line == *velocity-inlet* ]]
then
echo "attempted to assign outflow"
var=2 #inlet
elif [[ $line == *outflow* ]]
then
var=3 #outlet
fi
echo $var
if [[ $line == *type* && $var == 1 ]]
then
sed -i 's/.*type.*/type zeroGradient/' 0/T3
echo "Attempted wall zeroGradient"
elif [[ $line == *type* && $var == 2 ]]
then
sed -i 's/.*type.*/type fixedValue\nvalue uniform (3 0 0)/' 0/T3
elif [[ $line == type* && $var == 3 ]]
then
sed -i 's/.*type.*/type zeroGradient/' 0/T3
fi
sed -i '/nFaces*/d' 0/T3 #Deletes irrelevant stuff from boundary file copy
sed -i '/startFace*/d' 0/T3
done <0/T3.
For example, it is supposed to change:
velocity-inlet_1
{
type patch;
nFaces 336;
startFace 75515;
}
outflow_2
{
type patch;
nFaces 136;
startFace 75851;
}
To:
velocity-inlet_1
{
type fixedValue;
value uniform (3 0 0);
}
outflow_2
{
type zeroGradient;
}
But instead changes it wrongly changes it to:
velocity-inlet_1
{
type fixedValue;
value uniform (3 0 0);
}
outflow_2
{
type fixedValue;
value uniform (3 0 0);
}
Help me stack overflow, you're my only hope.
You have a few issues. sed will effect a whole line by default, and you're not telling it which line to modify in the first place. You're also modifying a file as you read it. I might go with something like this:
var="none"
while IFS= read -r line; do
if [[ "$line" =~ "wall" ]]; then
var=wall
elif [[ "$line" =~ "velocity-inlet" ]]; then
var=inlet
fi
if [[ "$line" =~ "type" && "$var" == "wall" ]]; then
echo "$line" | sed 's|\(type *\).*|\1zeroGradient|'
elif [[ "$line" =~ "type" && "$var" == "inlet" ]]; then
echo "$line" | sed 's|\(type *\).*|\1uniform (3 0 0)|'
else
echo "$line"
fi
done
And then do
script.sh < 0/T3 > 0/T3.mod
You can of course modify this to read/write from particular files as well, and you can avoid sed (see here...)

how to search for blank lines in bash

I am trying to create an if statement that performs an action when it reads a blank line.
I would assume it would be something like this : if ($line=='\n');then
where line is the line that it is reading from a text file. But this is not working.
while read line; do
if [ "$line" = "" ]; then
echo BLANK
fi
done < filename.txt
or a slight variation:
while read line; do
if [ "$line" ]; then
echo NOT BLANK
else
echo BLANK
fi
done < filename.txt
try this:
if [[ "x$line" == "x" ]]; then...
or
if [[ "$line" =~ "^$" ]]; ...
Or also:
grep -q '.' <<< $line
Returns 1 if line is empty, 0 if non-empty

How can I test if line is empty in shell script?

I have a shell script like this:
cat file | while read line
do
# run some commands using $line
done
Now I need to check if the line contains any non-whitespace character ([\n\t ]), and if not, skip it.
How can I do this?
Since read reads whitespace-delimited fields by default, a line containing only whitespace should result in the empty string being assigned to the variable, so you should be able to skip empty lines with just:
[ -z "$line" ] && continue
try this
while read line;
do
if [ "$line" != "" ]; then
# Do something here
fi
done < $SOURCE_FILE
bash:
if [[ ! $line =~ [^[:space:]] ]] ; then
continue
fi
And use done < file instead of cat file | while, unless you know why you'd use the latter.
cat i useless in this case if you are using while read loop. I am not sure if you meant you want to skip lines that is empty or if you want to skip lines that also contain at least a white space.
i=0
while read -r line
do
((i++)) # or $(echo $i+1|bc) with sh
case "$line" in
"") echo "blank line at line: $i ";;
*" "*) echo "line with blanks at $i";;
*[[:blank:]]*) echo "line with blanks at $i";;
esac
done <"file"
if ! grep -q '[^[:space:]]' ; then
continue
fi
blank=`tail -1 <file-location>`
if [ -z "$blank" ]
then
echo "end of the line is the blank line"
else
echo "their is something in last line"
fi
awk 'NF' file | while read line
do
# run some commands using $line
done
stole this answer to a similar question:
Delete empty lines using sed

Resources