I am trying to read lines from a file containing multiple lines. I want to identify lines that contain only spaces.
By definition, an empty line is empty and does not contain anything (including spaces).
I want to detect lines that seems to be empty but they are not (lines that contain spaces only)
while read line; do
if [[ `echo "$line" | wc -w` == 0 && `echo "$line" | wc -c` > 1 ]];
then
echo "Fake empty line detected"
fi
done < "$1"
But because read ignores spaces in the start and in the end of a string my code isn't working.
an example of a file
hi
hi
(empty line, no spaces or any other char)
hi
(two spaces)
hey
Please help me to fix the code
Disable word splitting by clearing the value of IFS (the internal field separator):
while IFS= read -r line; do
....
done < "$1"
The -r isn't strictly necessary, but it is good practice.
Also, a simpler way to check the value of line (I assume you're looking for a line with nothing but whitespace):
if [[ $line =~ ^$ ]]; then
echo "Fake empty line detected"
fi
Following your code, it can be improved.
while read line; do
if [ -z "$line" ]
then
echo "Fake empty line detected"
fi
done < "$1"
The test -z checks if $line is empty.
Output:
Fake empty line detected
Fake empty line detected
Related
I am processing a developer commit message in git hook.
let's say the file has the following content
\n new lines here
# this is a sample commit
# only for the developers
Ticket-ID: we fix old bugs and introduces new ones
we always do this stuff
so cool, not really :P
# company name
My intention is to get only this line Ticket-ID : we fix old bugs and introduces new ones
User123's comment is nice and terse: grep -E "^[[:alnum:]]" file |head -n 1 however is does not catch lines of text that start with non-alphanumeric characters that are not a # such as commit messages that start with an emoji, dashes, parenthesis, etc..
🚀 yeah this line is an exception
--> This is also an edge case
(so is this)
To catch all edge cases you can loop through the file and check each $line with a negated ! regexp operator =~ for:
Not being a newline ! $line =~ (^[^\n ]*$)
Not starting with a pound sign ! $line =~ ^#
Not being a line consisting of all spaces ! $line =~ (^[ ]*$)
Then just echo the $line and break if those conditions are met:
# file parse.sh
#!/bin/bash
if [[ -f $1 ]]; then
while IFS= read -r line
do
[[ ! $line =~ (^[^\n ]*$) && ! $line =~ ^# && ! $line =~ (^[ ]*$) ]] && echo "$line" && break
done < "$1"
fi
# file commit .txt
# this is a sample commit
# only for the developers
Ticket-ID: we fix old bugs and introduces new ones
we always do this stuff
so cool, not really :P
# company name
Now you can invoke the parse.sh like this
bash parse.sh commit.txt
Or save the results to a variable using a subshell
result=$(bash parse.sh commit.txt); echo "$result"
Below single line grep should work as per your requirement:
grep -E "^[[:alnum:]]" file |head -n 1
Explanation:
^[[:alnum:]] :: to capture only the line starting with any alphanumeric character[0-9A-Za-z]
head -n 1 :: to capture the first occurrence
I have an exercise where I have a file and at the begin of it I have something like
#!usr/bin/bash
# tototata
#tititutu
#ttta
Hello world
Hi
Test test
#zabdazj
#this is it
And I have to take each first line starting with a # until the line where I don't have one and stock it in a variable. In case of a shebang, it has to skip it and if there's blank space between lines, it has to skip them too. We just want the comment between the shebang and the next character.
I'm new to bash and I would like to know if there's a way to do it please ?
Expected output:
# tototata
#tititutu
#ttta
Try in this easy way to better understand.
#!/bin/bash
sed 1d your_input_file | while read line;
do
check=$( echo $line | grep ^"[#;]" )
if ([ ! -z "$check" ] || [ -z "$line" ])
then
echo $line;
else
exit 1;
fi
done
This may be more correct, although your question was unclear about weather the input file had a script shebang, if the shebang had to be skipped to match your sample output, or if the input file shebang was just bogus.
It is also unclear for what to do, if the first lines of the input file are not starting with #.
You should really post your assignment's text as a reference.
Anyway here is a script that does collects first set of consecutive lines starting with a sharp # into the arr array variable.
It may not be an exact solution to your assignment (witch you should be able to solve with what your previous lessons taught you), but will get you some clues and keys to iterate reading lines from a file and testing that lines starts with a #.
#!/usr/bin/env bash
# Our variable to store parsed lines
# Is an array of strings with an entry per line
declare -a arr=()
# Iterate reading lines from the file
# while it matches Regex: ^[#]
# mean while lines starts with a sharp #
while IFS=$'\n' read -r line && [[ "$line" =~ ^[#] ]]; do
# Add line to the arr array variable
arr+=("$line")
done <a.txt
# Print each array entries with a newline
printf '%s\n' "${arr[#]}"
How about this (not tested, so you may have to debug it a bit, but my comments in the code should explain what is going on):
while read line
do
# initial is 1 one the first line, and 0 after this. When the script starts,
# the variable is undefined.
: ${initial:=1}
# Test for lines starting with #. Need to quote the hash
# so that it is not taken as comment.
if [[ $line == '#'* ]]
then
# Test for initial #!
if (( initial == 1 )) && [[ $line == '#!'* ]]
then
: # ignore it
else
echo $line # or do whatever you want to do with it
fi
fi
# stop on non-blank, non-comment line
if [[ $line != *[^\ ]* ]]
then
break
fi
initial=0 # Next line won't be an initial line
done < your_file
I am trying to do some math on 2nd column of a txt file , but some lines are not numbers , i only want to operate on the lines which have numbers .and keep other line unchanged
txt file like below
aaaaa
1 2
3 4
How can I do this?
Doubling the second column in any line that doesn't contain any alphabetic content might look a bit like the following in native bash:
#!/bin/bash
# iterate over lines in input file
while IFS= read -r line; do
if [[ $line = *[[:alpha:]]* ]]; then
# line contains letters; emit unmodified
printf '%s\n' "$line"
else
# break into a variable for the first word, one for the second, one for the rest
read -r first second rest <<<"$line"
if [[ $second ]]; then
# we extracted a second word: emit it, doubled, between the first word and the rest
printf '%s\n' "$first $(( second * 2 )) $rest"
else
# no second word: just emit the whole line unmodified
printf '%s\n' "$line"
fi
fi
done
This reads from stdin and writes to stdout, so usage is something like:
./yourscript <infile >outfile
thanks all ,this is my second time to use this website ,i find it is so helpful that it can get the answer very quickly
I also find a answer below
#!/bin/bash
FILE=$1
while read f1 f2 ;do
if[[$f1 != *[!0-9]*]];then
f2=`echo "$f2 -1"|bc` ;
echo "$f1 $f2"
else
echo "$f1 $f2"
fi
done< %FILE
I have a problem with writing bash script. The problem is in comparison of strings. When I launch it, there's no errors. However in result, it is always changing the variable client.
So if for an example we have two lines in file
apple A
orange D
and if I give the who=A I expect to see in result apple, or if at D - orange
But no matter of what I choose A or D it is always giving me the result - orange
No matter of the strings, it always change the variable client, like ignoring the comparison. Please help.
while read line
do
IFS=" "
set -- $line
echo $2" "$who":"$1
if [[ "$2"="$who" ]]
then
echo "change"
client=$1
fi
done < $file
echo $client
So now I changed the code as in one of the comment below, but now the caparison always false therefore the variable client is always empty
while read -r line
do
#IFS=" "
#set -- $line
#echo $2" "$who":"$1
#if [[ "$2" = "$who" ]]
a="${line% *}"
l="${line#* }"
if [[ "$l" == "$who" ]]
then
echo "hi"
client="$a"
fi
done < $file
If you have data in a file with each line like apple D and you want to read the file and separate then items, the parameter expansion/substring extraction is the correct way to process the line. For example (note $who is taken from your problem statement):
while read -r line
do
fruit="${line% *}" # remove from end to space
letter="${line#* }" # remove from start to space
if [[ "$letter" == "$who" ]]
then
echo "change"
client="$fruit"
fi
done < $file
Short Example
Here is a quick example of splitting the words with parameter expansion/substring extraction:
#!/bin/bash
while read -r line
do
fruit="${line% *}"
letter="${line#* }"
echo "fruit: $fruit letter: $letter"
done
exit 0
input
$ cat dat/apple.txt
Apple A
Orange D
output
$ bash apple.sh <dat/apple.txt
fruit: Apple letter: A
fruit: Orange letter: D
Change if [[ "$2"="$who" ]] to
if [[ "$2" = "$who" ]]
spaces around =
Example (for clarification):
who=A
while read line
do
IFS=" "
set -- $line
echo $2" "$who":"$1
if [[ "$2" = "$who" ]]
then
echo "change"
client=$1
fi
done < file #this is the file I used for testing
echo $client
Output:
A A:apple
change
D A:orange
apple
For who=D:
A D:apple
D D:orange
change
orange
You do need spaces around that = operator.
However, I think you're facing yet another issue as you're trying to change the value of the client variable from inside the while loop (which executes in a subshell). I don't think that will work; see this quesion for details.
I'm attempting to read a config file that is formatted as follows:
USER = username
TARGET = arrows
I realize that if I got rid of the spaces, I could simply source the config file, but for security reasons I'm trying to avoid that. I know there is a way to read the config file line by line. I think the process is something like:
Read lines into an array
Filter out all of the lines that start with #
search for the variable names in the array
After that I'm lost. Any and all help would be greatly appreciated. I've tried something like this with no success:
backup2.config>cat ~/1
grep '^[^#].*' | while read one two;do
echo $two
done
I pulled that from a forum post I found, just not sure how to modify it to fit my needs since I'm so new to shell scripting.
http://www.linuxquestions.org/questions/programming-9/bash-shell-program-read-a-configuration-file-276852/
Would it be possible to automatically assign a variable by looping through both arrays?
for (( i = 0 ; i < ${#VALUE[#]} ; i++ ))
do
"${NAME[i]}"=VALUE[i]
done
echo $USER
Such that calling $USER would output "username"? The above code isn't working but I know the solution is something similar to that.
The following script iterates over each line in your input file (vars in my case) and does a pattern match against =. If the equal sign is found it will use Parameter Expansion to parse out the variable name from the value. It then stores each part in it's own array, name and value respectively.
#!/bin/bash
i=0
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
name[i]=${line%% =*}
value[i]=${line#*= }
((i++))
fi
done < vars
echo "total array elements: ${#name[#]}"
echo "name[0]: ${name[0]}"
echo "value[0]: ${value[0]}"
echo "name[1]: ${name[1]}"
echo "value[1]: ${value[1]}"
echo "name array: ${name[#]}"
echo "value array: ${value[#]}"
Input
$ cat vars
sdf
USER = username
TARGET = arrows
asdf
as23
Output
$ ./varscript
total array elements: 2
name[0]: USER
value[0]: username
name[1]: TARGET
value[1]: arrows
name array: USER TARGET
value array: username arrows
First, USER is a shell environment variable, so it might be better if you used something else. Using lowercase or mixed case variable names is a way to avoid name collisions.
#!/bin/bash
configfile="/path/to/file"
shopt -s extglob
while IFS='= ' read lhs rhs
do
if [[ $lhs != *( )#* ]]
then
# you can test for variables to accept or other conditions here
declare $lhs=$rhs
fi
done < "$configfile"
This sets the vars in your file to the value associated with it.
echo "Username: $USER, Target: $TARGET"
would output
Username: username, Target: arrows
Another way to do this using keys and values is with an associative array:
Add this line before the while loop:
declare -A settings
Remove the declare line inside the while loop and replace it with:
settings[$lhs]=$rhs
Then:
# set keys
user=USER
target=TARGET
# access values
echo "Username: ${settings[$user]}, Target: ${settings[$target]}"
would output
Username: username, Target: arrows
I have a script which only takes a very limited number of settings, and processes them one at a time, so I've adapted SiegeX's answer to whitelist the settings I care about and act on them as it comes to them.
I've also removed the requirement for spaces around the = in favour of ignoring any that exist using the trim function from another answer.
function trim()
{
local var=$1;
var="${var#"${var%%[![:space:]]*}"}"; # remove leading whitespace characters
var="${var%"${var##*[![:space:]]}"}"; # remove trailing whitespace characters
echo -n "$var";
}
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
setting_name=$(trim "${line%%=*}");
setting_value=$(trim "${line#*=}");
case "$setting_name" in
max_foos)
prune_foos $setting_value;
;;
max_bars)
prune_bars $setting_value;
;;
*)
echo "Unrecognised setting: $setting_name";
;;
esac;
fi
done <"$config_file";
Thanks SiegeX. I think the later updates you mentioned does not reflect in this URL.
I had to edit the regex to remove the quotes to get it working. With quotes, array returned is empty.
i=0
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
name[i]=${line%% =*}
value[i]=${line##*= }
((i++))
fi
done < vars
A still better version is .
i=0
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
name[i]=`echo $line | cut -d'=' -f 1`
value[i]=`echo $line | cut -d'=' -f 2`
((i++))
fi
done < vars
The first version is seen to have issues if there is no space before and after "=" in the config file. Also if the value is missing, i see that the name and value are populated as same. The second version does not have any of these. In addition it trims out unwanted leading and trailing spaces.
This version reads values that can have = within it. Earlier version splits at first occurance of =.
i=0
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
name[i]=`echo $line | cut -d'=' -f 1`
value[i]=`echo $line | cut -d'=' -f 2-`
((i++))
fi
done < vars