so in a bash script, have a while loop that read lines from the outpt of some pippelined sort commands. i get an error: line 13: syntax error near unexpected token 'done'
line 13 is the last line, where the "done" and the pipeline is. my script:
#!/bin/bash
while read -a line; do
if [[ ${line[2]} < $1 ]]; then
continue
fi
if [[ $2 != -MM ]]; then
if [[ ${line[3]} = N ]]; then
continue
fi
fi
echo -n "${line[0]} "
echo ${line[1]}
done < <(sort -b rooms | sort -sk2 | sort -sk4 | sort -sk3)
tnx.
I would first try to write the date to temp so I can see what the sorting is doing by inspecting the temp file. Then read that in line by line.
#!/usr/bin/env bash
TMPF="/tmp/tmp-file.dat"
sort -b rooms | sort -sk2 | sort -sk4 | sort -sk3 > "${TMPF}"
while IFS= read -r line ;do
if [[ ${line[2]} -ge $1 ]] && [[ $2 != -MM ]] && [[ ${line[3]} = N ]]; then
echo -n "${line[0]} "
echo ${line[1]}
fi
done < "${TMPF}"
You can use without using ()
like done < rooms |sort -b | sort -sk2 | sort -sk4 | sort -sk3
Related
I have a bash while read line block reading from a text file specified by $filename:
IFS=''
while read -r line
do
...
done < $filename
Instead of reading the whole file each time, I would like to supply different inputs in the redirect depending on the arguments supplied to the script.
Whole file: done < "$filename"
start at line x: done < <(tail -n +"$x" "$filename")
line x to line y: done < <(tail -n +"$x" "$filename" | head -n "$y")
start to line y: done < <(head -n "$y" "$filename")
How can I assign these inputs to a variable ahead of time to be read by the while loop?
My input file is ~4GB with some 58M lines (all with different lengths), and may grow or shrink from time to time. Reading https://unix.stackexchange.com/questions/47407/cat-line-x-to-line-y-on-a-huge-file it appears that tail | head is the fastest method to read from the middle of a file, so given the file size, I'm deliberately avoiding awk and sed for the most part.
Your data is too big to read in whole. The good news is that the contents of a process substitution is a shell script, so you can write:
while IFS= read -r line; do
...
done < <(
if [[ $x && $y ]]; then tail -n +"$x" "$filename" | head -n "$y"
elif [[ $x ]]; then tail -n +"$x" "$filename"
elif [[ $y ]]; then head -n "$y" "$filename"
else cat "$filename"
fi
)
One thing I don't like about process substitutions is that code follows the loop for which it is input. It would be nice if it appeared first. I think this will work, but is untested:
# set up file descriptor 3
exec 3< <(
if [[ $x && $y ]]; then tail -n +"$x" "$filename" | head -n "$y"
elif [[ $x ]]; then tail -n +"$x" "$filename"
elif [[ $y ]]; then head -n "$y" "$filename"
else cat "$filename"
fi
)
# iterate over lines read from fd 3
while IFS= read -u3 -r line; do
...
done
# close fd 3
exec 3<&-
I might handle all of these as part of the loop condition, with an explicitly maintained line counter.
start=10
end=30
i=0
while ((i <= end )) && IFS= read -r line; do
(( i++ >= start )) || continue
...
done < "$filename"
However, if you might skip a significant number of lines at the beginning, it might be more efficient to use sed
while IFS= read -r line; do
...
done < <(sed -n "$start,$stop p" "$filename")
or awk:
while IFS= read -r line; do
...
done < <(awk -v start "$start" -v end "$end" 'NR >= start && NR <= end' "$filename")
This then raises the question of how much of the body of the while loop can be moved into awk itself.
For a homework assignment I have to Take the results from the grep command, and write out up to the first 5 of them, numbering them from 1 to 5. (Print the number, then a space, then the line from grep.) If there are no lines, print a message saying so. So far I managed to store the grep command in an array but this is where I've gotten stuck: Can anyone provide guidance as to how to proceed in printing this as stated above
pattern="*.c"
fileList=$(grep -l "main" $pattern)
IFS=$"\n"
declare -a array
array=$fileList
for x in "${array[#]}"; do
echo "$x"
done
you can grep options -c and -l
pattern="*.c"
searchPattern="main"
counter=1
while read -r line ; do
IFS=':' read -r -a lineInfo <<< "$line"
if [[ $counter > 5 ]]; then
exit 1
fi
if [[ ${lineInfo[1]} > 0 ]]; then
numsOfLine=""
while read -r fileline ; do
IFS=':' read -r -a fileLineInfo <<< "$fileline"
numsOfLine="$numsOfLine ${fileLineInfo[0]} "
done < <(grep -n $searchPattern ${lineInfo[0]})
echo "$counter ${lineInfo[0]} match on lines: $numsOfLine"
let "counter += 1"
else
echo "${lineInfo[0]} no match lines"
fi
done < <(grep -c $searchPattern $pattern)
If you're only allowed to use grep and bash(?):
pattern="*.c"
fileList=($(grep -l "main" $pattern))
if test ${#fileList[#]} = 0 ; then
echo "No results"
else
n=0
while test $n -lt ${#fileList[#]} -a $n -lt 5 ; do
i=$n
n=$(( n + 1 ))
echo "$n ${fileList[$i]}"
done
fi
If you are allowed to use commands in addition to grep, you can pipe the results through nl to add line numbers, then head to limit the results to the first 5 lines, then a second grep to test if there were any lines. For example:
if ! grep -l "main" $pattern | \
nl -s ' ' | sed -e 's/^ *//' | \
head -n 5 | grep '' ; then
echo "No results"
fi
I want to separate directories from files in a list. I would like them to appear as follows:
DirectoryName1
DirectoryNameA
DirectoryName_Two
--
FileName1
FileNameA
FileName_Two
Basically, I want two or three dashes in between my directories and files.
Here is what the following code looks like.
DirectoryName1
DirectoryNameA
DirectoryName_Two
FileName1
FileNameA
FileName_Two
Here is my code:
#!/bin/bash
if [[ $# -ge 1 ]]; then
cd "$1" 2> /dev/null
if [[ $? = 1 ]]; then
echo "Please enter a valid directory."
else
ls -a | sort -k 1 | awk '{printf "(%d) %s\n", NR, $0;}'
fi
else
ls -a | sort -k 1| awk '{printf "(%d) %s\n", NR, $0;}'
fi
Here's one possible solution:
#!/bin/bash
if [[ $# -ge 1 ]]; then
dir_to_list=$1
if [[ ! -d ${dir_to_list} ]]; then
echo "Please enter a valid directory."
exit
fi
else
dir_to_list="."
fi
files=`ls --group-directories-first $dir_to_list`
DIRS="TRUE"
i=0
for f in ${files}; do
if [[ ${DIRS} == "TRUE" && ! -d ${dir_to_list}/${f} ]]; then
# First non-directory entry
echo ----
DIRS="FALSE"
fi
(( i++ ))
echo ${i}. ${f}
done
Cheers
Update: fixed bug for listing other directories
I am trying to create a bash dictionary script that accepts first argument and creates file named after that, then script accepts next arguments (which are files inside same folder) and outputs their content into file (first argument). It also sorts, deletes symbols etc., but main problem is, that sometimes ouptut file is empty (I am passing one non empty file and one non existing file), after deleting and running script few more times it is sometimes empty sometimes not.
#!/bin/bash
numberoffileargs=$(( $# - 1 ))
exitstat=0
counterexit=0
acceptingstdin=0;
> "$1";
#check if we have given input files given
if [ "$#" -gt 1 ]; then
#for cycle going through input files
for i in "${#:2}"
do
#check whether input file is readable
if [ -r "${i}" ]; then
cat "${i}" >> "$1"
#else redirect to standard output
else
exitstat=2
counterexit=$((counterexit + 1))
echo "file does not exist" 1>&2
fi
done
else
echo "stdin code to be done"
acceptingstdin=1
#stdin input to output file
#stdin=$(cat)
fi
#one word for each line, alphabetical sort, alphabet only, remove duplicates
#all lowercase
#sort -u >> "$1"
if [ "$counterexit" -eq "$numberoffileargs" ] && [ "$acceptingstdin" -eq 0 ]; then
exitstat=3
fi
cat "$1" | sed -r 's/[^a-zA-Z\-]+/ /g' | tr A-Z a-z | tr ' ' '\n' | sort -u | sed '/^$/d' > "$1"
echo "$numberoffileargs"
echo "$counterexit"
echo "$exitstat"
exit $exitstat
Here is your script with some syntax improvement. Your trouble came from the fact that the dictionary was both on input and output on your pipeline; I added a temp file to fix it.
#!/bin/bash
(($# >= 1)) || { echo "Usage: $0 dictionary file ..." >&2 ; exit 1;}
dict="$1"
shift
echo "Creating $dict ..."
>| "$dict" || { echo "Failed." >&2 ; exit 1;}
numberoffileargs=$#
exitstat=0
counterexit=0
acceptingstdin=0
if (($# > 0)); then
for i ; do
#check whether input file is readable
if [ -r "${i}" ]; then
cat "${i}" >> "$dict"
else
exitstat=2
let counterexit++
echo "file does not exist" >&2
fi
done
else
echo "stdin code to be done"
acceptingstdin=1
fi
if ((counterexit == numberoffileargs && acceptingstdin == 0)); then
exitstat=3
fi
sed -r 's/[^a-zA-Z\-]+/ /g' < "$dict" | tr '[:upper:]' '[:lower:]' | tr ' ' '\n' |
sort -u | sed '/^$/d' >| tmp$$
mv -f tmp$$ "$dict"
echo "$numberoffileargs"
echo "$counterexit"
echo "$exitstat"
exit $exitstat
The pipeline might be improved.
Please excuse this extremely inefficient script, I am new to shell scripting. I am receiving an error near the if clause in the function matchFS(). I have posted the error down below. Can anyone offer me some guidance?
#!/bin/bash
function matchFS() {
usage=$(df -h | tail -n +2 | awk '{print $5}' | sed 's/%//g')
usagearr=( $usage )
for i in "${usagearr[#]}"
do
if [[ $1 eq "${usagearr[$i]}" ]]; then
# print matching row from df -h
fi
done
}
usage=$(df -h | tail -n +2 | awk '{print $5}' | sed 's/%//g')
usagearr=( $usage )
len=${#usagearr[#]}
for (( i=0; i<$len; i++ )) # we have to use (( )) here to represent the c style for loop
do
if [ "${usagearr[$i]}" -gt "10" ]; then
matchFS ${usagearr[$i]}
fi
done
Error: line 13: conditional binary operator expected
line 13: syntax error near eq'
line 13: if [[ $1 eq "49 ]]; then'
If you look at help test you'll quickly realize that eq is not one of the choices. At least, not without adding something else to it.
#!/bin/bash
function matchFS() {
### duplicate definition, these are already known to the function.
usage=$(df -h | tail -n +2 | awk '{print $5}' | sed 's/%//g')
usagearr=( $usage )
### you probably did want to use another variable here,
### because the "i" is also shared with the caller
for i in "${usagearr[#]}"
do
### -eq instead of eq
if [[ $1 -eq "${usagearr[$i]}" ]]; then
### the if statement can not be empty
# print matching row from df -h
fi
done
}
usage=$(df -h | tail -n +2 | awk '{print $5}' | sed 's/%//g')
usagearr=( $usage )
len=${#usagearr[#]}
for (( i=0; i<$len; i++ )) # we have to use (( )) here to represent the c style for loop
do
if [ "${usagearr[$i]}" -gt "10" ]; then
matchFS ${usagearr[$i]}
fi
done