I have a list that looks like this:
sharename:shareX
comment:commentX
sharename:shareY
comment:commentY
sharename:shareZ
comment:commentZ
and so on...
And this is how I would like the list to look like:
shareX;commentX
shareY;commentY
shareZ;commentZ
How can I accomplish that in bash?
Pure Bash:
IFS=':'
while read a b; read c d; do # read 2 lines
echo -e "$b:$d"
done < "$infile"
one liner:
odd=0; for i in `cat list | cut -d":" -f2`; do if [ $odd -eq 0 ]; then echo -ne $i; odd=1; else echo $i; odd=0; fi; done
formatted:
odd=0;
for i in `cat list | cut -d":" -f2`;
do
if [ $odd -eq 0 ];
then
echo -ne $i";";
odd=1;
else
echo $i;
odd=0;
fi;
done
untested, the sed part may be wrong
paste -d ';' - - < filename | sed -r 's/(^|;)[^:]:/\1/g'
This might work for you:
sed '$!N;s/[^:]*:\([^\n]*\)\n[^:]*:/\1;/' file
shareX;commentX
shareY;commentY
shareZ;commentZ
Related
Complete novice in Bash. Trying to iterate thru 1000 gzip files, may be GNU parallel is the solution??
#!/bin/bash
ctr=0
echo "file_name,symbol,record_count" > $1
dir="/data/myfolder"
for f in "$dir"/*.gz; do
gunzip -c $f | while read line;
do
str=`echo $line | cut -d"|" -f1`
if [ "$str" == "H" ]; then
if [ $ctr -gt 0 ]; then
echo "$f,$sym,$ctr" >> $1
fi
ctr=0
sym=`echo $line | cut -d"|" -f3`
echo $sym
else
ctr=$((ctr+1))
fi
done
done
Any help to speed the process will be greatly appreciated !!!
#!/bin/bash
ctr=0
export ctr
echo "file_name,symbol,record_count" > $1
dir="/data/myfolder"
export dir
doit() {
f="$1"
gunzip -c $f | while read line;
do
str=`echo $line | cut -d"|" -f1`
if [ "$str" == "H" ]; then
if [ $ctr -gt 0 ]; then
echo "$f,$sym,$ctr"
fi
ctr=0
sym=`echo $line | cut -d"|" -f3`
echo $sym >&2
else
ctr=$((ctr+1))
fi
done
}
export -f doit
parallel doit ::: *gz 2>&1 > $1
The Bash while read loop is probably your main bottleneck here. Calling multiple external processes for simple field splitting will exacerbate the problem. Briefly,
while IFS="|" read -r first second third rest; do ...
leverages the shell's built-in field splitting functionality, but you probably want to convert the whole thing to a simple Awk script anyway.
echo "file_name,symbol,record_count" > "$1"
for f in "/data/myfolder"/*.gz; do
gunzip -c "$f" |
awk -F "\|" -v f="$f" -v OFS="," '
/H/ { if(ctr) print f, sym, ctr
ctr=0; sym=$3;
print sym >"/dev/stderr"
next }
{ ++ctr }'
done >>"$1"
This vaguely assumes that printing the lone sym is just for diagnostics. It should hopefully not be hard to see how this can be refactored if this is an incorrect assumption.
I am reading a file with lines like:
folder=abc
name=xyz
For some lines line I would like set a variable e.g name=xyz corresponding to the line I have read.
Cutting it down, with name=xyz and folder=abc, I have tried:
while read -r line; do
$line
echo $name
done < /etc/testfile.conf
This gives an error message ./test: line 4: folder=abc: command not found etc.
I have tried "$line" and $($line) and it is the same. Is it possible to do what I whant?
I have succeeded by doing:
while read -r line; do
if [[ "$line" == 'folder'* ]]; then
folder="$(echo "$line" | cut -d'=' -f 2)"
fi
if [[ "$line" == 'name'* ]]; then
name="$(echo "$line" | cut -d'=' -f 2)"
fi
done < /etc/testfile.conf
but this seems messy
for your sample, declare is the safest option:
while read -r line; do
declare "$line"
done
$ echo "$folder"
abc
$ echo "$name"
xyz
Direct approach, use eval.
Different approach, try with source or .:
$ echo "$line"
folder=abc
$ . <(echo "$line")
$ echo "$folder"
abc
But probably the good answer will be to tackle the problem in a different way.
You can clean up your approach a bit without resorting to eval.
while IFS="=" read -r name value; do
case $name in
folder) folder=$value ;;
name) name=$value ;;
esac
done < /etc/testfile.conf
why not only source de file ?
$ . infile ; echo "$name"
xyz
For a homework assignment I have to Take the results from the grep command, and write out up to the first 5 of them, numbering them from 1 to 5. (Print the number, then a space, then the line from grep.) If there are no lines, print a message saying so. So far I managed to store the grep command in an array but this is where I've gotten stuck: Can anyone provide guidance as to how to proceed in printing this as stated above
pattern="*.c"
fileList=$(grep -l "main" $pattern)
IFS=$"\n"
declare -a array
array=$fileList
for x in "${array[#]}"; do
echo "$x"
done
you can grep options -c and -l
pattern="*.c"
searchPattern="main"
counter=1
while read -r line ; do
IFS=':' read -r -a lineInfo <<< "$line"
if [[ $counter > 5 ]]; then
exit 1
fi
if [[ ${lineInfo[1]} > 0 ]]; then
numsOfLine=""
while read -r fileline ; do
IFS=':' read -r -a fileLineInfo <<< "$fileline"
numsOfLine="$numsOfLine ${fileLineInfo[0]} "
done < <(grep -n $searchPattern ${lineInfo[0]})
echo "$counter ${lineInfo[0]} match on lines: $numsOfLine"
let "counter += 1"
else
echo "${lineInfo[0]} no match lines"
fi
done < <(grep -c $searchPattern $pattern)
If you're only allowed to use grep and bash(?):
pattern="*.c"
fileList=($(grep -l "main" $pattern))
if test ${#fileList[#]} = 0 ; then
echo "No results"
else
n=0
while test $n -lt ${#fileList[#]} -a $n -lt 5 ; do
i=$n
n=$(( n + 1 ))
echo "$n ${fileList[$i]}"
done
fi
If you are allowed to use commands in addition to grep, you can pipe the results through nl to add line numbers, then head to limit the results to the first 5 lines, then a second grep to test if there were any lines. For example:
if ! grep -l "main" $pattern | \
nl -s ' ' | sed -e 's/^ *//' | \
head -n 5 | grep '' ; then
echo "No results"
fi
I have written this script to remove text from each line before ::
#!/bin/bash
txt=test.txt
COUNT=$(cat $txt | wc -l)
while [ $COUNT -gt 0 ]; do
data=$(sed -n ${count}p $txt)
sed '$count \c
"${data#*:}"' $txt
let COUNT=COUNT-1
done
I think I have an issue with using variables in commands without spaces. Can anyone tell me what I have done wrong?
I think you are over complicating it. To do this you just need cut:
cut -d':' -f2- file
-d sets the field separator.
-f indicates what fields to use. By saying 2- we indicate "all from the 2nd one on".
Test
$ cat a
hello
hello:man i am here:or there
and:you are here
$ cut -d':' -f2- a
hello
man i am here:or there
you are here
Some comments regarding your code:
#!/bin/bash
txt=test.txt
COUNT=$(cat $txt | wc -l) # you can directly say 'wc -l < "$txt"'
while [ $COUNT -gt 0 ]; do
data=$(sed -n ${count}p $txt) # you are using "count", not "COUNT"
sed '$count \c # same here. And I don't know what
"${data#*:}"' $txt # this sed is supposed to work like
let COUNT=COUNT-1 # you have to say let "COUNT=COUNT-1"
done
Also, it is good to indent the code, so that it shows like:
while ...
do
... things ...
done
All together, I would do:
#!/bin/bash
txt=a
count=$(wc -l < "$txt")
while (( count-- > 0 )); do
data=$(sed -n "${count}p" "$txt")
#sed '$COUNT \c "${data#*:}"' $txt # not using it
echo "${data#*:}"
done
Since you are reading the file from the bottom and done some conditions around it, you could just drop it and just use tac to print the file on reverse:
while IFS= read -r data do
echo "${data#*:}"
done < <(tac file)
I have a find script that automatically opens a file if just one file is found. The way I currently handle it is doing a word count on the number of lines of the search results. Is there an easier way to do this?
if [ "$( cat "$temp" | wc -l | xargs echo )" == "1" ]; then
edit `cat "$temp"`
fi
EDITED - here is the context of the whole script.
term="$1"
temp=".aafind.txt"
find src sql common -iname "*$term*" | grep -v 'src/.*lib' >> "$temp"
if [ ! -s "$temp" ]; then
echo "ΓΈ - including lib..." 1>&2
find src sql common -iname "*$term*" >> "$temp"
fi
if [ "$( cat "$temp" | wc -l | xargs echo )" == "1" ]; then
# just open it in an editor
edit `cat "$temp"`
else
# format output
term_regex=`echo "$term" | sed "s%\*%[^/]*%g" | sed "s%\?%[^/]%g" `
cat "$temp" | sed -E 's%//+%/%' | grep --color -E -i "$term_regex|$"
fi
rm "$temp"
Unless I'm misunderstanding, the variable $temp contains one or more filenames, one per line, and if there is only one filename it should be edited?
[ $(wc -l <<< "$temp") = "1" ] && edit "$temp"
If $temp is a file containing filenames:
[ $(wc -l < "$temp") = "1" ] && edit "$(cat "$temp")"
Several of the results here will read through an entire file, whereas one can stop and have an answer after one line and one character:
if { IFS='' read -r result && ! read -n 1 _; } <file; then
echo "Exactly one line: $result"
else
echo "Either no valid content at all, or more than one line"
fi
For safely reading from find, if you have GNU find and bash as your shell, replace <file with < <(find ...) in the above. Even better, in that case, is to use NUL-delimited names, such that filenames with newlines (yes, they're legal) don't trip you up:
if { IFS='' read -r -d '' result && ! read -r -d '' -n 1 _; } \
< <(find ... -print0); then
printf 'Exactly one file: %q\n' "$result"
else
echo "Either no results, or more than one"
fi
Well, given that you are storing these results in the file $temp this is a little easier:
[ "$( wc -l < $temp )" -eq 1 ] && edit "$( cat $temp )"
Instead of 'cat $temp' you can do '< $temp', but it might take away some readability if you are not very familiar with redirection 8)
If you want to test whether the file is empty or not, test -s does that.
if [ -s "$temp" ]; then
edit `cat "$temp"`
fi
(A non-empty file by definition contains at least one line. You should find that wc -l agrees.)
If you genuinely want a line count of exactly one, then yes, it can be simplified substantially;
if [ $( wc -l <"$temp" ) = 1 ]; then
edit `cat "$temp"`
fi
You can use arrays:
x=($(find . -type f))
[ "${#x[*]}" -eq 1 ] && echo "just one || echo "many"
But you might have problems in case of filenames with whitespace, etc.
Still, something like this would be a native way
no this is the way, though you're making it over-complicated:
if [ "`wc -l $temp | cut -d' ' -f1`" = "1" ]; then
edit "$temp";
fi
what's complicating it is:
useless use of cat,
unuseful use of xargs
and I'm not sure if you really want the editcat $temp`` which is editing the file at the content of $temp