I have a program which transposes a matrix. It works properly when passed a file as a parameter, but it gives strange output when given input via stdin.
This works:
$ cat m1
1 2 3 4
5 6 7 8
$ ./matrix transpose m1
1 5
2 6
3 7
4 8
This doesn't:
$ cat m1 | ./matrix transpose
5
[newline]
[newline]
[newline]
This is the code I'm using to transpose the matrix:
function transpose {
# Set file to be argument 1 or stdin
FILE="${1:-/dev/stdin}"
if [[ $# -gt 1 ]]; then
print_stderr "Too many arguments. Exiting."
exit 1
elif ! [[ -r $FILE ]]; then
print_stderr "File not found. Exiting."
exit 1
else
col=1
read -r line < $FILE
for num in $line; do
cut -f$col $FILE | tr '\n' '\t'
((col++))
echo
done
exit 0
fi
}
And this code handles the argument passing:
# Main
COMMAND=$1
if func_exists $COMMAND; then
$COMMAND "${#:2}"
else
print_stderr "Command \"$COMMAND\" not found. Exiting."
exit 1
fi
I'm aware of this answer but I can't figure out where I've gone wrong. Any ideas?
for num in $line; do
cut -f$col $FILE | tr '\n' '\t'
((col++))
echo
done
This loop reads $FILE over and over, once for each column. That works fine for a file but isn't suitable for stdin, which is a stream of data that can only be read once.
A quick fix would be to read the file into memory and use <<< to pass it to read and cut.
matrix=$(< "$FILE")
read -r line <<< "$matrix"
for num in $line; do
cut -f$col <<< "$matrix" | tr '\n' '\t'
((col++))
echo
done
See An efficient way to transpose a file in Bash for a variety of more efficient one-pass solutions.
#!/bin/bash
file="/home/vdabas2/file2"
while IFS='' read -r line || [[ -n "$line" ]];
do
pbreplay -O "$line" >> output
done < "$file"
I am able to read a file line by line and the output of each line processed is being redirected to output by using the above shell script.
But I need a different file as a redirected output for each line processed and save it like output1, output2 and so on. So, if there are 10 lines in that file which are being passed on as arguments then I need 10 output files.
#!/bin/bash
file="/home/vdabas2/file2"
i=1
while IFS='' read -r line || [[ -n "$line" ]];
do
pbreplay -O "$line" >> output.${i}
((i++))
done < "$file"
Add increment for example.
I have a file that contains 10 lines with this sort of content:
aaaa,bbb,132,a.g.n.
I wanna walk throw every line, char by char and put the data before the " , " is met in an output file.
if [ $# -eq 2 ] && [ -f $1 ]
then
echo "Read nr of fields to be saved or nr of commas."
read n
nrLines=$(wc -l < $1)
while $nrLines!="1" read -r line || [[ -n "$line" ]]; do
do
for (( i=1; i<=$n; ++i ))
do
while [ read -r -n1 temp ]
do
if [ temp != "," ]
then
echo $temp > $(result$i)
else
fi
done
paste -d"\n" $2 $(result$i)
done
nrLines=$($nrLines-1)
done
else
echo "File not found!"
fi
}
In parameter $2 I have an empty file in which I will store the data from file $1 after I extract it without the " , " and add a couple of comments.
Example:
My input_file contains:
a.b.c.d,aabb,comp,dddd
My output_file is empty.
I call my script: ./script.sh input_file output_file
After execution the output_file contains:
First line info: a.b.c.d
Second line info: aabb
Third line info: comp
(yes, without the 4th line info)
You can do what you want very simply with parameter-expansion and substring-removal using bash alone. For example, take an example file:
$ cat dat/10lines.txt
aaaa,bbb,132,a.g.n.
aaaa,bbb,133,a.g.n.
aaaa,bbb,134,a.g.n.
aaaa,bbb,135,a.g.n.
aaaa,bbb,136,a.g.n.
aaaa,bbb,137,a.g.n.
aaaa,bbb,138,a.g.n.
aaaa,bbb,139,a.g.n.
aaaa,bbb,140,a.g.n.
aaaa,bbb,141,a.g.n.
A simple one-liner using native bash string handling could simply be the following and give the following results:
$ while read -r line; do echo ${line%,*}; done <dat/10lines.txt
aaaa,bbb,132
aaaa,bbb,133
aaaa,bbb,134
aaaa,bbb,135
aaaa,bbb,136
aaaa,bbb,137
aaaa,bbb,138
aaaa,bbb,139
aaaa,bbb,140
aaaa,bbb,141
Paremeter expansion w/substring removal works as follows:
var=aaaa,bbb,132,a.g.n.
Beginning at the left and removing up to, and including, the first ',' is:
${var#*,} # bbb,132,a.g.n.
Beginning at the left and removing up to, and including, the last ',' is:
${var##*,} # a.g.n.
Beginning at the right and removing up to, and including, the first ',' is:
${var%,*} # aaaa,bbb,132
Beginning at the left and removing up to, and including, the last ',' is:
${var%%,*} # aaaa
Note: the text to remove above is represented with a wildcard '*', but wildcard use is not required. It can be any allowable text. For example, to only remove ,a.g.n where the preceding number is 136, you can do the following:
${var%,136*},136 # aaaa,bbb,136 (all others unchanged)
To print 2016 th line from a file named file.txt u have to run a command like this-
sed -n '2016p' < file.txt
More-
sed -n '2p' < file.txt
will print 2nd line
sed -n '2011p' < file.txt
2011th line
sed -n '10,33p' < file.txt
line 10 up to line 33
sed -n '1p;3p' < file.txt
1st and 3th line
and so on...
For more detail, please have a look in this tutorial and this answer.
In native bash the following should do what you want, assuming you replace the contents of your script.sh with the below:
#!/bin/bash
IN_FILE=${1}
OUT_FILE=${2}
IFS=\,
while read line; do
set -- ${line}
for ((i=1; i<=${#}; i++)); do
((${i}==4)) && continue
((n+=1))
printf '%s\n' "Line ${n} info: ${!i}"
done
done < ${IN_FILE} > ${OUT_FILE}
This will not print the 4th field of each line within the input file, on a new line in the output file (I assume this is your requirement as per your comment?).
[wspace#wspace sandbox]$ awk -F"," 'BEGIN{OFS="\n"}{for(i=1; i<=NF-1; i++){print "line Info: "$i}}' data.txt
line Info: a.b.c.d
line Info: aabb
line Info: comp
This little snippet can ignore the last field.
updated:
#!/usr/bin/env bash
if [ ! -f "$1" -o $# -ne 2 ];then
echo "Usage: $(basename $0) input_file out_file"
exit 127
fi
input_file=$1
output_file=$2
: > $output_file
if [ "$(wc -l < $1)" -ne 0 ];then
while true
do
read -r -n1 char
if [ "$char" == "" ];then
break
elif [ $char != "," ];then
temp=$temp$char
else
echo "line info: $temp" >> $output_file
temp=""
fi
done < $input_file
else
echo "file $1 is empty"
fi
Maybe this is what you want
Did you try
sed "s|,|\n|g" $1 | head -n -1 > $2
I assume that only the last word would not have a comma on its right.
Try this (tested with you sample line) :
#!/bin/bash
# script.sh
echo "Number of fields to save ?"
read nf
while IFS=$',' read -r -a arr; do
newarr=${arr[#]:0:${nf}}
done < "$1"
for i in ${newarr[#]};do
printf "%s\n" $i
done > "$2"
Execute script with :
$ ./script.sh inputfile outputfile
Number of fields ?
3
$ cat outputfile
a.b.c.d
aabb
comp
All words separated with commas are stored into an array $arr
A tmp array $newarr removes last $n element ($n get the read command).
It loops over new array and prints result in $2, the outputfile.
Using bash shell:
I am trying to read a file line by line.
and every line contains two meaning full file names delimited by "``"
file:1 image_config.txt
bbbbb.mp4``thumb/hashdata.gif
bbbbb.mp4``thumb/hashdata2.gif
Shell Script
#!/bin/bash
filename="image_config.txt"
while IFS='' read -r line || [[ -n "$line" ]]; do
IFS='``' read -r -a array <<< "$line"
if [ "$line" = "" ]; then
echo lineempty
else
file=${array[0]}
hash=${array[2]}
echo $file$hash;
output=$(ffmpeg -v warning -ss 2 -t 0.8 -i $file -vf scale=200:-1 -gifflags +transdiff -y $hash);
echo $output;
# echo ${array[0]}${array[1]}${array[2]}
fi;
done < "$filename"
first time executed successfully but when loop executes second time.
variable file lost bbbbb from bbbbb.mp4
and following output comes out
Output :
user#domain [~/public_html/Videos]$ sh imager.sh
bbbbb.mp4thumb/hashdata.gif
.mp4thumb/hashdata2.gif
.mp4: No such file or directory
lineempty
Please check out Bash FAQ 89 - I'm using a loop which runs once per line of input but it only seems to run once; everything after the first line is ignored? which seems to be helpful in your case.
Aside:
There is no point in using the same character twice in IFS.
IFS=\`
Is enough.
Check out this:
var='abc``def'
IFS=\`\` read -ra arr <<< "$var"
printf '<%s>\n' "${arr[#]}"
Output:
<abc>
<>
<def>
As you can see, arr[0] is abc, arr[1] is empty and arr[2] is def, and not arr[0] is abc and arr[1] is def as one might expect.
Taken from the IFS wiki of Greycat and Lhunath Bash Guide :
The IFS variable is used in shells (Bourne, POSIX, ksh, bash) as the input field separator (or internal field separator). Essentially, it is a string of special characters which are to be treated as delimiters between words/fields when splitting a line of input.
Here is how you could do differently, avoiding a read in the read:
#!/bin/bash
filename="image_config.txt"
while IFS='' read -r line || [[ -n "$line" ]]; do
if [ "$line" = "" ]; then
echo lineempty
else
file=$( echo ${line} | awk -F \` ' { print $1 } ' )
hash=$( echo ${line} | awk -F \` ' { print $3 } ' )
echo $file$hash;
output=$(ffmpeg -v warning -ss 2 -t 0.8 -i $file -vf scale=200:-1 -gifflags +transdiff -y $hash);
echo $output;
fi;
done < "$filename"
I have one file Length.txt having multiples names (40) line by line.
I want to write a small shell script where it will count the character count of each line of the file and if the count is less than 9 replace those lines with adding extra 8 spaces and 1 at the end of each line.
For example, if the name is
XXXXXX
replace as
XXXXXX 1
I tried with the below coding. It is working for me, however whenever it's replacing the line it is displaying all the lines at a time.
So suppose I have 40 lines in Length.txt and out of that 4 lines having less than 9 character count then my output has 160 lines.
Can anyone help me to display only 40 line output with the 4 changed lines?
#!/usr/bin/sh
#set -x
while read line;
do
count=`echo $line|wc -m`
if [ $count -lt 9 ]
then
Number=`sed -n "/$line/=" Length.txt`;
sed -e ""$Number"s/$line/$line 1/" Length4.txt
fi
done < Length.txt
A single sed command can do that:
sed -E 's/^.{,8}$/& 1/' file
To modify the contents of the file add -i:
sed -iE 's/^.{,8}$/& 1/' file
Partial output:
94605320 1
105018263
2475218231
7728563 1
1
* Fixed to include only add 8 spaces not 9, and include empty lines. If you don't want to process empty lines, use {1,8}.
$ cat foo.input
I am longer than 9 characters
I am also longer than 9 characters
I am not
Another long line
short
$ while read line; do printf "$line"; (( ${#line} < 9 )) && printf " 1"; echo; done < foo.input
I am longer than 9 characters
I am also longer than 9 characters
I am not 1
Another long line
short 1
Let me show you what is wrong with your script. The only thing missing from your script is that you need to use sed -i to edit file and re-save it after making the replacement.
I'm assuming Length4.txt is just a copy of Length.txt?
I added sed -i to your script and it should work now:
cp Length.txt Length4.txt
while read line;
do
count=`echo $line|wc -m`
if [ $count -lt 9 ]
then
Number=`sed -n "/$line/=" Length.txt`
sed -ie ""$Number"s/$line/$line 1/" Length4.txt
fi
done < Length.txt
However, you don't need sed or wc. You can simplify your script as follows:
while IFS= read -r line
do
count=${#line}
if (( count < 9 ))
then
echo "$line 1"
else
echo "$line"
fi
done < Length.txt > Length4.txt
$ awk -v FS= 'NF<9{$0=sprintf("%s%*s1",$0,8,"")} 1' file
XXXXXX 1
Note how simple it would be to check for a number other than 9 characters and to print some sequence of blanks other than 8.