This question already has answers here:
Take nth column in a text file
(6 answers)
Closed 2 years ago.
I have written a simple code that takes data from a text file( which has space-separated columns and 1.5 million rows) gives the output file with the specified column. But this code takes more than an hr to execute. Can anyone help me out to optimize runtime
a=0
cat 1c_input.txt/$1 | while read p
do
IFS=" "
for i in $p
do
a=`expr $a + 1`
if [ $a -eq $2 ]
then
echo "$i"
fi
done
a=0
done >> ./1.c.$2.column.freq
some lines of sample input:
1 ib Jim 34
1 cr JoHn 24
1 ut MaRY 46
2 ti Jim 41
2 ye john 6
2 wf JoHn 22
3 ye jOE 42
3 hx jiM 21
some lines of sample output if the second argument entered is 3:
Jim
JoHn
MaRY
Jim
john
JoHn
jOE
jiM
I guess you are trying to print just 1 column, then do something like
#! /bin/bash
awk -v c="$2" '{print $c}' 1c_input.txt/$1 >> ./1.c.$2.column.freq
If you just want something faster, use a utility like cut. So to
extract the third field from a single space delimited file bigfile
do:
cut -d ' ' -f 3 bigfile
To optimize the shell code in the question, using only builtin shell
commands, do something like:
while read a b c d; echo "$c"; done < bigfile
...if the field to be printed is a command line parameter, there are
several shell command methods, but they're all based on that line.
I have a case where I have to replace the number 1 with number 3 at 10th location of various lines in a stored text file. I am unable to find a way to do that. Below is sample file and code.
Sample file:
$ cat testdata.txt
1 43515 8 1 victor Samuel 20190112
3215736 4 6 Michael pristine 20180923
1 56261 1 1 John Carter 19880712
#!/bin/sh
filename=testdata.txt
echo "reading number of line"
nol=$(cat $filename | wc -l)
flag[$nol]=''
echo "reading content of file"
for i in (1..$nol)
do
flag=($cut -c10-11 $filename)
if($flag==1)
sed 's/1/3/2'
fi
done
But this is not working.
Please help to resolve this.
Updated:
Sample Output:
1 43515 8 1 victor Samuel 20190112
3215736 4 6 Michael pristine 20180923
1 56261 3 1 John Carter 19880712
try this
sed "s/^\(.\{8\}\) 1 \(.*\)$/\1 3 \2/g" testdata.txt > new_testdata.txt
If sed supports the option -i you can also edit inplace.
sed -i "s/^\(.\{8\}\) 1 \(.*\)$/\1 3 \2/g" testdata.txt
output
1 43515 8 1 victor Samuel 20190112
3215736 4 6 Michael pristine 20180923
1 56261 3 1 John Carter 19880712
explanation
s # substitute
/^\( # from start of line, save into arg1
.\{8\} # the first 8 characters
\) 1 \( # search pattern ' 1 '
.* # save the rest into arg2
\)$/ # to the end of the line
\1 3 \2 # output: arg1 3 arg2
/g # global on whole line
I have 2 text files as below
A.txt (with 2 rows):
abc-1234
tik-3456
B.txt (with 4 rows)
123456
234567
987
12
I want to combine these 2 to get the below file in CSV format:
column-1 column-2
abc-1234 123456
tik-3456 234567
987
12
I am trying below command. However, not achieving the above result.
paste -d "," A.txt B.txt > C.csv
It is giving below output:
abc-1234
,123456
tik-3456,234567
,987
,12
Can anyone please let me know, what I am missing here?
In linux we have one utility that does one think very good. So:
paste merges files
column with -t creates tables
The following:
paste -d',' /tmp/1 /tmp/2 | column -t -N 'column-1,column-2' -s',' -o' '
outputs the desired result.
This question already has answers here:
Length of string in bash
(11 answers)
Closed 6 years ago.
Is it even possible? I currently have a one-liner to count the number of words in a file. If I output what I currently have it looks like this:
3 abcdef
3 abcd
3 fec
2 abc
This is all done in 1 line without loops and I was thinking if I could add a column with length of each word in a column. I was thinking I could use wc -m to count the characters, but I don't know if I can do that without a loop?
As seen in the title, no AWK, sed, perl.. Just good old bash.
What I want:
3 abcdef 6
3 abcd 4
3 fec 3
2 abc 3
Where the last column is length of each word.
while read -r num word; do
printf '%s %s %s\n' "$num" "$word" "${#word}"
done < file
You can do something like this also:
File
> cat test.txt
3 abcdef
3 abcd
3 fec
2 abc
Bash script
> cat test.txt.sh
#!/bin/bash
while read line; do
items=($line) # split the line
strlen=${#items[1]} # get the 2nd item's length
echo $line $strlen # print the line and the length
done < test.txt
Results
> bash test.txt.sh
3 abcdef 6
3 abcd 4
3 fec 3
2 abc 3
I have a very basic shell script here:
for file in Alt_moabit Book_arrival Door_flowers Leaving_laptop
do
for qp in 10 12 15 19 22 25 32 39 45 60
do
for i in 0 1
do
echo "$file\t$qp\t$i" >> psnr.txt
./command > $file-$qp-psnr.txt 2>> psnr.txt
done
done
done
command calculates some PSNR values and writes a detailed summary to a file for each combination of file, qp and i. That's fine.
The 2>> outputs one line of information that I really need. But when executed, I get:
Alt_moabit 10 0
total 47,8221 50,6329 50,1031
Alt_moabit 10 1
total 47,8408 49,9973 49,8197
Alt_moabit 12 0
total 47,0665 50,1457 49,6755
Alt_moabit 12 1
total 47,1193 49,4284 49,3476
What I want, however, is this:
Alt_moabit 10 0 total 47,8221 50,6329 50,1031
Alt_moabit 10 1 total 47,8408 49,9973 49,8197
Alt_moabit 12 0 total 47,0665 50,1457 49,6755
Alt_moabit 12 1 total 47,1193 49,4284 49,3476
How can I achieve that?
(Please feel free to change the title if you think there's a more appropriate one)
You could pass the -n option to your first echo command, so it doesn't output a newline.
As a quick demonstration, this :
echo "test : " ; echo "blah"
will get you :
test :
blah
With a newline between the two outputs.
While this, with a -n for the first echo :
echo -n "test : " ; echo "blah"
will get you the following output :
test : blah
Without any newline between the two output.
The (GNU version of) echo utility has a -n option to omit the trailing newline. Use that on your first echo. You'll probably have to put some space after the first line or before the second for readability.
You can use printf instead of echo, which is better for portability reasons.
printf is the correct way to solve your problem (+1 kurumi), but for completeness, you can also do:
echo "$file\t$qp\t$i $( ./command 2>&1 > $file-$qp-psnr.txt )" >> psnr.txt
While echo -n may work if you just want the print the output to console, it won't work if you want the output redirected to file.
If you want the concatenated output to be redirected to a file, this will work:
echo "Str1: `echo "Str2"`" >> file
I was also facing the same problem.
To define my problem, I have a script which is using the echo function like this:
echo -n "Some text here"
echo -n "Some text here"
The output I was getting is like this:
-n Some text here
-n Some text here
and I want the text to be in same line and it is also printing -n option in the output.
Note :- According to man Page, -n option do not print the trailing newline character.
The way I solved it using by adding the shebang in the starting of the script file like this.
#!/bin/bash
echo -n "Some text here"
echo -n "Some text here"
This will print the desired output like this:
Some text here Some text here
Hope this helps!