This question already has answers here:
How to extract one column of a csv file
(18 answers)
Closed 1 year ago.
I have a .csv file with three columns. I want to keep the first column only. I have been trying to work with a command similar to the one below.
cut -f 1,4 output.csv > output.txt
No matter what I do, my output remains the same- giving me all three columns. Can anyone give me some insight?
Thanks!
read file one line at a time, trim everything right of that first comma:
while read -r line; do echo ${line%%,*}; done < output.csv > output.txt
Related
This question already has answers here:
How can I use a file in a command and redirect output to the same file without truncating it?
(14 answers)
Closed 8 months ago.
I am using this command
sort -k1 -n source-g5.txt
to sort the content of file tmp-source-g5.txt (n rows, 2 columns) according to the numerical value of the first column.
When I run that line, the terminal prints out the desired result, but as I try to save the result into the same file,
sort -k1 -n source-g5.txt > source-g5.txt
the file shows no difference from before.
What am I doing wrong?
SOLVED
From this thread it turns out that redirecting the output of sort into the same file from which sort reads as source will not work since
the shell is makes the redirections (not the sort(1) program) and the
input file (as being the output also) will be erased just before
giving the sort(1) program the opportunity of reading it.
So I have split my command into two
sort -k1 -n source-g5.txt > tmp-source-g5.txt
mv tmp-source-g5.txt > source-g5.txt
This question already has answers here:
What's the most robust way to efficiently parse CSV using awk?
(6 answers)
Closed 3 years ago.
I have a CSV file with several rows and columns, some rows have 4 columns and some have 5 columns. I want to add to those with 4 columns, one more column so all the rows have 5 columns. The info that I must add must be in the 3rd row, or at the end.
The CSV file looks like this:
name;ip;supplier;os;manufact
How can this be done in bash?
You can use read to split the CSV, then output the desired number of columns.
cat test.csv | while read line; do
IFS=\; read -ra COL <<< "$line"
echo "${COL[0]};${COL[1]};${COL[2]};${COL[3]};${COL[4]};"
done
For example, with test.csv containing:
1;2;3;4
1;2;3;4;5
1;2;3;4;
1;2;3;4;5;
The above script outputs:
1;2;3;4;;
1;2;3;4;5;
1;2;3;4;;
1;2;3;4;5;
This question already has answers here:
Fast way of finding lines in one file that are not in another?
(11 answers)
Difference between sh and Bash
(11 answers)
Closed 4 years ago.
I want to compare with online users inside freeradius database and users inside mikrotik devices , so I retrieved online users from freeradius and insert them inside file1, get online users from mikrotik and insert them inside file2, now I want to get the users that are found in file2 but not found inside file1
so I used this code
I am trying to get users that are found in file2 but not found in file 1.
comm -23 <(sort < file2) <(sort file1)
but I had this error
Syntax error: "(" unexpected.
So where is my error?
actually I don't know much things about bash and shell , so please help me
Assuming your files contain 1 username per line in file1, a simple one liner like the below should do what you need, without having to sort the files :
while read line; do if grep -q "$line" file2; then echo "user in both :" "$line"; fi; done < file1
This question already has answers here:
Remove the last line from a file in Bash
(16 answers)
Closed 5 years ago.
I asked a question regarding popping the last line of a text file in PHP, and now, is it possible to re-write the logic in shell script?
I tried this to obtain the last line:
tail -n 1 my_log_file.log
but I am not sure how can I remove the last line and save the file.
P.S. given that I use Ubuntu server.
To get the file content without the last line, you can use
head -n-1 logfile.log
(I am not sure this is supported everywhere)
or
sed '$d' logfile.log
What you want is truncate the file just before the last line without having to read the file entirely.
truncate -s -"$(tail -n1 file | wc -c)" file
That's assuming the file is not currently being written to.
truncate is part of the GNU coreutils (so generally found on recent Linux distributions) and is not a standardized Unix or POSIX command. Many "dd" implementations can be used to truncate a file as well.
(Solution is based on sch's answer so credit should go to him/her)
This approach will allow you to efficiently retrieve the last line of the file and truncate the file to remove that line. This can better deal with large inputs as the file is not read sequentially.
# retrieve last line from file
LAST=$(tail -n 1 my_log_file.log)
# truncate file
let TRUNCATE_SIZE="${#LAST} + 1"
truncate -s -"$TRUNCATE_SIZE" my_log_file.log
# ... $LAST contains 'popped' last line
Note that this will not work as expected if the file is modified between the calls to tail and truncate.
One way is:
sed '$d' < f1 > f2 ; mv f2 f1
This question already has answers here:
find difference between two text files with one item per line [duplicate]
(11 answers)
Closed 9 years ago.
I have a Masters.txt (all records) and a New.txt file. I want to process New.txt against Masters.txt and output all the lines from New.txt that do not exist in Masters.txt
i'm not sure if this is something the sort -u command can do.
Sort both files first using sort and then use the comm command to list the lines that exist only in new.txt and not in masters.txt. Something like:
sort masters.txt >masters_sorted.txt
sort new.txt >new_sorted.txt
comm -2 -3 new_sorted.txt masters_sorted.txt
comm produces three columns in its output by default; column 1 contains lines unique to the first file, column 2 contains lines unique to the second file; column 3 contains lines common to both files. The -2 -3 switches suppress the second and third columns.
see the linux comm command:
http://unstableme.blogspot.com/2009/08/linux-comm-command-brief-tutorial.html