How to copy a .c file to a numbered listing - bash
I simply want to copy my .c file into a line-numbered listing file. Basically generate a .prn file from my .c file. I'm having a hard time finding the right bash command to do so.
Do you mean nl?
nl -ba filename.c
The -ba means to number all lines, not just non-empty ones.
awk '{print FNR ":" $0}' file1 file2 ...
is one way.
FNR is FileNumberRecord (the current line number per file).
You can change the ":" per your needs.
$0 means "the-whole-line-of-input"
Or you can do
cat -n file1 file2 ....
IHTH
On my linux system, I occasionally use pr -tn to prefix line numbers for listings. The -t option suppresses headers and footers; -n says to prefix line numbers. -n allows optional format and digit specifiers; see man page. Anyhow, to print file xyz.c to xyz.prn with line numbering, use:
pr -tn xyz.c > xyz.prn
Note, this is not as compact and handy as cat -n xyz.c > xyz.prn (using cat -n as suggested in a previous answer); but pr has numerous other options, and I most often use it when I want to both number the lines and put them into multiple columns or print multiple files side by side. Eg for a 2-column numbered listing use:
pr -2 -tn xyz.c > xyz.prn
I think shellter has the right idea. However, if your require output written to files with prn extensions, here's one way:
awk '{ sub(/\.c$/, "", FILENAME); print FNR ":" $0 > FILENAME ".prn" }' file1.c file2.c ...
To perform this on all files in the present working directory:
for i in *.c; do awk '{ sub(/\.c$/, "", FILENAME); print FNR ":" $0 > FILENAME ".prn" }' "$i"; done
Related
Command to remove all but select columns for each file in unix directory
I have a directory with many files in it and want to edit each file to only contain a select few columns.
I have the following code which will only print the first column
for i in /directory_path/*.txt; do awk -F "\t" '{ print $1 }' "$i"; done
but if I try to edit each file by adding >'$I' as below then I lose all the information in my files
for i in /directory_path/*.txt; do awk -F "\t" '{ print $1 }' "$i" > "$i"; done
However I want to be able to remove all but a select few columns in each file for example 1 and 3.
Given:
cat file
1 2 3
4 5 6
You can do in place editing with sed:
sed -i.bak -E 's/^([^[:space:]]*).*/\1/' file
cat file
1
4
If you want freedom to work with multiple columns and have in place editing, use GNU awk that also supports in place editing:
gawk -i inplace '{print $1, $3}' file
cat file
1 3
4 6
If you only have POSIX awk or wanted to use cut you generally do this:
Modify the file with awk, cut, sed, etc
Redirect the output to a temp file
Rename the temp file back to the original file name.
Like so:
awk '{print $1, $3}' file >tmp_file; mv tmp_file file
Or with cut:
cut -d ' ' -f 1,3 file >tmp_file; mv tmp_file file
To do a loop on files in a directory, you would do:
for fn in /directory_path/*.txt; do
awk -F '\t' '{ print $1 }' "$fn" >tmp_file
mv tmp_file "$fn"
done
Just to add a little more to #dawg's perfectly well working answer according to my use case.
I was dealing with CSVs, and standard CSV can have , in some values as long as it's in double quotes like for example, the below-mentioned row will be a valid CSV row.
col1,col2,col2
1,abc,"abc, inc"
But the command above was treating the , between the double quotes as delimiter too.
Also, the output file delimiter wasn't specified in the command.
These are the modifications I had to make for it handle the above two problems:
for fn in /home/ubuntu/dir/*.csv; do
awk -F ',' '{ FPAT = "([^,]*)|(\"[^\"]+\")"; OFS=","; print $1,$2 }' "$fn" >tmp_file
mv tmp_file "$fn"
done
The OSF delimiter will be the diameter of the output/result file.
The FPAT handles the case of , between quotation mark.
The regex and the information for that is mentioned ins awk's official documentation in section 4.7 Defining Fields by Content.
I was led to that solution through this answer.
AWK remove blank lines and append empty columns to all csv files in the directory
Hi I am looking for a way to combine all the below commands together. Remove blank lines in the csv file (comma delimited) Add multiple empty columns to each line up to 100th column Perform action 1 & 2 on all the files in the folder I am still learning and this is the best I could get: awk '!/^[[:space:]]*$/' x.csv > tmp && mv tmp x.csv awk -F"," '($100="")1' OFS="," x.csv > tmp && mv tmp x.csv They work out individually but I don't know how how to put them together and I am looking for ways to have it run through all the files under the directory. Looking for concrete AWK code or shell script calling AWK. Thank you! An example input would be: a,b,c x,y,z Expected output would be: a,b,c,,,,,,,,,, x,y,z,,,,,,,,,,
you can combine in one script without any loops
$ awk 'BEGIN{FS=OFS=","} FNR==1{close(f); f=FILENAME".updated"} NF{$100=""; print > f}' files...
it won't overwrite the original files.
You can pipe the output of the first to the other:
awk '!/^[[:space:]]*$/' x.csv | awk -F"," '($100="")1' OFS="," > new_x.csv
If you wanted to run the above on all the files in your directory, you would do:
shopt -s nullglob
for f in yourdirectory/*.csv; do
awk '!/^[[:space:]]*$/' "${f}" | awk -F"," '($100="")1' OFS="," > new_"${f}"
done
The shopt -s nullglob is so that an empty directory won't give you a literal *. Quoted from a good source for about looping through files
With recent enough GNU awk you could:
$ gawk -i inplace 'BEGIN{FS=OFS=","}/\S/{NF=100;$1=$1;print}' *
Explained:
$ gawk -i inplace ' # using GNU awk and in-place file editing
BEGIN {
FS=OFS="," # set delimiters to a comma
}
/\S/ { # gawk specific regex operator that matches any character that is not a space
NF=100 # set the field count to 100 which truncates fields above it
$1=$1 # edit the first field to rebuild the record to actually get the extra commas
print # output records
}' *
Some test data (the first empty record is empty, the second empty record has a space and a tab, trust me bro):
$ cat file
1,2,3
1,2,3,4,5,6,
1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101
Output of cat file after the execution of the GNU awk program:
1,2,3,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
1,2,3,4,5,6,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100
bash one liner to combine text files line by line except for the first line in every file except for the first file
I am currently doing it using this script: head -1 file001.txt > all.txt; tail -n +2 -q *.txt >> all.txt it is doing alright, gettting the header from first file and then concatenating the others. Problem is: the other lines are all in one line, rather than each being in separate line. Please help to correct this problem. if you have any other one liner that does the job, perfect, it is welcome. the desired answer will be: get the header from any of the files with that extension, I don't want to give any specific file name like file001.txt, i'd prefer it to take it from any file, because anyway they have the same headers. But it is possible that my file names don't be the pattern file001.txt, that is why I say any file with that extension.
Use awk: awk 'FNR==1&&!ctr++;FNR!=1' *.txt Explanation: FNR is the line number of the current input file ctr is a variable that starts at 0 and is incremented every time we see the first line of a file ctr is only zero for the first input file, so !ctr is only true for the first file. Without an explicit action, the current input line is printed. The first condition prints line 1 of each file if ctr is false. The second condition prints a line if it is not the first line of a file. Demonstration: $ printf '%s\n' hdr f1 f2 > f1.txt $ printf '%s\n' hdr g1 g2 > f2.txt $ printf '%s\n' hdr h1 h2 > f3.txt $ awk 'FNR==1&&!ctr++;FNR!=1' *.txt hdr f1 f2 g1 g2 h1 h2 A slightly simpler command: awk '!ctr++ || FNR!=1' *.txt which prints a line if either ctr is zero (which is only true for the first line of the first file) or the line is not the first line of a file.
try this and it might work for you:
sed -e '2,${/^YOUR_HEADER/d' -e '}' *.txt > all.txt
Using awk to extract specific line from all text files in a directory
I have a folder with 50 text files and I want to extract the first line from each of them at the command line and output this to a result.txt file.
I'm using the following command within the directory that contains the files I'm working with:
for files in *; do awk '{if(NR==1) print NR, $0}' *.txt; done > result.txt
When I run the command, the result.txt file contains 50 lines but they're all from a single file in the directory rather than one line per file. The common appears to be looping over a single 50 times rather than over each of the 50 files.
I'd be grateful if someone could help me understand where I'm going wrong with this.
try this -
for i in *.txt;do head -1 $i;done > result.txt
OR
for files in *.txt;do awk 'NR==1 {print $0}' $i;done > result.txt
Your code has two problems:
You have an outer loop that iterates over *, but your loop body doesn't use $files. That is, you're invoking awk '...' *.txt 50 times. This is why any output from awk is repeated 50 times in result.txt.
Your awk code checks NR (the number of lines read so far), not FNR (the number of lines read within the current file). NR==1 is true only at the beginning of the very first file.
There's another problem: result.txt is created first, so it is included among *.txt. To avoid this, give it a different name (one that doesn't end in .txt) or put it in a different directory.
A possible fix:
awk 'FNR==1 {print NR, $0}' *.txt > result
Why not use head? For example with find:
find midir/ -type f -exec head -1 {} \; >> result.txt
If you want to follow your approach you need to specify the file and not use the wildcard with awk:
for files in *; do awk '{if(NR==1) print NR, $0}' "$files"; done > result.txt
Is there a simple one-line approach to combine the output of an unix command with a file?
I want to combine the header of one text file with another text file. I know I can do it with head -n1 file1.txt > header.txt cat header.txt file2.txt > file2_withHeader.txt but that generates an unnecessary intermediate file and it's two steps. Is there a way to make it one step without generating an intermediate file? I was thinking of this cat $(head -n1 file1.txt) file2.txt > file2_withHeader.txt but it does not work because $(head -n1 file1.txt) is not a file so it cannot concatenate. Thanks
All three of the other answers to this question are good, and since each answers in a different way, it'd be a good exercise to learn each one. LC-datascientist's and Diego's answer each spawn a subshell, so λuser's two-line approach, perhaps with && between the two commands, is the most ideal.
If you really want this in one command that doesn't launch a sub-shell, you can use awk (mawk):
awk 'BEGIN { getline < file1.txt; print } 1' file2.txt
gawk can do it even more elegantly thanks to the nextfile command:
gawk '1; NR == FNR { nextfile }' file1.txt file2.txt
These do the same thing: the first line of file1.txt and prints it, then prints the entirety of file2.txt.
The mawk code uses getline to read a single line from a given input into $0 (in this case; see the man page), we can do that before reading file2.txt. 1 is true, so this second clause always fires, triggering the default action (print $0).
The gawk code prints first, then uses NR == FNR to determine that you're looking at the first file (the overall number of records (lines) and the current file's number of records is the same), in which case it's already time to move to the second file.
For the first three lines, mawk needs a loop:
awk 'BEGIN { for (i=0;i<3;i++) { getline < file1.txt; print } } 1' file2.txt
and gawk merely needs another condition:
gawk '1; NR == FNR && NR >= 3 { nextfile }' file1.txt file2.txt
You can actually append with the redirections: head -n1 file1.txt > result.txt cat file2.txt >> result.txt
in bash you can also do cat <(head -n 1 file1.txt) file2.txt > file2_withHeader.txt which I think it was you were looking for.
Thank you, λuser. That's one option I can do, too. Actually, I just figured out the one-liner that I was hoping for. (head -n1 file1.txt; cat file2.txt) > file2_withHeader.txt Sorry, I realized this is the same question as others asking to combine two commands in unix, so this has actually been answered.