I have a plain text file with two columns. I need to take each line which contains two columns and send them to a command.
The source file looks like this:
potato potato2
the line needs to be sent to another command so it looks like this
command potato potato2
output I can just have to std out.
Been such a long time that I've tried a simple bash script...
I assume that your file contains two columns per line, separated by either spaces or tabs.
xargs -n 2 command < file.txt
See: man xargs
Looks like you just need to read a file line by line, so the following code should do:
while read -r line
do
echo "$line" | xargs your-other-command #Use xargs to convert input into arguments
done < source-file.txt
Related
I am an undergrad CS student.
I just started on bash and I have a question.
How do I redirect input from a file to a command that requires two arguments like 'cmp', 'diff', 'grep'?
Is it possible that I have a txt file with
"a1 a2"
and redirect this file to the input of cmp?
Thank you,
It is possible. You can split the redirected input of a command into multiple arguments with xargs, for example:
echo pattern file | xargs grep
will execute grep pattern file. The input redirection can also come from a file, like this:
xargs grep < inputfile
xargs can do a lot more, basically it will take the standard input, split it into multiple arguments, and feed them to the given command.
I'm new to OSX command line tools.
I am trying to find a block of text in a file and append this text at the end of all lines in another text file. At run time I don't know what this text will be, I just know it will be located within "BEGINHMM" and "ENDHMM". Also, I don't know the makeup of the destination file, except for that it will not be an empty text file.
The command which finds the block of text of interest is:
sed -n '/<BEGINHMM>/,/<ENDHMM>/p' proto
where "proto" is a text file containing the text of interest.
I've been trying to pipe the output of the above command to another 'sed' command, in the following manner:
xargs -I '{}' sed -i .bak 's/$/{}/' monophones0.txt
but I am getting some bizarre results, I see the "{}" inserted in the text for example.
I've also tried piping to:
xargs -0 sed -i .bak 's/$/&/' monophones0.txt
but I just get the printout (similar to terminal echo) of the text I am trying to grab.
Ultimately I want to loop over several 'proto' files in multiple directories and copy the text between the "BEGINHMM", "ENDHMM" block in each directory, and append the selected text to that directory's monophones.txt lines.
I am running the commands in the terminal, bash, OSX 10.12.2
Any help would be appreciated.
(1) Your sed command is of the form sed -n '/A/,/B/p'; this will include the lines on which A and B occur, even if these strings do not appear at the beginning of the line. This form may have other surprises in store for you as well (what do expect will happen if B is missing or repeated?), but the remainder of this post assumes that's what you want.
(2) It's not clear how you intend to specify the "proto" files, but you do indicate they might be in several directories, so for the remainder of this post, I'll assume they are listed, one per line, in a file named proto.txt in each directory. This will ensure that you don't run into any limitations on command-line length, but the following can easily be modified if you don't want to create such a file.
(3) Here is a script which will use the sed command you've mentioned to copy segments from each of the "proto" files specified in a directory to monophones0.txt in the directory in which the script is executed.
#!/bin/bash
OUT=monophones0.txt
cat proto.txt | while read file
do
if [ -r "$file" ] ; then
sed -n '/<BEGINHMM>/,/<ENDHMM>/p' "$file" >> $OUT
elif [ -n "$file" ] ; then
echo "NOT FOUND: $file" >&2
fi
done
Just like what you did before. tmpfile=$(mktemp); sed -n '/<BEGINHMM>/,/<ENDHMM>/p' proto >$tmpfile; sed -i .bak "r $tmpfile" monophones0.txt; rm $tmpfile. This is the basic idea; there are other checks you need to perform to make this a robust script.
– 4ae1e1
I need to write the output of a command to a specific line in a document. I can not just append it like so COMMAND | cat >> file, I need it to be added between two lines without replacing one or the other. I'm sure you must be able to do this via sed.
The following solution works when the output of COMMAND is only 1 line (inserting to line 4):
COMMAND | sed -i "4i \`cat` FILE"
Use that command:
command | sed -i '3r /dev/stdin' file
That inserts text after the 3rd line and reads from stdin (all output from command).
I have a file containing valid xmls (one per line) and I want to execute a utility (xpath) on each line one by one.
I tried xargs but that seems doesn't seem to have an option to pass the line as stdin :-
% cat <xmls-file> | xargs -p -t -L1 xpath -p "//Path/to/node"
Cannot open file '//Path/to/node' at /System/Library/Perl/Extras/5.12/XML/XPath.pm line 53.
I also tried parallel --spreadstdin but that doesn't seem to work either :-
% cat <xmls-file> | parallel --spreadstdin xpath -p "//Path/to/node"
junk after document element at line 2, column 0, byte 1607
If you want every line of a file to be split off and made stdin for a utility
you could use a for loop in bash shell:
cat xmls-file | while read line
do ( echo $f > /tmp/input$$;
xpath -p "//Path/to/node" </tmp/input$$
rm -f /tmp/input$$
);
done
The $$ appends the process id number, creating a unique name
I assume xmls-file contains, on each line, what you want iterated into $f and that you want this as stdin for a command line, not as a parameter to the command.
On the other hand, your specification may be incorrect and maybe instead you need each line
to be part of a command. In that case, delete the echo and rm lines, and change the xpath command to include $f wherever the line from the file is needed.
I've not done much XML so the do command may need to be edited.
You are very close with the GNU Parallel version; only -n1 missing:
cat <xmls-file> | parallel -n1 --spreadstdin xpath -p "//Path/to/node"
I've seen the technique before, but don't know what it's called and forget the exact syntax. Let's say I need to pipe in a file to a program like: command < input-file. However, I want to directly pass these lines of the input file into the command without the intermediate input file. It looks something like this, but it doesn't work:
command < $(file-line1; file-line2; file-line3)
Can someone tell me what this is called and how to do it?
This is called Process Substitution
command < <(printf "%s\n" "file-line1" "file-line2" "file-line3")
With the above, command will think its being input a file with a name much like /dev/fd/XX where 'XX' is some number. As you mentioned, this is a temporary file (actually a file descriptor) but it will contain the 3 lines you passed in to the printf command.
Herestring.
command <<< $'line 1\nline 2\nline 3\n'
Or heredoc.
command << EOF
line 1
line 2
line 3
EOF
I think you are referring to a "here document". Like this
#!/bin/sh
cat <<EOF
This is
the
lines.
EOF
How about:
cat myfile.txt | head -n3 | command