I have a little command line utility rjp2tif that extracts radiometric data from a jpeg file into a tiff file. I was hoping to be able to pipe the filepath to ImageJ on the command line and have ImageJ open the tiff file. To this end, rjp2tif writes the filepath of the tiff file to standard output. I tried the following in bash:
$ rjp2tif /path/to/rjpeg | open -a imagej
and
$ rjp2tif /path/to/rjpeg | open -a imagej -f
The first opens ImageJ but doesn't open the file.
The second opens ImageJ with a text window with the filepath in it.
This is on macOS Monterey, but I don't think that matters.
Anyone tried to do this and been successful? TIA.
Assuming the rjp2tif command returns a file-path in standard output, and you want to pass this output as a regular CLI argument to another command, you may be interested in the xargs command. But note that in the general case, you may hit some issue if the file-path contains spaces or so:
Read space, tab, newline and end-of-file delimited arguments from standard input and execute the specified utility with them as arguments.
The arguments are typically a long list of filenames (generated by ls or find, for example) that get passed to xargs via a pipe.
So in this case, assuming each file-path takes only one line (which is obviously the case if there's only one line overall), you can use the following NUL-based tip relying on the tr command.
Here is the command you'd obtain:
rjp2tif /path/to/rjpeg | tr '\n' '\0' | xargs -0 open -a imagej
Note: I have a GNU/Linux OS, so can you please confirm it does work under macOS?
FTR, below is a comprehensive shell code allowing one to test two different modes of xargs: generating one command per line-argument (-n1), or a single command with all line-arguments in one go:
$ printf 'one \ntwo\nthree and four' | tr '\n' '\0' | xargs -0 -n1 \
bash -c 'printf "Run "; for a; do printf "\"$a\" "; done; echo' bash
Run "one "
Run "two"
Run "three and four"
$ printf 'one \ntwo\nthree and four' | tr '\n' '\0' | xargs -0 \
bash -c 'printf "Run "; for a; do printf "\"$a\" "; done; echo' bash
Run "one " "two" "three and four"
######################################
# or alternatively (with no for loop):
######################################
$ printf 'one \ntwo\nthree and four' | tr '\n' '\0' | xargs -0 -n1 \
bash -c 'printf "Run "; printf "\"%s\" " "$#"; echo' bash
Run "one "
Run "two"
Run "three and four"
$ printf 'one \ntwo\nthree and four' | tr '\n' '\0' | xargs -0 \
bash -c 'printf "Run "; printf "\"%s\" " "$#"; echo' bash
Run "one " "two" "three and four"
Related
I have the following two scripts:
#script1.sh:
#!/bin/bash
this_chunk=(1 2 3 4)
printf "%s\n" "${this_chunk[#]}" | ./script2.sh
#script2.sh:
#!/bin/bash
while read -r arr
do
echo "--$arr"
done
When I execute script1.sh, the output is as expected:
--1
--2
--3
--4
which shows that I was able to pipe the elements of the array this_chunk as arguments to script2.sh. However, if I change the line calling script2.sh to
printf "%s\n" "${this_chunk[#]}" | xargs ./script2.sh
there is no output. My question is, how to pass the array this_chunk using xargs, rather than simple piping? The reason is that I will have to deal with large arrays and thus long argument lists which will be a problem with piping.
Edit:
Based on the answers and comments, this is the correct way to do it:
#script1.sh
#!/bin/bash
this_chunk=(1 2 3 4)
printf "%s\0" "${this_chunk[#]}" | xargs -0 ./script2.sh
#script2.sh
#!/bin/bash
for i in "${#}"; do
echo $i
done
how to pass the array this_chunk using xargs
Note that xargs by default interprets ' " and \ sequences. To disable the interpretation, either preprocess the data, or better use GNU xargs with -d '\n' option. -d option is not part of POSIX xargs.
printf "%s\n" "${this_chunk[#]}" | xargs -d '\n' ./script2.sh
That said, with GNU xargs prefer zero terminated streams, to preserve newlines:
printf "%s\0" "${this_chunk[#]}" | xargs -0 ./script2.sh
Your script ./script2.sh ignores command line arguments, and your xargs spawns the process with standard input closed. Because the input is closed, read -r arr fails, so your scripts does not print anything, as expected. (Note that in POSIX xargs, when the spawned process tries to read from stdin, the result is unspecified.)
I'm working on some script to help organize some stuff. I have simplified my code down to this from my original:
$ echo $(find -type d -maxdepth 5 | grep -E "\./([^/])*/([^/])*/([^/])*/([^/])*"
| cut -d'/' -f 3 | sort | uniq -d | xargs -I R sh -c 'echo -e "alpha \n"')
(R actually doesn't do anything here but it's used in the original)
Particularly, I think something is wrong with my xargs
xargs -I R sh -c 'echo -e "alpha \n"'
What I would look to see happen is alpha to be printed several times, each on a newline. However, my output looks like
alpha alpha alpha...
I've been scouring around the internet trying to find how to fix this but it's no use. I've just started experimenting with bash, can someone please point out what I'm doing wrong?
This is a special case of I just assigned a variable, but echo $variable shows something else.
Just running a command, no echo:
$ printf '%s\n' "first line" "second line"
first line
second line
Running a command in an unquoted command substitution, as your code is currently written:
$ echo $(printf '%s\n' "first line" "second line")
first line second line
Running a command in a quoted command substitution:
$ echo "$(printf '%s\n' "first line" "second line")"
first line
second line
...so: If you don't have a reason for the outer echo $(...), just remove it; and if you do have a reason, add quotes.
I am trying to use basename utility in xargs piped from printf as below:
printf "%s" "$ACTUAL_FILES" | xargs -d ' ' -i printf "%s\n" "$(basename {})"
Here $ACTUAL_FILES is an array of absolute file paths, each delimited with a space.
With the above snippet I am trying to print filename without path in each line. But the output I am getting is same as in $ACTUAL_FILES with each element in new line.
I know that we can achieve this with bash sub shell and echo with xargs, but I was informed to use printf with xargs.
How can I use basename or any other utility to get the filename.
You need to strip the path after processing xargs (I write your var in lowercase):
printf "%s" "${actual_files}" | xargs -d ' ' -i printf "%s\n" "{}" | sed 's#.*/##'
Processing can be easier when you start with replacing spaces by newlines.
tr ' ' '\n' <<< "${actual_files}"| sed 's#.*/##'
You can avoid tr with
grep -Eo "[^/]*( |$)" <<< "${actual_files}"
xargs sends your arguments directly to the command. There is no shell intervention anymore when xargs makes the arguments and sends them to the command. Using xargs you cannot make use of a subshell call ($(..)). You can either preparse them before sending them to xargs, or you can let xargs make a shell instead in which you can make use of all the shell features.
printf "%s" "$ACTUAL_FILES" | xargs -d ' ' -i bash -c 'printf "%s\n" "$(basename {})"'
If it is possible for you to use, GNU parallel comes with much more features in building a command.
printf "%s" "$ACTUAL_FILES" | parallel -q printf "%s\n" "{/}"
In here {/} automatically is the basename of the input arguments and -q preserves the quoting used.
using xargs and echo to output result of samtools to new line in output.txt file
samtools view $SAMPLE\.bam | cut -f3 | uniq -c | sort -n | \
xargs -r0 -n1 echo -e "Summarise mapping...\n" >> ../output.txt
This adds the result on a new line after the echo but also adds a space before the result on the first new line, how can i stop this?
It's not xargs which is adding the space. It's the echo command:
The echo utility arguments shall be separated by single <space> characters and a <newline> character shall follow the last argument. (Text from Posix standard; emphasis added.)
If you want more control, use printf:
...
xargs -r0 -n1 printf "Summarise mapping...\n%s\n" >> ../output.txt
Unlike printf does not automatically add a newline at the end, so it needs to be included in the format.
Note that printf automatically interprets escape sequence like \n in the format string (but not in interpolated arguments). As an additional bonus for using printf, you could leave out the -n1 option since printf automatically repeats the format until all arguments are consumed.
What is the purpose of xargs_tasks in the following command?:
cat source.ndjson | tr '\n' '\0' | xargs -0 -n 1 -P 8 sh -c './theprocess.py $1' xargs_tasks >> errors.log 2>&1
Namely, after reading the man page details of the -c option to sh, you can actually get the same results by running:
cat source.ndjson | tr '\n' '\0' | xargs -0 -n 1 -P 8 sh -c './theprocess.py $0' >> errors.log 2>&1
That is, with -c, any arguments that come after the string in quotes are assigned to the variables $0, $1, and so forth. xargs_tasks in the first command is set to $0, and the arguments piped to xargs one at a time get set to $1. Hence my second command works exactly the same since I directly have $0 inside the string instead of $1.
My guess was that xargs_tasks gives you a short string to filter with in something like htop, but that's a stretch.
When you use the -c option to sh, the next argument after the command is taken as $0 in the command, and the arguments after it become $1, $2, etc.
$0 is supposed to contain the name of the shell script, while arguments start at $1. So xargs_task is there just to be a placeholder to prevent the first argument from being put into $0.