How can I use command in sed replace with groups capturing? - bash

Question

I have a file like this in content :
#INCLUDE<~/boulou/billy.txt>
And from a bash/sed script/command, I want to replace this line by the content of the file ~/boulou/billy.txt
This is my current command that can find the file path :
sed -E "s/#INCLUDE\<(.*)\>/\1/g" test.sh
This shows me the file path but when I want to fetch the content with this :
sed -E "s/#INCLUDE\<(.*)\>/$(cat \1)/g" test.sh
I get the error "cat: 1: No such file or directory"
Let me know if you want more information.

Answer 1

Your code doesn't work because it first runs cat \1 (which is the same as cat 1), inserts the output into the sed command, then runs sed -E "s/...//g" test.sh (empty replacement because cat 1 outputs nothing on stdout).
This is because the shell processes $( ... ) first, then runs the resulting command line.
For what you're trying to do, you have to read the file from inside the search/replace command, not up front.
I don't know how to do that with sed, so I'd simply use Perl:
perl -pe 's{#INCLUDE<([^<>]*)>}{ open my $fh, "<", $1 or die "$1: $!"; local $/; readline $fh }eg' test.sh
However, this will not work with your example because ~/boulou/billy.txt does not exist (you most likely do not have a directory literally called ~ in your current working directory). To fix this (and simplify the code considerably), I'd use:
perl -MPath::Tiny -pe 's{#INCLUDE<([^<>]*)>}{ path($1)->slurp }eg' test.sh
However, this requires the Path::Tiny module, which is not part of the core perl distribution.
It's possible to expand ~ manually, but that makes the code even messier (this is the point at which I'd consider putting it in a separate script file):
perl -pe 's{#INCLUDE<([^<>]*)>}{ my $p = $1; $p =~ s{^~/}{$ENV{HOME}/}; open my $fh, "<", $p or die "$p: $!"; local $/; readline $fh }eg' test.sh
(Disclaimer: Somewhat hacky, does not handle ~user notation, only replaces a leading ~/ by the contents of the HOME environment variable.)

Answer 2

You can try something like that :
$ sed -r 's/#INCLUDE<(.*)>/printf "%b" "$(cat \1)"/e' test.sh
It should work pretty smooth :)

Answer 3

This might work for you (GNU sed):
sed 's/^#INCLUDE<\(.*\)>$/cat \1/e' file
This evaluates the expression on the right side of the substitution command. In this case it will replace the matched line by the contents of the file named between < and >.

Answer 4

The problem here is that the shell is replacing $(cat \1) way early. That is, even before the sed command is executed and passed that parameters, the shell is doing the interpolation.
If you are just doing #INCLUDE substitutions, you may consider using m4. Awk may be a reasonable way of approaching this if you need to write your own.

Answer 5

This will work robustly with any input file contents running any awk in any shell on any UNIX system:
INPUT:
$ cat file1
before
foo [#INCLUDE<file2>] bar
after
$ cat file2
stuff
& nonsense
here
SCRIPT:
$ cat tst.awk
match($0,/#INCLUDE<[^<>]*>/) {
file = substr($0,RSTART+9,RLENGTH-10)
rep = sep = ""
while ( (getline line < file) > 0 ) {
rep = rep sep line
sep = ORS
}
$0 = substr($0,1,RSTART-1) rep substr($0,RSTART+RLENGTH)
}
{ print }
OUTPUT:
$ awk -f tst.awk file1
before
foo [stuff
& nonsense
here] bar
after
Here's what the currently accepted answer will do with that same input:
$ sed -r 's/#INCLUDE<(.*)>/printf "%s" $(cat \1)/e' file1
before
/bin/sh: foo: command not found
after
and here's what it'll do if the INCLUDE is on a line of it's own:
$ cat file1
before
#INCLUDE<file2>
after
$ sed -r 's/#INCLUDE<(.*)>/printf "%s" $(cat \1)/e' file1
before
stuff&nonsensehere
after
This simpler script would be closer to correct if INCLUDE was on a line of its own:
$ sed -r 's/#INCLUDE<(.*)>/cat \1/e' file1
before
stuff
& nonsense
here
after
but would also fail if any other text was on the line as it'd still try to execute it!