This question already has answers here:
Bash and filenames with spaces
(6 answers)
Closed 8 years ago.
I'm writing a script to do variable substitution into a Java properties file, of the format name=value. I have a source file, source.env like this:
TEST_ENV_1=test environment variable one
TEST_ENV_2=http://test.environment.com/one
#this is a comment with an equal sign=blah
TEST_ENV_3=/var/log/test/env/2.log
My script will replace every occurence of TEST_ENV_1 in the file dest.env with "test environment variable one", and so on.
I'm trying to process a line at a time, and having problems because looping on output from a command like sed or grep tokenizes on white space rather than the entire line:
$ for i in `sed '/^ *#/d;s/#.*//' source.env`; do
echo $i
done
TEST_ENV_1=test
environment
variable
one
TEST_ENV_2=http://test.environment.com/one
TEST_ENV_3=/var/log/test/env/2.log
How do I treat them as lines? What I want to be able to do is split each line apart on the "=" sign and make a sed script with a bunch of substitution regex's based on the source.env file.
sed '/^ *#/d;s/#.*//' source.env | while read LINE; do
echo "$LINE"
done
An alternative is to change $IFS as per #Jim's answer. It's better to avoid backticks in this case as they'll cause the entire file to be read in at once, whereas piping the output of sed to while above will allow the file to be processed line by line without reading the whole thing in to memory.
Related
This question already has answers here:
Difference between single and double quotes in Bash
(7 answers)
sed command not replacing ip address via variables [duplicate]
(1 answer)
Closed 1 year ago.
I'm familiar with sed's ability to replace in-place in a file, as answered here: sed edit file in place.
I.e. sed -i 's/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g' filename
I'm having trouble expanding this concept to replacing a string-literal that takes the form of a shell-variable -- how does one do this?
To illustrate with an example:
Given file file.txt:
# file.txt
set(FOO ${FOO})
...and shell environment variable ${FOO}:
$ FOO=bar
$ echo ${FOO}
bar
...how can I use sed to replace string-literal "${FOO}" in file.txt with the value of shell-variable ${FOO} i.e. "bar"? I.e. I'd like the resulting content of file.txt to be:
# file.txt
set(FOO bar)
I have a mental block thinking past the obviously-incorrect sed -i 's/${FOO}/${FOO}/g' file.txt
I gravitate towards sed because of past experience, and might prefer a sed-centric answer for the same reason. But any solution is probably okay, but with a preference for POSIX-compliance, if shell-native. To be even more specific, this is going to be run in a docker container with a debian-10.3 base...so I suppose solutions that work with any tools included in that distro should be okay as well.
This question already has an answer here:
How to delete a line by passing line number as variable?
(1 answer)
Closed 3 years ago.
The line is specicied by the user so I have the number of the line in a variable
sed $input 'd' file.txt > file.txt
The problem here is that I don't know where and how to put the variable $input. I have tried lot's of combinations and there are all wrong.
I know that if I put a single integer it works but I don't know the way with a variable
sed -i "${input}d" file.txt
The variable needs to be in braces, use double quotes to prevent matching problems, and use the -i switch to act directly on the file
This question already has answers here:
Capturing multiple line output into a Bash variable
(7 answers)
Closed 5 years ago.
I have a line in my shell script like below
output_logs=`sh script2.sh $1 $2`
script2.sh produce lots of lines of logs. I want to grep specific lines of its output. The problem I have is $output_logs has the entire output of script2.sh as a single line string and grep produces strange results because of this. How to get the logs from script2.sh to be stored in individual lines?
Maybe you should try something like this.
output_logs=`sh script2.sh $1 $2`
echo "$output_logs"
Note that this is different from echo $output_logs.
The double-quoted version of the variable preserves internal spacing of the value exactly as it is represented in the variable — newlines, tabs, multiple blanks and all — whereas the unquoted version replaces each sequence of one or more blanks, tabs and newlines with a single space.
This question already has answers here:
How do I use shell variables in an awk script?
(7 answers)
Closed 6 years ago.
I have a variable LINE and want to use it with awk to pull out the contents of the line numbered LINE from table.txt and make that a new variable called NAME which is then used to grep another file.
NAME=`awk 'FNR==$LINE' < table.txt`
echo "this is $NAME"
Seems to be close, but not quite the syntax.
If I use:
NAME=`awk 'FNR==1' < table.txt`
echo "this is $NAME"
Then echo gives me the first line of table.txt, if I use 2 I get the 2nd line, 3 the 3rd line, then I stopped variations.
Thanks for any advice.
EDITed first post formatting faux pas.
You're looking for:
NAME=`awk -v line="$LINE" 'FNR==line' < table.txt`
but the backticks notation is obsolete so this is better:
NAME=$(awk -v line="$LINE" 'FNR==line' < table.txt)
and you should never use all-upper-case for variable names unless they are exported (in shell) to avoid clashing with builtin names so really it should be:
name=$(awk -v line="$line" 'FNR==line' < table.txt)
but whatever you're doing is almost certainly the wrong approach and should be done entirely within awk. Make sure you fully understand everything discussed in why-is-using-a-shell-loop-to-process-text-considered-bad-practice if you're considering using shell to manipulate text.
To complement Ed Morton's helpful awk-based answer:
If you only need to extract a single line by index, sed allows for a more concise solution that is also easier to optimize (note that I've changed the variable names to avoid all-uppercase variable names):
name=$(sed -n "$line {p;q;}")
-n tells sed not to print (possibly modified) input lines by default
$line, assuming this shell variable expands it to a positive integer (see caveat below), only matches the input line with that (1-based) index.
{p;q;}, prints (p) the matching line, then exits the overall script immediately (q) as an optimization (no need to read the remaining lines).
Note:
For more complex sed scripts it is NOT a good idea to use a double-quoted shell string with shell-variable expansion as the sed script, because understanding what is interpreted by the shell up front vs. what sed ends up seeing as a result can become confusing.
Heed Ed's point that you're likely better off solving your problem with awk alone, given that awk can do its own regex matching (probably no need for grep).
This question already has answers here:
Replace a string in shell script using a variable
(12 answers)
Closed 7 years ago.
I want to use sed command in a loop passing a variable say a such that it searches for a and in the line it gets a it replaces "true" to "false".
I have a text file containing 3000 different names and another xml file containing 15000 lines. in the lines in which these 3000 entries are there i need to make changes.
I have written a code snippet but that is not giving expected output. Can anyone help. Thanks in advance.
for i in {1..3000}; do
a=`awk NR==$i'{print $1}' names.txt`
# echo $a
sed -e '/$\a/ s/true/false/' abc.xml > abc_new.xml
done
You have to replace single-quotes(') around sed's parameters with double-quotes("). In bash, single-quote won't allow variable expansion. Also, you might want to use sed's in-place edit (pass -i option) in your for loop.
So the one liner script will look like:
for a in `cat names.txt`; do sed -i.bak -e "/$a/s/true/false/" abc.xml ; done