I am not sure if I am asking the question correctly.
This is what I working with:
a=foo
b=bar
cat 1.txt > new1.txt
Contents of 1.txt:
$a
$b
When run this, my new1.txt looks like this:
$a
$b
What I want:
foo
bar
With envsubst:
export a="foo"
export b="bar"
envsubst < 1.txt > new1.txt
Output to new1.txt:
foo
bar
With sed
sed "s/\$a/$a/; s/\$b/$b/" 1.txt > new1.txt
Related
I am using Linux and bash.
I have a simple text file like below:
VAR1=100
VAR2=5
VAR3=0
VAR4=99
I want to extract by means of bash the value of VAR2, that is 5.
How could I do that?
Assuming the file is called vars.txt
sed -n 's/^VAR2=\(.*\)/\1/p' < vars.txt
You can use the value elsewhere like this using single back quotes
echo VAR2=`sed -n 's/^VAR2=\(.*\)/\1/p' < txt`
The simplest way might be to use source or simply . to read and execute the file. This would work with your example, because there are no spaces in the variable values. Otherwise you need to use grep + cut or awk, as stated in other answers.
. /path/to/your/file
echo $VAR2
[edit]
As stated by dawg, this would make the other variables available in your script too, and possibly overwrite existing variables.
Given:
$ echo "$txt"
VAR1=100
VAR2=5
VAR3=0
VAR4=99
You can use awk:
$ echo "$txt" | awk -F= '/^VAR2/ { print $2 }'
5
Or grep and cut:
$ echo "$txt" | egrep '^VAR2=\d+' | cut -d = -f 2
5
On Bash, you can insert the value of those assignments into the current shell using source and filter the lines you wish to use. In this case, only the line VAR2=5 will be used. You need to write that to a file and then source that file:
$ echo "$txt" | grep '^VAR2' > tmp && source tmp && rm tmp
$ echo $VAR2
5
For the files as described, you can just source the file as bash script which will run it's content and update you workspace environment with it. For example:
source file.txt
echo $VAR2
Assume this as your txt file, named test.txt
VAR2 = 5
VAR3 = 0
VAR4 = 99
you can cat test.txt | grep 'VAR2' | awk '{printf $3}'
and then your output will be: 5
Here, cat test.txt will display the content of test.txt in your terminal,grep 'VAR2' will list lines containing 'VAR2' and awk '{printf $3}' will print the value of the variable
Sample data in a stackoverflow.csv file I have:
foo
foo
foo
foo
bar
bar
bar
bar
baz
baz
baz
baz
I know with sed -n /foo/p stackoverflow.csv > foo.csv
I'll get all records matching foo directed to that file, but I don't want to specify the matching pattern on the cli, I'd rather put it in a script and have all records (foo, bar and baz) sent to their own file.
Basically this in a script:
sed -n /foo/p stackoverflow.csv > foo.csv
sed -n /bar/p stackoverflow.csv > bar.csv
sed -n /baz/p stackoverflow.csv > baz.csv
Like this:
#!/bin/sh
sleep 2
sed -n /foo/p > foo.csv
sed -n /bar/p > bar.csv
sed -n /baz/p > baz.csv
This creates the files but they're all empty.
Also, if the script were to have just one print statement, it works.
Any input?
You missed the input filename
#!/bin/sh
sleep 2
sed -n /foo/p stackoverflow.csv > foo.csv
sed -n /bar/p stackoverflow.csv > bar.csv
sed -n /baz/p stackoverflow.csv > baz.csv
You can provide input file as an argument to your script, in that case change the script to read an argument and pass it to the sed command.
Run the script like this: ./script.sh input_filename, where you can specify different input files as argument.
#!/bin/sh
file=${1}
sed -n /foo/p ${file} > foo.csv
sed -n /bar/p ${file} > bar.csv
sed -n /baz/p ${file} > baz.csv
Here's a solution with grep in a loop where you don't need to provideeach term in advance. Assuming a bash shell:
for i in $(sort stackoverflow.csv | uniq); do grep $i testfile > $i; done
The w command in sed allows you to write to a named file.
sed -n -e '/foo/wfoo.csv' -e '/bar/wbar.csv' \
-e '/baz/wbaz.csv' stackoverflow.csv
Not all sed dialects accept multiple -e arguments; a multi-line string with the commands separated by newlines may be the most portable solution.
sed -n '/foo/wfoo.csv
/bar/wbar.csv
/baz/wbaz.csv' stackoverflow.csv
This examines the input file just once, and writes out the matching lines as it proceeds through the file.
To create a script which accepts one or more file names as arguments, replace the hard-coded file name with "$#".
#!/bin/sh
sed -n ... "$#"
Related questions:
This question ended up not requiring a resolution.
This question had a resolution not involving timeout, but suggested that timeout should work for this purpose when available.
My question:
This command produces no output into foo.txt:
$ cat foo.sh
#!/bin/sh
sh -c "echo foo; sleep 5; echo bar" | awk '/foo/ {print $1}'
$ timeout 2 ./foo.sh > foo.txt
If I don't redirect into foo.txt, I see foo print out immediately as expected.
On the other hand, the following produces "foo" int the file foo.txt as expected:
$ timeout 2 sh -c "echo foo; sleep 5; echo bar" > foo.txt
$ cat foo.txt
foo
Does anyone know why this may be happening and how best to resolve it? This is a toy example, but the actual script I'm running that led to this problem produces around 100 lines of output on the command line, but also leaves foo.txt empty if it times out before terminating.
I found a solution to this. The key is to add fflush() inside the awk script, which seemed to be buffering the output:
#!/bin/sh
sh -c "echo foo; sleep 5; echo bar" | awk '/foo/ {print $1; fflush()}'
$ timeout 2 ./foo.sh > foo.txt
In my experience, this is because pipe "|" wait "echo foo; sleep 5; echo bar" run complete .So after 5s awk can get the output, but timeout terminate the command in 2s so it cannot get the text.
Edit:
Maybe this helps, you can move char (") to the end like this:
$ cat foo.sh
#!/bin/sh
sh -c "echo foo; sleep 5; echo bar | awk '/foo/ {print $1;}'"
$ timeout 2 ./foo.sh > foo.txt
$ cat foo.txt
foo
Redirection to a file is very usefull to append a string as a new line to a file, like
echo "foo" >> file.txt
echo "bar" >> file.txt
Result:
foo
bar
But is it also possible to redirect a string to the same line in the file ?
Example:
echo "foo" <redirection-command-for-same-line> file.txt
echo "bar" <redirection-command-for-same-line> file.txt
Result:
foobar
The newline is added by echo, not by the redirection. Just pass the -n switch to echo to suppress it:
echo -n "foo" >> file.txt
echo -n "bar" >> file.txt
-n do not output the trailing newline
An alternate way to echo results to one line would be to simply assign the results to variables. Example:
j=$(echo foo)
i=$(echo bar)
echo $j$i
foobar
echo $i $j
bar foo
This is particularly useful when you have more complex functions, maybe a complex 'awk' statement to pull out a particular cell in a row, then pair it with another set.
I have a file called a.txt. with values like
1
2
3
...
I want to overwrite this file but
echo "$var" >> a.txt
echo "$var1" >> a.txt
echo "$var2" >> a.txt
...
just appends. Using > is not useful as well. How can i overwrite with using >> operator in shell script?
You may want to use > for the first redirection and >> for subsequent redirections:
echo "$var" > a.txt
echo "$var1" >> a.txt
echo "$var2" >> a.txt
> truncates the file if it exists, and would do what you originally asked.
>> appends to the file if it exists.
If you want to overwrite the content of a file (not truncate it), use 1<>
e.g.:
[23:58:27 0 ~/tmp] $ echo foobar >a
[23:58:28 0 ~/tmp] $ cat a
foobar
[23:58:50 0 ~/tmp] $ echo -n bar 1<>a
[23:58:53 0 ~/tmp] $ cat a
barbar
In what way is using > not useful? That explicitly does what you want by overwriting the file, so use > for the first and then >> to append future values.
echo "$var
$var1
$var2" > a.txt
or
echo -e "$var\n$var1\n$var2" > a.txt