A test file has the following string:
$ cat testfile
x is a \xtest string
The following script attempts to replace escape sequence: \x occurrences with yy using sed
#!/bin/bash
echo "Printing directly to stdout"
sed -e "s/\\\x/yy/g" testfile
var1=`sed -e "s/\\\x/yy/g" testfile`
echo "Printing from variable"
echo "${var1}"
As it can be seen below, the results are different when it is printed with and without saving to a temporary variable. Could someone help me understand why this happens?
I'd want the variable to hold the string that has replaced only \x
Printing directly to stdout
x is a yytest string
Printing from variable
yy is a \yytest string
Platform: macOS
You should put your command inside a $(...) like that:
#!/bin/bash
echo "Printing directly to stdout"
sed -e "s/\\\x/yy/g" testfile
var1=$(sed -e "s/\\\x/yy/g" testfile)
echo "Printing from variable"
echo "${var1}"
Related
I'm trying to write a shell script that takes in a file(ex. file_1_2.txt) and replaces any "_" with "."(ex. file.1.2.txt). This is what I have but its giving me a blank output when I run it.
read $var
x= `echo $var | sed 's/\./_/g'`
echo $x
I'm trying to store the changed filename in the variable "x" and then output x to the console.
I am calling this script by writing
./script2.sh < file_1_2.txt
There is two problems. First, your code has some bugs:
read var
x=`echo $var | sed 's/_/\./g'`
echo $x
will work. You had an extra $ in read var, a space too much (as mentioned before) and you mixed up the replacement pattern in sed (it was doing the reverse of what you wanted).
Also if you want to replace the _ by . in the filename you should do
echo "file_1_2.txt" | ./script2.sh
If you use < this will read the content of `file_1_2.txt" into your script.
Another solution, with bash only:
$ x=file_1_2.txt; echo "${x//_/.}"
file.1.2.txt
(See “Parameter expansion” section in bash manual page for details)
And you can also do this with rename:
$ touch file_1_2.txt
$ ls file*
file_1_2.txt
$ rename 'y/_/\./' file_1_2.txt
$ ls file*
file.1.2.txt
Threre is not need for sed as bash supports variable replacement:
$ cat ./script2
#!/bin/bash
ofile=$1
nfile=${ofile//_/./}
echo mv "$ofile" "$nfile"
$ ./script2 file_1_2.txt
mv "file_1_2.txt" "file.1.2.txt"
Then just remove echo if you are satisfied with the result.
I want to be able to add newline characters before every occurences of some tokens appearing in some .tex files that I possess, some of those tokens are '\itemQ', '\pagebreakQ'. I created a procedure that ends up creating a command for sed stored in $sedInpt:
~$ echo "$sedInpt"
-e s/\(\\itemQ\)/\n\1/ -e s/\(\\pagebreakQ\)/\n\1/
I want to use "$sedInpt" as a command for sed:
echo "$inputText" | eval "sed ${sedInpt}"
but if I do the following as a test:
echo 'hello\itemQ' | eval "sed ${sedInpt}"
hello\itemQ
you can see there ain't any newline that has been added before \itemQ.
So I've tried debugging this way of doing thing by calling bash -x to see what's happened in detail:
~$ bash -x
~$ echo "hello\itemQ" | eval "sed ${sedInpt}"
+ echo 'hello\itemQ'
+ eval 'sed -e s/\(\\itemQ\)/\n\1/ -e s/\(\\pagebreakQ\)/\n\1/'
++ sed -e 's/(\itemQ)/n1/' -e 's/(\pagebreakQ)/n1/'
hello\itemQ
you can see that the backslashes of \n and \1 and even the ones before ( and ) that I had placed in "$sedInpt" seem to have disappeared when parsed by eval.
So I am bit lost on what to do next to do what I want.. any ideas?
You could also just combine them into a single command, which in my opinion is more straightforward:
$ cat /tmp/sed.sh
sedInpt='s/\(\\itemQ\)/\n\1/; s/\(\\pagebreakQ\)/\n\1/'
echo "hello\itemQ" | sed "$sedInpt"
$ /tmp/sed.sh
hello
\itemQ
Edit: As #123 rightly points out, storing commands in variables is dangerous and should be avoided if possible. If you have complete control over what is stored, it should be safe, but if it comes from any sort of user input, it is a "Command Injection" vulnerability.
Following #Inian advice I managed to achieve what I wanted to do in this way:
~$ sedInpt=( -e 's/\(\\itemQ\)/\n\1/' -e 's/\(\\pagebreakQ\)/\n\1/' )
~$ echo "hello\itemQ" | sed "${sedInpt[#]}"
hello
\itemQ
I came across an interesting thing in Shell Scripting and not 100% sure why the behaviour is like this
I tried the below script:
#!/bin/sh
CMD="curl -XGET http://../endpoint";
var1=eval $CMD | sed -e 's/find/replace/g';
echo $var1; # Output: printed the value on this line
echo $var1; # Output: blank/no data printed (Why it is blank?)
I had to change the command in variable enclosing with back-tick ` to print the variable as many time as I wanted.
CMD="curl -XGET http://../endpoint";
var1=`eval $CMD | sed -e 's/find/replace/g'`;
echo $var1; # Output: printed the value on this line
echo $var1; # Output: printed the value on this line
Why I have to surround my command with ` to assign it's o/p to the variable in subsequent variable usage?
I have a feeling that it has something to do with the variable-command scope.
Shedding light on my understanding will be appreciated!
UPDATE:
I tried the below command and it is working in my env.
#!/bin/sh
CMD="curl -XGET http://www.google.com/";
var1=eval $CMD | sed -e 's/find/replace/g';
echo $var1; # Output: printed the value on this line
echo "######";
echo $var1; # Output: blank/no data printed (Why it is blank?)
sh/bash allows you to run a command with a variable in its environment, without permanently modifying the variable in the shell. This is great, because you can e.g. run a command in a certain language just one time without having to change your entire user's or system's language:
$ LC_ALL=en_US.utf8 ls foo
ls: cannot access foo: No such file or directory
$ LC_ALL=nb_NO.utf8 ls foo
ls: cannot access foo: Ingen slik fil eller filkatalog
However, this means that when you try to do
var=this is some command
you're trigger this syntax.
It means "run the command is a command and tell it that the variable var is set to this"
It does not assign "this is my string" to the variable, and it definitely does not evaluate "this is a string" as a command, and then assign its output to var.
Given this, we can look at what actually happened:
CMD="curl -XGET http://../endpoint";
var1=eval $CMD | sed -e 's/find/replace/g'; # No assignment, output to screen
echo $var1; # Output: blank/no data printed
echo $var1; # Output: blank/no data printed
There is no scope issue and no inconsistency: the variable is never assigned, and is never written by an echo statement.
var=`some command` (or preferably, var=$(some command)) works because this is valid syntax to assign output from a program to a variable.
The first example isn't doing what you think it is.
Neither echo is printing anything. Make them echo "[$var1]" to see that.
You need the backticks to run the command and capture its output.
Your first attempt was running the $CMD | sed -e 's/find/replace/g'; pipeline with the environment of $CMD containing var1 set to a value of eval.
You also shouldn't be putting commands inside strings (or using eval in general). See http://mywiki.wooledge.org/BashFAQ/001 for more on why.
I am trying, so far unsuccessfully to read and print a tab character from a file in a Bourne shell script.
For example, here is my file, in.txt (stackoverflow won't let me write a tab, so replace [tabcharacter] with a tab):
[tabcharacter]Hello World!
My script as as follows:
#!/bin/sh
while read line
do
echo -e "${line}" >> out.txt
/bin/echo -e "${line}" >> out.txt
done < "./in.txt"
The out.txt I get is:
-e hello!
hello!
Whereas I would expect from one of these the output to be the same as in.txt.
I think it's a problem with the way I use the read command. But I'm not sure how I can get it to read tabs.
Any help much appreciated.
#!/bin/sh
export IFS=
while read line
do
echo -e "$line" >> out.txt
/bin/echo -e "$line" >> out.txt
done < "./in.txt"
I seted the IFS variable to a empty string, now its working, please test it!
When we use shell expansion, it gives all the expanded word in one line. For example:
#!/bin/bash
data="Hello\ {World,Rafi}"
eval echo $data
This produces the following output:
Hello World Hello Rafi
Is it possible to output each line on a separate line like this?
Hello World
Hello Rafi
If I understand you right, you want to generate multiple words using brace expansion ({...}), then print each word on a separate line.
If you don't absolutely have to store "Hello\ {World,Rafi}" in a variable, you can do this with printf shell-builtin
printf "%s\n" "Hello "{Rafi,World}
Some explanation:
The format string (here: %s\n) is reused until all the arguments to printf is used up (Reference).
%s\n consumes 1 argument
"Hello "{Rafi,World} returns 2 words/arguments i.e. "Hello Rafi" and "Hello World"
So, this printf command is equivalent to
printf "%s\n%s\n" "Hello Rafi" "Hello World"
except you don't have to type all that up.
#!/bin/bash
data="Hello\ {World'\n',Rafi'\n',Kamal'\n'}"
eval echo -e "$data"
echo -e will evaluate newline characters.
Same as Antarus'a answer, except that echo has "-n". From http://unixhelp.ed.ac.uk/CGI/man-cgi?echo
-n do not output the trailing newline
#!/bin/bash
data="Hello\ {World,Rafi}'\n'"
eval echo -n -e "$data"
Actually, your problem is not the expansion but the echo command. Depending on your system, you might get what you want by
#!/bin/bash
data="Hello\ {World\\\\n,Rafi}"
eval echo -e "$data"
It is different solution but a very clean one.
#!/bin/bash
names="World Rafi"
for name in $names
do
echo Hello $name
done
Instead of using eval (which is dangerous and really not a good practice — see my comment in your post), another strategy would be to use an array. The following will do exactly what you want, in a clean and safe way:
data=( "Hello "{World,Rafi} )
printf "%s\n" "${data[#]}"