I can't understand this...
I have .sh file with variables (varsource.sh)
var1=AppleOrange
var2=Mango
Now I am sourcing my varsource.sh in my test.sh script
#!/bin/ksh
. ./varsource.sh
appended=$var1$var2
echo $appended
varloc1=aPPLEoRANGE
varloc2=mANGO
locappended=$varloc1$varloc2
echo $locappended
The output of above script is
MangoOrange
aPPLEoRANGEmANGO
I expected similar behavior when I use variables from sourced file and variables local to my script.
In case of variables from sourced file, the second variable is replacing characters of the first variable instead of appending.
More observations:
. ./varsource.sh
appended=${var1}xx
echo $appended
Output: xxpleOrange
But appending to left end of the variable is working
. ./varsource.sh
appended=xx$var1
echo $appended
Output:xxAppleOrange
Could some one help me understand this behaviour? What should I do to perform the appending in case of sourced variables?
You're on a Windows machine, or the sourced file was created on a Windows machine.
Your line endings are CRLF, carriage return and line feed, DOS/Windows style. The shell treats the carriage return as a regular character, not as part of the 'end of line'.
Remove the carriage returns and all will go back to normal.
Related
I have a file that I pass to a bash command that will create an output in a loop like so:
for file in /file/list/*
do
command
done
I wish to save the output that would have gone to standard out of each loop to a text file in my working directory. Currently I am trying this:
for file in /file/list/*
do
command | tee "$file_command output.txt"
done
What I expect to see are new files created in my current directory titled file1.txt_commandoutput.txt, file2.txt_commandoutput.txt, etc. The output of the command should be saved as a different file for each file. However I get only one file created and it's called ".txt" and can't be opened by any standard software on Mac. I am new to bash scripting, so help would be much appreciated!
Thanks.
Your problem comes from the variable name you're using:
"$file_command_output.txt" looks for a variable named file_command_output (the dot cannot be in the variable name, but the alphanumerical characters and the underscore all can).
What you're looking for is "${file}_command_output.txt" to make the variable name more explicit.
You have two issues in your script.
First, the wrong parameter/variable is expanded (file_command instead of file) because it's followed by a character that can be interpreted as part of the name (the underscore, _). To fix it, enclose the parameter name in braces, like this: ${file}_command (see Shell Parameter Expansion in bash manual).
Second, even with fixed variable name expansion, the file won't be created in your working directory, because the file holds an absolute pathname (/file/list/name). To fix it, you'll have to strip the directory from the pathname. You can do that with either basename command, or even better with a modified shell parameter expansion that will strip the longest matching prefix, like this: ${file##*/} (again, see Shell Parameter Expansion, section on ${parameter##word}).
All put together, your script now looks like:
#!/bin/bash
for file in /file/list/*
do
command | tee "${file##*/}_command output.txt"
done
Also, to just save the command output to a file, without printing it in terminal, you can use a simple redirection, instead of tee, like this: command > "${file##*/}_com...".
If you are not aware of xargs, try this:
$ ls
file
$ cat > file
one
two
three
$ while read this; do touch $this; done < ./file
$ ls
file one three two
Consider the following script:
#!/bin/bash
CMD="echo hello world > /tmp/hello.out"
${CMD}
The output for this is:
hello world > /tmp/hello.out
How can I modify CMD so that the output gets redirected to hello.out?
For my use case, it is not feasible to either do this:
${CMD} > /tmp/hello.out
or to add this at the top of the script:
exec > /tmp/hello.out
No, there is no way to make a redirection happen from a variable.
Why?
The first thing the shell does with a command line is:
Each line that the shell reads from the standard input or a script is called a pipeline; it contains one or more commands separated by zero or
more pipe characters (|). For each pipeline it reads, the shell breaks it up into commands, sets up the I/O for the pipeline, then does the following for each command (Figure 7-1):
From: Learning the bash Shell Unix Shell Programming . Chapter Preview / Figure . Pdf
That means that even before starting with the first word of a command line, the redirections are set up.
The "Parameter Expansion" happens quite a lot latter (in step 6 of the Figure).
There is no way to set up redirections after a variable is expanded.
Unless ...
The "command line is reprocessed" using eval.
eval "$CMD"
But this comes with a lot of danger.
The command line is changed by the first processing in the 12 steps detailed in the book (quotes are removed, variables expanded, words split, etc.).
It is usually quite difficult to estimate all the changes and consequences before the line is actually processed.
And then, it is processed again.
You can use eval to instruct the shell to reinterpret the variable content as a shell command:
eval $CMD
This question already has answers here:
Bash syntax error: unexpected end of file
(21 answers)
Closed 8 years ago.
If I set a variable in a shell script and then try to use it to create another variable, it doesn't seem to get substituted correctly. Example:
#!/bin/bash
X=/software/xxx
echo variable X = $X
echo path using variable: $X/yyy
echo path without variable: /software/xxx/yyy
This outputs:
variable X = /software/xxx
/yyy using variable: /software/xxx
path without variable: /software/xxx/yyy
I had expected the second output line to be:
path using variable: /software/xxx/yyy
I have tried various combinations of quotes and using ${X}, but all to no avail.
I'm new to shell scripting (coming from a Windows background), so I'm sure there is something really simple that I'm missing here.
In case anyone wonders why I want to do this, the background to this is that I need to write a shell script that takes a relative pathname parameter, determines its absolute pathname, then sets CLASSPATH with a number of jar files in that directory before invoking a Java program:
#!/bin/bash
DIR=`readlink -fn $1`
export CLASSPATH=$DIR/x.jar:$DIR/y.jar
java progname
You have carriage return (CR) character at the end of your X= variable declaration:
#!/bin/bash
X=/software/xxx^M
echo variable X = $X
echo path using variable: $X/yyy
(where ^M represents ONE special character - CR). I assume your editor doesn't show it. Try to open your script with vim. Windows and Linux marks newline in a different way (check wiki) . I suppose you're using some Windows editor.
Please read Bash Pitfalls webpage. I'm sure you can learn a lot from there.
btw. one way to enter ^M at vim: go to insert-mode i, than press: CTRL+k, M, ENTER
Example one
#!/bin/sh
# purpose: print out current directory name and contents
pwd
ls
Example two
# purpose: print out current directory name and contents
#!/bin/sh
pwd
ls
What is the difference – if I make the first line a comment(#), with #!/bin/sh as the second line, what will happen?
What is meaning of #!/bin/sh ?
Normally a shell script is run by your default shell defined in the /etc/passwd file. But you can define explicitly a program which can run your script.
Unices uses a common method to determine what program needed to run a specific script (man execve(2)). If the script has the proper execute rights set and in a script the first line starts with a #! characters, it will run by the program defined afterwards.
For example if the first line is #!/usr/bin/awk -f then the rest of the file will be passed to the awk program (so it has to use awk syntax). Or if a Makefile starts with #!/usr/bin/make -f then the rest of the file will be passed to make. You can start the script as a normal program and the script can be written in awk or make (or whatever defined) syntax.
If execve does not find #! as the first two character of the file, it will consider as a normal script file and it will run as it is.
So using #! You can determine the script language and You do not need to know what shell is used by the other user using your script. In any other line #! will be interpretered your default shell, which is usually just a comment line.
what is difference between 1st & 2nd shell scripts..?
No difference in output. But the time to execute both will be little different as the interpreter reads line one by one.
if i give comment(#) in 1st line after #!/bin/sh in 2nd line so what will happen ?
Any line started with (#) except the shebang(#!) is treated as a comment in shell script.
what is meaning of #!/bin/sh ?
Its the path(here - /bin/sh) to the interpreter used after the shebang (#!) . Shell will try to use the interpreter language mentioned after the shebang to execute the script.
I am trying to write a script which reads the hostname of the remote machine and then uses that result in following commands. However, the variable seems to be corrupted or something.
Here is an example of what is happening:
sbaker#eye004:~/workspace/fire_trunk$ REMOTE_HOSTNAME="`ssh $REMOTE 'hostname'`"
sbaker#eye004:~/workspace/fire_trunk$ echo "before $REMOTE_HOSTNAME after"
prints (note the prefixing whitespace):
" after sbaker-PC"
sbaker#eye004:~/workspace/fire_trunk$ echo $REMOTE_HOSTNAME
prints:
"sbaker-PC"
I am wondering why the variable seems unusable and doing weird things (if the after word is longer than the before word, it writes over the top of the characters).
I would expect the first echo to print: "before sbaker-PC after".
Am I just doing something stupid here?
I am using bash on ubuntu 11.
If you put it through od -c you'll see that it's actually returning sbaker-PC\r. The CR at the end is causing it to return the cursor to the first column before echoing the rest of the text, obscuring the "before". As for why it's adding \r, perhaps the file giving the hostname on the other side was saved with DOS line endings (CRLF) instead of *nix line endings (LF).