Here is the offending part of my script:
read -d '' TEXT <<'EOF'
Some Multiline
text that
I would like
in
a
var
EOF
echo "$TEXT" > ~/some/file.txt
and the error:
read: 175: Illegal option -d
I use this read -d all over the place and it works fine. Not sure why its not happy now. I'm running the script on Ubuntu 10.10
Fixes? Workarounds?
If you run sh and then try that command, you get:
read: 1: Illegal option -d
If you do it while still in bash, it works fine.
I therefore deduce that your script is not running under bash.
Make sure that your script begins with the line:
#!/usr/bin/env bash
(or equivalent) so that the correct shell is running the script.
Alternatively, if you cannot do that (because the script is not a bash one), just be aware that -d is a bash feature and may not be available in other shells. In that case, you will need to find another way.
The -d option to read is a feature unique to bash, not part of the POSIX standard (which only specifies -r and -p options to read). When you run your script with sh on Ubuntu, it's getting run with dash, which is a POSIX shell, and not bash. If you want the script to run under bash then you should run it with bash, or give it a #!/bin/bash shebang. Otherwise, it should be expected to run under any POSIX sh.
Related
I want to check if a command exists on my machine (RedHat) inside a perl script.
Im trying to check if compgen -c contains the desired command, but running it from inside a script just gives me an empty output. Other commands work fine.
example.pl:
my $x = `compgen -c`;
print $x;
# empty output
my $y = `ls -a`;
print $y;
# .
# ..
# example.pl
Are there possible solutions for this? Or is there a better way to check for commands on my machine?
First, Perl runs external commands using /bin/sh, which is nowadays a link to a shell that is a default-of-sorts on your system. Much of the time that is bash, but not always; on RedHat it is.
This compgen is a bash builtin. One way to discover that is to run man compgen (in bash) -- and the bash manual pops up. Another way is type as Dave shows.
To use builtins we generally need to run an explicit shell for them, and they have a varied behavior in regards to whether the shell is "interactive" or not.† I can't find a discussion of that in bash documentation for this builtin but experimentation reveals that you need
my #completions = qx(bash -c "compgen -c")
The quotes are needed so to pass a complete command to a shell that will be started.
Note that this way you don't catch any STDERR out of those commands. That will come out on the terminal, and it can get missed that way. Or, you can redirect that stream in the command, by adding 2>&1 (redirect to STDOUT) at the end of it.
This is one of the reasons to use one of a number of good libraries for running and managing external commands instead of the builtin "backticks" (the qx I use above is an operator form of it.)
† This can be facilitated with -i
my #output_lines = qx(bash -i -c "command with arguments")
It's because compgen is a bash built-in command, not an external command. And when you run a command using backticks, you get your system's default shell - which is probably going to be /bin/sh, not bash.
The solution is to explicitly run bash, using the -c command-line option to give it a command to run.
my $x = `bash -c compgen -c`;
From a bash prompt, you can use type to see how a command is implemented.
$ type ssh
ssh is /usr/bin/ssh
$ type compgen
compgen is a shell builtin
I tried running this code:
#!/bin/bash
read -s "Password: " password
With command:
run sh init.sh
it throws an error: read: Illegal option -s. Any help.
I take it you're using Debian/Ubuntu, or a BSD-derivative?
When you execute a command like run sh init.sh (although I'm not myself familiar with this run command) you are overriding the #!/bin/bash shebang. In your case sh is a strictly compliant POSIX shell like dash, where, in fact, the only argument to read that is not an extension is -r.
So maybe you'd want to use run bash init.sh instead?
#!/bin/bash
echo -n "Hurry up and type something! > "
if read -t 10 response ; then
echo "Greate, you made it in time!"
else
echo "sorry, you are too slow!"
fi
I have written above code in terminal and got error "read: Illegal option -t".
Bash supports -t, so it looks like you're trying to execute it with sh or some other shell, which is odd, since you have the correct shebang.
Make sure you run it with ./script or path_to_script/script. If you just run it in the terminal, first start bash.
I had the same problem and then I figured out that I was using #!/bin/sh instead of #!/bin/bash. After changing the shebang everything worked as desired.
bash supports the -t option for the read builtin since version bash-2.04 (see ChangeLog), so either you are using an ancient version of bash (<= 2.03) or are not really running your script under bash.
Run bash --version to check the version and double-check that your shebang really looks like #!/bin/bash in your script.
#!/bin/bash
export HIVE_OPTS="$HIVE_OPTS -hiveconf mapred.job.queue.name=hdmi-technology"
export HIVE_AUX_JARS_PATH=/home/hadoop/lib/HiveUDF.jar
hive -S -e 'set mapred.job.queue.name=hdmi-technology'
hive -S -e 'SELECT count(*) from testingtable2' > attachment.txt
Whenever I try to run the above shell script(count.sh) like below, I always get errors, I have no idea what wrong I am doing as I am new to shell script and I am not sure how I can add environment variables in shell script.
bash-3.00$ sh count.sh
count.sh: HIVE_OPTS= -hiveconf mapred.job.queue.name=hdmi-technology^M: is not an
identifier
Is there anything wrong I am doing in my shell script, by the way I am adding environment variables in first two lines? Any help will be appreciated.
After all the changes I did as per the below comments,
mv count.sh count,
chmod +x count,
./count
whenever I try to do this in my prompt directly export HIVE_AUX_JARS_PATH=/home/hadoop/lib/HiveUDF.jar it works fine, but whenever I try to add this in my shell script as mentioned in my question, I always get 'java.io.FileNotFoundException(File file:/home/hadoop/lib/HiveUDF.jar does not exist. Why is it so?
You're running a bash script with a sh command that is not bash. Run it with ./count.sh so the shebang line will take effect, or just say bash count.sh.
You edited your shell script on a DOS or Windows machine, which uses the CRLF pair (\r\n) instead of the Unix-style LF ('\n') line endings.
The ^M is the tell-tale -- it is a \r carriage return character.
This should fix it:
tr -d '\r' < count.sh > count.sh.fixed
mv count.sh.fixed count.sh
Another option:
sed -i 's/\r//g' count.sh
I can't seem to find the difference between a script run two different ways.
Here's the script (named test.sh):
#! /bin/bash
printf "%b\n" "\u5A"
When the script is sourced:
. test.sh
> Z ## Result I want ##
When the script is run:
./test.sh
> \u5A ## Result I get ##
I want the run script to give the results of the sourced script... what setting do I need to set/change?
You are probably getting different versions of printf; the script you are sourcing from is probably a /bin/sh script, not a Bash script proper?
Shouldn't you be using \x instead of \u? printf "%b\n" "\x5A" works fine in both cases for me.
(Totally different idea here, so I'm posting it as another answer.)
Try running these at the command line:
builtin printf "%b\n" "\u5A"
/usr/bin/env printf "%b\n" "\u5A"
printf is both a shell builtin and an executable, and you may be getting different ones depending on whether you source or run the script. To find out, insert this in the script and run it each way:
type printf
While you're at it, you may as well insert this line too:
echo $SHELL
That will reveal if you're getting different shells, per tripleee.
HAHA!!! I finally traced down the problem! Read ahead if interested (leave the page if not).
These are the only command that will translate \u properly:
. ./test.sh ## Sourcing the script, hash-bang = #! /bin/sh
. ./test.bash ## Sourcing the script, hash-bang = #! /bin/bash
./test ## Running the script with no hash-bang
All of the following produce identical results in that they do NOT translate \u:
./test.sh ## Script is run from an interactive shell but in a non-interactive shell
## test.sh has first line: #! /bin/sh
/bin/sh -c "./test.sh" ## Running the script in a non-interactive sh shell
/bin/sh -lc "./test.sh" ## Running the script in a non-interactive, login sh shell
/bin/sh -c ". ./test.sh" ## Sourcing the file in a non-interactive sh shell
/bin/sh -lc ". ./test.sh" ## Sourcing the file in a non-interactive, login sh shell
## test.bash has first line: #! /bin/bash
/bin/bash -c "./test.bash" ## Running the script in a non-interactive bash shell
/bin/bash -lc "./test.bash" ## Running the script in a non-interactive, login bash shell
/bin/bash -c ". ./test.bash" ## Sourcing the file in a non-interactive bash shell
/bin/bash -lc ". ./test.bash" ## Sourcing the file in a non-interactive, login bash shell
## And from ***tripleee*** (thanks btw):
/bin/sh --norc; . ./test.sh ## Sourcing from an interactive sh shell without the ~/.bashrc file read
/bin/bash --norc; . ./test.bash ## Sourcing from an interactive bash shell without the ~/.bashrc file read
The only way to get proper translation is to run the script without a hash-bang... and I finally figured out why! Without a hash-bang my system chooses the default shell, which btw is NOT /bin/bash... it turns out to be /opt/local/bin/bash... two different versions of bash!
Finally, I removed the OSX /bin/bash [v3.2.48(1)] and replaced it with the MacPorts /opt/local/bin/bash [v4.2.10(2)] and now running the script works! It actually solved about 10-15 other problems I've had (like ${var,,}, read sN1 char, complete -EC "echo ' '", and a host of other commands I have scattered throughout my scripts, ~/.bashrc amd ~/.profile). Honestly, I really should have noticed when my scripts using associative arrays suddenly crapped out on me... how stupid can I get!?
I've been using bash v4 for a looong time now, and my Lion upgrade went and down-graded bash back to v3 (get with the program Apple!)... ugh, I feel so ashamed! Everyone still using bash v3, upgrade!! bash v4 is has many, many beautiful upgrades over version 3. Type bash --version to see what version you are running. One advantage is now bash can translate \uHEX into Unicode!
Try removing the space in the first line, I seem to recall that can cause problems. Offhand I'd guess that because of that space, you're not getting bash, but sh.
Glad you solved it. Still, you might be looking for a portable solution.
Assuming you are always using the same formatting string, we can just discard it, and use something like this;
printf () {
# Discard format string
shift
perl -CSD -le '
print map { s/^\\u//; chr(hex($_)) } #ARGV' "$#"
}
Edit to add: You would simply add this function definition at the beginning of your existing script, overriding the builtin printf. Obviously, if you also use printf for other stuff, this special-purpose replacement isn't good enough.
You could rename the function to uprintf or something, still. It merely translates a sequence of hex codes to the corresponding Unicode characters, discarding any \u prefix.