I am trying to create an alias on Hadoop machine and run it from Hive JVM.
When I explicitly run the command from Hive with ! prefix it works, however when I add the alias, source the .bashrc file and call the alias from Hive, I get an error. Example:
.bashrc content:
# Environment variables required by hadoop
export JAVA_HOME=/usr/lib/jvm/java-7-oracle
export HADOOP_HOME_WARN_SUPPRESS=true
export HADOOP_HOME=/home/hadoop
export PATH=$PATH:/home/hadoop/bin
alias load-table='java -cp /home/hadoop/userlib/MyJar.jar com.MyClass.TableLoader';
Call on Hive:
!load-table;
Output:
Exception raised from Shell command Cannot run program "load-table": error=2, No such file or directory
Aliases have several limitations compared to shell functions (e.g. by default you cannot call an alias from a non-interactive shell).
Define in your ~/.bashrc:
function load-table() {
# Make sure the java executable is accessible
if which java > /dev/null 2>&1; then
java -cp /home/hadoop/userlib/MyJar.jar com.MyClass.TableLoader
else
echo "java not found! Check your PATH!"
fi
}
export -f load-table # to export the function (BASH specific)
Source you .bashrc to apply the changes. Then, call load-table.
Related
I want to run a single line command via NiFi ExecuteStreamCommand processor. I want to run a gsutil command and before doing that I want to export environment variable GOOGLE_APPLICATION_CREDENTIALS.
So the command would be
export GOOGLE_APPLICATION_CREDENTIALS='/temp/abc.json'
However, NiFi needs the path of the command. On the server when I checked which export I do not get its path:
[user#server1 ~]$ which export
/usr/bin/which: no export in (/opt/teradata/client/14.10/tbuild/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin:/home/user/.local/bin:/home/user/bin:/usr/local/google-cloud-sdk/bin/)
If its a builtin command, how do I get NiFi to run it?
how do I get NiFi to run it?
Use env to run a command with modified environment.
/usr/bin/env GOOGLE_APPLICATION_CREDENTIALS='/temp/abc.json' gsutil
I want to set env variable in shell script. Shell script content is:
#!/bin/bash
export XDEBUG_CONFIG="idekey=PHPSTORM"
I tried both bash bin/enable_debug and bin/enable_debug. After both command I get:
$ echo $XDEBUG_CONFIG
$
However if I run export XDEBUG_CONFIG="idekey=PHPSTORM" directly in cli it works. What's wrong with my method?
You can try running your script as below:
. bin/enable_debug
OR
source bin/enable_debug
as indicated by #Aserre
I am new to the oracle apps form development.
i am unable to generate .FMX file using below command in putty.
frmcmp_batch.sh module=/u01/install/APPS/apps/apps_st/appl/au/12.0.0/forms/US/EMP.fmb
userid=apps/apps
output_file=/u01/install/APPS/apps/apps_st/appl/po/12.0.0/forms/US/EMP.fmx module_type=form
Please help me on the same.
Thanks&Regards,
Vivek
you may call a script from the command line
$ appCompile.sh EMP.fmb
where appCompile.sh maybe like below one
ORACLE_HOME=/u01/install/APPS/apps/apps_st/appl/au/12.0.0/forms/US; export ORACLE_HOME
export NLS_LANG=american_america.we8iso8859p9 #for Turkish
NLS_DATE_FORMAT=DD/MM/YYYY; export NLS_DATE_FORMAT
FORMS_PATH=/data/aski_kodlar/standard; export FORMS_PATH
alias oh='cd $ORACLE_HOME'
LD_LIBRARY_PATH=/u01/install/APPS/apps/apps_st/appl/au/12.0.0/forms/US/lib:/u01/install/APPS/apps/apps_st/appl/au/12.0.0/forms/US/jdk/jre/lib/sparcv9:
/u01/install/APPS/apps/apps_st/appl/au/12.0.0/forms/US/jdk/jre/lib/sparcv9/server:/u01/install/APPS/apps/apps_st/appl/au/12.0.0/forms/US/jdk/jre/lib/sparcv9/native_threads
export LD_LIBRARY_PATH
export ORACLE_TERM=vt220
export TERM=xterm
type=$2
if test "$type" = ""
then
type=form
fi
echo Compiling Form $1 ....
filename=`echo $1|cut -f1 -d.`
/u01/install/APPS/apps/apps_st/appl/scripts/frmcmp_batch.sh userid=apps/apps#db_name Module_Type=$type compile_all=yes window_state=minimize batch=yes Module=$1
Before compile you must SET the environment variables in linux, it depends on which kind of environment you are logged in, if it is Oracle On-Demand or Custom.
For custom:
Search a file extension .env
Usually located in /u01/oracle/EBS/app, run that file to SET environment variables.
For Oracle On-Demand:
In Linux SSH, run comand below where XXXX is the database
pbrun impdba -u apXXXX
Afer that you must run your compleation script.
Put your promp in
cd $AU_TOP/forms/US
export PATH=$PATH:$AU_TOP/resource:$AU_TOP/forms/US
Run compilation script by replacing APPS_PASSWORD, XXCUST_TOP, XX_FORM_FILE.
frmcmp_batch module=$XXHMS_TOP/forms/US/XX_FORM_FILE.fmb userid=apps/APPS_PASSWORD output_file=$XXCUST_TOP/forms/US/XX_FORM_FILE.fmx compile_all=special batch=yes
It will create a LOG file with .err extension.
this would help :
frmcmp_batch module=/disk5/PROD/apps/apps_st/appl/au/12.0.0/forms/US/EMP.fmb userid=apps/apps output_file=/disk5/PROD/apps/apps_st/appl/ont/12.0.0/forms/US/EMP.fmx module_type=form batch=yes
I'm very confused about how my shell is reading bash_profile.
In root, my ~/.bash_profile looks like so
# .bash_profile
# Get the aliases and functions
if [-f ~/.bashrc ]; then
.~/.bashrc
fi
PATH=$PATH:$HOME/bin:$HOME/sbin:$HOME/usr/sbin:$HOME/usr/bin:/usr/sbin
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
export PATH=$PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH
unset USERNAME
There is no ~/.profile file.
In a user called maruhan, my ~/.bash_profile looks like so
# .bash_profile
# Get the aliases and functions
if [-f ~/.bashrc ]; then
.~/.bashrc
fi
PATH=$PATH:$HOME/bin:$HOME/sbin:$HOME/usr/sbin:$HOME/usr/bin:/usr/sbin
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/maruhan/Desktop/issac:/usr/local/lib
ASDF=$ASDF:/home
export PATH=$PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH
export ASDF=$ASDF
unset USERNAME
And my ~/.profile looks like so
LD_LIBRARY_PATH=/home/maruhan/Desktop/issac:/usr/local/lib:$LD_LIBRARY_PATH
ASDF=/home:$ASDF
export ASDF=$ASDF
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH
You can clearly see that ASDF is not defined in root's bash_profile.
However when I call export, I get this in root.
declare -x ASDF=":/home"
but nothing about LD_LIBRARY_PATH.
Strangely in maruhan, running export shows both ASDF and LD_LIBRARY_PATH.
Also, nothing about ASDF or LD_LIBRARY_PATH exist in /etc/environment. I also don't have a /etc/bash_profile file.
Running echo $0 gives me bash for both root and maruhan.
How come LD_LIBRARY_PATH disappeared in root while ASDF is there?
The rules are a bit complicated. According to bash's man page:
INVOCATION
A login shell is one whose first character of argument zero is a -, or one
started with the --login option.
An interactive shell is one started without non-option arguments (unless -s is
specified) and without the -c option whose standard input and error are both
connected to terminals (as determined by isatty(3)), or one started with the -i
option. PS1 is set and $- includes i if bash is interactive, allowing a shell
script or a startup file to test this state.
... ...
When bash is invoked as an interactive login shell, or as a non-interactive
shell with the --login option, it first reads and executes commands from the
file /etc/profile, if that file exists. After reading that file, it looks for
~/.bash_profile, ~/.bash_login, and ~/.profile, in that order, and reads and
executes commands from the first one that exists and is readable. The --noprofile
option may be used when the shell is started to inhibit this behavior.
... ...
When an interactive shell that is not a login shell is started, bash reads and
executes commands from ~/.bashrc, if that file exists. This may be inhibited by
using the --norc option. The --rcfile file option will force bash to read and
execute commands from file instead of ~/.bashrc.
... ...
Note that on some systems bash may be customized so that it would also execute a system wide rc file (e.g. /etc/bash.bashrc) before sourcing ~/.bashrc for an interactive shell that's not a login shell.
Shells started by a login mechanism (usually with a username/password prompt, like console login, telnet, ssh, ...) are usually login shells. For a login shell, $0 is usually -bash.
[local] % ssh user#host <-- The user is trying to login
Password: P#ssw0rd
[remote] % echo $0
-bash <-- This is a login shell
[remote] % bash <-- This is not a login (no username/password)
[remote] % echo $0
bash <-- Not a login shell
[remote] %
To make life easier I would put all rc things in ~/.bashrc and source ~/.bashrc in ~/.bash_profile. For example:
% cat ~/.bash_profile
[[ -f ~/.bashrc ]] && source ~/.bashrc
% cat ~/.bashrc
# return immediately if not in an interactive shell
[[ $- != *i* ]] && return 0
export FOO=bar
PATH=$PATH:/my/path
%
Below is my shell script that I am trying to execute using PLINK on MachineB from MachineA(Windows Machine).
#!/bin/bash
export HIVE_OPTS="$HIVE_OPTS -hiveconf mapred.job.queue.name=hdmi-technology"
hive -S -e 'SELECT count(*) from testingtable1' > attachment22.txt
I am using plink to execute the shell script like below,
C:\PLINK>plink uname#MachineB -m test.sh
Using keyboard-interactive authentication.
Password:
Using keyboard-interactive authentication.
Your Kerberos password will expire in 73 days.
And this is the below error I always get whenever I try to run like above.
sh: HIVE_OPTS= -hiveconf mapred.job.queue.name=hdmi-technology: is not
an identifier
Something wrong with my shell script? or some trailing spaces? I am not able to figure it out. I am running PLINK from windows machine
The sh: prefix on the error message indicates that the script is being executed by sh, not bash.
bash lets you combine setting a variable and exporting it into a single command:
export foo=bar
sh, or at least some older versions of it, require these two actions to be separated:
foo=bar ; export foo
A version of sh that doesn't recognize the export foo=bar syntax will interpret the string foo=bar as a variable name (and an illegal one, since it isn't an identifier).
Either arrange for the script to be executed by bash, or change this:
export HIVE_OPTS="$HIVE_OPTS -hiveconf mapred.job.queue.name=hdmi-technology"
to this:
HIVE_OPTS="$HIVE_OPTS -hiveconf mapred.job.queue.name=hdmi-technology"
export HIVE_OPTS
For that matter, since you're referring to $HIVE_OPTS at the very beginning of your script, it's almost certainly already exported, so you could just drop the export.
(You'll also need to avoid any other bash-specific features.)
So why is the system invoking the shell with sh? The #!/bin/bash syntax is specific to Unix-like systems. Windows generally decides how to execute a script based on the file extension; apparently your system is configured to invoke *.sh files using sh. (You could configure your system, using Folder Options, to invoke *.sh files using bash, but that might introduce other problems.)
I think the -m option to plink is for reading commands to execute on the remote machine from a local file. If my comment about line endings doesn't work, try
plink uname#MachineB test.sh
Make sure test.sh is executable by running
chmod +x test.sh
on MachineB.