I am a newer in shell script, following is a shell script:
$JAVA_HOME/bin/java -Dpid=MyJava \
-Xms${HEAP_MIN}m -Xmx${HEAP_MAX}m -cp ${CPG_CLASSPATH} \
-Dconfig=${CFG_FILE} \
-Dcom.test.eps.configpath=${my_config}/ \
-Dcom.test.eps.rt.config=${my_config}/ \
-Dlog4j.configuration=file:///${my_config}/log4j.properties.ewf.rt \
com.test.MyJava &
Can anyone tell me the code meaning of every line above?
$JAVA_HOME/bin/java — invokes the JRE binary located at the bin path of the folder designated by the $JAVA_HOME variable set in the current user's enviornment. That is, it runs Java, specifically the version pointed at by JAVA_HOME.
The trailing \s are escape charecters which escape the newline at the end of the line. Normally in a shell program a newline at the end of the line tells the shell that you're done with the command and it can interpret it now. Ending a line with \ tells the shell that the command will actually continue on the next line, i.e. this is all one command.
-Dpid=myJava — sets a system property for the jre named pid with value myJava. Java programs can basically ask, getProperty("pid") at runtime and it will return "myJava" and then choose its behavior appropriately, this is a way of configuring the programs the the JRE runs.
-Xms${HEAP_MIN}m — sets javas min heap size to value in ${HEAP_MIN} env var. The heap size is how much memory the jre sets aside to store its stack trace.
-Xmx${HEAP_MAX}m — sets Java's max heap size to value in ${HEAP_MAX} env var.
-cp ${CPG_CLASSPATH} — sets the Java classpath to the value in the ${CPG_CLASSPATH} env var.
-Dconfig=${CFG_FILE} — sets a system property for the JRE named config with value ${CFG_FILE}.
-Dcom.test.eps.configpath=${my_config}/ — sets a system property for the JRE named com.test.eps.configpath with value ${my_config}.
-Dcom.test.eps.rt.config=${my_config}/ — sets a system property for the JRE named com.test.eps.rt.config with value ${my_config}.
-Dlog4j.configuration=file:///${my_config}/log4j.properties.ewf.rt — sets a system property for the JRE named log4j.configuration with value file:///${my_config}/log4j.properties.ewf.rt.
com.test.MyJava is a Java class essentially located on the classpath at com/test/MyJava.class which presumably has a main function. After the JRE initializes with all of the previous configurations set, it will run this class and run its main function.
& tells the OS to run this command in its own process and not to wait for it to return before the cli gives control back to the user. It's basically telling the OS to run this program in a process separate from the one which is running your shell.
Assuming your on a unix based system, check this out:
http://www.manpagez.com/man/1/java/
That is complicated, so maybe you should learn the basics of shell scripting first:
http://www.freeos.com/guides/lsst/
As shell goes, there is nothing intrinsically complex there; there is, however, an awful lot of the same sort of stuff over and over.
At the highest level, it is a single command. The backslashes at the ends of the lines continue the command with the information on the next line. It invokes the JVM (Java Virtual Machine, which is the key component of the JRE, Java Runtime Environment) found in the directory $JAVA_HOME/bin, where $JAVA_HOME is (I trust you recognize), a shell variable; indeed, it should be an environment variable. The JVM is invoked with 10 arguments (unless I've miscounted again), and is run in the background (that's the & at the end). That means the shell script launches the JVM and does not wait for it to complete.
Some of the arguments also contain the expanded form of shell variables, some of which may be environment variables (you cannot tell definitively which variables are environment variables simply by looking at them, but conventionally, environment variables are all upper-case, like $JAVA_HOME). The alternative notation ${HEAP_MAX} expands a variable HEAP_MAX, just as $HEAP_MAX does. However, in that argument, you cannot (sensibly) write:
-Xmx$HEAP_MAXm # Wrong (in this context)
because that looks for a variable $HEAP_MAXm; you use the braces to delimit the name of the variable. There are also a lot of other things you can do with the name in braces, such as ${JAVA_HOME:?} which says that if $JAVA_HOME is not set, generate an error message and stop the script. See the manual for 'Parameter Expansion'.
For the meanings of the arguments to the JVM, you'll need to look at the man page. As an exercise in shell, though, the other arguments are all variations on what has been discussed already, using different variables to identify the locations of files needed by the JVM or the program it is running.
Related
Here's a simple test script (named "testps")
#!/bin/bash
echo PS4=$PS4
and I have set and exported PS4 like this:
export PS4='$LINENO:'
When I run it, either with ./testps or bash ./testps, the result is:
PS4=+
Looks like the value of "PS4" has been reset.
The only way I have found so far to customize PS4 is to run the script with bash -l after having added export PS4='$LINENO:' in .bashrc.
What have I missed here?
Note also that when using ksh, PS4 is initialized with its value from the environment if any.
This behavior doesn't happen with all users -- a copy of PS4 from the environment is only ignored when running as root, since shell version 4.4.
Quoting from CHANGES in the bash source:
g. Shells running as root no longer inherit PS4 from the environment, closing a security hole involving PS4 expansion performing command substitution.
This was done because environment variables are passed to setuid executables, and some setuid executables (unwisely) use system(), popen(), etc. to invoke a shell. While ld.so ignores LD_LIBRARY_PATH, LD_PRELOAD and similar when running in setuid, bash historically did not do so with environment variables that could cause arbitrary execution.
As described in https://www.openwall.com/lists/oss-security/2016/09/26/9, this issue could be exploited as follows:
env -i SHELLOPTS=xtrace PS4='$(id)' ./test
...would run id, instead of printing $(id) as part of xtrace logs, even if invocation of ./test crosses a privilege boundary.
Personally, I set PS4 inside my scripts -- it's ignored unless they're run with set -x, after all, so why not establish a meaningful value regardless?
If you want to force it to happen with fewer side effects than you get from bash -l or bash -i, set BASH_ENV to have the name of a file that can be sourced to perform your desired initialization.
When I source my .cshrc file and run the Tcl script it is working fine:
$ source .cshrc-sample
$ tclsh invoke.tcl
Following is the .cshrc file:
setenv AUTOTEST "/auto/isbutest/frt"
setenv ATS_EASY "$AUTOTEST"
setenv ATS_USER_PATH "$AUTOTEST"
setenv PATH "${AUTOTEST}/bin:${PATH}"
But when I tried setting the environment variable in Tcl itself and run the script,
I get the following error:
$ tclsh invoke.tcl
can't find package ha
while executing
"package require ha"
(file "invoke.tcl" line 8)
My Tcl script - invoke.tcl:
global env
set env(AUTOTEST) "/auto/isbutest/frt"
set env(ATS_EASY) "/auto/isbutest/frt"
set env(ATS_USER_PATH) "/auto/isbutest/frt"
set env(PATH) "$env(PATH):/auto/isbutest/frt/bin:";
package require ha
How can I run the script without sourcing the .cshrc?
The thing is setting environment variable is not possible using scripts, the lifetime of the variable is within the runtime of the script. When I tried printing the PATH variable it shows what is needed, but I don't know why it is not working. Is there any other workaround for this?
There's a few possibilities. The key things to look at are whether there are any other environment variables that you've missed out, whether the Tcl auto_path global variable is correct immediately before the package require, and whether there is anything else going on.
The easiest way from the Tcl side is to add:
puts "auto_path=$auto_path"
parray env
immediately before the package require that has the error. That should print out plenty of information. (Pay particular attention to if you are setting the TCL_LIBRARY or TCLLIBPATH environment variables differently.)
Aside from that, it's possible that there is something set in the ~/.tclshrc file, which is only sourced in interactive mode (it happens before you get your prompt). That could cause observable changes. Another option is if the ha package's pkgIndex.tcl script is written to use abbreviated commands, which only work when Tcl is in interactive mode. Errors in the package index definition script will make the code that describes how to actually load/source the package's implementation not register, and could give you the error state you see. If the script is assuming it can use abbreviations, fix it as that's always a bug. Abbreviations are a convenience when using Tcl interactively, and should never be put in proper saved code.
You might want to check whether the list of packages is complete. Use this code for that:
catch {package require NoSuchPackage}; # Force immediate population of the list of packages
puts Packages:\n\t[join [lsort -dictionary [package names]] \n\t]
Again, put this in after any setting of global variables and before the problem package require.
In side tcl script, you can simply do setenv as, setenv AUTOTEST="/auto/isbutest/frt".
if you want to set a variable, use set VARNAME "/auto/isbutest/frt".
if you want to get any environment variable, use $::env(AUTOTEST).
and any variable declared using set command can be accessed using $VARNAME.
Execution of
user#EWD-MacBook-Pro:~$ export property.name=property.value
Gives me
-bash: export: `property.name=property.value': not a valid identifier
Is it possible to have system properties with dot inside? If so how do that?
As others have said, bash doesn't allow it so you'll have to use your favourite scripting language to do it. For example, in Perl:
perl -e '$ENV{"property.name"} = "property.value"; system "bash"'
This will fire up a subshell bash with the property.name environment variable set, but you still can't access that environment variable from bash (although your program will be able to see it).
Edit: #MarkEdgar commented that the env command will work too:
env 'property.name=property.value' bash # start a subshell, or
env 'property.name=property.value' command arg1 arg2 ... # Run your command
As usual, you only require quotes if you need to protect special characters from the shell or want to include spaces in the property name or value.
I spent better part of this afternoon trying to figure out how to access some property set by Jenkins (to pass a job parameters jenkins uses property format with a dot) - this was a good hint from Adrian and yes it works for reading properties in the script too. I was at a loss as to what to do but then I tried:
var=`perl -e 'print $ENV{"property.name"};print "\n";'`
This worked quite well actually. But of course that works in a shell that was started with the property set in the environment already i.e. in Adrian's example this could work in a script started from bash instance invoked in perl example he provided. It would not if this perl sniplet was put in the same shell only directly after his perl example.
At least I learnt something this afternoon so not all this time is a waste.
If you export those properties to run an application, some programs can support setting system property as an option, and allow . in the property name.
In Java world, most of tools support setting system property by -D option, e.g. you can set system property with dot like this -Dproperty.name=property.value.
Bash only permits '_' and alpha numeric characters in variable names. The '.' isn't permitted.
http://tldp.org/LDP/abs/html/gotchas.html
I happen to run some commands blindly, in order to get things done.
I started to work with Jenkins recently, and then I had to use this export command to run the Jenkins WAR archive. What does the export command do in general, and why do we need to run this command, while running Jenkins (after the Jenkins home is set)?
export in sh and related shells (such as Bash), marks an environment variable to be exported to child-processes, so that the child inherits them.
export is defined in POSIX:
The shell shall give the export attribute to the variables corresponding to the specified names, which shall cause them to be in the environment of subsequently executed commands. If the name of a variable is followed by = word, then the value of that variable shall be set to word.
I guess you're coming from a Windows background. So I'll contrast them (I'm kind of new to Linux too). I found a user's reply to my comment, to be useful in figuring things out.
In Windows, a variable can be permanent or not. The term environment variable includes a variable set in the cmd shell with the SET command, as well as when the variable is set within the Windows GUI, thus set in the registry, and becoming viewable in new cmd windows.
E.g., the documentation for the set command in Windows "Displays, sets, or removes environment variables. Used without parameters, set displays the current environment settings."
In Linux, set does not display environment variables. It displays shell variables which it doesn't call/refer to as environment variables. Also, Linux doesn't use set to set variables (apart from positional parameters and shell options, which I explain as a note at the end), only to display them and even then only to display shell variables. Windows uses set for setting and displaying, e.g., set a=5, but Linux doesn't.
In Linux, I guess you could make a script that sets variables on bootup, e.g., /etc/profile or /etc/.bashrc, but otherwise, they're not permanent. They're stored in RAM.
There is a distinction in Linux between shell variables, and environment variables. In Linux, shell variables are only in the current shell, and environment variables, are in that shell and all child shells.
You can view shell variables with the set command (though note that, unlike Windows, variables are not set in Linux with the set command).
set -o posix; set (doing that set -o posix once first, helps not display too much unnecessary stuff). So set displays shell variables.
You can view environment variables with the env command.
Shell variables are set with, e.g., just a = 5.
Environment variables are set with export. Export also sets the shell variable.
Here you see shell variable zzz set with zzz = 5, and see it shows when running set, but it doesn't show as an environment variable.
Here we see yyy set with export, so it's an environment variable. And see it shows under both shell variables and environment variables:
$ zzz=5
$ set | grep zzz
zzz=5
$ env | grep zzz
$ export yyy=5
$ set | grep yyy
yyy=5
$ env | grep yyy
yyy=5
$
Other useful QnAs:
https://unix.stackexchange.com/questions/176001/how-can-i-list-all-shell-variables
https://askubuntu.com/questions/26318/environment-variable-vs-shell-variable-whats-the-difference
Note: One point which elaborates a bit and is somewhat corrective to what I've written, is that, in Linux bash, 'set' can be used to set "positional parameters" and "shell options/attributes", and technically both of those are variables, though the man pages might not describe them as such.
But still, as mentioned, set won't set shell variables or environment variables). If you do set asdf then it sets $1 to asdf, and if you do echo $1 you see asdf.
If you do set a=5 it won't set the variable a, equal to 5. It will set the positional parameter $1 equal to the string of "a=5". So if you ever saw set a=5 in Linux it's probably a mistake unless somebody actually wanted that string a=5, in $1.
The other thing that Linux's set can set, is shell options/attributes. If you do set -o you see a list of them. And you can do for example set -o verbose, off, to turn verbose on (by the way, the default happens to be off, but that makes no difference to this). Or you can do set +o verbose to turn verbose off. Windows has no such usage for its set command.
In simple terms, environment variables are set when you open a new shell session. At any time if you change any of the variable values, the shell has no way of picking that change. That means the changes you made become effective in new shell sessions.
The export command, on the other hand, provides the ability to update the current shell session about the change you made to the exported variable. You don't have to wait until new shell session to use the value of the variable you changed.
Is there are way to check that all variables declared in a bash script are being used? Something analogous to -Wall in c++?
Cheers
There's no built-in way to do this, no, and it can never be done perfectly, because of situations like these:
Suppose the user's Bash profile exports CLASSPATH as an environment variable. Then a Bash script might include a statement like CLASSPATH=... but never refer to CLASSPATH, if its only purpose in assigning to CLASSPATH is to modify the behavior of some program that uses the environment variable (such as java).
Suppose that I create variables a and b, and a variable c whose value is obtained from the user and can be either a or b. I can then use ${!c} to obtain the value of the user-specified variable; a given run of the script might never refer to b (because c is set to 'a'), but a different run of the script might do differently.
That said, you might be interested in the -u option to the set builtin. If your script contains this command:
set -u
then from that point on, it will be an error to refer to a variable or parameter that has not been set. This can help detect typos in variable-names and whatnot. This is obviously much less than what gcc -Wall does (since gcc always gives an error-message when you refer to an undeclared variable), but you may find it beneficial in the same way.