Make environment variables accessible to Gradle task - bash

I need to run a Gradle task inside a shell environment, which must be created before the task is launched. Using commandLine or executable is not appropriate, as I need to run the task in the same process as the shell script. Originally, I called the script directly inside gradlew, but later I decided to source it from build.gradle.kts and call subsequent tasks through gradlew:
val setupRosEnv by tasks.creating(Exec::class) {
executable = "bash"
args("-c", "source $rosPath/setup.sh && source gradlew myTask")
}
I can build everything by running ./gradlew setupRosEnv from the CLI. Besides sourcing the script then running gradlew, is there a way to achieve this using the Gradle API? The current solution seems a bit hacky, and is clunky for other tasks to depend on setupRosEnv, as this will lead to an infinite loop or must be explicitly handled to prevent tasks being run more than once.
As the shell script itself is generated by ROS, it cannot be translated to Gradle or easily parsed.

It depends how your gradle task myTask use the environment. If it use the environment by System.getenv, you can do use the following step.
parse the bash environment file env.sh and load all variables into Properties
append the environment variables to current process in java.lang.ProcessEnvironment by reflection
use the injected environment variables in your build task
Below is just roughly an example code copied from Java with minor modification, but it works fine in gradle build task.
task myEnvironInjected << {
println("task with injected enviroment")
}
task myBuildTask(dependsOn: myEnvironInjected) << {
def v1 = System.getenv("V1")
println("my build task running V1=${v1}")
}
myEnvironInjected.doFirst {
final Map<String, String> bashEnvMap = new HashMap<>();
try {
// a simple simulation of bash command source env.sh
// use it carefully
BufferedReader reader = new BufferedReader(new InputStreamReader(new FileInputStream
("/path/to/env.sh")));
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(outputStream));
String line;
while ((line = reader.readLine()) != null) {
if (line.length() > 0 && line.startsWith("export ")) {
String newLine = line.trim().replaceFirst("^export ", "");
// remove single or double quote from the value if it has
int quoteIndex = newLine.indexOf('=')+1;
if (quoteIndex < newLine.length() && (newLine.charAt(quoteIndex) == ('"' as char) ||
newLine.charAt(quoteIndex) == ('\'' as char))) {
newLine = newLine.substring(0, quoteIndex) + newLine.substring(quoteIndex+1, newLine.length()-1);
}
writer.write(newLine);
writer.newLine();
}
}
writer.flush();
writer.close();
InputStream inputStream = new ByteArrayInputStream(outputStream.toByteArray());
Properties properties = new Properties();
properties.load(inputStream);
for (Map.Entry<Object, Object> entry : properties.entrySet()) {
bashEnvMap.put(((String) entry.getKey()), ((String) entry.getValue()));
}
outputStream.close();
inputStream.close();
} catch (IOException e) {
e.printStackTrace();
}
for (Class<?> aClass : Collections.class.getDeclaredClasses()) {
if ("java.util.Collections\$UnmodifiableMap".equals(aClass.getName())) {
try {
Field mapField = aClass.getDeclaredField("m");
mapField.setAccessible(true);
Object mapObject = mapField.get(System.getenv());
Map<String, String> environMap = ((Map<String, String>) mapObject);
environMap.putAll(bashEnvMap);
} catch (NoSuchFieldException e) {
e.printStackTrace();
} catch (IllegalAccessException e) {
e.printStackTrace();
}
}
}
for (Map.Entry<String, String> envEntry : System.getenv().entrySet()) {
System.out.println(envEntry.getKey() + "=" + envEntry.getValue());
}
}
My test file env.sh looks like this.
export V1="v1 vvv"
export V2='v 2222'
export V3=v33333
If your build task use the environment variables not by System.getenv, your hacking method might be the best solution.
task myBuildTaskWithSourceEnv(type: Exec) {
commandLine '/bin/bash'
setArgs(['-c', 'source ../env.sh;set;../gradlew :app:assembleDebug'])
}

Related

Implement `Process.waitFor(long timeout, TimeUnit unit)` in Java 6

I am working on a legacy (Java 6/7) project that uses ProcessBuilder to request a UUID from the machine in an OS-agnostic way. I would like to use the Process.waitFor(long timeout, TimeUnit unit) method from Java 8, but this isn't implemented in Java 6. Instead, I can use waitFor(), which blocks until completion or an error.
I would like to avoid upgrading the version of Java used to 8 if possible as this necessitates a lot of other changes (migrating code away from removed internal APIs and upgrading a production Tomcat server, for example).
How can I best implement the code for executing the process, with a timeout? I was thinking of somehow implementing a schedule that checks if the process is still running and cancelling/destroying it if it is and the timeout has been reached.
My current (Java 8) code looks like this:
/** USE WMIC on Windows */
private static String getSystemProductUUID() {
String uuid = null;
String line;
List<String> cmd = new ArrayList<String>() {{
add("WMIC.exe"); add("csproduct"); add("get"); add("UUID");
}};
BufferedReader br = null;
Process p = null;
SimpleLogger.debug("Attempting to retrieve Windows System UUID through WMIC ...");
try {
ProcessBuilder pb = new ProcessBuilder().directory(getExecDir());
p = pb.command(cmd).start();
if (!p.waitFor(TIMEOUT, SECONDS)) { // No timeout in Java 6
throw new IOException("Timeout reached while waiting for UUID from WMIC!");
}
br = new BufferedReader(new InputStreamReader(p.getInputStream()));
while ((line = br.readLine()) != null) {
if (null != line) {
line = line.replace("\t", "").replace(" ", "");
if (!line.isEmpty() && !line.equalsIgnoreCase("UUID")) {
uuid = line.replace("-", "");
}
}
}
} catch (IOException | InterruptedException ex) {
uuid = null;
SimpleLogger.error(
"Failed to retrieve machine UUID from WMIC!" + SimpleLogger.getPrependedStackTrace(ex)
);
// ex.printStackTrace(System.err);
} finally {
if (null != br) {
try {
br.close();
} catch (IOException ex) {
SimpleLogger.warn(
"Failed to close buffered reader while retrieving machine UUID!"
);
}
if (null != p) {
p.destroy();
}
}
}
return uuid;
}
You can use the following code which only uses features available under Java 6:
public static boolean waitFor(Process p, long t, TimeUnit u) {
ScheduledExecutorService ses = Executors.newSingleThreadScheduledExecutor();
final AtomicReference<Thread> me = new AtomicReference<Thread>(Thread.currentThread());
ScheduledFuture<?> f = ses.schedule(new Runnable() {
#Override public void run() {
Thread t = me.getAndSet(null);
if(t != null) {
t.interrupt();
me.set(t);
}
}
}, t, u);
try {
p.waitFor();
return true;
}
catch(InterruptedException ex) {
return false;
}
finally {
f.cancel(true);
ses.shutdown();
// ensure that the caller doesn't get a spurious interrupt in case of bad timing
while(!me.compareAndSet(Thread.currentThread(), null)) Thread.yield();
Thread.interrupted();
}
}
Note that unlike other solutions you can find somewhere, this will perform the Process.waitFor() call within the caller’s thread, which is what you would expect when looking at the application with a monitoring tool. It also helps the performance for short running sub-processes, as the caller thread will not do much more than the Process.waitFor(), i.e. does not need to wait for the completion of background threads. Instead, what will happen in the background thead, is the interruption of the initiating thread if the timeout elapsed.

How can i fail jmeter test if there is at least one false condision in FOR loop in beanshell script?

I have test with small beanshell-script when i get from csv-file some names (like cars,telephones,blabla) and i have to check this names in html from previous test-step. Result of every check i have to write in other file. My issue here is mark this step to red color if at least one name wasn't find in html. Code here:
String Response = prev.getResponseDataAsString();
try {
File file = new File(vars.get("pathtocsv"));
FileReader fr = new FileReader(file);
BufferedReader reader = new BufferedReader(fr);
String line = reader.readLine();
if (line != null) {
String[] parts = line.split(",");
try{
FileWriter fw = new FileWriter(vars.get("pathtoresults"), true);
BufferedWriter bw = new BufferedWriter(fw);
PrintWriter pw = new PrintWriter(bw);
for(String i : parts) {
String utf8String= new String(i.getBytes("windows-1251"), "UTF-8");
if(Response.contains(utf8String)){
pw.println("Response contain element: " + i);
}
else{
pw.println("!!! Response doesn't contain element: " + i);
Failure=true;
FailureMessage = "!!! Response doesn't contain element: " + i;
log.warn( "!!! Response doesn't contain element " + utf8String);
prev.setResponseCode("400");
}
}
pw.close();
}
catch (IOException e){
e.printStackTrace();
}
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
prev is a shorthand to SampleResult class instance so you can conditionally mark sampler as failed from the PostProcessor using prev.setSuccessful(false) method.
Be aware that Beanshell is not the recommended scripting option, starting from JMeter version 3.1 users are strongly encouraged to switch to JSR223 Test Elements and Groovy language as Groovy is more Java compliant, has a lot of JDK enhancements and performs much better. See Apache Groovy - Why and How You Should Use It for more details.
You can fail parent sampler by marking success as false:
prev.setSuccessful(false);

How to call .sh script from a JavaFX button`s action handler? [duplicate]

It is quite simple to run a Unix command from Java.
Runtime.getRuntime().exec(myCommand);
But is it possible to run a Unix shell script from Java code? If yes, would it be a good practice to run a shell script from within Java code?
You should really look at Process Builder. It is really built for this kind of thing.
ProcessBuilder pb = new ProcessBuilder("myshellScript.sh", "myArg1", "myArg2");
Map<String, String> env = pb.environment();
env.put("VAR1", "myValue");
env.remove("OTHERVAR");
env.put("VAR2", env.get("VAR1") + "suffix");
pb.directory(new File("myDir"));
Process p = pb.start();
You can use Apache Commons exec library also.
Example :
package testShellScript;
import java.io.IOException;
import org.apache.commons.exec.CommandLine;
import org.apache.commons.exec.DefaultExecutor;
import org.apache.commons.exec.ExecuteException;
public class TestScript {
int iExitValue;
String sCommandString;
public void runScript(String command){
sCommandString = command;
CommandLine oCmdLine = CommandLine.parse(sCommandString);
DefaultExecutor oDefaultExecutor = new DefaultExecutor();
oDefaultExecutor.setExitValue(0);
try {
iExitValue = oDefaultExecutor.execute(oCmdLine);
} catch (ExecuteException e) {
System.err.println("Execution failed.");
e.printStackTrace();
} catch (IOException e) {
System.err.println("permission denied.");
e.printStackTrace();
}
}
public static void main(String args[]){
TestScript testScript = new TestScript();
testScript.runScript("sh /root/Desktop/testScript.sh");
}
}
For further reference, An example is given on Apache Doc also.
I think you have answered your own question with
Runtime.getRuntime().exec(myShellScript);
As to whether it is good practice... what are you trying to do with a shell script that you cannot do with Java?
I would say that it is not in the spirit of Java to run a shell script from Java. Java is meant to be cross platform, and running a shell script would limit its use to just UNIX.
With that said, it's definitely possible to run a shell script from within Java. You'd use exactly the same syntax you listed (I haven't tried it myself, but try executing the shell script directly, and if that doesn't work, execute the shell itself, passing the script in as a command line parameter).
Yes it is possible to do so. This worked out for me.
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import org.omg.CORBA.portable.InputStream;
public static void readBashScript() {
try {
Process proc = Runtime.getRuntime().exec("/home/destino/workspace/JavaProject/listing.sh /"); //Whatever you want to execute
BufferedReader read = new BufferedReader(new InputStreamReader(
proc.getInputStream()));
try {
proc.waitFor();
} catch (InterruptedException e) {
System.out.println(e.getMessage());
}
while (read.ready()) {
System.out.println(read.readLine());
}
} catch (IOException e) {
System.out.println(e.getMessage());
}
}
Here is my example. Hope it make sense.
public static void excuteCommand(String filePath) throws IOException{
File file = new File(filePath);
if(!file.isFile()){
throw new IllegalArgumentException("The file " + filePath + " does not exist");
}
if(isLinux()){
Runtime.getRuntime().exec(new String[] {"/bin/sh", "-c", filePath}, null);
}else if(isWindows()){
Runtime.getRuntime().exec("cmd /c start " + filePath);
}
}
public static boolean isLinux(){
String os = System.getProperty("os.name");
return os.toLowerCase().indexOf("linux") >= 0;
}
public static boolean isWindows(){
String os = System.getProperty("os.name");
return os.toLowerCase().indexOf("windows") >= 0;
}
Yes, it is possible and you have answered it! About good practises, I think it is better to launch commands from files and not directly from your code. So you have to make Java execute the list of commands (or one command) in an existing .bat, .sh , .ksh ... files.
Here is an example of executing a list of commands in a file MyFile.sh:
String[] cmd = { "sh", "MyFile.sh", "\pathOfTheFile"};
Runtime.getRuntime().exec(cmd);
To avoid having to hardcode an absolute path, you can use the following method that will find and execute your script if it is in your root directory.
public static void runScript() throws IOException, InterruptedException {
ProcessBuilder processBuilder = new ProcessBuilder("./nameOfScript.sh");
//Sets the source and destination for subprocess standard I/O to be the same as those of the current Java process.
processBuilder.inheritIO();
Process process = processBuilder.start();
int exitValue = process.waitFor();
if (exitValue != 0) {
// check for errors
new BufferedInputStream(process.getErrorStream());
throw new RuntimeException("execution of script failed!");
}
}
As for me all things must be simple.
For running script just need to execute
new ProcessBuilder("pathToYourShellScript").start();
The ZT Process Executor library is an alternative to Apache Commons Exec. It has functionality to run commands, capturing their output, setting timeouts, etc.
I have not used it yet, but it looks reasonably well-documented.
An example from the documentation: Executing a command, pumping the stderr to a logger, returning the output as UTF8 string.
String output = new ProcessExecutor().command("java", "-version")
.redirectError(Slf4jStream.of(getClass()).asInfo())
.readOutput(true).execute()
.outputUTF8();
Its documentation lists the following advantages over Commons Exec:
Improved handling of streams
Reading/writing to streams
Redirecting stderr to stdout
Improved handling of timeouts
Improved checking of exit codes
Improved API
One liners for quite complex use cases
One liners to get process output into a String
Access to the Process object available
Support for async processes ( Future )
Improved logging with SLF4J API
Support for multiple processes
This is a late answer. However, I thought of putting the struggle I had to bear to get a shell script to be executed from a Spring-Boot application for future developers.
I was working in Spring-Boot and I was not able to find the file to be executed from my Java application and it was throwing FileNotFoundFoundException. I had to keep the file in the resources directory and had to set the file to be scanned in pom.xml while the application was being started like the following.
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
<includes>
<include>**/*.xml</include>
<include>**/*.properties</include>
<include>**/*.sh</include>
</includes>
</resource>
</resources>
After that I was having trouble executing the file and it was returning error code = 13, Permission Denied. Then I had to make the file executable by running this command - chmod u+x myShellScript.sh
Finally, I could execute the file using the following code snippet.
public void runScript() {
ProcessBuilder pb = new ProcessBuilder("src/main/resources/myFile.sh");
try {
Process p;
p = pb.start();
} catch (IOException e) {
e.printStackTrace();
}
}
Hope that solves someone's problem.
Here is an example how to run an Unix bash or Windows bat/cmd script from Java. Arguments can be passed on the script and output received from the script. The method accepts arbitrary number of arguments.
public static void runScript(String path, String... args) {
try {
String[] cmd = new String[args.length + 1];
cmd[0] = path;
int count = 0;
for (String s : args) {
cmd[++count] = args[count - 1];
}
Process process = Runtime.getRuntime().exec(cmd);
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
try {
process.waitFor();
} catch (Exception ex) {
System.out.println(ex.getMessage());
}
while (bufferedReader.ready()) {
System.out.println("Received from script: " + bufferedReader.readLine());
}
} catch (Exception ex) {
System.out.println(ex.getMessage());
System.exit(1);
}
}
When running on Unix/Linux, the path must be Unix-like (with '/' as separator), when running on Windows - use '\'. Hier is an example of a bash script (test.sh) that receives arbitrary number of arguments and doubles every argument:
#!/bin/bash
counter=0
while [ $# -gt 0 ]
do
echo argument $((counter +=1)): $1
echo doubling argument $((counter)): $(($1+$1))
shift
done
When calling
runScript("path_to_script/test.sh", "1", "2")
on Unix/Linux, the output is:
Received from script: argument 1: 1
Received from script: doubling argument 1: 2
Received from script: argument 2: 2
Received from script: doubling argument 2: 4
Hier is a simple cmd Windows script test.cmd that counts number of input arguments:
#echo off
set a=0
for %%x in (%*) do Set /A a+=1
echo %a% arguments received
When calling the script on Windows
runScript("path_to_script\\test.cmd", "1", "2", "3")
The output is
Received from script: 3 arguments received
It is possible, just exec it as any other program. Just make sure your script has the proper #! (she-bang) line as the first line of the script, and make sure there are execute permissions on the file.
For example, if it is a bash script put #!/bin/bash at the top of the script, also chmod +x .
Also as for if it's good practice, no it's not, especially for Java, but if it saves you a lot of time porting a large script over, and you're not getting paid extra to do it ;) save your time, exec the script, and put the porting to Java on your long-term todo list.
I think with
System.getProperty("os.name");
Checking the operating system on can manage the shell/bash scrips if such are supported.
if there is need to make the code portable.
String scriptName = PATH+"/myScript.sh";
String commands[] = new String[]{scriptName,"myArg1", "myArg2"};
Runtime rt = Runtime.getRuntime();
Process process = null;
try{
process = rt.exec(commands);
process.waitFor();
}catch(Exception e){
e.printStackTrace();
}
Just the same thing that Solaris 5.10 it works like this ./batchstart.sh there is a trick I don´t know if your OS accept it use \\. batchstart.sh instead. This double slash may help.
for linux use
public static void runShell(String directory, String command, String[] args, Map<String, String> environment)
{
try
{
if(directory.trim().equals(""))
directory = "/";
String[] cmd = new String[args.length + 1];
cmd[0] = command;
int count = 1;
for(String s : args)
{
cmd[count] = s;
count++;
}
ProcessBuilder pb = new ProcessBuilder(cmd);
Map<String, String> env = pb.environment();
for(String s : environment.keySet())
env.put(s, environment.get(s));
pb.directory(new File(directory));
Process process = pb.start();
BufferedReader inputReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
BufferedWriter outputReader = new BufferedWriter(new OutputStreamWriter(process.getOutputStream()));
BufferedReader errReader = new BufferedReader(new InputStreamReader(process.getErrorStream()));
int exitValue = process.waitFor();
if(exitValue != 0) // has errors
{
while(errReader.ready())
{
LogClass.log("ErrShell: " + errReader.readLine(), LogClass.LogMode.LogAll);
}
}
else
{
while(inputReader.ready())
{
LogClass.log("Shell Result : " + inputReader.readLine(), LogClass.LogMode.LogAll);
}
}
}
catch(Exception e)
{
LogClass.log("Err: RunShell, " + e.toString(), LogClass.LogMode.LogAll);
}
}
public static void runShell(String path, String command, String[] args)
{
try
{
String[] cmd = new String[args.length + 1];
if(!path.trim().isEmpty())
cmd[0] = path + "/" + command;
else
cmd[0] = command;
int count = 1;
for(String s : args)
{
cmd[count] = s;
count++;
}
Process process = Runtime.getRuntime().exec(cmd);
BufferedReader inputReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
BufferedWriter outputReader = new BufferedWriter(new OutputStreamWriter(process.getOutputStream()));
BufferedReader errReader = new BufferedReader(new InputStreamReader(process.getErrorStream()));
int exitValue = process.waitFor();
if(exitValue != 0) // has errors
{
while(errReader.ready())
{
LogClass.log("ErrShell: " + errReader.readLine(), LogClass.LogMode.LogAll);
}
}
else
{
while(inputReader.ready())
{
LogClass.log("Shell Result: " + inputReader.readLine(), LogClass.LogMode.LogAll);
}
}
}
catch(Exception e)
{
LogClass.log("Err: RunShell, " + e.toString(), LogClass.LogMode.LogAll);
}
}
and for usage;
ShellAssistance.runShell("", "pg_dump", new String[]{"-U", "aliAdmin", "-f", "/home/Backup.sql", "StoresAssistanceDB"});
OR
ShellAssistance.runShell("", "pg_dump", new String[]{"-U", "aliAdmin", "-f", "/home/Backup.sql", "StoresAssistanceDB"}, new Hashmap<>());

Permission Issues When Running Shell Script From Android App

I am using the following code to run a shell script through an android application. I am able to run the script but only echo command is getting executed. Commands like netcfg, cat, route, setprop are not getting executed. Are there any permissions which I am missing out or any other thing. Please let me know.
public void run() {
// TODO Auto-generated method stub
String path;
path="/system/bin/sh /persist/eth0.sh";
Process p;
try {
p = Runtime.getRuntime().exec(path);
InputStream is = p.getInputStream();
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String line;
while ((line = br.readLine()) != null) {
System.out.println(line);
}
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
}
Android has its own linux kerneal and does not ship with all common commands found in desktop distros.

EC2 Java SDK - User data script

I'm looking for a way to attach a user data script to an EC2 RunRequest in the Java SDK (the equivalent of ec2-run-instances ami-1234567 -f startup-script.zip for the command line tool).
Several things I've read indicate that anything user data string with "#! " will execute, but this doesn't seem to be the case.
Is this even possible?
FYI: here's my test class:
public class AWSTest {
public static void main(String[] args) {
AWSCredentials credentials = new BasicAWSCredentials("access-key","secret-access-key");
AmazonEC2Client ec2 = new AmazonEC2Client(credentials);
RunInstancesRequest request = new RunInstancesRequest();
request.setInstanceType(InstanceType.M1Small.toString());
request.setMinCount(1);
request.setMaxCount(1);
request.setImageId("ami-84db39ed");
request.setKeyName("linux-keypair");
request.setUserData(getUserDataScript());
ec2.runInstances(request);
}
private static String getUserDataScript(){
ArrayList<String> lines = new ArrayList<String>();
lines.add("#! /bin/bash");
lines.add("curl http://www.google.com > google.html");
lines.add("shutdown -h 0");
String str = new String(Base64.encodeBase64(join(lines, "\n").getBytes()));
return str;
}
static String join(Collection<String> s, String delimiter) {
StringBuilder builder = new StringBuilder();
Iterator<String> iter = s.iterator();
while (iter.hasNext()) {
builder.append(iter.next());
if (!iter.hasNext()) {
break;
}
builder.append(delimiter);
}
return builder.toString();
}
}
Unfortunately, after I run this, I'm able to SSH into the box, and confirm that
It hasn't shut down and
It didn't download the file
Any assistance is greatly appreciated.
Best,
Zach
This works to insert user data in an instance run request, in this case specifically to join an ECS cluster:
private static String getECSuserData(String clusterName) {
String userData = "";
userData = userData + "#!/bin/bash" + "\n";
userData = userData + "echo ECS_CLUSTER=" + clusterName + " ";
userData = userData + ">> /etc/ecs/ecs.config";
String base64UserData = null;
try {
base64UserData = new String( Base64.encodeBase64( userData.getBytes( "UTF-8" )), "UTF-8" );
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
return base64UserData;
}
It could be possible that the AMI your using does not support user-data script?
Please use the AMI's found at www.alestic.com.
A good reference also http://alestic.com/2009/06/ec2-user-data-scripts

Resources