From inside jShell script, is it possible to access or register variables that are defined in code that's also creating JShell?
Currently there seems to be no mechanism to either access or register a variable to Shell instance, or return none string types from inside JShell (like objects or lambda etc.)
ex:
import jdk.jshell.JShell;
import jdk.jshell.JShellException;
import jdk.jshell.SnippetEvent;
import java.util.List;
public class Main {
public static void main(String[] args) throws JShellException {
var localVar = 1;
JShell shell = JShell.create();
// How to register localVar variable with shell instance or access variables from scope
List events = shell.eval("var x = localVar;");
SnippetEvent event = events.get(0);
System.out.println("Kind: " + event.snippet().kind() + ", Value: " + event.value());
}
}
While you can't access local names like in your example, you can create a JShell instance that executes in the same JVM that created it. For this you would use the LocalExecutionControl. Using this execution control you could move localVar to a static field in your Main class and then access it from "inside" the JShell code with Main.localVar.
Unfortunately as the API is designed to support execution providers that could be in a different process or even a different machine, the return type is a string. If you are interested in a hack, the IJava jupyter kernel needed to an implementation of eval that returned an Object which ended up using an ExecutionControl implementation based on the DirectExecutionControl that stored the result of an eval call in a map and returned a unique id to reference that result. Then using the shell you would have to lookup the result from the id returned by eval (think of something like results.get(eval(sourceCode))). That implementation is on github in IJavaExecutionControl.java and IJavaExecutionControlProvider.java with a sample usage in CodeEvaluator.java#L72 if you are interested in taking any of it (MIT license).
Related
I just need to know if a global public class variable, used in a SparkStreaming process will be considered as a broadcasted variable.
For now, I succeeded to use a pre-setted variable "inventory" into a JavaDStream transformation.
class Foo {
public static Map<String,String> inventory;
public static void main(String args) {
inventory = Inventory.load(); // here i set the variable
SparkSession sparkSession = ...
JavaStreamingContext ssc = ... // here i initialize the Spark Streaming Context
JavaInputDStream<ConsumerRecord<String, String>> records = ...
JavaDStream<Map<String,Object>> processedRecords = records.flatMap(rawRecord->{
return f(rawRecord,inventory); // just an example...
}
}
}
What I understand is that the part into the lambda expression (rawRecord) is a distributed one, and then I presume that "inventory" is broadcasted to each executor that performs the process, is that so?
A global class variable is different from a broadcast variable .
Using a class variable is fine but this can be inefficient, especially
for large variables such as a lookup table or a machine learning
model. The reason for this is that when you use a variable in a
closure or a class variable in your case , it must be deserialised on
the worker nodes many times (one per task). Moreover, if you use the
same variable in multiple Spark actions and jobs, it will be re-sent
to the workers with every job instead of once.
Broadcast variables are shared, immutable variables that
are cached on every machine in the cluster instead of serialized with every single task
All you need to do is
Broadcast<Map<String,String>> broadcast = ssc.sparkContext().broadcast(inventory);
and access it
broadcast.value().get(key)
Yes, you have to broadcast that variable to keep available for all the executors in the distributed environment.
While I was able to define methods, defining variables using Jenkins Shared library seems to an unresolved quest so far.
I added /vars/True.groovy with this body
def call() {
return true
}
And now inside Jenkinsfile, I tried to test if it works as expected:
println "evaluation ${ true == True }"
But surprise, this fails as it considers true != class True. My impression is that true == True() may work but that's not the point, I need a real variable because that whole purpose was to avoid some errors caused by people using wrong case.
So what is the magic trick?
I've found a way to achieve this, but with a slight caveat: the variables must be defined/wrapped within a class. However, this does have the upside of providing better organization, in order not to pollute the global space too much.
For example, we often reuse four standard "magic strings" for build statuses, which I wanted to save as global constants to facilitate interoperability. So I created a global status class, defined in vars/status.groovy:
class status {
final String STARTED = "STARTED"
final String SUCCESS = "SUCCESS"
final String FAILURE = "FAILURE"
final String ABORTED = "ABORTED"
}
The constants can then be used by referring to their parent class:
echo status.STARTED
echo status.SUCCESS
echo status.FAILURE
echo status.ABORTED
Specials thanks to #mkobit for pointing me in the right direction!
It looks like Global Variables defined in the vars directory must be lower/camel/maybe some other special casing. This isn't stated anywhere in on the Defining global variables section, but there is this note at the top:
The vars directory hosts scripts that define global variables accessible from Pipeline. The basename of each *.groovy file should be a Groovy (~ Java) identifier, conventionally camelCased. The matching *.txt, if present, can contain documentation, processed through the system’s configured markup formatter (so may really be HTML, Markdown, etc., though the txt extension is required).
Here is what I tried:
vars/MyTrue.groovy
class MyTrue implements Serializable {
}
vars/myTrue.groovy
class myTrue implements Serializable {
}
vars/mytrue.groovy
class mytrue implements Serializable {
}
vars/doesCasingMatter.groovy
class DoesCasingMatter implements Serializable {
}
And in my pipeline script to test if they are instances or Class types (no script security enabled here):
echo("MyTrue: ${Class.isInstance(MyTrue)}")
echo("myTrue: ${Class.isInstance(myTrue)}")
echo("mytrue: ${Class.isInstance(mytrue)}")
echo("What bindings are there?: ${binding.variables}")
This prints out:
[Pipeline] echo
MyTrue: true
[Pipeline] echo
myTrue: false
[Pipeline] echo
mytrue: false
[Pipeline] echo
What bindings are there?: [steps:org.jenkinsci.plugins.workflow.cps.DSL#e96256, myTrue:myTrue#8a1ddc5, mytrue:mytrue#392ff649]
Which seems to indicate that something about the class name determines how it gets compiled and created. The first example, which is similar to your vars/True.groovy, is only imported as a class and not instantiated. The other two are compiled and instantiated and are bound to the script with their defined class names. I think you will have to define your classes differently if you want them to be global variables.
Variable True is definitely a variable but it holds a reference to object of type True(that you defined in /vars/True.groovy). You have two options
The good one:
Use it this way
println "evaluation ${ true == True() }"
The strange one:
You can override equals() method in /vars/True.groovy
public boolean equals(obj) {
return obj == true;
}
Then this should work
println "evaluation ${ true == True }"
But it would be really strange and can lead to misunderstandings.
I'm still figuring out how to use p5.js. In regular java you have to declare each variable using its data type, ex. int foo = 0.
In p5, I know you can just use var foo but you can also declare variables using this.foo. If someone could clarify when is the proper time to use var and when i can use this, that would be very helpful.
For example, if I want to declare a variable inside a method, should i use var foo = thing or could I declare it using this.foo = thing? What should I use when declaring global variables or when referring to objects passed into methods?
Thanks!
First of all, p5 is not a language, it is a Javascript library, you are coding in Javascript, not p5.
Coming to your question, if you want to use some function as a data type, similar to a class in java, and want all the "instances" of that to have their own different variables, you use this. If they are just variables you use in someway but don't need to be specific for each instance, or if the function is not a constructor function and is not to be used as a data type, you will just use var then.
Again, there is no class stuff in javascript, you will have to write what is called a constructor function in order to "simulate" a java class, but be aware that a constructor function should not return anything. Here is an example of car class in java:
class car {
int speed = ___;
String model = ___;
static int numOfWheels = ___;
}
This is what it will look like in javascript (a constructor function):
function car() {
this.speed = ____;
this.model = ____;
var numOfWheels = ___;
}
If you declare a variable without this, it can be roughly compared to a static variable in a java class in the sense that it will be constant among all the instances.
So basically, at least in most cases, you will use this.varName usually inside constructor functions, i.e., functions that you will use to construct objects.
What should I use when declaring global variables or when referring to objects passed into methods?
Global variables will almost always be var something = something. When referring to objects passed into functions, just use the dot notation to refer to its properties like passedObject.someProperty
I would recommend you to learn Javascript before jumping into p5 directly, here are some resources that I found useful when I started learning Javascript-
w3 School
JavaScript Info Website
TheNewBoston
To invoke Java from JS you can use Java.type. Is there a way to bind a java class in the Bindings?
So replace:
scriptEngine.eval("Java.type('my.own.AwesomeObj')");
with something like:
Bindings bindings = new SimpleBindings();
bindings.put("AwesomeObj", my.own.AwesomeObj.class);
scriptEngine.setBindings(bingings, ScriptContext.GLOBAL_SCOPE);
I am working on a framework where I want to make a lot of classes available for the js scripts, and preferably not use a string concatenation and an eval. Currently it throws an exception with message: AwesomeObj is not a function, what makes sense.
Nashorn distinguishes a type from a java.lang.Class instance, just like Java does (in Java language, my.own.AwesomeObj is a type, while my.own.AwesomeObj.class is an instance of java.lang.Class. You can use a type to access static members, or as a constructor. You can't use a Class object for that purpose. The bad news is, you can't directly obtain the object that Nashorn uses for representing types; it's an instance of jdk.internal.dynalink.beans.StaticClass and it lives in a restricted package. However, you can evaluate it in script with
engine.eval("Java.type('my.own.AwesomeObj')");
and put the result of that in the bindings. Incidentally, within Nashorn, if you put the Class object into bindings under name AwesomeObjClass, you can use the synthetic property .static to obtain the type, e.g.:
var AwesomeObj = AwesomeObjClass.static;
In this sense, .static on a Class object is the dual of .class on a type object (.static obviously doesn't exist in Java, where types aren't reified as runtime objects).
var stringType = Java.type("java.lang.String");
var stringClass = stringType.class
print(stringClass instanceof java.lang.Class); // true
print(stringType === stringClass.static); // true
Hope this helps.
since Java 9 you can use jdk.dynalink.beans.StaticClass.forClass as it is not internal anymore:
Bindings bindings = engine.createBindings();
bindings.put("AwesomeObj", StaticClass.forClass(my.own.AwesomeObj.class));
then you can utilize the binding in the JS code with :
var awesome = new AwesomeObj();
In your top-level script of your framework do:
var AwesomeObj = Java.type("my.own.AwesomeObj");
I am unable to send object to direct view workers.
Here is what I want to do:
class Test:
def __init__(self):
self.id = 'ddfdf'
from IPython.parallel import Client
rc = Client()
dv = rc[:]
t = Test()
dv['t'] = t
print dv['t']
NameError: name 't' is not defined
This would work if I try to push pandas object or any of the build in objects.
What is the way to do it with custom object?
I tried:
dv['Test'] = Test
dv['t'] = t
print dv['t']
UnpicklingError: NEWOBJ class argument isn't a type object
For interactively defined classes (in __main__), you do need to push the class definition, or use dill. But even this doesn't appear to work for old-style classes, which is a bug in IPython's handling of old-style classes[1]. This code works fine if you use a new-style class:
class Test(object):
...
instead of on old-style class. Note that old-style classes are not available in Python 3.
It's generally a good idea to always use new-style classes anyway.