If I invoke my R script's compute_score function in R studio, the geography score returned is 68.18, which is what I expect. However when I invoke the same function in Renjin it is returning 60.0, which is incorrect. I have used the exact same value for the function argument in both cases. Any idea what could be causing this? Is there any chance this could be a Renjin bug?
The end of my compute_score function looks like this...
compute_score <- function(CIF)
{
...
response = list(
geographyScore = as.numeric(geographyScore),
industryScore = as.numeric(industryScore),
productScore = as.numeric(productScore),
channelScore = as.numeric(channelScore),
clientTypeScore = as.numeric(clientTypeScore),
transactionScore = as.numeric(transactionScore),
tags = tags
)
return(response)
}
And I invoke it in my Java class using the following lines of code...
ScriptEngine engine = new RenjinScriptEngineFactory().getScriptEngine();
File file = new File(getClassLoader().getResource("sample_interface.R").getFile());
engine.eval(new java.io.FileReader(file));
ListVector resList = (ListVector) engine.eval("compute_score(" + buildRRequest(buildRequest(ACCOUNT_HOLDER, NP_RETAIL, ISO_SWISS, ISO_WESTERN_SAHARA, false)) + ")");
And this is what resList evaluates to when I check in Intellij debugger:
list(geographyScore = 60.0, industryScore = 75.0, productScore = 100.0, channelScore = 50.0, clientTypeScore = 25.0, transactionScore = 100.0, tags = list())
All the other fields return correctly other than geography, that should definitely be 68.18182, would someone have any idea why it isn't?
Just to remind you, passing in the exact same value when invoking the function in RStudio gives me 68.18182, invoking the same function with the same argument value using renjin's eval operation gives me a score of 60.0, at first glance it looks like it is rounding but Im not so sure, any help would be appreciated greatly.
Related
I am using the function corrplot from corrplot package to generate a plot of a correlation matrix created with cor.test (psych package).
When I try to save the result into a variable, the variable is NULL.
Anyone could advice, please?
library(corrplot)
library(psych)
library(ggpubr)
data(iris)
res_pearson.c_setosa<-iris%>%
filter(Species=="setosa")%>%
select(Sepal.Length:Petal.Width)%>%
corr.test(., y = NULL, use = "complete",method="pearson",adjust="bonferroni", alpha=.05,ci=TRUE,minlength=5)
corr.a<-corrplot(res_pearson.c_setosa$r[,1:3],
type="lower",
order="original",
p.mat = res_pearson.c_setosa$p[,1:3],
sig.level = 0.05,
insig = "blank",
col=col4(10),
tl.pos = "ld",
tl.cex = .8,
tl.srt=45,
tl.col = "black",
cl.cex = .8)+
my.theme #this is a theme() piece, but if I take this away, the result is a list rather than a plot
You can create your own function where you put recordPlot at the end to save the plot. After that you can save the output of the function in a variable. Here is a reproducible example:
library(corrplot)
library(psych)
library(ggpubr)
library(dplyr)
data(iris)
res_pearson.c_setosa<-iris%>%
filter(Species=="setosa")%>%
select(Sepal.Length:Petal.Width)%>%
corr.test(., y = NULL, use = "complete",method="pearson",adjust="bonferroni", alpha=.05,ci=TRUE,minlength=5)
your_function <- function(ff){
corr.a<-corrplot(ff$r[,1:3],
type="lower",
order="original",
p.mat = ff$p[,1:3],
sig.level = 0.05,
insig = "blank",
#col=col4(10),
tl.pos = "ld",
tl.cex = .8,
tl.srt=45,
tl.col = "black",
cl.cex = .8)
#my.theme #this is a theme() piece, but if I take this away, the result is a list rather than a plot
recordPlot() # save the latest plot
}
your_function(res_pearson.c_setosa)
p <- your_function(res_pearson.c_setosa)
p
Created on 2022-07-13 by the reprex package (v2.0.1)
As you can see, the variable p outputs the plot.
I am new to transformer based models. I am trying to fine-tune the following model (https://huggingface.co/Chramer/remote-sensing-distilbert-cased) on my dataset. The code:
enter image description here
and I got the following error:
enter image description here
I will be thankful if anyone could help.
The preprocessing steps I followed:
input_ids_t = []
attention_masks_t = []
for sent in df_train['text_a']:
encoded_dict = tokenizer.encode_plus(
sent,
add_special_tokens = True,
max_length = 128,
pad_to_max_length = True,
return_attention_mask = True,
return_tensors = 'tf',
)
input_ids_t.append(encoded_dict['input_ids'])
attention_masks_t.append(encoded_dict['attention_mask'])
# Convert the lists into tensors.
input_ids_t = tf.concat(input_ids_t, axis=0)
attention_masks_t = tf.concat(attention_masks_t, axis=0)
labels_t = np.asarray(df_train['label'])
and i did the same for testing data. Then:
train_data = tf.data.Dataset.from_tensor_slices((input_ids_t,attention_masks_t,labels_t))
and the same for testing data
It sounds like you are feeding the transformer_model 1 input instead of 3. Try removing the square brackets around transformer_model([input_ids, input_mask, segment_ids])[0] so that it reads transformer_model(input_ids, input_mask, segment_ids)[0]. That way, the function will have 3 arguments and not just 1.
I have the following ServiceNow script which inserts the record into live_message table.
(function executeRule(current, previous/*null when async*/) {
var requestBody;
var responseBody;
var status;
var request;
var response;
try {
request = new sn_ws.RESTMessageV2("LiveMessageWebhook", "post");
request.setRequestHeader("Content-Type", "application/x-www-form-urlencoded");
var parameters = "&chat_message=" + current.chat_message.toString();
parameters = parameters + "&context=" + current.context.toString();
parameters = parameters + "&formatted_message=" + current.formatted_message.toString();
parameters = parameters + "&has_attachments=" + current.has_attachments.toString();
parameters = parameters + "&has_links=" + current.has_links.toString();
parameters = parameters + "&has_tags=" + current.has_tags.toString();
parameters = parameters + "&ID=" + current.id.toString();
parameters = parameters + "&in_reply_to=" + current.in_reply_to.toString();
parameters = parameters + "&isLiked=" + current.is_liked.toString();
parameters = parameters + "&lastActivity=" + current.last_activity.toString();
parameters = parameters + "&lastMessage=" + current.last_message.toString();
parameters = parameters + "&likeCount=" + current.like_count.toString();
parameters = parameters + "&message=" + current.message.toString();
parameters = parameters + "&next_reply_order_chunk=" + current.next_reply_order_chunk.toString();
parameters = parameters + "&order=" + current.order.toString();
parameters = parameters + "&poll=" + current.poll.toString();
parameters = parameters + "&private=" + current.private_message.toString();
parameters = parameters + "&profile=" + current.profile.toString();
parameters = parameters + "&reflected_field=" + current.reflected_field.toString();
parameters = parameters + "&reply_to=" + current.reply_to.toString();
parameters = parameters + "&state=" + current.state.toString();
parameters = parameters + "&group_type=" + current.sys_class_name.toString();
parameters = parameters + "&created_by=" + current.sys_created_by.toString();
parameters = parameters + "&created_on=" + current.sys_created_on.toString();
parameters = parameters + "&domain=" + current.sys_domain.toString();
parameters = parameters + "&domain_path=" + current.sys_domain_path.toString();
parameters = parameters + "&sys_ID=" + current.sys_id.toString();
parameters = parameters + "&to_profile=" + current.sys_domain.toString();
parameters = parameters + "&updates=" + current.sys_mod_count.toString();
parameters = parameters + "&updated_by=" + current.sys_updated_by.toString();
parameters = parameters + "&updated_on=" + current.sys_updated_on.toString();
request.setRequestBody(encodeURI(parameters));
var l = request.getRequestBody();
response = request.execute();
responseBody = response.haveError()
? response.getErrorMessage()
: response.getBody();
status = response.getStatusCode();
{
var gr = new GlideRecord('live_message');
gr.initialize();
gr.chat_message = current.chat_message;
gr.context = current.context.toString();
gr.formatted_message = "abc";
gr.group = current.group;
gr.has_attachments = current.has_attachments;
gr.has_links = current.has_links;
gr.has_tags = current.has_tags;
gr.id = current.id;
gr.in_reply_to = "admin";
gr.is_liked = current.is_liked;
gr.last_activity = current.last_activity;
gr.last_message = current.last_message;
gr.like_count = current.like_count;
gr.message = "abc";
gr.next_reply_order_chunk = current.next_reply_order_chunk;
gr.order = current.order;
gr.poll = current.poll;
gr.private_message = current.private_message;
gr.profile = current.profile;
gr.reflected_field = current.reflected_field;
gr.reply_to = current.reply_to;
gr.state = current.state;
gr.sys_class_name = current.sys_class_name;
gr.sys_created_by = current.sys_created_by;
gr.sys_created_on = current.sys_created_on;
gr.sys_domain = current.sys_domain;
gr.sys_domain_path = current.sys_domain_path;
gr.sys_mod_count = current.sys_mod_count;
gr.sys_updated_by = current.sys_updated_by;
gr.sys_updated_on = current.sys_updated_on;
gr.to_profile = current.to_profile;
gr.insertWithReferences();
}
} catch (ex) {
responseBody = 'Exception: ' + ex;
status = '900';
requestBody = request
? request.getRequestBody()
: null;
} finally {
gs.info("Final: Request Body: " + requestBody);
gs.info("Final: Response: " + responseBody);
gs.info("Final: HTTP Status: " + status);
gs.addInfoMessage('Final: Finished');
}
})(current, previous);
But after inserting the record it again goes back to the try block again and does the whole thing again and now with the inserted record
How is this business rule configured?
Is it possible that the changes made by this script, could also trigger the script to run?
Here's an example of what I mean:
Imagine I have a business rule that executes whenever a record in the table "u_arbitrary_counter" is updated.
Imagine that the function of this business rule is that whenever such a record is updated, we increase the value of the "u_counter" field by one, like so:
current.setValue('u_counter', parseInt(current.getValue('u_counter')) + 1);
current.update();
By using .update(), I'm forcing an update to the database, EVEN if this is a "before" business rule.
Pro Tip: Note that "before" business rules run on the data that's about to be
saved to the database, BEFORE the actual database operation has taken
place. Any changes to the "current" object in a "before" business rule
will be saved even without using current.update, because you're
modifying the data that's about to be sent to the database anyway, in
the natural course of this operation.
So using current.update() in a BEFORE business rule isn't a great idea. For the same reason though, performing any other operation which necessarily leads to a database update which could trigger this same business rule is a bad idea. It doesn't have to be current.update() -- Instead, imagine if rather than updating the current record, I did something like...
var gr = new GlideRecord('u_arbitrary_counter'); //the same table this BR is running on
gr.initialize();
gr.setValue('u_counter', parseInt(current.getValue('u_counter')) + 1);
gr.insert(); //This triggers a database action!
This is no good for the same reason.
Ditto for any REST calls which would trigger a DB action on this table which may trigger this business rule.
The solution
If this is indeed the cause of your issue (and without knowing the configuration of your business rule, I can't be sure if it is or not), there is actually a way to tell the system not to run any further business rules as a result of the operation your script performs. Right before your gr.insertWithReferences(); line (line 91), IF that block of code is the issue (which I'm fairly confident it is), add:
gr.setWorkflow(false);
This prevents business rules from running.
Unfortunately, since you're placing the record into the live_message table, that may not be a good idea either, since the messenger may require some business rule to propagate or present that information once inserted; I'm not sure.
If that is the case, I would recommend adding a condition to your business rule so that it only runs under certain circumstances, and then craft your inserted record to not meet those criteria.
Pro Tip: PS - I notice you access a table field using current.field_name.toString() in some places, and directly access
current.field_name in others. While the former is acceptable (not
that I recommend it in most cases), the latter should almost never be
used -- except in the case of journal fields. Instead, I strongly
recommend using the appropriate getter/setter methods, to avoid
pass-by-reference issues and confusion, and to ensure you're not
relying on JavaScript's coersion, which doesn't always play nice with
Mozilla Rhino, which ServiceNow runs on the back-end. Example issue
here. For more info, please see my article on using getValue()
and setValue() in ServiceNow.
pymc.__version__ = '3.0'
theano.__version__ = '0.6.0.dev-RELEASE'
I'm trying to use PyMC3 with a complex likelihood function:
First question: Is this possible?
Here's my attempt using Thomas Wiecki's post as a guide:
import numpy as np
import theano as th
import pymc as pm
import scipy as sp
# Actual data I'm trying to fit
x = np.array([52.08, 58.44, 60.0, 65.0, 65.10, 66.0, 70.0, 87.5, 110.0, 126.0])
y = np.array([0.522, 0.659, 0.462, 0.720, 0.609, 0.696, 0.667, 0.870, 0.889, 0.919])
yerr = np.array([0.104, 0.071, 0.138, 0.035, 0.102, 0.096, 0.136, 0.031, 0.024, 0.035])
th.config.compute_test_value = 'off'
a = th.tensor.dscalar('a')
with pm.Model() as model:
# Priors
alpha = pm.Normal('alpha', mu=0.3, sd=5)
sig_alpha = pm.Normal('sig_alpha', mu=0.03, sd=5)
t_double = pm.Normal('t_double', mu=4, sd=20)
t_delay = pm.Normal('t_delay', mu=21, sd=20)
nu = pm.Uniform('nu', lower=0, upper=20)
# Some functions needed for calculation of the y estimator
def T(eqd):
doses = np.array([52.08, 58.44, 60.0, 65.0, 65.10,
66.0, 70.0, 87.5, 110.0, 126.0])
tmt_times = np.array([29,29,43,29,36,48,22,11,7,8])
return np.interp(eqd, doses, tmt_times)
def TCP(a):
time = T(x)
BCP = pm.exp(-1E7*pm.exp(-alpha*x*1.2 + 0.69315/t_delay(time-t_double)))
return pm.prod(BCP)
def normpdf(a, alpha, sig_alpha):
return 1./(sig_alpha*pm.sqrt(2.*np.pi))*pm.exp(-pm.sqr(a-alpha)/(2*pm.sqr(sig_alpha)))
def normcdf(a, alpha, sig_alpha):
return 1./2.*(1+pm.erf((a-alpha)/(sig_alpha*pm.sqrt(2))))
def integrand(a):
return normpdf(a,alpha,sig_alpha)/(1.-normcdf(0,alpha,sig_alpha))*TCP(a)
func = th.function([a,alpha,sig_alpha,t_double,t_delay], integrand(a))
y_est = sp.integrate.quad(func(a, alpha, sig_alpha,t_double,t_delay), 0, np.inf)[0]
likelihood = pm.T('TCP', mu=y_est, nu=nu, observed=y_tcp)
start = pm.find_MAP()
step = pm.NUTS(state=start)
trace = pm.sample(2000, step, start=start, progressbar=True)
which produces the following message regarding the expression for y_est:
TypeError: ('Bad input argument to theano function with name ":42" at index 0(0-based)', 'Expected an array-like object, but found a Variable: maybe you are trying to call a function on a (possibly shared) variable instead of a numeric array?')
I've overcome various other hurdles to get this far, and this is where I'm stuck. So, provided the answer to my first question is 'yes', then am I on the right track? Any guidance would be helpful!
N.B. Here is a similar question I found, and another.
Disclaimer: I'm very new at this. My only previous experience is successfully reproducing the linear regression example in Thomas' post. I've also successfully run the Theano test suite, so I know it works.
Yes, its possible to make something with a complex or arbitrary likelihood. Though that doesn't seem like what you're doing here. It looks like you have a complex transformation of one variable into another, the integration step.
Your particular exception is that integrate.quad is expecting a numpy array, not a pymc Variable. If you want to do quad within pymc, you'll have to make a custom theano Op (with derivative) for it.
I'm having trouble getting this API working in .net 3.5 (works fine in 4.0). Basically I have following code
List<ParameterExpression> parameters = new List<ParameterExpression>();
parameters.Add(Expression.Parameter(typeof(double), "R0C6"));
parameters.Add(Expression.Parameter(typeof(double), "R0C7"));
parameters.Add(Expression.Parameter(typeof(double), "R0C8"));
parameters.Add(Expression.Parameter(typeof(double), "R0C9"));
parameters.Add(Expression.Parameter(typeof(double), "R0C10"));
parameters.Add(Expression.Parameter(typeof(double), "R0C11"));
parameters.Add(Expression.Parameter(typeof(double), "R0C12"));
LambdaExpression e = DynamicExpressionBuilder.ParseLambda(
parameters.ToArray(), typeof(double), "R0C6 + R0C7 + R0C8 + R0C9 + R0C10 + R0C11 + R0C12");
var result = e.Compile().DynamicInvoke(1, 1, 1, 1, 1, 1, 1);
Console.WriteLine(result);
When running this code I get ArgumentException. The reason being .NET 3.5 does not have Func delegate which takes more than 4 parameters. Can someone suggest me change in Dynamic.cs so that I can get able code working in 3.5?
Yes thats one alternative. Here is the code that worked for me
List<ParameterExpression> parameters = new List<ParameterExpression>();
parameters.Add(Expression.Parameter(typeof(Holder), "H"));
LambdaExpression e = DynamicExpressionBuilder.ParseLambda(parameters.ToArray(), typeof(double), "H.R0C6 + H.R0C7 + H.R0C8 + H.R0C9 + H.R0C10 + H.R0C11 + H.R0C12");
var result = e.Compile().DynamicInvoke(new Holder { R0C6 = 1, R0C7 = 1, R0C8 = 1, R0C9 = 1, R0C10 = 1, R0C11 = 1, R0C12 = 1 });
Console.WriteLine(result);
But surely there should be better way of achieving my original implementation.
Is there a reason that you can't combine all the doubles into a single parameter object that contains fields or properties for the values? Then you'd only need one parameter in your Func for the expression.