I have a ModelBatchPredictOp component in my pipeline. This component generates 3 artifacts: batchpredictionjob, big_query_table, and gcs_output_directory. The pipeline is running fine.
What I need is a way to access the tableId property of artifact big_query_table, so I can use it in the next component (a BigqueryQueryJobOp).
That is what I want:
The URI information would be good as well since it contains the full path for the created table and I can extract the desired part.
This is my 2nd component which is creating the batch prediction and the 3rd component which should consume the previous outputs.
# Component to do the batch prediction
batch_predict_op = ModelBatchPredictOp(
project=project_id,
location=DEFAULT_VERTEX_REGION,
instances_format = 'bigquery',
predictions_format = 'bigquery',
model=importer_spec.outputs['artifact'],
job_display_name='teste_batch_predict',
bigquery_source_input_uri=f'bq://{input_data_table_ref}',
bigquery_destination_output_uri= f'bq://{output_bq}',
).after(input_data_table_op)
top_predictions_table_ref = f'{project_id}.{bigquery_dataset}.test'
# Component to create the table based on previous component
top_predictions_op = bq.BigqueryQueryJobOp(
project_id,
location = bigquery_job_location,
query = predict_dataset.get_query(
output_table = top_predictions_table_ref,
source_table = batch_predict_op.outputs['bigquery_output_table'],
query_name = 'query_top_100.sql',
DEBUG = DEBUG)
).after(batch_predict_op)
Related
I am been trying to search online for a solution but due to lack of knowledge in HP ALM I am not able to search proper hit tags
Set RunFactory = tsTest.RunFactory
Set obj_theRun = RunFactory.AddItem(CStr(testrunname))
obj_theRun.Status = sExecutionStatus '"Passed" '-- Status to be updated
obj_theRun.Tester = strTesterName
Getting error in this line object does not support obj_theRun.Tester
I just want to update the Tester column(Not Responsible tester) in Test set via vbscript. Please refer to the attached image at the very last column (TesterAny help is appreciated. Thank you in advance.
The documentation for ALM says the Tester Name can be specified by passing an array as the argument to AddItem.
https://admhelp.microfocus.com/alm/api_refs/ota/Content/ota/topic8805.html?Highlight=tester
An array consisting of the following elements:
Name - The name of the run (string. required).
Tester - The name of the user responsible (string. optional)
Location - The host name (string. optional). The default is the host name of the current machine
So change your code to this:
Set runFactory = tsTest.RunFactory
Dim newRunArgs(3)
newRunArgs(0) = testrunname ' `testrunname` is already a string so you don't need CStr.
newRunArgs(1) = "tester name goes here"
Set newRunArgs(2) = Nothing
Set newRun = RunFactory.AddItem( newRunArgs )
newRun.Status = sExecutionStatus '"Passed" '-- Status to be updated
I have a Application entity and a User entity. Theres a one to many relation from application to users, i.e one application can have multiple users.
i update the user and change the application assigned to him. So far i have below:
Diff diff = javers.compare(oldUser, newUser);
List<ReferenceChange> referenceChanges = diff.getChangesByType(ReferenceChange.class);
for (ReferenceChange referenceChange : referenceChanges) {
Object oldRef = referenceChange.getLeftObject().get();
Object newRef = referenceChange.getRightObject().get();
Diff refDiff = javers.compare(oldRef,newRef);
}
Please find the below result while debugging:
referenceChanges List:
ReferenceChange{globalId:'com.ds.appmanager.services.domain.User/3', property:'application', oldRef:'com.ds.appmanager.services.domain.Application/7', newRef:'com.ds.appmanager.services.domain.Application/3'}
oldRef
Application [applicationId=7, applicationName=KYC, applicationDesc=DEV_TEST, applicationLaunch=2016-12-31 00:00:00.0, live=false]
newRef
Application [applicationId=3, applicationName=KYC, applicationDesc=hellp, applicationLaunch=Sat Mar 12 00:00:00 IST 2016, live=true]
When i pass both objects to javers.compare(oldRef,newRef), it again gives me same result as referenceChanges. I thought it should give me Value Changes of each property e.g, applicationDesc, oldVal = DEV_TEST, newVal = hellp.
Am i missing something here? All i want to do is to get the value changes of properties of the Application object which is referenced inside User object.
P.S: If i change Object oldRef = referenceChange.getLeftObject().get(); to Object oldRef = referenceChange.getLeftObject();, i get JaversException: CLASS_EXTRACTION_ERROR JaVers bootstrap error - Don't know how to extract Class from type 'T'.
UPDATE
Just noted i am not getting ValueChanges while doing Diff refDiff = javers.compare(oldRef,newRef);
Debug shows follwing changes:
Diff:
1. NewObject{globalId:'com.ds.appmanager.services.domain.Application/7'}
2. ObjectRemoved{globalId:'com.ds.appmanager.services.domain.User/5'}
3. ObjectRemoved{globalId:'com.ds.appmanager.services.domain.Application/3'}
4. ObjectRemoved{globalId:'com.ds.appmanager.services.domain.User/6'}
5. ObjectRemoved{globalId:'com.ds.appmanager.services.domain.User/7'}
6. ReferenceChange{globalId:'com.ds.appmanager.services.domain.User/3', property:'application', oldRef:'com.ds.appmanager.services.domain.Application/3', newRef:'com.ds.appmanager.services.domain.Application/7'}
The user tutorial says
Navigate to Data > View All
Choose to filter by the model key
Hit Save Model
Input for path: /data/h2o-training/...
Hit Submit
The problem is that I do not have this menu (H2o, 3.0.0.26, web interface)
I am, unfortunately, not familiar with the web interface but I can offer a workaround involving H2O in R. The functions
h2o.saveModel(object, dir = "", name = "", filename = "", force = FALSE)
and
h2o.loadModel(path, conn = h2o.getConnection())
Should offer what you need. I will try to have a look at H2O Flow.
Update
I cannot find the possibility to explicitly save a model either. What you can do instead is save the 'Flow'. You ergo could upload/import your file, build the model and then save / load the status :-)
When viewing the model in H2O Flow, you will see an 'Export' button as an action that can be taken against a model
From there, you will be prompted to specify a path in 'Export Model' dialog. Specify the path and hit the 'Export' button. That will save you model to disk.
I'm referring to H2O version 3.2.0.3
A working example that I've used recently while building a deep learning model in version 2.8.6 in h2o.The model was saved in hdfs.For latest version you probably have to remove the classification=T switch and have to replace data with training_frame
library(h2o)
h = h2o.init(ip="xx.xxx.xxx.xxx", port=54321, startH2O = F)
cTrain.h2o <- as.h2o(h,cTrain,key="c1")
cTest.h2o <- as.h2o(h,cTest,key="c2")
nh2oD<-h2o.deeplearning(x =c(1:12),y="tgt",data=cTrain.h2o,classification=F,activation="Tanh",
rate=0.001,rho=0.99,momentum_start=0.5,momentum_stable=0.99,input_dropout_ratio=0.2,
hidden=c(12,25,11,11),hidden_dropout_ratios=c(0.4,0.4,0.4,0.4),
epochs=150,variable_importances=T,seed=1234,reproducible = T,l1=1e-5,
key="dn")
hdfsdir<-"hdfs://xxxxxxxxxx/user/xxxxxx/xxxxx/models"
h2o.saveModel(nh2oD,hdfsdir,name="DLModel1",save_cv=T,force=T)
test=h2o.loadModel(h,path=paste0(hdfsdir,"/","DLModel1"))
This should be what you need:
library(h2o)
h2o.init()
path = system.file("extdata", "prostate.csv", package = "h2o")
h2o_df = h2o.importFile(path)
h2o_df$CAPSULE = as.factor(h2o_df$CAPSULE)
model = h2o.glm(y = "CAPSULE",
x = c("AGE", "RACE", "PSA", "GLEASON"),
training_frame = h2o_df,
family = "binomial")
h2o.download_pojo(model)
http://h2o-release.s3.amazonaws.com/h2o/rel-slater/5/docs-website/h2o-docs/index.html#POJO%20Quick%20Start
How to save models in H2O Flow:
go to "List All Models"
in the model details, you will find an "Export" option
enter the model name you want to save it as
import it back again
How to save a model trained in h2o-py:
# say "rf" is your H2ORandomForestEstimator object. To export it
>>> path = h2o.save_model(rf, force=True) # save_model() returns the path
>>> path
u'/home/user/rf'
#to import it back again(as a new object)
>>> rafo = h2o.load_model(path)
>>> rafo # prints model details
Model Details
=============
H2ORandomForestEstimator : Distributed Random Forest
Model Key: drf1
Model Summary:
######Prints model details...................
I am working on BPT framework on QTP and for a testcase i am using same component multiple times. I want to fetch the Business component instance number from QC ->Testplan->TestCase->Business component. Based on the instance number i will read the required component parameters from excel sheet.
Thanks.
Below code worked for me:
Set TestTree = TestSet.TSTestFactory
Set TestList = TestTree.NewList("")
For Each TSTest In TestList
Set Test = TSTest.Test
Set objTestFactory = tdc.TestFactory
Set tFilter = objTestFactory.Filter
tFilter.Filter("TS_TEST_ID") = Test.ID
Set tList = objTestFactory.NewList(tFilter.Text)
Dim objTest
For Each tst In tList
If tst.Type = "BUSINESS-PROCESS" Then
Set objTest = tst
Exit For
End If
Next
If objTest Is Nothing Then
Else
Dim BPTest As BusinessProcess
Set BPTest = objTest
BPTest.Load
' Get list if component instance from the Test
Set components = BPTest.BPComponents
Set Component = components.Item(1)
For i = 1 To BPTest.BPComponents.Count
componentName = components.Item(i).Name
componentInstanceId = components.Item(i).ID
ComponentId = components.Item(i).Component.ID
Next
End if
Next
This code will fetch the details from the test lab level and go until the business component level. Currently it is extracting the component name and id. You can modify the code as per your need.
Visit tutorial at knowlzz . It has explained the solution nicely with each and every detail.
I have created a JMeter functional test that essentially:
creates a user;
logs in with the user;
deletes the user.
Now, I need to be able to thread this, and dynamically generate a username with a default prefix and a numerically incremented suffix (ie TestUser_1, TestUser_2, ... etc).
I used the counter, and things were working fine until I really punched up the number of threads/loops. When I did this, I was getting a conflict with the counter. Some threads were trying to read the counter, but the counter had already been incremented by another thread. This resulted in trying to delete a thread that was just created, then trying to log in with a thread that was just deleted.
The project is set up like this:
Test Plan
Thread group
Counter
User Defined Variables
Samplers
I was hoping to solve this problem by using the counter to append a number to the user defined variables upon thread execution, but the counter cannot be accessed in the user defined variables.
Any ideas on how I can solve this problem?
Thank you in advance.
I've used the following scheme successfully with any amount of test users:
1. Generate using beanshell-script (in BeanShell Sampler e.g.) csv-file with test-user details, for example:
testUserName001,testPwd001
testUserName002,testPwd002
. . .
testUserName00N,testPwd00N
with the amount of entries you need for the test-run.
This is done once per "N users test-run", in separate Thread Group, in setUp Thread Group or maybe even in separate jmx-script... makes no difference.
You can please find working beanshell-code below.
2. Create your test users IN TEST APPLICATION using previously created users-list.
If you don't need create in application you may skip this.
Thread Group
Number of Threads = 1
Loop Count = 1
. . .
While Controller
Condition = ${__javaScript("${newUserName}"!="",)} // this will repeat until EOF
CSV Data Set Config
Filename = ${__property(user.dir)}${__BeanShell(File.separator,)}${__P(users-list,)} // path to generated users-list
Variable Names = newUserName,newUserPwd // these are test-users details read from file into pointed variables
Delimiter = '
Recycle on EOF? = False
Stop thread on EOF? = True
Sharing Mode = Current thread group
[CREATE TEST USERS LOGIC HERE] // here are actions to create separate user in application
. . .
3. Perform multi-user logic.
Schema like the given above one but Thread Group executed not for 1 but for N threads.
Thread Group
Number of Threads = ${__P(usersCount,)} // set number of users you need to test
Ramp-Up Period = ${__P(rampUpPeriod,)}
Loop Count = X
. . .
While Controller
Condition = ${__javaScript("${newUserName}"!="",)} // this will repeat until EOF
CSV Data Set Config
Filename = ${__property(user.dir)}${__BeanShell(File.separator,)}${__P(users-list,)} // path to generated users-list
Variable Names = newUserName,newUserPwd // these are test-users details read from file into pointed variables
Delimiter = '
Recycle on EOF? = False
Stop thread on EOF? = True
Sharing Mode = Current thread group
[TEST LOGIC HERE] // here are test actions
. . .
The key idea is in using Thread Group + While Controller + CSV Data Set Config combination:
3.1. CSV Data Set Config reads details for each the test users from generated file:
. . . a. only once - because of "Stop thread on EOF? = True";
. . . b. doesn't block file for further access (in another thread groups, e.g., if there are any) - because of "Sharing Mode = Current thread group";
. . . c. pointed variables - "Variable Names = newUserName,newUserPwd" - you will use in further test-actions;
3.2. While Controller forces CSV Data Set Config to read all the entries from generated file - because of defined condition (until the EOF).
3.3. Thread Group will start all the threads with defined ramp-up - or simultaneously if ramp-up = 0.
You can take here template script for described schema: multiuser.jmx.
Beanshell script to generate test-users details looks like below and takes the following args:
- test-users count
- test-user name template ("TestUser_" in your case)
- test-user name format (e.g. 0 - to get TestUser_1, 00 - to get TestUser_01, 000- for TestUser_001, etc... you can simply hardcode this orexclude at all)
- name of generated file.
import java.text.*;
import java.io.*;
import java.util.*;
String [] params = Parameters.split(",");
int count = Integer.valueOf(params[0]);
String testName = params[1];
String nameFormat = params[2];
String usersList = params[3];
StringBuilder contents = new StringBuilder();
try {
DecimalFormat formatter = new DecimalFormat(nameFormat);
FileOutputStream fos = new FileOutputStream(System.getProperty("user.dir") + File.separator + usersList);
for (int i = 1; i <= count; i++) {
String s = formatter.format(i);
String testUser = testName + s;
contents.append(testUser).append(",").append(testUser);
if (i < count) {
contents.append("\n");
}
}
byte [] buffer = contents.toString().getBytes();
fos.write(buffer);
fos.close();
}
catch (Exception ex) {
IsSuccess = false;
log.error(ex.getMessage());
System.err.println(ex.getMessage());
}
catch (Throwable thex) {
System.err.println(thex.getMessage());
}
All together it will look like:
Sorry if answer is too overloaded.
Hope this helps.
The "User Defined Variables" config element does not pick up the reference variable from the "Counter" config element. I think this is a bug in JMeter. I have verified this behavior in version 3.2.
I added a "BeanShell Sampler" element to work around the issue.
Notice that the reference name of the "Counter" element is INDEX
The RUN_FOLDER variable gets set to a combination of the TESTS_FOLDER variable and the INDEX variable in the "BeanShell Sampler"
The "Debug Sampler" simply gathers a snapshot of the variables so I can see them in the "View Results Tree" listener element. Notice how the RUN_FOLDER variable has the INDEX variable value (5 in this case) appended.