Apache jena to update remote endpoint - insert

The current procedure we now often perform is to extract data from an endpoint, perform computational analysis, generate RDF files and manually load them back into the endpoint.
Now I was looking into automating this procedure using Apache Jena ARQ as these dependencies are currently used for the retrieval of information.
I manage to get it partially to work using INSERT statements but performing thousands if not millions of inserts one by one seem a bit inefficient to me. The second issue is that we sometimes have regex or " in a string and this needs to escaped but there are many exceptions.
Is there a way to either parse or iterate over an internal apache jena model statements and inject this directly into an endpoint?
We currently use graphdb but it would be great if this can be applied using a universal approach.
Update
I have updated the code to handle 10 statements at once not sure yet what eventually the limit will be...
public class EndpointTests extends TestCase {
// Only for local testing
public void testEndpoint() throws Throwable {
String endpoint = "http://10.117.11.77:7200/repositories/Test";
Domain domain = new Domain("file:///Users/jasperkoehorst/diana_interproscan_head.nt");
StmtIterator statements = domain.getRDFSimpleCon().getModel().listStatements();
String strInsert = "INSERT DATA { ";
int insertCounter = 0;
while (statements.hasNext()) {
insertCounter = insertCounter + 1;
Statement statement = statements.nextStatement();
String subject = statement.getSubject().getURI();
String predicate = statement.getPredicate().getURI();
String object = statement.getObject().toString();
if (statement.getObject().isURIResource()) {
object = "<" + statement.getObject().toString() + ">";
}
if (statement.getObject().isLiteral()) {
object = statement.getObject().asLiteral().getString();
object = object.replaceAll("\\\\", "\\\\\\\\");
object = object.replaceAll("\"","\\\\\"");
}
if (object.startsWith("http")) {
object = "<" + object + ">";
} else {
object = "\"" + object + "\"";
}
strInsert = strInsert + "<" + subject + "> <" + predicate + "> " + object + " . ";
if (insertCounter % 10 == 0) {
System.out.println(insertCounter);
strInsert = strInsert + " } ";
UpdateRequest updateRequest = UpdateFactory.create(strInsert);
UpdateProcessor updateProcessor = UpdateExecutionFactory.createRemote(updateRequest, endpoint + "/statements");
updateProcessor.execute();
strInsert = "INSERT DATA { ";
}
}
}
}

Related

Spring Data JPA Query by Example with access to nested Objects Attributes

I use Query by Example and want to know how I can find objects with certain properties in the nested objects.
A plan anyone?
Here is my example Code:
ExampleMatcher matcher = ExampleMatcher.matching()
.withMatcher("offer2product.id.productId", match -> match.exact()
);
Offer2ProductId id = new Offer2ProductId();
id.setProductId(1337L);
Offer2Product offer2Product = new Offer2Product();
offer2Product.setId(id);
Set<Offer2Product> offer2productSet = new HashSet<>();
offer2productSet.add(offer2Product);
Offer probe = new Offer();
probe.setOffer2productSet(offer2productSet);
Example<Offer> example = Example.of(probe, matcher);
List<Offer> offerList = offerRepository.findAll(example);
Quoting Spring data documentation: https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#query-by-example
Currently, only SingularAttribute properties can be used for property matching.
In your example, you want to search by a property that is a Set<> (offer2productSet), which is a PluralAttribute - it is not possible to search by this field. It will be ignored when building a query, as can be seen here:
https://github.com/spring-projects/spring-data-jpa/blob/master/src/main/java/org/springframework/data/jpa/convert/QueryByExamplePredicateBuilder.java#L112
man actually can do it, follow the example below, where demand has a list of labels, I don't have time to explain everything but soon I'll update this post.
#Repository
#RequiredArgsConstructor
public class DemandaFilterRepository {
private final EntityManager entityManager;
public Page<Demanda> findAll(DemandaFilterDTO demandaFilter, Pageable pageable) {
String query = "select d from Demanda d where 1=1";
Map<String, Object> parameters = new HashMap<>();
if (demandaFilter.getId() != null) {
query += " and d.id = :id";
parameters.put("id", demandaFilter.getId());
}
if (!StringUtils.isEmpty(demandaFilter.getTenant())) {
query += " and upper(d.tenant) = upper(:tenant)";
parameters.put("tenant", demandaFilter.getTenant());
}
if (!StringUtils.isEmpty(demandaFilter.getAssunto())) {
query += " and upper(d.assunto) like upper(:assunto)";
parameters.put("assunto", "%" + demandaFilter.getAssunto() + "%");
}
if (!StringUtils.isEmpty(demandaFilter.getDescricao())) {
query += " and upper(d.descricao) like upper(:descricao)";
parameters.put("descricao", "%" + demandaFilter.getDescricao() + "%");
}
if (!StringUtils.isEmpty(demandaFilter.getEtiqueta())) {
query = query.replace("Demanda d", "Demanda d inner join d.etiquetas etiqueta");
query += " and upper(etiqueta.descricao) = upper(:etiqueta)";
parameters.put("etiqueta", demandaFilter.getEtiqueta());
}
if (!StringUtils.isEmpty(demandaFilter.getNomeDemandante())) {
query += " and upper(d.demandante.nome) like upper(:nomeDemandante)";
parameters.put("nomeDemandante", "%" + demandaFilter.getNomeDemandante() + "%" );
}
if (!StringUtils.isEmpty(demandaFilter.getDtInclusao())) {
query += " d.dtInclusao like :dtInclusao";
parameters.put("dtInclusao", "%" + demandaFilter.getDtInclusao() + "%");
}
query += " ORDER BY d.id DESC ";
TypedQuery<Demanda> typedQuery = entityManager.createQuery(query, Demanda.class)
.setMaxResults(pageable.getPageSize())
.setFirstResult(pageable.getPageNumber() * pageable.getPageSize());
parameters.forEach(typedQuery::setParameter);
return new PageImpl<>(typedQuery.getResultList());
}
}
this is a class for one of a small project that I'm working on ...

get workflow malfunction exception with java api

Does anyone know how to get a workflow malfunction error message using the java pe api? I am running the QueueSample java code provided by IBM and it is not clear to me how to do this. Any help would be appreciated!
I found the malfunction error message for my workflow in the VWParticipantHistory.getLogFields() array. I modified the example code from the Developing Applications with IBM FileNet P8 APIs redbook:
// Create session object and log onto Process Engine
...
// Get the specific work item
...
// Get VWProcess object from work object
VWProcess process = stepElement.fetchProcess();
// Get workflow definitions from the VWProcess
VWWorkflowDefinition workflowDefinition =
process.fetchWorkflowDefinition(false);
// Get maps for each workflow definition
VWMapDefinition[] workflowMaps = workflowDefinition.getMaps();
// Iterate through each map in the workflow Definition
for (int i = 0; i < workflowMaps.length; i++) {
// Get map ID and map name for each map definition
int mapID = workflowMaps[i].getMapId();
String mapName = workflowMaps[i].getName();
// Get workflow history information for each map
VWWorkflowHistory workflowHistory = process.fetchWorkflowHistory(mapID);
String workflowOriginator = workflowHistory.getOriginator();
// Iterate through each item in the Workflow History
while (workflowHistory.hasNext()) {
// Get step history objects for each workflow history
VWStepHistory stepHistory = workflowHistory.next();
String stepName = stepHistory.getStepName();
System.out.println("step history name = " + stepName);
// Iterate through each item in the Step History
while (stepHistory.hasNext()) {
// Get step occurrence history
// objects for each step history object
VWStepOccurrenceHistory stepOccurenceHistory = stepHistory.next();
Date stepOcurrenceDateReceived = stepOccurenceHistory.getDateReceived();
Date stepOcurrenceDateCompleted = stepOccurenceHistory.getCompletionDate();
while (stepOccurenceHistory.hasNext()) {
// Get step work object information
// for each step occurrence
VWStepWorkObjectHistory stepWorkObjectHistory = stepOccurenceHistory.next();
stepWorkObjectHistory.resetFetch();
// Get participant information for each work object
while (stepWorkObjectHistory.hasNext()) {
VWParticipantHistory participantHistory = stepWorkObjectHistory.next();
String opName = participantHistory.getOperationName();
System.out.println("operation name = " + opName);
Date participantDateReceived = participantHistory.getDateReceived();
String participantComments = participantHistory.getComments();
String participantUser = participantHistory.getUserName();
String participantName = participantHistory.getParticipantName();
VWDataField[] logFields = participantHistory.getLogFields();
System.out.println("** start get log fields **");
for (int index=0; index<logFields.length; index++){
VWDataField dataField = logFields[index];
String name = dataField.getName();
String val = dataField.getStringValue();
System.out.println("name = " + name + " , value = " + val);
}
System.out.println("** end get log fields **");
} // while stepWorkObjectHistory
} // while stepOccurenceHistory
} // while stepHistory
} // while workflowHistory
} // for workflow maps

Xamarin UILabel not updating

I've a basic for loop that's basically download files. It's supposed to update the label as long as it progress.
By searching here at Stack Overflow, I found an orientation to use SetNeedsDisplay(). But it's still refuses to update. Any idea ?
for (int i = 0; i < files.Length; i++)
{
status.Text = "Downloading file " + (i + 1) + " of " + files.Length + "...";
status.SetNeedsDisplay();
string remoteFile = assetServer + files[i];
var webClient2 = new WebClient();
string localFile = files[i];
string localPath3 = Path.Combine(documentsPath, localFile);
webClient2.DownloadFile(remoteFile, localPath3);
}
As previously suggested try to avoid blocking the UI when doing heavy transactions in it. WebClient already has a async method which you can use.
webClient2.DownloadFileasync(new System.Uri(remoteFile), localPath3);
and to prevent you from accessing the UI from a different thread use the built-in method InvokeOnMainThread when accessing UI elements.
InvokeOnMainThread (() => {
status.Text = "Downloading file " + (i + 1) + " of " + files.Length + "...";
status.SetNeedsDisplay ();
});
and finally use the using statement to help you with the resources disposal.
using (var webClient2 = new WebClient ())
{
webClient2.DownloadFileAsync (new System.Uri (remoteFile), localPath3);
}
You could also have the iteration inside the using statement this way you don't have to create a WebClient object for each file instead you will use the same object to download all files available in your files array.

What exactly am I sending through the parameters?

When doing a XMLHttpRequest and using POST as the form method, what exactly am I sending? I know it should be like send(parameters), parameters = "variable1=Hello", for example. But what if I want to do this:
parameters = "variable1=" + encodeURIComponent(document.getElementById("variable1").value);
variable1 being the id of the HTML form input.
Can I do it like this or do I need to assign the encodeURIComponent value to a javascript variable and send that variable:
var variable2;
parameters = "variable2=" + encodeURIComponent(document.getElementById("variable1").value);
You're suppose to send the object and it's value, but, is it an object from the HTML form, a javascript object or a php object? The problem is I already tried it and I still can't get the encoded input in my database, all I get is the raw input from the user.
BTW, I know it's a pretty dull question, but I feel the need to understand exactly what I'm doing if I want to come up with a solution.
g
function createObject()
{
var request_type;
var browser = navigator.appName;
if(browser == "Microsoft Internet Explorer")
{
request_type = new ActiveXObject("Microsoft.XMLHTTP");
}
else
{
request_type = new XMLHttpRequest();
}
return request_type;
}
var http = createObject();
//INSERT
function insert()
{
var Faculty2 = encodeURIComponent(document.getElementById("Faculty").value);
var Major2 = encodeURIComponent(document.getElementById("Major").value);
var Professor2 = encodeURIComponent(document.getElementById("Professor").value);
var Lastname2 = encodeURIComponent(document.getElementById("Lastname").value);
var Course2 = encodeURIComponent(document.getElementById("Course").value);
var Comments2 = encodeURIComponent(document.getElementById("Comments").value);
var Grade2 = encodeURIComponent(document.getElementById("Grade").value);
var Redflag2 = encodeURIComponent(document.getElementById("Redflag").value);
var Owner2 = encodeURIComponent(document.getElementById("Owner").value);
//Location and parameters of data about to be sent are defined
//Required: verify that all fields are not empty. Use encode URI() to solve some issues about character encoding.
var params = "Faculty=" + Faculty2 + "&Major=" + Major2 + "&Professor=" + Professor2 + "&Lastname=" + Lastname2 + "&Course=" + Course2 + "&Comments=" + Comments2 + "&Grade=" + Grade2 + "&Redflag=" + Redflag2 + "&Owner=" + Owner2;
var url = "prehp/insert.php";
http.open("POST", url, true);
//Technical information about the data
http.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
http.setRequestHeader("Content-length", params.length);
//Now, we send the data
http.onreadystatechange = function()
{
if(http.readyState == 4)
{ var answer = http.responseText;
document.getElementById('insert_response').innerHTML = "Ready!" + answer;
}
else
{document.getElementById('insert_response').innerHTML = "Error";
}}
http.send(params);
}
PHP code:
$insertAccounts_sql = "INSERT INTO Information (Faculty, Major, Professor, Lastname, Course, Comments, Grade, Redflag, Owner)
VALUES('$_POST[Faculty]','$_POST[Major]','$_POST[Professor]','$_POST[Lastname]','$_POST[Course]','$_POST[Comments]','$_POST[Grade]','$_POST[Redflag]','$_POST[Owner]')";
$dbq = mysql_query($insertAccounts_sql, $dbc);
if($dbq)
{
print "1 record added: Works very well!";
}
else
if(!$dbq)
{
die('Error: ' . mysql_error());
}
$dbk = mysql_close($dbc);
if($dbk)
{
print "Database closed!";
}
else
if(!$dbk)
{
print "Database not closed!";
}
I did that but the value that the database got was the raw input and not the encoded input. I'm running out of ideas, don't know what else to try. Could it be the settings of the database, can the database be decoding the input before storing it? That seems far-fetched to me, but I've been looking at this from all angles and still can't come up with a fresh answer.
PS: Sorry for posting my comments on the answer area, first timer here.
when creating query strings, it has really nothing to do with objects or anything like that. All you want to be sending is key/value pairs. how you construct that is up to you, but it often neater and more manageable to assign your values to variables first. i.e.
var myVar1Value = encodeURIComponent(document.getElementById('variable1').value);
var myVar2Value = encodeURIComponent(document.getElementById('variable2').value);
var url = "http://www.mydomain.com?" + "var1=" + myVar1Value + "&var2=" + myVar2Value;
It's called a query string, so it's just a string. what you do with it on the server side is what makes the 'magic' happen.
edit: If you're having problems with values, then you should print them to console to verify you are getting what you expect.

What is the use of "AsyncPattern" property of "OperationContractAttribute" + wcf?

Thus for used ajax enabled wcf services to get records from DB and display it in client without using AsyncPattern property of OperationContractAttribute....
When should i consider AsyncPattern property?
Sample of my operationcontract methods,
[OperationContract]
public string GetDesignationData()
{
DataSet dt = GetDesignationViewData();
return GetJSONString(dt.Tables[0]);
}
public string GetJSONString(DataTable Dt)
{
string[] StrDc = new string[Dt.Columns.Count];
string HeadStr = string.Empty;
for (int i = 0; i < Dt.Columns.Count; i++)
{
StrDc[i] = Dt.Columns[i].Caption;
HeadStr += "\"" + StrDc[i] + "\" : \"" + StrDc[i] + i.ToString() + "¾" + "\",";
}
HeadStr = HeadStr.Substring(0, HeadStr.Length - 1);
StringBuilder Sb = new StringBuilder();
Sb.Append("{\"" + Dt.TableName + "\" : [");
for (int i = 0; i < Dt.Rows.Count; i++)
{
string TempStr = HeadStr;
Sb.Append("{");
for (int j = 0; j < Dt.Columns.Count; j++)
{
if (Dt.Rows[i][j].ToString().Contains("'") == true)
{
Dt.Rows[i][j] = Dt.Rows[i][j].ToString().Replace("'", "");
}
TempStr = TempStr.Replace(Dt.Columns[j] + j.ToString() + "¾", Dt.Rows[i][j].ToString());
}
Sb.Append(TempStr + "},");
}
Sb = new StringBuilder(Sb.ToString().Substring(0, Sb.ToString().Length - 1));
Sb.Append("]}");
return Sb.ToString();
}
public DataSet GetDesignationViewData()
{
try
{
string connectionString = System.Configuration.ConfigurationManager.ConnectionStrings["connectionString"].ConnectionString;
return SqlHelper.ExecuteDataset(connectionString, CommandType.StoredProcedure, DataTemplate.spDesignation_View);
}
catch (Exception err)
{
throw err;
}
}
AsyncPattern has a few uses- it's mainly a server performance optimization that allows you to free up worker pool request threads on blocking operations. For example, when a long-running blocking operation like DB access occurs, if you're using an async DB API on the server with AsyncPattern, the worker thread can return to the pool and service other requests. The original request is "awakened" later on another worker thread when the DB access completes, and the rest of the work is done (the service client just patiently waits- this is all transparent to it unless you're using an AsyncPattern-aware client and binding). This CAN allow your service to process more requests, if done carefully. To take advantage, you need to be using APIs on the server that have native async implementations. The only one I see that might be a candidate is the DB call that's happening in your SQLHelper.ExecuteDataset method- you'd have to read up on the underlying API to make sure a TRUE asynchronous option is available (presence of BeginXXX/EndXXX methods doesn't necessarily mean it's a TRUE async impl). The System.SqlClient stuff is truly async.
A word of caution: you have to be processing a lot of requests to make this worthwhile- there's a significant cost to code complexity and readability to split things up this way. You also need to understand multi-threaded programming very well- there are numerous pitfalls around locking, error handling, etc, that are well outside the scope of a SO post.
Good luck!

Resources