So I have a Apache Camel route that reads Data elements from a JPA endpoint, converts them to DataConverted elements and stores them into a different database via a second JPA endpoint. Both endpoints are Oracle databases.
Now I want to set a flag on the original Data element that it got copied successfully. What is the best way to achieve that?
I tried it like that: saving the ID in the context and then reading it and accessing a dao method in the .onCompletion().onCompleteOnly().
from("jpa://Data")
.onCompletion().onCompleteOnly().process(ex -> {
var id = Long.valueOf(getContext().getGlobalOption("id"));
myDao().setFlag(id);
}).end()
.process(ex -> {
Data data = ex.getIn().getBody(Data.class);
DataConverted dataConverted = convertData(data);
ex.getMessage().setBody(data);
var globalOptions = getContext().getGlobalOptions();
globalOptions.put("id", data.getId().toString());
getContext().setGlobalOptions(globalOptions);
})
.to("jpa://DataConverted").end();
However, this seems to trigger a deadlock, the dao method is stalling on the commit of the update. The only explanation could be that the Data object gets locked by Camel and is still locked in the .onCompletion().onCompleteOnly() part of the route, therefore it can't get updated there.
Is there a better way to do it?
Have you tried using the recipient list EIP where first destination is the jpa:DataConverted endpoint and the second destination will be the endpoint to set the flag. This way both get the same message and will be executed sequentially.
https://camel.apache.org/components/3.17.x/eips/recipientList-eip.html
from("jpa://Data")
.process(ex -> {
Data data = ex.getIn().getBody(Data.class);
DataConverted dataConverted = convertData(data);
ex.getIn().setBody(data);
})
.recipientList(constant("direct:DataConverted","direct:updateFlag"))
.end();
from("direct:DataConverted")
.to("jpa://DataConverted")
.end();
from("direct:updateFlag")
.process(ex -> {
var id = ((MessageConverted) ex.getIn().getBody()).getId();
myDao().setFlag(id);
})
.end();
Keep in mind, you might want to make the route transactional by adding .transacted()
https://camel.apache.org/components/3.17.x/eips/transactional-client.html
Related
In our app, we are synchronizing some of the data to elasticsearch, and some of this data is users' records. The app is grails 5.1 and we are using Elasticsearch Java API Client for elasticsearch integration.
The indexing is working perfectly fine, and an example of user data looks like this:
Now, we have this following function that suppose to get the list of users by their ids:
PublicUser[] getAllByIds(Long[] ids) {
MgetRequest request = new MgetRequest.Builder()
.ids(ids.collect { it.toString() }.toList())
.index("users")
.build()
MgetResponse<PublicUser> response = elasticSearchClientProviderService.getClient().mget(
request,
PublicUser.class
)
response.docs().collect {
it.result().source()
}
}
And when the response holds at least one user record, we are getting a list of PulicUser objects -> as expected.
However, if the search result is empty, the eventual return from this function is a list with one null element.
Some investigation
response.docs() holds a single non-existing document (looks like this one is filled with the request data).
And, as a result, the return from this function is (as I mentioned above) list of one null element.
Another observation:
I expected that response object will have .hits(), for the actual results are accessible through: response.hits().hits(). But now of that exist.
The only season I started looking into docs() directly is because if this documentation: https://www.elastic.co/guide/en/elasticsearch/reference/master/docs-multi-get.html
There is a lack of Elasticsearch Java API Client docs. They mostly refer to REST API docs.
What is the correct way to get the list of results from mget request?
For now, I am solving this the following way. Will be glad to see if there is a better way, though.
PublicUser[] getAllByIds(Long[] ids) {
MgetRequest request = new MgetRequest.Builder()
.ids(ids.collect { it.toString() }.toList())
.index("users")
.build()
MgetResponse<PublicUser> response = elasticSearchClientProviderService.getClient().mget(
request,
PublicUser.class
)
List<PublicUser> users = []
response.docs().each {
if (it.result().found()) {
users.add(it.result().source())
}
}
users
}
I have a question regarding a small issue that I'm having. I've created a widget that will live on the Service Portal to allow an admin to Accept or Reject requests.
The data for the widget is pulling from the Approvals (approval_approver) table. Under my GlideRecord, I have a query that checks for the state as requested. (Ex. addQuery('state', 'requested'))
To narrow down the search, I tried entering addQuery('sys_id', current.sys_id). When I use this query, my script breaks and I get an error on the Service Portal end.
Here's a sample of the GlideRecord script I've written to Accept.
[//Accept Request
if(input && input.action=="acceptApproval") {
var inRec1 = new GlideRecord('sysapproval_approver');
inRec1.addQuery('state', 'requested');
//inRec1.get('sys_id', current.sys_id);
inRec1.query();
if(inRec1.next()) {
inRec1.setValue('state', 'Approved');
inRec1.setValue('approver', gs.getUserID());
gs.addInfoMessage("Accept Approval Processed");
inRec1.update();
}
}][1]
I've research the web, tried using $sp.getParameter() as a work-around and no change.
I would really appreciate any help or insight on what I can do different to get script to work and filter the right records.
If I understand your question correctly, you are asking how to get the sysId of the sysapproval_approver record from the client-side in a widget.
Unless you have defined current elsewhere in your server script, current is undefined. Secondly, $sp.getParameter() is used to retrieve URL parameters. So unless you've included the sysId as a URL parameter, that will not get you what you are looking for.
One pattern that I've used is to pass an object to the client after the initial query that gets the list of requests.
When you're ready to send input to the server from the client, you can add relevant information to the input object. See the simplified example below. For the sake of brevity, the code below does not include error handling.
// Client-side function
approveRequest = function(sysId) {
$scope.server.get({
action: "requestApproval",
sysId: sysId
})
.then(function(response) {
console.log("Request approved");
});
};
// Server-side
var requestGr = new GlideRecord();
requestGr.addQuery("SOME_QUERY");
requestGr.query(); // Retrieve initial list of requests to display in the template
data.requests = []; // Add array of requests to data object to be passed to the client via the controller
while(requestsGr.next()) {
data.requests.push({
"number": requestsGr.getValue("number");
"state" : requestsGr.getValue("state");
"sysId" : requestsGr.getValue("sys_id");
});
}
if(input && input.action=="acceptApproval") {
var sysapprovalGr = new GlideRecord('sysapproval_approver');
if(sysapprovalGr.get(input.sysId)) {
sysapprovalGr.setValue('state', 'Approved');
sysapprovalGr.setValue('approver', gs.getUserID());
sysapprovalGr.update();
gs.addInfoMessage("Accept Approval Processed");
}
...
Is there any way to track changes to Metadata, like new fields, new entities and so on?
It is difficult to control a very large project in the same environment, so sometimes there are some customization that should not be deployed to productions (Mostly are mistakes or test in a development environment).
And there is a way to know who did that customization?
I am looking to know every possible change, not any in particular.
You have to use the RetrieveMetadataChangesRequest and it is not possible to know who made the change.
This is available only from Microsoft Dynamics CRM 2011 Update Rollup 12
This request is intended to be used to cache information from the metadata and be able to work offline, but we can use it to track changes to metadata in complex projects and complex teams
Examples on internet are not very friendly so this is how you can use the request:
The request can be completed only with filling one parameter
RetrieveMetadataChangesRequest req = new RetrieveMetadataChangesRequest()
{
ClientVersionStamp = null
};
var response = (RetrieveMetadataChangesResponse)service.Execute(req);
The first time you executed this request ClientVersionStamp needs to be null, because there was no request made to the metadata before and there is no ClientVersionStamp. This parameter is the last time you query for metadata changes and if it is null it will bring all customization from all time, so probably this request won't complete on time so we need to tune up.
var EntityFilter = new MetadataFilterExpression(LogicalOperator.And);
EntityFilter.Conditions.Add(new MetadataConditionExpression("SchemaName", MetadataConditionOperator.Equals, "ServiceAppointment"));
var entityQueryExpression = new EntityQueryExpression()
{
Criteria = EntityFilter
};
RetrieveMetadataChangesRequest req = new RetrieveMetadataChangesRequest()
{
Query = entityQueryExpression,
ClientVersionStamp = null
};
var response = (RetrieveMetadataChangesResponse)service.Execute(req);
This will query all metadata changes for "ServiceAppointment", feel free to use the entity you want, but what we need is the ServerTimeStamp from the response, it will looks like "22319800!09/13/2017 16:17:46", if you try to send this time stamp first, it will throw an exception, so it is necessary to query first to get a server time stamp.
Now you can use the request and the time stamp to retrieve all new changes since "22319800!09/13/2017 16:17:46"
RetrieveMetadataChangesRequest req = new RetrieveMetadataChangesRequest()
{
Query = entityQueryExpression,
ClientVersionStamp = #"22319800!09/13/2017 16:17:46"
};
var response = (RetrieveMetadataChangesResponse)service.Execute(req);
You can filter the query to match your needs, only search for specific entities, labels, relationship, keys and attributes or specific properties.
EntityQueryExpression entityQueryExpression = new EntityQueryExpression()
{
Criteria = EntityFilter,
Properties = EntityProperties,
RelationshipQuery = new RelationshipQueryExpression()
{
Properties = RelationshipProperties,
Criteria = RelationshipFilter
},
AttributeQuery = new AttributeQueryExpression()
{
Properties = AttributeProperties,
Criteria = AttributeFilter
}
};
Use this request and implement it the way you need.
A couple more options:
Register a plugin on Publish and Publish All, and track who did
the publish and when. That may help you narrow down who was making
changes, although someone could technically make a change without
publishing it, so not perfect information.
If you're using Dynamics OnPremise, the Metadata tables sometimes store information about who made a change that is not visible through a Metadata retrieve. I've found this to be very spotty though, not all Metadata has a Modified By user stored.
I have an http module where I'm adding a response filter below for compression. This works for all API calls except for 1, the call to MetaData. If I remove the [BreezeController] decoration it works fine. I think it has to do with action filter attribute that converts the string return type into an HttpResponse return type with string content.
The error I'm getting is " Exception message: The stream state of the underlying compression routine is inconsistent."
I've done some testing where a method thats defined to return an HttpResponse works fine. So I think its the scenario where the method is defined to return string, and then the action filter changes it to HttpResponse at runtime.
Any ideas how I can get this to work?
Here's the response filter being added in BeginRequest:
HttpApplication app = (HttpApplication)sender;
// Check the header to see if it can accept compressed output
string encodings = app.Request.Headers.Get("Accept-Encoding");
if (encodings == null)
return;
Stream s = app.Response.Filter;
encodings = encodings.ToLower();
if (encodings.Contains("gzip"))
{
app.Response.Filter = new GZipStream(s, CompressionMode.Compress);
app.Response.AppendHeader("Content-Encoding", "gzip");
}
Don't know the specifics of what you're doing but I know that the [BreezeController] attribute strips out filters and adds back just the ones that breeze wants.
One approach might be to define a separate controller (ModelMetadataController) that only serves the metadata. This controller doesn't have the [BreezeController] attribute; it's a plain old Web API controller.
Then you create a "Breeze controller" (ModelController) with all of the usual methods except the Metadata method.
You call the metadata controller from the client during app launch via MetadataStore.fetchMetadata just to get metadata.
Once you have populated a metadataStore in this fashion, you use it in your EntityManager which sends query and save requests to the "real" Web API data controller.
The client code might look something like this:
var ds = new breeze.DataService({
serviceName: 'breeze/Model' // the breeze query & save controller
});
var ms = new MetadataStore({
namingConvention: breeze.NamingConvention.camelCase, // assuming that's what you want
});
ms.addDataService(ds); // associate the metadata-to-come with the "real" dataService
var manager = new breeze.EntityManager({
dataService: ds,
metadataStore: ms
});
// the fun bit: fetch the metadata from a different controller
var promise = ms.fetchMetadata('breeze/ModelMetadata') // the metadata-only controller!
return promise; // wait on it appropriately
Following the OData samples created by the web api team, my controller has the following for supporting Patch:
public HttpResponseMessage Patch([FromODataUri] int key, Delta<Foo> item)
{
var dbVersion = myDb.GetById(key);
if(dbVersion == null)
throw Request.EntityNotFound();
item.Patch(dbVersion);
myDb.Update(dbVersion);
return Request.CreateResponse(HttpStatusCode.NoContent);
}
and using the auto-generated client (derived from DataServiceContext), I submit a patch request like this:
var foo = svcContainer.Foos.Where (f => f.Id == 1).SingleOrDefault();
foo.Description = "Updated Description";
svcContainer.UpdateObject(foo);
svcContainer.SaveChanges(SaveChangesOptions.PatchOnUpdate);
However, tracing the call in fiddler, I see that all other properties of Foo are serialized and sent to the service. Is that the correct behavior? I expected only the Id and Description to be sent over the wire. Also, if I debug the service method and call
GetChangedPropertyNames on item, all its property names are returned.
Should I be creating some sort of Delta instance on the client?
I understand the disconnected nature of the service and thus the service side does not have a context for tracking changes, but it seems to me the api team added support for patch for a reason, so I'd like to know if the client ought to be invoking the update in a different manner.
Update
The link YiDing provided explains how to create a true PATCH request from the client (using the Microsoft.OData.Client.DataServiceContext created by the Microsoft.OData.Client 6.2.0 and above.
For convenience, here is the code snippet:
var svcContainer = new Default.Container(<svcUri>);
var changeTracker = new DataServiceCollection<Foo>(svcContainer.Foos.Where(f => f.Id == 1));
changeTracker[0].Description = "Patched Description";
svcContainer.SaveChanges();
The DataServiceCollection implements property tracking, and using this pattern, only the updated properties are sent to the service.
Without using DataServiceCollection and simply using
svcContainer.UpdateObject(foo);
svcContainer.SaveChanges();
all properties are still sent over the wire despite documentation to the contrary, at least as of Microsoft.OData.Client 6.7.0
The client side property tracking is now supported from Microsoft.OData.Client version 6.2.0. It will detect only the modified properties of an entity and send the update request as PATCH instead of PUT to meet the requirement of your scenario. Please refer to this blog post for more details:
https://devblogs.microsoft.com/odata/tutorial-sample-client-property-tracking-for-patch/