Can a LINQ query with a where clause on the key to an OData service be done with filter query option instead of a canonical URL? - linq

The problem
I'm trying to query data from an OData V4 service. This is done with a C# client generated by the OData Connected Service extension for Visual Studio. The query is done with a LINQ expression with a where clause. The where clause contains criteria for the key fields of the entity being queried.
The query results in a DataServiceQueryException ("An error occurred while processing this request") with an inner DataServiceClientException ("NotFound"), both from the Microsoft.OData.Client namespace. See below for the full stack trace.
Analysis
Using Fiddler I can see that the request being sent is using a canonical URL (also called a by-key request). If the criteria values do not match any existing data, the response has the code 404 Not Found. This code seems to cause the exception.
When the where clause is changed to also include non-key fields, the request is sent using a $filter query option. In this case, if the criteria values do not match any existing data, the response has the code 200 OK. This does not cause an exception and returns null as result of the LINQ query.
Another workaround is to not use LINQ and instead specify explicitely that a filter query option should be used.
Comparison with the OData reference service TripPin showed that the 404 response does not seem to be the correct response in this case. TripPin instead returns 204 No Content. While the OData specification has several indications that this seems the correct response in this case, I could not find an explicit statement to that effect. In any case, this point is moot since I don't have control over the OData service and can't change its behavior.
Repro details
Unfortunately, the OData service in question is not publicly available. It may be possible to mock such a service or find a public service that shows the same behavior. I have not looked into this since I found a solution (see my answer).
Nevertheless, here is the code that causes the exception:
static void GetData()
{
Uri odataUri = new Uri("https://the-odata-service", UriKind.Absolute);
// Resources is a class generated by the OData Connected Service extension
// and extends Microsoft.OData.Client.DataServiceContext
Resources context = new Resources(odataUri);
var entity = context.Entities.Where(x => x.Key == 1).SingleOrDefault();
}
Producing this request and response:
GET https://the-odata-service/entities(1) HTTP/1.1
HTTP/1.1 404 Not Found
The exception:
Unhandled exception. Microsoft.OData.Client.DataServiceQueryException: An error occurred while processing this request.
---> Microsoft.OData.Client.DataServiceClientException: NotFound
at Microsoft.OData.Client.QueryResult.ExecuteQuery()
at Microsoft.OData.Client.DataServiceRequest.Execute[TElement](DataServiceContext context, QueryComponents queryComponents)
--- End of inner exception stack trace ---
at Microsoft.OData.Client.DataServiceRequest.Execute[TElement](DataServiceContext context, QueryComponents queryComponents)
at Microsoft.OData.Client.DataServiceQuery`1.GetEnumerator()
at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1 source, Boolean& found)
at System.Linq.Enumerable.SingleOrDefault[TSource](IEnumerable`1 source)
at Microsoft.OData.Client.DataServiceQueryProvider.ReturnSingleton[TElement](Expression expression)
at System.Linq.Queryable.SingleOrDefault[TSource](IQueryable`1 source)
at <my test program in the GetData method in the line of the SingleOrDefault call>
If I change the LINQ to
var entity = context.Entities
.Where(x =>
x.Key == 1
&& x.AnotherNonKeyField == "2")
.SingleOrDefault();
I get
GET https://the-odata-service/Entities?$filter=Key%20eq%201%20and%20AnotherNonKeyField%20eq%20'2'&$top=2 HTTP/1.1
HTTP/1.1 200 OK
{
"#odata.context":"https://the-odata-service/$metadata#Entities","value":[
]
}
which does not result in an exception, but entity being null.
The question
To sum up, while there are workarounds, I would prefer if I could query the odata service with LINQ and without having to add dummy criteria (which would not always be possible). Is there a way to do that?

TLDR
The KeyComparisonGeneratesFilterQuery property of the DataServiceContext can be used to generate a $filter query option.
Some more background
I spent some time researching this issue in context of LINQ and the client that was generated. In hindsight, it is obvious that the Microsoft OData Client library would have been a better place to start, since it throws the exception. But who has time to read a stack trace when instead you can furiously google and debug for a few hours *sigh* ?
Eventually I found my way to issue #851 DataServiceQuery makes a "by key" request when Where clause compares just the ID, causing exception instead of empty result if the entity is not found. and pull request #1762 Enable Where clause to generate $filter query options for key predicates. Especially the later does a much better job of explaining the purpose and how to use the KeyComparisonGeneratesFilterQuery property than the documentation.
With that, the above code can be fixed like this:
static void GetData()
{
Uri odataUri = new Uri("https://the-odata-service", UriKind.Absolute);
// Resources is a class generated by the OData Connected Service extension
// and extends Microsoft.OData.Client.DataServiceContext
Resources context = new Resources(odataUri);
context.KeyComparisonGeneratesFilterQuery = true;
var entity = context.Entities.Where(x => x.Key == 1).SingleOrDefault();
}
Which produces
GET https://the-odata-service/Entities?$filter=Key%20eq%201&$top=2 HTTP/1.1
HTTP/1.1 200 OK
{
"#odata.context":"https://the-odata-service/$metadata#Entities","value":[
]
}

Related

How to correctly get the results from MgetResponse object?

In our app, we are synchronizing some of the data to elasticsearch, and some of this data is users' records. The app is grails 5.1 and we are using Elasticsearch Java API Client for elasticsearch integration.
The indexing is working perfectly fine, and an example of user data looks like this:
Now, we have this following function that suppose to get the list of users by their ids:
PublicUser[] getAllByIds(Long[] ids) {
MgetRequest request = new MgetRequest.Builder()
.ids(ids.collect { it.toString() }.toList())
.index("users")
.build()
MgetResponse<PublicUser> response = elasticSearchClientProviderService.getClient().mget(
request,
PublicUser.class
)
response.docs().collect {
it.result().source()
}
}
And when the response holds at least one user record, we are getting a list of PulicUser objects -> as expected.
However, if the search result is empty, the eventual return from this function is a list with one null element.
Some investigation
response.docs() holds a single non-existing document (looks like this one is filled with the request data).
And, as a result, the return from this function is (as I mentioned above) list of one null element.
Another observation:
I expected that response object will have .hits(), for the actual results are accessible through: response.hits().hits(). But now of that exist.
The only season I started looking into docs() directly is because if this documentation: https://www.elastic.co/guide/en/elasticsearch/reference/master/docs-multi-get.html
There is a lack of Elasticsearch Java API Client docs. They mostly refer to REST API docs.
What is the correct way to get the list of results from mget request?
For now, I am solving this the following way. Will be glad to see if there is a better way, though.
PublicUser[] getAllByIds(Long[] ids) {
MgetRequest request = new MgetRequest.Builder()
.ids(ids.collect { it.toString() }.toList())
.index("users")
.build()
MgetResponse<PublicUser> response = elasticSearchClientProviderService.getClient().mget(
request,
PublicUser.class
)
List<PublicUser> users = []
response.docs().each {
if (it.result().found()) {
users.add(it.result().source())
}
}
users
}

Return different HTTP status codes from Application Service

I'm using the ASP.NET Boilerplate Framework and I've created a custom app service that inherits from AsyncCrudAppServiceBase and I've overridden the Get function.
public override async Task<ExampleDto> Get(EntityDto<int> input)
{
var example = await Repository.GetAll().FirstOrDefault(x => x.Id == input.Id);
return ObjectMapper.Map<ExampleDto>(example);
}
However, because I'm using FirstOrDefault if the record doesn't exist it returns null and the HTTP Status is 200 instead of 404, which I want.
If I change the LINQ function to First or, ideally, Single then and exception is thrown but the HTTP Status is 500.
I can check if the record exists but how do I return a different HTTP status from an Application Service?
You should consider using repository get method in your app service.
See
https://aspnetboilerplate.com/Pages/Documents/Repositories#base-repository-methods
Reason being Repository.Get() will throw EntityNotFoundException when FirstOrDefault return null.
See https://github.com/aspnetboilerplate/aspnetboilerplate/blob/6bce9a0572170e8daf1b21ee982a869714579e2d/src/Abp/Domain/Repositories/AbpRepositoryBase.cs#L73-L82
And when Abp wraps the response, EntityNotFoundException will have NotFound as the HttpStatusCode
See https://github.com/aspnetboilerplate/aspnetboilerplate/blob/15e5f33885442bfd0996936191fd964144a95652/src/Abp.AspNetCore/AspNetCore/Mvc/ExceptionHandling/AbpExceptionFilter.cs#L99-L102

What is the correct way to call patch from an OData client in Web Api 2

Following the OData samples created by the web api team, my controller has the following for supporting Patch:
public HttpResponseMessage Patch([FromODataUri] int key, Delta<Foo> item)
{
var dbVersion = myDb.GetById(key);
if(dbVersion == null)
throw Request.EntityNotFound();
item.Patch(dbVersion);
myDb.Update(dbVersion);
return Request.CreateResponse(HttpStatusCode.NoContent);
}
and using the auto-generated client (derived from DataServiceContext), I submit a patch request like this:
var foo = svcContainer.Foos.Where (f => f.Id == 1).SingleOrDefault();
foo.Description = "Updated Description";
svcContainer.UpdateObject(foo);
svcContainer.SaveChanges(SaveChangesOptions.PatchOnUpdate);
However, tracing the call in fiddler, I see that all other properties of Foo are serialized and sent to the service. Is that the correct behavior? I expected only the Id and Description to be sent over the wire. Also, if I debug the service method and call
GetChangedPropertyNames on item, all its property names are returned.
Should I be creating some sort of Delta instance on the client?
I understand the disconnected nature of the service and thus the service side does not have a context for tracking changes, but it seems to me the api team added support for patch for a reason, so I'd like to know if the client ought to be invoking the update in a different manner.
Update
The link YiDing provided explains how to create a true PATCH request from the client (using the Microsoft.OData.Client.DataServiceContext created by the Microsoft.OData.Client 6.2.0 and above.
For convenience, here is the code snippet:
var svcContainer = new Default.Container(<svcUri>);
var changeTracker = new DataServiceCollection<Foo>(svcContainer.Foos.Where(f => f.Id == 1));
changeTracker[0].Description = "Patched Description";
svcContainer.SaveChanges();
The DataServiceCollection implements property tracking, and using this pattern, only the updated properties are sent to the service.
Without using DataServiceCollection and simply using
svcContainer.UpdateObject(foo);
svcContainer.SaveChanges();
all properties are still sent over the wire despite documentation to the contrary, at least as of Microsoft.OData.Client 6.7.0
The client side property tracking is now supported from Microsoft.OData.Client version 6.2.0. It will detect only the modified properties of an entity and send the update request as PATCH instead of PUT to meet the requirement of your scenario. Please refer to this blog post for more details:
https://devblogs.microsoft.com/odata/tutorial-sample-client-property-tracking-for-patch/

servicestack - caching a service response using redis

I have a servicestack service which when called via the browser (restful) Url ex:http://localhost:1616/myproducts, it works fine.
The service method has RedisCaching enabled. So first time it hits the data repository and caches it for subsequent use.
My problem is when I try calling it from a c# client via Soap12ServiceClient. It returns the below error:
Error in line 1 position 183. Expecting element '<target response>'
from namespace 'http://schemas.datacontract.org/2004/07/<target namespace>'..
Encountered 'Element' with name 'base64Binary',
namespace 'http://schemas.microsoft.com/2003/10/Serialization/'.
Below is my Client code:
var endpointURI = "http://mydevelopmentapi.serverhostingservices.com:1616/";
using (IServiceClient client = new Soap12ServiceClient(endpointURI))
{
var request = new ProductRequest { Param1 = "xy23432"};
client.Send<ProductResponse>(request);
}
It seems that the soapwsdl used is giving the problem, but I appear to have used the defaults as generated by servicestack..
Any help will be much appreciated.
Update
I was able over come this error by changing the cache code at the service end:
Code that returned error at client end:
return RequestContext.ToOptimizedResultUsingCache(this.CacheClient, cacheKey,
() =>
new ProductResponse(){CreateDate = DateTime.UtcNow,
products = new productRepository().Getproducts(request)
});
Code that works now:
var result = this.CacheClient.Get<ProductResponse>(cacheKey);
if (result == null)
{
this.CacheClient.Set<ProductResponse>(cacheKey, productResult);
result = productResult;
}
return result;
But I am still curious to know why the first method (RequestContext.ToOptimizedResultUsingCache) returned error at c# client?
But I am still curious to know why the first method (RequestContext.ToOptimizedResultUsingCache) returned error at c# client?
From what I can tell, the ToOptimizedResultUsingCache is trying to pull a specific format (xml, html, json, etc) out of the cache based on the RequestContext's ResponseContentType (see code here and here). When using the Soap12ServiceClient the ResponseContentType is text/html (not sure if this is correct/intentional within ServiceStack). So what ToOptimizedResultUsingCache is pulling out of the cache is a string of html. The html string is being returned to the Soap12ServiceClient and causing an exception.
By pulling directly out of the cache you are bypassing ToOptimizedResultUsingCache's 'format check' and returning something the Soap12ServiceClient can handle.
** If you are using Redis and creating your key with UrnId.Create method you should see a key like urn:ProductResponse:{yourkey}.html
Thanks for your response paaschpa.
I revisited the code and I was able to fix it. Since your response gave me the direction, I have accepted your answer. Below is my fix.
I moved the return statement from RequestContext to the response DTO.
Code which throws error when used via c# client (code was returning entire requestcontext):
return RequestContext.ToOptimizedResultUsingCache(this.CacheClient, cacheKey,
() =>
new ProductResponse(){CreateDate = DateTime.UtcNow,
products = new productRepository().Getproducts(request)
});
Fixed Code (return moved to response DTO):
RequestContext.ToOptimizedResultUsingCache(this.CacheClient, cacheKey,
() => {
return new ProductResponse(){CreateDate = DateTime.UtcNow,
products = new productRepository().Getproducts(request)
}
});

Error updating/posting to Twitter

I tried with this code to post on the wall (Twitter) of a user
if (credentials.ConsumerKey == null || credentials.ConsumerSecret == null)
{
credentials.ConsumerKey = ConfigurationManager.AppSettings["twitterConsumerKey"];
credentials.ConsumerSecret = ConfigurationManager.AppSettings["twitterConsumerSecret"];
}
auth = new MvcAuthorizer
{
Credentials = credentials
};
auth.CompleteAuthorization(Request.Url);
if (!auth.IsAuthorized)
{
Uri specialUri = new Uri(Request.Url.ToString());
return auth.BeginAuthorization(specialUri);
}
twitterCtx = new TwitterContext(auth);
twitterCtx.UpdateStatus("Welcome");
Probleme : the first test goes well, I posted on the wall the second test shows this error:
Error while querying Twitter.
someone can help me to solve this problem
Thanks,
LINQ to Twitter throws a TwitterQueryException when detecting an error from Twitter. You can look at the Response property of the TwitterQueryException instance to see the message that Twitter is sending back. Another way to get a complete view of the query and Twitter's response is to use Fiddler2 to view the HTTP traffic and see what Twitter's response is.
In your case, I'm looking at the fact that you said the first post worked, but the second one doesn't. This might be caused by posting a duplicate message, which Twitter doesn't allow. If you look at any of the LINQ to Twitter demos that post a message, you'll notice that they contain a DateTime, which practically guarantees that the text of each message will be different. So, In your case, you could try this:
twitterCtx.UpdateStatus("Welcome - " + DateTime.Now.ToString());
You're welcome to provide more info by posting the contents of the Response property from the TwitterQueryException. Also, for more info, I've begun a FAQ at http://linqtotwitter.codeplex.com/wikipage?title=LINQ%20to%20Twitter%20FAQ&referringTitle=Documentation.

Resources