I am using openiddict library from https://github.com/openiddict.
I need to list all the scopes where display name matches specified filter criteria on request to api. However, I can't figure out how to consume the ListAsync method on IOpenIddictScopeManager .
Method signature is :
IAsyncEnumerable<TResult> ListAsync<TResult>(
Func<IQueryable<**object**>,IQueryable<TResult>> query, CancellationToken cancellationToken = default);
Scope manager uses a store internally and delegate calls to it.
IOpenIddictScopeStore.cs
Method signature is :
IAsyncEnumerable<TResult> ListAsync<TState, TResult>(
Func<IQueryable<**TScope**>, TState, IQueryable<TResult>> query,
TState state, CancellationToken cancellationToken);
Implementation for the MongDB store can be located at OpenIddictMongoDbScopeStore.
Update
I am able to get it working with SQL store :
Func<IQueryable<object>, IQueryable<object>> query =
(scopes) => scopes.Where(s => (s as OpenIddictEntityFrameworkCoreScope).DisplayName.StartsWith(request.ScopesFilter)
);
await scopeManager.ListAsync<object>(query, CancellationToken.None)
However, same thing won't work for MONGO store.
Func<IQueryable<object>, IQueryable<object>> query =
(scopes) => scopes.Where(s => (s as OpenIddictMongoDbScope).DisplayName.StartsWith(request.ScopesFilter)
);
await scopeManager.ListAsync<object>(query, CancellationToken.None)
The mongo store fails with below exceptions
a. when using above query with CountAsync(query)
"Unable to cast object of type 'MongoDB.Driver.Linq.MongoQueryableImpl2[OpenIddict.MongoDb.Models.OpenIddictMongoDbScope,System.Object]' to type 'MongoDB.Driver.Linq.IMongoQueryable1[OpenIddict.MongoDb.Models.OpenIddictMongoDbScope]'."
b. when using above query with ListAsync(query)
Unable to cast object of type 'AsyncStateMachineBox1[MongoDB.Driver.IAsyncCursor1[OpenIddict.MongoDb.Models.OpenIddictMongoDbScope],MongoDB.Driver.MongoCollectionImpl1+<UsingImplicitSessionAsync>d__1061[OpenIddict.MongoDb.Models.OpenIddictMongoDbScope,MongoDB.Driver.IAsyncCursor1[OpenIddict.MongoDb.Models.OpenIddictMongoDbScope]]]' to type 'System.Threading.Tasks.Task1[MongoDB.Driver.IAsyncCursor1[System.Object]]'. at MongoDB.Driver.Linq.MongoQueryProviderImpl1.ExecuteAsync[TResult](Expression expression, CancellationToken cancellationToken)
Seems like an issue with MongoDb driver. I upgraded MongDb driver binaries to a recent version and the query started working as expected.
Related
The problem
I'm trying to query data from an OData V4 service. This is done with a C# client generated by the OData Connected Service extension for Visual Studio. The query is done with a LINQ expression with a where clause. The where clause contains criteria for the key fields of the entity being queried.
The query results in a DataServiceQueryException ("An error occurred while processing this request") with an inner DataServiceClientException ("NotFound"), both from the Microsoft.OData.Client namespace. See below for the full stack trace.
Analysis
Using Fiddler I can see that the request being sent is using a canonical URL (also called a by-key request). If the criteria values do not match any existing data, the response has the code 404 Not Found. This code seems to cause the exception.
When the where clause is changed to also include non-key fields, the request is sent using a $filter query option. In this case, if the criteria values do not match any existing data, the response has the code 200 OK. This does not cause an exception and returns null as result of the LINQ query.
Another workaround is to not use LINQ and instead specify explicitely that a filter query option should be used.
Comparison with the OData reference service TripPin showed that the 404 response does not seem to be the correct response in this case. TripPin instead returns 204 No Content. While the OData specification has several indications that this seems the correct response in this case, I could not find an explicit statement to that effect. In any case, this point is moot since I don't have control over the OData service and can't change its behavior.
Repro details
Unfortunately, the OData service in question is not publicly available. It may be possible to mock such a service or find a public service that shows the same behavior. I have not looked into this since I found a solution (see my answer).
Nevertheless, here is the code that causes the exception:
static void GetData()
{
Uri odataUri = new Uri("https://the-odata-service", UriKind.Absolute);
// Resources is a class generated by the OData Connected Service extension
// and extends Microsoft.OData.Client.DataServiceContext
Resources context = new Resources(odataUri);
var entity = context.Entities.Where(x => x.Key == 1).SingleOrDefault();
}
Producing this request and response:
GET https://the-odata-service/entities(1) HTTP/1.1
HTTP/1.1 404 Not Found
The exception:
Unhandled exception. Microsoft.OData.Client.DataServiceQueryException: An error occurred while processing this request.
---> Microsoft.OData.Client.DataServiceClientException: NotFound
at Microsoft.OData.Client.QueryResult.ExecuteQuery()
at Microsoft.OData.Client.DataServiceRequest.Execute[TElement](DataServiceContext context, QueryComponents queryComponents)
--- End of inner exception stack trace ---
at Microsoft.OData.Client.DataServiceRequest.Execute[TElement](DataServiceContext context, QueryComponents queryComponents)
at Microsoft.OData.Client.DataServiceQuery`1.GetEnumerator()
at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1 source, Boolean& found)
at System.Linq.Enumerable.SingleOrDefault[TSource](IEnumerable`1 source)
at Microsoft.OData.Client.DataServiceQueryProvider.ReturnSingleton[TElement](Expression expression)
at System.Linq.Queryable.SingleOrDefault[TSource](IQueryable`1 source)
at <my test program in the GetData method in the line of the SingleOrDefault call>
If I change the LINQ to
var entity = context.Entities
.Where(x =>
x.Key == 1
&& x.AnotherNonKeyField == "2")
.SingleOrDefault();
I get
GET https://the-odata-service/Entities?$filter=Key%20eq%201%20and%20AnotherNonKeyField%20eq%20'2'&$top=2 HTTP/1.1
HTTP/1.1 200 OK
{
"#odata.context":"https://the-odata-service/$metadata#Entities","value":[
]
}
which does not result in an exception, but entity being null.
The question
To sum up, while there are workarounds, I would prefer if I could query the odata service with LINQ and without having to add dummy criteria (which would not always be possible). Is there a way to do that?
TLDR
The KeyComparisonGeneratesFilterQuery property of the DataServiceContext can be used to generate a $filter query option.
Some more background
I spent some time researching this issue in context of LINQ and the client that was generated. In hindsight, it is obvious that the Microsoft OData Client library would have been a better place to start, since it throws the exception. But who has time to read a stack trace when instead you can furiously google and debug for a few hours *sigh* ?
Eventually I found my way to issue #851 DataServiceQuery makes a "by key" request when Where clause compares just the ID, causing exception instead of empty result if the entity is not found. and pull request #1762 Enable Where clause to generate $filter query options for key predicates. Especially the later does a much better job of explaining the purpose and how to use the KeyComparisonGeneratesFilterQuery property than the documentation.
With that, the above code can be fixed like this:
static void GetData()
{
Uri odataUri = new Uri("https://the-odata-service", UriKind.Absolute);
// Resources is a class generated by the OData Connected Service extension
// and extends Microsoft.OData.Client.DataServiceContext
Resources context = new Resources(odataUri);
context.KeyComparisonGeneratesFilterQuery = true;
var entity = context.Entities.Where(x => x.Key == 1).SingleOrDefault();
}
Which produces
GET https://the-odata-service/Entities?$filter=Key%20eq%201&$top=2 HTTP/1.1
HTTP/1.1 200 OK
{
"#odata.context":"https://the-odata-service/$metadata#Entities","value":[
]
}
I am updating a record on my mongoDb database.
The update is performed in a wrapper for a series of reason, it goes like:
Contact contact = generateContact();
UpdateResult updateResult = performUpdateOne("collection",new Document("field",fieldValue), new Document("$push", new Document("contacts", contact)),updateOptions);
Please keep in mind the new Document("$set", new Document("contacts", contact)) parameter because is the only mongoDb document that contains a reference to a Pojo, this field is used in the the method performUpdateOne
private UpdateResult performUpdateOne(String collectionName, Document filter, Document set, UpdateOptions updateOptions) {
...
...
LOG.debugf("Performing MongoDb UpdateOne command, collection[%s] query[%s] update[%s].", collectionName, filter.toJson(), set.toJson());
UpdateResult updateResult = collection.updateOne(filter, set, updateOptions);
The set.toJson() call gives me the exception:
Exception: : org.bson.codecs.configuration.CodecConfigurationException: Can't find a codec for class ...Contact. at org.bson.internal.CodecCache.lambda$getOrThrow$1(CodecCache.java:52)
If I comment the LOG:debugf line the collection.updateOne(filter, set, updateOptions) gives me no problem.
Why is the set document serialized correctly in the updateOne method while giving an error when i call .toJson() on it?
Thanks a lot
I am using mongo sync java driver 4.3.4
When I try to run this code, it throws this error:
System.AggregateException: One or more errors occurred. (A second operation started on this context before a previous operation completed. This is usually caused by different threads using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913.)
---> System.InvalidOperationException: A second operation started on this context before a previous operation completed. This is usually caused by different threads using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913.
var returnFacilitators = facilitators.Select(async f => new
{
Name = f.DisplayNameLastFirstDegreeTitles(),
Value = f.FacilitatorId,
FirstName = f.FirstName,
LastName = f.LastName,
DegreeTitles = f.DegreeTitles,
CanDelete = await facilitatorService.CanRemoveFacilitatorAsync(f.FacilitatorId)
}).ToList();
CanRemoveFacilitator is an async Task<bool>.
NEST/Elasticsearch.Net version:5.6.5
Elasticsearch version:5.4.3
We are trying to fetch result from our index using the LowLevelClient. We are using the below SearchAsync API
var searchDescriptor = new SearchDescriptor<MyType>()
.Type("mytype")
.Index("myindex")
.Query(....)
.Aggregation(ag => ag.Terms(... Aggregation(ag1 => ag1.Min(...TopHits(...)))));
var memoryStream = new MemoryStream();
_client.Serializer.Serialize(searchDescriptor, memoryStream);
var response = await _client.LowLevel.SearchAsync<byte[]>(memoryStream.ToArray()).ConfigureAwait(false);
//_client - instance of Nest.ElasticClient
//Next Step - Deserialize the response
This is giving me results from other indices also(a combination of results from the various indices) and my deserialization is breaking. The client is ignoring type and index name and calling POST /_search API instead of POST /myindex/mytype/_search on the elastic search
Note:
We need to call a lower-level client because we are using a custom deserializer for performance concern
What is the issue here?
Found a workaround
The SearchAsync<>() method have overloaded method _client.LowLevel.SearchAsync<T>(string indexName, string typeName, T t)
Passing the index name and type name will narrow to search to that particular index.
But the question still remains, why it is not taking the index and type name from SearchDescriptor object.
The following code snippet is a MoreLikeThis query built using NEST API:
private class Temp
{
public string Content { get; set; }
public string TextToSearch { get; set; }
}
var temp = new Temp
{
TextToSearch = "empire",
};
var response = await model.ElasticClient.SearchAsync<Temp>(s => s
.Query(q => q
.MoreLikeThis(qd => qd
.Like(l => l.Text(temp.TextToSearch))
.MinTermFrequency(1)
.MinDocumentFrequency(1)
.Fields(fd => fd.Fields(r => r.Content)))));
After executing this code snippet response.Documents did not return any records. But when the following JSON payload is POSTed by POSTMAN, the results are received successfully:
{"query":{"more_like_this":{"fields":["content"],"like":["advanced technology"],"min_doc_freq":1,"min_term_freq":1}}}
This payload is generated by the C# code snippet above when enabling audit trail. While the credentials are passed in both cases properly why the NEST API version 6.5.0 does not receive documents from the elastic search instance?
Is there a bug in the library or we're missing a point?
Besides the TextToSearch being "empire" in the C# example and "advanced technology" in the JSON query DSL example, I strongly suspect that the issue here is that of the index and type being targeted in the NEST case.
When no index and type are provided in the API call:
For index,
Will look to see if there is a default index to use for Temp type configured with DefaultMappingFor<T> on ConnectionSettings
If no default index for Temp, will use the DefaultIndex configured on ConnectionSettings
If no default index is configured on ConnectionSettings, the API call will not be made and NEST will throw an exception to indicate that it does not have enough information to make the API call.
For type,
Will look to see if there is a default type name to use for Temp type configured with DefaultMappingFor<T> on ConnectionSettings
Will look to see if a type name convention is configured using DefaultTypeNameInferrer on ConnectionSettings. If none is configured, or the delegate it is configured with returns null or "" for a given type, then will continue
Will look to see if a default type name is specified with DefaultTypeName on ConnectionSettings. If none is specified, a type name will be inferred for a POCO type by lowercasing the type name. For Temp, this will be temp.
So, assuming you have a default index configured and no convention for type names, the request URI for your NEST example will be
<configured uri>/<default index>/temp/_search
which probably does not match what you are using in Postman.
Check out the documentation to see more details about Index name inference and Type name inference.