I store an object in mongodb which contains a Guid. Mongodb converts the Guid to a binary value
{
"_id" : ObjectId("52cf4a467b302a4797db23e8"),
"name" : "test",
"guid" : new BinData(3, "qZ8PQdmDv0+K500wnj6skA=="),
}
I get an empty result set when I use the Guid in a Linq expression and the count is always 0.
var queryable = _database.GetCollection<MyObject>("myname").AsQueryable();
var guid = new Guid("410f9fa9-83d9-4fbf-8ae7-4d309e3eac90");
var count = queryable.Where(x => x.Guid == guid).Count();
The reason is that Linq generates the following request
count: "member_variables", query: { panelid: "410f9fa9-83d9-4fbf-8ae7-4d309e3eac90" }
but the request should be
count: "member_variables", query: { panelid: new BinData(3, "qZ8PQdmDv0+K500wnj6skA==") }
I tried various Linq expressions, but none worked.
queryable.Where(x => x.Guid.ToString() == guid.ToString())
throws the exception: "Unable to determine the serialization information for the expression: x.Guid.ToString()."
queryable.Where(x => x.Guid == new BsonBinaryData(guid));
throws the exception: "Unsupported where clause: ((BsonBinaryData)x.Guid == UuidLegacy:0x24f7ceb84f06d143b6426e8f01cb7825)."
How can I request the documents?
Note: I need to use IQuerable and can't use Mongodb queries.
Related
I am trying to create an Elasticsearch MLT query using NEST's object initializer syntax. However, the final query when serialized, is ONLY missing the MLT part of it. Every other query is present though.
When inspecting the query object, the MLT is present. It's just not getting serialized.
I wonder what I may be doing wrong.
I also noticed that when I add Fields it works. But I don't believe fields is a mandatory property here that when it is not set, then the MLT query is ignored.
The MLT query is initialized like this;
new MoreLikeThisQuery
{
Like = new[]
{
new Like(new MLTDocProvider
{
Id = parameters.Id
}),
}
}
MLTDocProvider implements the ILikeDocument interface.
I expect the serialized query to contain the MLT part, but it is the only part that is missing.
This looks like a bug in the conditionless behaviour of more like this query in NEST; I've opened an issue to address. In the meantime, you can get the desired behaviour by marking the MoreLikeThisQuery as verbatim, which will override NEST's conditionless behaviour
var client = new ElasticClient();
var parameters = new
{
Id = 1
};
var searchRequest = new SearchRequest<Document>
{
Query = new MoreLikeThisQuery
{
Like = new[]
{
new Like(new MLTDocProvider
{
Id = parameters.Id
}),
},
IsVerbatim = true
}
};
var searchResponse = client.Search<Document>(searchRequest);
which serializes as
{
"query": {
"more_like_this": {
"like": [
{
"_id": 1
}
]
}
}
}
I want to query that returns document that two fields of it are equal
I found mongodb raw query from this question :
db.coll.find({ $where : "this.field1 == this.field2" } );
How can I perform it with spring criteria:
criteria = criteria.andOperator(
Criteria.where("successfulSent").is("true"),
Criteria.where("this.fieldOne == this.fieldTwo"));
but its not working beacuse generated query become :
{ ... "$and" : [ { "successfulSent" : "true"} , { "this.fieldOne == this.fieldOne " : { }}]}
You can try on this way:
Criteria.where("$where").is("this.field1 == this.field2")
Query toString() will be:
Query: { "$where" : "this.cts == this.uts"}, Fields: null, Sort: null
I am trying to build a query with Criteria Builder(LIKE), to look for a string in JSONARRAY field like this:
[
{
"family_class": "Electric",
"family_name": "lightBulb"
},
{
"family_class": "Others",
"family_name": "Oil"
}
]
one option would be to look for the family_name attribute, or maybe check if it contains the string there.
if (residues != null && residues.length > 0) {
List<Predicate> predicates = new ArrayList<Predicate>();
for (String residue : residues) {
predicates.add(cb.like(root.get("jsonColumn"), residue.toLowerCase()));
}
cr.select(root).where(predicates.toArray(new Predicate[] {}));
Query<SyncCollectionPoints> q = sess.createQuery(cr);
List<SyncCollectionPoints> result= q.getResultList();
This is the error i get:
Unrecognized token 'oil': was expecting ('true', 'false' or 'null')
All i want is to return the lines that have that string in the jsonColumn field.
I got it to work like this:
#Formula(value = "lower(jsonColumn::text)")
private String residuesToSearch;
just a simple cast did the trick
I have a JArray with this configuration
[
{
"name" : "text",
"age" : 32
},
{
"name" : "text2",
"age" : 33
},
]
and I want to use LINQ query to select a JArray containing JObjects with just the key value of a specified given key.
For example:
GetCollectionOfPropertiesByKey("name");
this would result in:
[
{
"name" : "text"
},
{
"name" : "text2"
},
]
using the same logic,
GetCollectionOfPropertiesByKey("age");
would result in:
[
{
"age" : 32
},
{
"age" : 33
},
]
You didn't specify whether you needed case-invariant filtering for properties by name, so here is an extension method with an optional StringComparison argument:
public static class JsonExtensions
{
public static JArray GetCollectionOfPropertiesByKey(this JArray array, string key, StringComparison comparison = StringComparison.Ordinal)
{
if (array == null)
return null;
var query = array
// Filter out array items that are not objects
.OfType<JObject>()
// Select the value JToken with the specified key using the specified StringComparison
.Select(o => o.GetValue(key, comparison))
// Filter for null (property was not found)
.Where(v => v != null)
// And select a new JObject containing just the JPropery parent of the selected value
.Select(v => new JObject(v.Parent));
return new JArray(query);
}
}
If a case-insensitive match is required, pass StringComparison.OrdinalIgnoreCase.
Notes:
The query filters for all items in the input array that are of type JObject, then uses JObject.GetValue(String, StringComparison) to get the corresponding value by key name.
Then, if a value is found, it constructs a new JObject using the parent of the selected value, which should be a JProperty.
If case insensitivity is never required, you could simplify the query a bit by using JObject.Property(String) to get the JProperty by key name, and then constructing a JObject containing it, like so:
var query = array
// Filter out array items that are not objects
.OfType<JObject>()
// Select the JProperty with the specified key
.Select(o => o.Property(key))
// Filter for null (property was not found)
.Where(p => p != null)
// And select a new JObject with just this property
.Select(p => new JObject(p));
Sample working .Net fiddle.
I have three main questions I need help answering.
How do you correctly map and store a nested map?
How do you search a nested part of a document?
How do you bulk index?
I'm using Nest version 2 and have been looking over the new documentation which can be found Here. The documentation has been useful in creating certain parts of the code but unfortunately doesn't explain how they fit together.
Here is the class I'm trying to map.
[ElasticsearchType(Name = "elasticsearchproduct", IdProperty = "ID")]
public class esProduct
{
public int ID { get; set; }
[Nested]
public List<PriceList> PriceList { get; set; }
}
[ElasticsearchType(Name = "PriceList")]
public class PriceList
{
public int ID { get; set; }
public decimal Price { get; set; }
}
and my mapping code
var node = new Uri(HOST);
var settings = new ConnectionSettings(node).DefaultIndex("my-application");
var client = new ElasticClient(settings);
var map = new CreateIndexDescriptor("my-application")
.Mappings(ms => ms
.Map<esProduct>(m => m
.AutoMap()
.Properties(ps => ps
.Nested<PriceList>(n => n
.Name(c => c.PriceList)
.AutoMap()
)
)
)
);
var response = client.Index(map);
This is the response I get:
Valid NEST response built from a succesful low level call on POST: /my-application/createindexdescriptor
So that seems to work. next index.
foreach (DataRow dr in ProductTest.Tables[0].Rows)
{
int id = Convert.ToInt32(dr["ID"].ToString());
List<PriceList> PriceList = new List<PriceList>();
DataRow[] resultPrice = ProductPriceTest.Tables[0].Select("ID = " + id);
foreach (DataRow drPrice in resultPrice)
{
PriceList.Add(new PriceList
{
ID = Convert.ToInt32(drPrice["ID"].ToString()),
Price = Convert.ToDecimal(drPrice["Price"].ToString())
}
esProduct product = new esProduct
{
ProductDetailID = id,
PriceList = PriceList
};
var updateResponse = client.Update<esProduct>(DocumentPath<esProduct>.Id(id), descriptor => descriptor
.Doc(product)
.RetryOnConflict(3)
.Refresh()
);
var index = client.Index(product);
}
}
Again this seems to work but when I come to search it does seem to work as expected.
var searchResults = client.Search<esProduct>(s => s
.From(0)
.Size(10)
.Query(q => q
.Nested(n => n
.Path(p => p.PriceList)
.Query(qq => qq
.Term(t => t.PriceList.First().Price, 100)
)
)
));
It does return results but I was expecting
.Term(t => t.PriceList.First().Price, 100)
to look move like
.Term(t => t.Price, 100)
and know that is was searching the nested PriceList class, is this not the case?
In the new version 2 documentation I can't find the bulk index section. I tried using this code
var descriptor = new BulkDescriptor();
***Inside foreach loop***
descriptor.Index<esProduct>(op => op
.Document(product)
.Id(id)
);
***Outside foreach loop***
var result = client.Bulk(descriptor);
which does return a success response but when I search I get no results.
Any help would be appreciated.
UPDATE
After a bit more investigation on #Russ advise I think the error must be with my bulk indexing of a class with a nested object.
When I use
var index = client.Index(product);
to index each product I can use
var searchResults = client.Search<esProduct>(s => s
.From(0)
.Size(10)
.Query(q => q
.Nested(n => n
.Path(p => p.PriceList)
.Query(qq => qq
.Term(t => t.PriceList.First().Price, 100)
)
)
)
);
to search and return results, but when I bulk index this no long works but
var searchResults = client.Search<esProduct>(s => s
.From(0)
.Size(10)
.Query(q => q
.Term(t => t.PriceList.First().Price, 100)
)
);
will work, code b doesn't work on the individual index method. Does anyone know why this has happened?
UPDATE 2
From #Russ suggested I have taken a look at the mapping.
the code I'm using to index is
var map = new CreateIndexDescriptor(defaultIndex)
.Mappings(ms => ms
.Map<esProduct>(m => m
.AutoMap()
.Properties(ps => ps
.Nested<PriceList>(n => n
.Name(c => c.PriceList)
.AutoMap()
)
)
)
);
var response = client.Index(map);
Which is posting
http://HOST/fresh-application2/createindexdescriptor {"mappings":{"elasticsearchproduct":{"properties":{"ID":{"type":"integer"},"priceList":{"type":"nested","properties":{"ID":{"type":"integer"},"Price":{"type":"double"}}}}}}}
and on the call to http://HOST/fresh-application2/_all/_mapping?pretty I'm getting
{
"fresh-application2" : {
"mappings" : {
"createindexdescriptor" : {
"properties" : {
"mappings" : {
"properties" : {
"elasticsearchproduct" : {
"properties" : {
"properties" : {
"properties" : {
"priceList" : {
"properties" : {
"properties" : {
"properties" : {
"ID" : {
"properties" : {
"type" : {
"type" : "string"
}
}
},
"Price" : {
"properties" : {
"type" : {
"type" : "string"
}
}
}
}
},
"type" : {
"type" : "string"
}
}
},
"ID" : {
"properties" : {
"type" : {
"type" : "string"
}
}
}
}
}
}
}
}
}
}
}
}
}
}
fresh-application2 returned mapping doesn't mention nested type at all, which I'm guessing is the issue.
The mapping my working nested query looks more like this
{
"my-application2" : {
"mappings" : {
"elasticsearchproduct" : {
"properties" : {
"priceList" : {
"type" : "nested",
"properties" : {
"ID" : {
"type" : "integer"
},
"Price" : {
"type" : "double"
}
}
},
"ID" : {
"type" : "integer"
},
}
}
}
}
}
This has the nested type returned. I think the one which isn't returning nested as a type is when I started using .AutoMap() , am I using it correctly?
UPDATE
I have fixed my mapping problem. I have changed my mapping code to
var responseMap = client.Map<esProduct>(ms => ms
.AutoMap()
.Properties(ps => ps
.Nested<PriceList>(n => n
.Name(c => c.PriceList)
.AutoMap()
)
)
);
Whilst you're developing, I would recommend logging out requests and responses to Elasticsearch so you can see what is being sent when using NEST; this'll make it easier to relate to the main Elasticsearch documentation and also ensure that the body of the requests and responses match your expectations (for example, useful for mappings, queries, etc).
The mappings that you have look fine, although you can forgo the attributes since you are using fluent mapping; there's no harm in having them there but they are largely superfluous (the type name for the esProduct is the only part that will apply) in this case because .Properties() will override inferred or attribute based mapping that is applied from calling .AutoMap().
In your indexing part, you update the esProduct and then immediately after that, index the same document again; I'm not sure what the intention is here but the update call looks superfluous to me; the index call will overwrite the document with the given id in the index straight after the update (and will be visible in search results after the refresh interval). The .RetryOnConflict(3) on the update will use optimistic concurrency control to perform the update (which is effectively a get then index operation on the document inside of the cluster, that will try 3 times if the version of the document changes in between the get and index). If you're replacing the whole document with an update i.e. not a partial update then the retry on conflict is not really necessary (and as per previous note, the update call in your example looks unnecssary altogether since the index call is going to overwrite the document with the given id in the index).
The nested query looks correct; You specify the path to the nested type and then the query to a field on the nested type will also include the path.I'll update the NEST nested query usage documentation to better demonstrate.
The bulk call looks fine; you may want to send documents in batches e.g. bulk index 500 documents at a time, if you need to index a lot of documents. How many to send in one bulk call is going to depend on a number of factors including the document size, how it is analyzed, performance of the cluster, so will need to experiment to get a good bulk size call for your circumstances.
I'd check to make sure that you are hitting the right index, that the index contains the count of documents that you expect and find a document that you know has a PriceList.Price of 100 and see what is indexed for it. It might be quicker to do this using Sense while you're getting up an running.