ElasticSearch NEST - DeleteByQuery(...) does not work - elasticsearch

I am trying to delete the specific documents from ElasticSearch using DeleteByQuery(..) method but it does not work. The response i get shows 200 status.
I can get the results if i search same query.
Below is my query -
FilterContainer[] container = new FilterContainer[2];
FilterContainer TypeFilter = new TermFilter
{
Field = "TYPE",
Value = TableName
};
FilterContainer BRConnectionIDFilter = new TermFilter
{
Field = "BRCONNECTIONID",
Value = BRConnectionID
};
container[0] = TypeFilter;
container[1] = BRConnectionIDFilter;
IDeleteResponse response = objElasticNestClient.DeleteByQuery<dynamic>(s => s.Index(ExtractionContext.ElasticSearchIndex).Query(b => b.Filtered(q => q.Query(a => a.MatchAll()).Filter(f => f.Bool(m => m.Must(container))))));
if (!response.IsValid && response.ConnectionStatus.HttpStatusCode == 200)
{
throw new Exception("Delete failed for object " + TableName + ". Error: " + response.ServerError);
}
I have used INDEX as NOT_ANALYZED for all fields.
Can anyone please guide me on this?

It worked after replacing query like below -
IDeleteResponse response = objElasticNestClient.DeleteByQuery<dynamic>(s => s.Index(ExtractionContext.ElasticSearchIndex).Type(TableName).Query(b => b.Filtered(q => q.Query(a => a.MatchAll()).Filter(f => f.Term("BRCONNECTIONID", BRConnectionID)))));
if (!response.IsValid && response.ConnectionStatus.HttpStatusCode == 200)
{
throw new Exception("Delete failed for object " + TableName + ". Error: " + response.ServerError);
}

Related

LinqToDB Exception cannot be converted to SQL

Hello I have an issue with this query
var queryGrouped = queryFiltered
.GroupBy(c => new { c.Id, c.TableOneId, c.TableOneName, c.TableTwoId, c.TableTwoName, c.TableTwoCode, c.TableThreeId, c.TableThreeName, c.Description, c.EffectiveDate, c.CreatedBy, c.ServiceGroupName })
.DisableGuard()
.Select(cg => new Model
{
Id = cg.Key.Id,
TableOneId = cg.Key.TableOneId,
TableOneName = cg.Key.TableOneName,
TableTwoId = cg.Key.TableTwoId,
TableTwoCode = cg.Key.TableTwoCode,
TableTwoName = cg.Key.TableTwoName,
TableThreeId = cg.Key.TableThreeId,
TableThreeName = cg.Key.TableThreeName,
Description = cg.Key.Description,
EffectiveDate = cg.Key.EffectiveDate,
EffectiveDateText = cg.Key.EffectiveDate != null ? cg.Key.EffectiveDate.Value.ToString("MM/dd/yyyy") : string.Empty,
ServiceGroupName = string.Join(", ", cg.Select(g => g.ServiceGroupName).Distinct()),
CreatedBy = cg.Key.CreatedBy
}).OrderBy(x => x.ServiceGroupName).ToListAsync();
If i run this when try to order by the field ServiceGroup it returns this message
LinqToDB.Linq.LinqException: ''Join(", ", cg.Select(g => g.ServiceGroupName).Distinct())' cannot be converted to SQL.'
So I don't know how to order by this field ServiceGroupName, thanks for any answer.
I would suggest to make grouping on the client side. Note that I have removed ServiceGroupName from grouping key.
var data = await queryFiltered
.Select(c => new {
c.Id,
c.TableOneId,
c.TableOneName,
c.TableTwoId,
c.TableTwoName,
c.TableTwoCode,
c.TableThreeId,
c.TableThreeName,
c.Description,
c.EffectiveDate,
c.CreatedBy,
c.ServiceGroupName
})
.ToListAsync();
var queryGrouped = data
.GroupBy(c => new { c.Id, c.TableOneId, c.TableOneName, c.TableTwoId, c.TableTwoName, c.TableTwoCode, c.TableThreeId, c.TableThreeName, c.Description, c.EffectiveDate, c.CreatedBy })
.Select(cg => new Model
{
Id = cg.Key.Id,
TableOneId = cg.Key.TableOneId,
TableOneName = cg.Key.TableOneName,
TableTwoId = cg.Key.TableTwoId,
TableTwoCode = cg.Key.TableTwoCode,
TableTwoName = cg.Key.TableTwoName,
TableThreeId = cg.Key.TableThreeId,
TableThreeName = cg.Key.TableThreeName,
Description = cg.Key.Description,
EffectiveDate = cg.Key.EffectiveDate,
EffectiveDateText = cg.Key.EffectiveDate != null ? cg.Key.EffectiveDate.Value.ToString("MM/dd/yyyy") : string.Empty,
ServiceGroupName = string.Join(", ", cg.Select(g => g.ServiceGroupName).Distinct()),
CreatedBy = cg.Key.CreatedBy
})
.OrderBy(x => x.ServiceGroupName)
.ToList();

ElasticSearch NEST ObjectInitializer syntax to fluent syntax translation not working

Given this ObjectInitializer NEST query
var mustClauses = new List<QueryContainer>
{
new QueryStringQuery
{
Query = queryFilter.Query,
Lenient = true
},
new MatchQuery
{
Field = new Field("status"),
Query = queryFilter.Status,
Lenient = true,
Operator = Operator.And
},
new DateRangeQuery
{
Field = new Field("timeSent"),
LessThanOrEqualTo = now,
GreaterThanOrEqualTo = GetDateTimeFor(queryFilter.TimeCriteria, now)
}
};
return client.SearchAsync<Ingestion.Entities.ElasticSearch.MessageData>(sd => sd.Query(q => q.Bool(b => b.Must(mustClauses.ToArray())))
.Sort(x => x.Descending(b => b.TimeSent))
.From(from)
.Size(pageSize));
which works, and outputs the following query to my Visual Studio Output window:
{"from":0,"query":{"bool":{"must":[{"match":{"status":{"lenient":true,"operator":"and","query":"Success"}}},{"range":{"timeSent":{"gte":"2021-12-14T03:39:26.5126419Z","lte":"2021-12-21T03:39:26.5126419Z"}}}]}},"size":20,"sort":[{"timeSent":{"order":"desc"}}]}
I am trying to convert it to fluent query syntax like this:
return client.SearchAsync<Ingestion.Entities.ElasticSearch.MessageData>(sd => sd.Query(q => q.Bool(b => b.Must(
mu => mu
.QueryString(qs => qs
.Query(queryFilter.Query)
.Lenient(true)),
mu =>
{
if (string.IsNullOrEmpty(queryFilter.Status))
return null;
return mu
.Match(ma => ma.Field(f => f.Status == queryFilter.Status)
.Lenient(true)
.Operator(Operator.And));
},
mu =>
{
if (queryFilter.TimeCriteria == TimeCriteria.All)
return null;
return mu.DateRange(dr => dr
.Field(f => f.TimeSent)
.LessThanOrEquals(now)
.GreaterThanOrEquals(GetDateTimeFor(queryFilter.TimeCriteria, now)));
})))
.Sort(x => x.Descending(b => b.TimeSent))
.From(from)
.Size(pageSize));
and it's not working. When I run this query, the Match query on that Status field does not appear in the NEST output.
Any guidance/help would be appreciated.
The status lambda expression should be
mu =>
{
if (string.IsNullOrEmpty(queryFilter.Status))
return null;
return mu
.Match(ma => ma.Field(f => f.Status)
.Query(queryFilter.Status)
.Lenient(true)
.Operator(Operator.And));
},

Dynamic Bool Query for NEST (ElasticSearch)

Good day:
I'm trying to achieve a dynamic Bool Query. Example...
QueryContainer addressQuery = null;
QueryContainer typeQuery = null;
BoolQueryDescriptor <Facility> boolQuery = new BoolQueryDescriptor<Facility>();
QueryContainerDescriptor<Facility> sh = new QueryContainerDescriptor<Facility>();
SearchDescriptor<Facility> mainSh = new SearchDescriptor<Facility>();
if (search.searchTerm != null)
{
addressQuery = sh.Term(f => f.Address, search.searchTerm) ||
sh.Match(m => m.Field(f => f.Address).Query(search.searchTerm) ) ||
sh.Term(f => f.ZipCode, search.searchTerm) ||
sh.Match(m => m.Field(f => f.State).Query(search.searchTerm));
boolQuery = boolQuery.Should(b => addressQuery);
}
if (search.type != null) {
typeQuery = sh.Term(f => f.Types.First(), search.type);
boolQuery = boolQuery.Must(b => typeQuery && addressQuery);
}
if (boolQuery != null)
{
mainSh = mainSh.Query(q => q.Bool(b => boolQuery));
request = await this._elasticClient.SearchAsync<Facility>(s => mainSh.Size(search.size));
}
This query is turning into a OR between AddressQuery and TypeQuery however, I'd like to achieve an And condition between both query. Effectively making the query dynamic. Is this possible? Example: dynamic addressQuery && typeQuery.
Thanks.
Fire it out..you can && or || query containers...sorter like:
queryContainer1=queryContainer1 && queryContainer2
Then .bool(b => b(queryContainer1)).

Linq GroupBy and Aggregate

Given the following list:
var data = new[]
{
new {category = "Product", text = "aaaa"},
new {category = "Product", text = "bbbb"},
new {category = "Product", text = "bbbb"},
};
how do I group it by category and return an object with category and a description of the different text put together ??
ie. would like to end yp with:
{
categroy="Product"
description = "aaaa,bbbb,cccc"
}
tried the following GroupBy and Aggregate, but something is not right
data.GroupBy(x => x.category).Select(g => new
{
category = g.Key,
description = g.Aggregate((s1, s2) => s1 + "," + s2)
});
TIA
Why don't you use String.Join(IEnumerable) method?
data.GroupBy(x => x.category).Select(g => new
{
category = g.Key,
description = String.Join(",", g.Select(x => x.text))
});
With Aggregate you should do following:
description = g.Aggregate(string.Empty, (x, i) => x + "," + i.text)
First parameter sets seed start value to String.Empty. Second parameter defines method to concatenate current seed value (string) with current element (anonymous_type).
data.GroupBy(x => x.category).Select(g => new
{
category = g.Key,
description = g.Select(x => x.text).Aggregate((s1, s2) => s1 + "," + s2)
});

LINQ-2-SQL How-To "Reuse" sub selections

Let's assume:
I have an EVENT table
I have a CLIENT table
And ADDRESS table have some column: UnitNo, StreetNo, StreetName, City, AddressType etc
A CLIENT has many EVENTs and CLIENT can have many ADDRESSes also
So if I want to query list of events with client HOME address street name, I just go
var qry = db.Events
.Select(evt => new {
EventAddress =
evt.Client
.Addresses
.FirstOrDefault(a => a.AddressType.Equals("HOME")).StreetName
});
However, if I want to get the full address I will need to concatenate sereval address fields. At the moment I am trying something like
var qry = db.Events
.Select(evt => new {
EventAddress =
evt.Client
.Addresses
.FirstOrDefault(a => a.AddressType.Equals("HOME")).StreetNo + " " +
evt.Client
.Addresses
.FirstOrDefault(a => a.AddressType.Equals("HOME")).StreetName + " " +
evt.Client
.Addresses
.FirstOrDefault(a => a.AddressType.Equals("HOME")).City
});
It doesn't work and looks ugly too
Is there a better way to make the
evt.Client.Addresses.FirstOrDefault(a => a.AddressType.Equals("HOME")) "reusable" so I can just go
var qry =
db.Events
.Select(evt => new {
EventAddress =
STUFF.StreetNo + " " + STUFF.StreetName + " " + STUFF.City
});
Many thanks in advance!
Hugh
UPDATE:
Thanks Ilian for the answer, it works well. And based on that I have created the extension version of the answer
var qry =
db.Events
.Select(evt => new {
EventAddress =
db.Addresses.Select(a => new
{
ClientId = a.ClientId,
AddressType = a.AddressType,
FullAddress = (a.addStreetNo ?? "") + (a.addStreetName ?? "")
})
.FirstOrDefault(a => a.ClientId == e.Client.ClientId && a.AddressType.Equals("HOME"))
.FullAddress
});
Use the query syntax:
var qry =
from evt in db.Events
let firstAddress = evt.Client.Addresses.FirstOrDefault(a => a.AddressType.Equals("HOME"))
where firstAddress != null
select new
{
EventAddress = firstAddress.StreetNo + " " +
firstAddress.StreetName + " " +
firstAddress.City
}

Resources