I am trying to insert\bulk insert data to Elastic using NEST API.
Can someone provide me the example using NEST?
Thanks,
Sameer
NEST documantation contains examples how to do this e.g.:
var descriptor = new BulkDescriptor();
foreach (var i in Enumerable.Range(0, 1000))
{
descriptor.Index<ElasticSearchProject>(op => op
.Document(new ElasticSearchProject {Id = i})
);
}
var result = client.Bulk(descriptor);
Also you can use IndexMany which is quite useful
var documents= new List<ElasticSearchProject> {...};
client.IndexMany(documents);
Good luck.
Related
I am writing backend in C# for a website. I'd like to recreate index with little downtime.
After reading these two posts:
Nest Client c# 7.0 for elastic search removing Aliases
Recreate ElasticSearch Index with Nest 7.x
I come up with this:
var alias_exist = await _client.Indices.ExistsAsync(index_string_alias);
if (alias_exist.Exists)
{
var oldIndices = await _client.GetIndicesPointingToAliasAsync(index_string_alias);
var oldIndexName = oldIndices.First().ToString();
await _client.Indices.BulkAliasAsync(new BulkAliasRequest
{
Actions = new List<IAliasAction>
{
new AliasRemoveAction {Remove = new AliasRemoveOperation {Index = oldIndexName, Alias = index_string_alias}},
new AliasAddAction {Add = new AliasAddOperation {Index = index_string_unique, Alias = index_string_alias}}
}
});
} else
{
var putAliasResponse = await _client.Indices.PutAliasAsync(new PutAliasRequest(index_string_unique, index_string_alias));
}
}
I'd like to remove index_string_alias if exists and assign the alias to the newly created index_string_unique.
Also, I'd like to confirm that I can treat the alias as the index name in my other queries.
I am really new to Elastic Search and wonder how people figure out these things. I searched through the official documentation and found little information about the async functions in NEST. Where should I look for explanations for functions?
Hi I would like to get all items of a container in a database in cosmos DB. I can see that there are already a lot of methods available but I don't see any getAllItems or a lookalike.
Is there an easy way to do it like using LINQ or something?
Thanks
If you want to do this using Linq, you can do the following (As suggested in this answer here: How can I use LINQ in CosmosDB SDK v3.0 async query?):
var db = Client.GetDatabase(databaseId);
var container = db.GetContainer(containerId);
var q = container.GetItemLinqQueryable<Person>();
var iterator = q.ToFeedIterator();
var results = await iterator.ReadNextAsync();
If you want a non-Linq solution, use this (taken from the v3 SDK samples: https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/Microsoft.Azure.Cosmos.Samples/Usage/ItemManagement/Program.cs#L259:
QueryDefinition query = new QueryDefinition(
"select * from sales s where s.AccountNumber = #AccountInput ")
.WithParameter("#AccountInput", "Account1");
FeedIterator<SalesOrder> resultSet = container.GetItemQueryIterator<SalesOrder>(
query,
requestOptions: new QueryRequestOptions()
{
PartitionKey = new PartitionKey("Account1"),
MaxItemCount = 1
});
while (resultSet.HasMoreResults)
{
FeedResponse<SalesOrder> response = await resultSet.ReadNextAsync();
// Work through iterator list here
}
Hope this helps!
You can use LINQ like described in Microsoft Azures documentation.
Something like this should be the solution of your problem:
var db = Client.GetDatabase(databaseId);
var container = db.GetContainer(containerId);
//Get iterator or use some LINQ queries here
var iterator = container.GetItemLinqQueryable<YourType>().GetEnumerator();
I'm using the NEST .NET client (6.3.1), and trying to compose a search query that is based on a number of (optional) parameters.
Here's what i've got so far:
var searchResponse = await _client.SearchAsync<Listing>(s => s
.Query(qq =>
{
var filters = new List<QueryContainer>();
if (filter.CategoryType.HasValue)
{
filters.Add(qq.Term(p => p.CategoryType, filter.CategoryType.Value));
}
if (filter.StatusType.HasValue)
{
filters.Add(qq.Term(p => p.StatusType, filter.StatusType.Value));
}
if (!string.IsNullOrWhiteSpace(filter.Suburb))
{
filters.Add(qq.Term(p => p.Suburb, filter.Suburb));
}
return ?????; // what do i do her?
})
);
filter is an object with a bunch of nullable properties. So, whatever has a value i want to add as a match query.
So, to achieve that i'm trying to build up a list of QueryContainer's (not sure that's the right way), but struggling to figure out how to return that as a list of AND predicates.
Any ideas?
Thanks
Ended up doing it by using the object initialisez method, instead of the Fluent DSL"
var searchRequest = new SearchRequest<Listing>
{
Query = queries
}
queries is a List<QueryContainer>, which i just build up, like this:
queries.Add(new MatchQuery
{
Field = "CategoryType",
Query = filter.CategoryType
}
I feel like there's a better way, and i don't like how i have to hardcode the 'Field' to a string... but it works. Hopefully someone shows me a better way!
I have Elastic Search Nest library code and need to mock the response i am getting from elastic search index.
var obj = service.Search<TestDocument>(new student().Query());
var Name= obj.Aggs.Terms("Name");
For Testing :
I am creating the Nest object after doing quick watch but facing issue -Aggregations - is a internal protected property and i am not able to set this value.
new Nest.KeyedBucket<object>
{
Key="XYZ school",
KeyAsString=null,
Aggregations=new Dictionary<string, IAggregationContainer>{}
}
Please suggest solution or any other approach i can use to mock elastic search nest object .
If you really want to stub the response from the client, you could do something like the following with Moq
var client = new Mock<IElasticClient>();
var searchResponse = new Mock<ISearchResponse<object>>();
var aggregations = new AggregateDictionary(new Dictionary<string, IAggregate> {
["Name"] = new BucketAggregate
{
Items = new List<KeyedBucket<object>>
{
new Nest.KeyedBucket<object>(new Dictionary<string, IAggregate>())
{
Key = "XYZ school",
KeyAsString = null,
DocCount = 5
}
}.AsReadOnly()
}
});
searchResponse.Setup(s => s.Aggregations).Returns(aggregations);
client.Setup(c => c.Search<object>(It.IsAny<Func<SearchDescriptor<object>, ISearchRequest>>()))
.Returns(searchResponse.Object);
var response = client.Object.Search<object>(s => s);
var terms = response.Aggregations.Terms("Name");
Another way would be to use the InMemoryConnection and return known JSON in response to a request..
For testing purposes however, it may be better to have an instance of Elasticsearch running, and perform integration tests against it. Take a look at Elastic.Xunit which provides an easy way to spin up an Elasticsearch cluster for testing purposes. This is used by the client in integration tests.
You can get Elastic.Xunit from the Appveyor feed.
I am trying to replacing the documents on ES using NEST. I am seeing the following options are available.
Option #1:
var documents = new List<dynamic>();
`var blkOperations = documents.Select(doc => new BulkIndexOperation<T>`(doc)).Cast<IBulkOperation>().ToList();
var blkRequest = new BulkRequest()
{
Refresh = true,
Index = indexName,
Type = typeName,
Consistency = Consistency.One,
Operations = blkOperations
};
var response1 = _client.Raw.BulkAsync<T>(blkRequest);
Option #2:
var descriptor = new BulkDescriptor();
foreach (var eachDoc in document)
{
var doc = eachDoc;
descriptor.Index<T>(i => i
.Index(indexName)
.Type(typeName)
.Document(doc));
}
var response = await _client.Raw.BulkAsync<T>(descriptor);
So can anyone tell me which one is better or any other option to do bulk updates or deletes using NEST?
You are passing the bulk request to the ElasticsearchClient i.e. ElasticClient.Raw, when you should be passing it to ElasticClient.BulkAsync() or ElasticClient.Bulk() which can accept a bulk request type.
Using BulkRequest or BulkDescriptor are two different approaches that are offered by NEST for writing queries; the former uses an Object Initializer Syntax for building up a request object while the latter is used within the Fluent API to build a request using lambda expressions.
In your example, BulkDescriptor is used outside of the context of the fluent API, but both BulkRequest and BulkDescriptor implement IBulkRequest so can be passed to ElasticClient.Bulk(IBulkRequest).
As for which to use, in this case it doesn't matter so whichever you prefer.