How to disable source field and use store with Nest? - elasticsearch

I want to disable source as it is done here initially. I am not sure how to achieve similar like in the sample
PUT tweets
{
"mappings": {
"tweet": {
"_source": {
"enabled": false
}
}
}
}
I tried below but when I enter http://localhost:9200/_plugin/head/; I can see that all the properties are stored. I expect to store and index only id and name properties.
var node = new Uri("http://localhost:9200");
var settings = new ConnectionSettings(node, defaultIndex: "mydatabase");
var client = new ElasticClient(settings);
var createIndexResult = client.CreateIndex("mydatabase");
var mapResult = client.Map<Product>(c => c.MapFromAttributes().SourceField(s=>s.Enabled(false)).IgnoreConflicts().Type("product").Indices("mydatabase"));
client.Index(sampleproduct);
[ElasticType(Name ="product", IdProperty = "ProductId" )]
[Table("Product")]
public partial class Product
{
[ElasticProperty(Name = "id",Index = FieldIndexOption.NotAnalyzed, Store = true)]
public int ProductId { get; set; }
[ElasticProperty(Index = FieldIndexOption.Analyzed, Store = true)]
public string Name { get; set; }
[ElasticProperty(Index = FieldIndexOption.No, Store = false)]
public int? ProductTypeId { get; set; }
[ElasticProperty(Index = FieldIndexOption.No, Store = false)]
public int? ManufacturerId { get; set; }
}
EDIT: without adding any documents after creating index, index metadata looks like in the image. i dont see any source enabled false.
EDIT2: after changing creating index first then mapping. Name field is displaying store=true as in the image but there is no value stored. I debugged and i am surely passing value where i index the sampleproduct

You need to add the mapping first and then index the document. Only then will you see the get the expected mapping. By indexing first without the mapping, a mapping gets dynamically created with default options (source being enabled).

Related

C# Mongo db driver updating an existing document

I have an existing document in mongo db with certain properties. When i am trying to update this document with a document with the same id but having more properties it is not updating it. I am using mongo db driver version 2.4.4
Following is the existing document in my db
{
"_id" : "1234",
"AccountNumber" : "3453535345354",
"Name" : "new1",
"PhoneNumber" : "34534535353534543",
"ETag" : "6ba32e6e-3808-41f5-9b28-0ea882d9c629",
"Id" : "1234"
}
Now when i am trying to update/ replace this document with the following document it does not work:
{
"AccountNumber": "3453535345354",
"Name":"new1",
"PhoneNumber":"34534535353534543",
"TempProperty":"something new",
"ETag" : "6ba32e6e-3808-41f5-9b28-0ea882d9c629",
"Id" : "1234",
"_id" : "1234"
}
Following is the C# code for my upsert method:
public async Task UpsertDocument(string collectionId, string documentId, BsonDocument item)
{
var collection = database.GetCollection<BsonDocument>(collectionId);
var initialCorrelationId = item["ETag"].AsString;
item["ETag"] = Guid.NewGuid().ToString();
var builders = Builders<BsonDocument>.Filter;
var filter = builders.Eq(x => x["Id"], documentId) & builders.Eq(x => x["ETag"], initialCorrelationId);
var options = new FindOneAndReplaceOptions<BsonDocument, BsonDocument>
{
ReturnDocument = ReturnDocument.Before,
IsUpsert = true
};
try
{
item["Id"] = documentId;
item["_id"] = documentId;
await collection.FindOneAndReplaceAsync(filter, item, options);
}
catch (MongoCommandException ex) when (ex.Code == 11000)
{
throw new ConcurrencyException(
$"Error upserting item with id {documentId} and etag {item["Etag"]}{Environment.NewLine}{item.ToJson()}");
}
}
Any help would be much appreciated.
Assuming you have a class as follows:
public class Thing
{
public ObjectId Id { get; set; }
public string AccountNumber { get; set; }
public string Name { get; set; }
public string PhoneNumber { get; set; }
public string ETag { get; set; }
public string SecondId { get; set; }
}
The following code does what you want:
var context = new Context();
var tempProperty = "newValue";
var id = "1234"; // This should be unique
var builder = Builders<Thing>.Filter;
var filter = builder.Eq(x => x.SecondId, id);
var update = Builders<Thing>.Update
.Set("TempProperty", tempProperty);
context.ThingCollection.UpdateOne(filter, update, new UpdateOptions() {IsUpsert = true});
I just renamed Id to SecondId.

NEST elasticsearch DateRange query

I am trying to get results from elasticsearch on a predefined range of dates using NEST api but i dont manage. This example code i am trying to retrieve a week of data but it is ignoring my date Filter and returning older data than my initial date.
Thanks in advance for the help!
var node = new Uri("http://localhost:9200");
var settings = new ConnectionSettings(node);
var client = new ElasticClient(settings);
settings.DefaultIndex("wholelog-2017.04*");
client.CreateIndex("wholelog-2017.04*",
create => create.Mappings(
mappings => mappings.Map<LogLine>(type =>
type.AutoMap()
)
));
DateTime InitDate = DateTime.Now.AddDays(-7);
var filterClauses = new List<QueryContainer>();
filterClauses.Add(new DateRangeQuery
{
Field = new Field("logTime"),
LessThanOrEqualTo = DateTime.Now,
GreaterThanOrEqualTo = InitDate
});
var searchRequest = new SearchRequest<LogLine>()
{
Size = 10000,
From = 0,
Scroll = "1m",
Query = new BoolQuery
{
Filter = filterClauses
}
};
var searchResult = client.Search<LogLine>(searchRequest);
[ElasticsearchType(Name="logLine")]
public class LogLine
{
public string Message {get;set;}
public DateTime logTime {get;set;}
public string type {get;set;}
public string logServer {get;set;}
}
logstash config file
output {
elasticsearch{
hosts => "localhost"
#user => ####
#password => ####
index => "wholelog-%{+YYYY.MM.dd}"
}
}
I assume that you didn't explicitly put your mapping and just started adding documents to your index. Try to create a new index like this
client.CreateIndex(indexName,
create => create.Mappings(
mappings => mappings.Map<LogLine>(
type => type.AutoMap()
)
)
);
And use DateTime for your field. DateMath is intended for building search queries
[ElasticsearchType(Name = "logLine")]
public class LogLine
{
public string Message { get; set; }
public DateTime logTime { get; set; }
public string type { get; set; }
public string logServer { get; set; }
}
After explicitly putting your mapping try to add docs and search against new index.

Elasticsearch - Trying to index MS Word attachment & making a full text search within

As the title is already indicating, I am trying to index MS Word documents and making a full text search within.
I have seen several examples, but I am not able to figure out what I am doing incorrectly.
Relevant Code:
[ElasticsearchType(Name = "AttachmentDocuments")]
public class Attachment
{
[String(Name = "_content")]
public string Content { get; set; }
[String(Name = "_content_type")]
public string ContentType { get; set; }
[String(Name = "_name")]
public string Name { get; set; }
public Attachment(Task<File> file)
{
Content = file.Result.FileContent;
ContentType = file.Result.FileType;
Name = file.Result.FileName;
}
}
The "Content" property above is set to "file.Result.FileContent" in the constructor. The "Content" property is a base64 string.
public class Document
{
[Number(Name = "Id")]
public int Id { get; set; }
[Attachment]
public Attachment File { get; set; }
public String Title { get; set; }
}
Below is the method for indexing documents to elasticsearch database.
public void IndexDocument(Attachment attachmentDocument)
{
// Create the index if it does not already exist
var indexExists = _client.IndexExists(new IndexExistsRequest(ElasticsearchIndexName));
if (!indexExists.Exists)
{
var indexDescriptor =
new CreateIndexDescriptor(new IndexName {Name = ElasticsearchIndexName}).Mappings(
ms => ms.Map<Document>(m => m.AutoMap()));
_client.CreateIndex(indexDescriptor);
}
var doc = new Document()
{
Id = 1,
Title = "Test",
File = attachmentDocument
};
_client.Index(doc);
}
Based on the code above, the document get indexed into the correct index(Screenshot from Elasticsearch host - Searchly):
Searchly Screenshot
The content in the file is : "VCXCVXCVXCVXCVXVXCVXCV" and with the following query I get zero hits in return:
QueryContainer queryContainer = null;
queryContainer |= new MatchQuery()
{
Field = "file",
Query = "VCXCVXCVXCVXCVXVXCVXCV"
};
var searchResult =
await _client.LowLevel.SearchAsync<string>(ApplicationsIndexName, "document", new SearchRequest()
{
From = 0,
Size = 10,
Query = queryContainer,
Aggregations = GetAggregations()
});
I would appericiate if someone could hint me what I am doing incorrectly or should look into?
Providing screenshot of mapping in my Elasticsearch database:
Elasticsearch - Mapping
Because you refer to wrong field. Field should be file.content
queryContainer |= new MatchQuery()
{
Field = "file.content",
Query = "VCXCVXCVXCVXCVXVXCVXCV"
};

Elastic search Nest index query always returning false

var local = new Uri("http://localhost:9200");
var settings = new ConnectionSettings(local, null);
var elastic = new ElasticClient(settings);
var res = elastic.CreateIndex(ci => ci
.Index("my_first_index_final2")
.AddMapping<BlogPost>(m => m.MapFromAttributes()));
Console.WriteLine(res.RequestInformation.Success);
var blogPost = new BlogPost
{
Id = Guid.NewGuid(),
Title = "First blog post",
Body = "This is very long blog post!"
};
var firstId = blogPost.Id;
var result = elastic.Index(blogPost, p => p
.Index("my_first_index_final2")
.Id(blogPost.Id.ToString())
.Refresh());
Console.WriteLine(result.RequestInformation.Success);
Blogpost class:
[ElasticType(IdProperty = "Id", Name = "blog_post")]
public class BlogPost
{
[ElasticProperty(Name = "_id", Index = FieldIndexOption.NotAnalyzed, Type = FieldType.String)]
public Guid? Id { get; set; }
[ElasticProperty(Name = "title", Index = FieldIndexOption.Analyzed, Type = FieldType.String)]
public string Title { get; set; }
[ElasticProperty(Name = "body", Index = FieldIndexOption.Analyzed, Type = FieldType.String)]
public string Body { get; set; }
public override string ToString()
{
return string.Format("Id: '{0}', Title: '{1}', Body: '{2}'", Id, Title, Body);
}
}
This is my code. Everytime it returns:
true
false
Meaning, it creates index but unable to insert document into the index. I don't understand the reason.
Also, I have to rename my index name everytime i run this demo console application as i think we cannot insert index with same name. how can i avoid doing this?
I am following this tutorial:
https://www.devbridge.com/articles/getting-started-with-elastic-using-net-nest-library-part-two/
Any other resource for learning nest and elastic search, please feel free to suggest.
My guess is you are using Elasticsearch 2.x. Your code won't break in Elasticsearch 1.x. The problem is, you are trying to add a field _id inside a document. It being one of the metadata fields, Elasticsearch 2.x prohibits you from indexing it inside the document. To make your code work, simply change the name of the Id field from _id to something different, say, id.

Getting an Enum to display on client side

I'm having hard time understanding how to convert an Enum value to it's corresponding name. My model is as follows:
public class CatalogRule
{
public int ID { get; set; }
[Display(Name = "Catalog"), Required]
public int CatalogID { get; set; }
[Display(Name = "Item Rule"), Required]
public ItemType ItemRule { get; set; }
public string Items { get; set; }
[Display(Name = "Price Rule"), Required]
public PriceType PriceRule { get; set; }
[Display(Name = "Value"), Column(TypeName = "MONEY")]
public decimal PriceValue { get; set; }
[Display(Name = "Exclusive?")]
public bool Exclude { get; set; }
}
public enum ItemType
{
Catalog,
Category,
Group,
Item
}
public enum PriceType
{
Catalog,
Price_A,
Price_B,
Price_C
}
A sample result from .net API:
[
{
$id: "1",
$type: "XYZ.CMgr.Models.CatalogRule, XYZ.CMgr",
ID: 1,
CatalogID: 501981,
ItemRule: 0,
Items: "198",
PriceRule: 1,
PriceValue: 0.5,
Exclude: false
},
{
$id: "2",
$type: "XYZ.CMgr.Models.CatalogRule, XYZ.CMgr",
ID: 2,
CatalogID: 501981,
ItemRule: 2,
Items: "9899",
PriceRule: 2,
PriceValue: 10.45,
Exclude: false
}
]
So in this example, I need to get Catalog for results[0].ItemRule & Price A for results[0].PriceRule. How can I accomplish this in BreezeJS??
This is easy to do in ASP.NET Web API, because it is an out-of-box feature in the default JSON serializer (Json.NET).
To see strings instead of enum numbers in JSON, just add an instance of StringEnumConverter to JSON serializer settings during app init:
var jsonFormatter = GlobalConfiguration.Configuration.Formatters.JsonFormatter;
jsonFormatter.SerializerSettings.Converters.Add(new Newtonsoft.Json.Converters.StringEnumConverter());
UPDATE: Yep, you right, this is not help with Breeze.js. Ok, you can anyway do a little magic to make enums work like strings (while new version with fix is not released).
Create a custom ContextProvider which updates all integer enum values in metadata to strings. Here it is:
public class StringEnumEFContextProvider<T> : EFContextProvider<T>
where T : class, new()
{
protected override string BuildJsonMetadata()
{
XDocument xDoc;
if (Context is DbContext)
{
xDoc = GetCsdlFromDbContext(Context);
}
else
{
xDoc = GetCsdlFromObjectContext(Context);
}
var schemaNs = "http://schemas.microsoft.com/ado/2009/11/edm";
foreach (var enumType in xDoc.Descendants(XName.Get("EnumType", schemaNs)))
{
foreach (var member in enumType.Elements(XName.Get("Member", schemaNs)))
{
member.Attribute("Value").Value = member.Attribute("Name").Value;
}
}
return CsdlToJson(xDoc);
}
}
And use it instead of EFContextProvider in your Web API controllers:
private EFContextProvider<BreezeSampleContext> _contextProvider =
new StringEnumEFContextProvider<BreezeSampleContext>();
This works well for me with current Breeze.js version (1.1.3), although I haven't checked other scenarios, like validation...
UPDATE: To fix validation, change data type for enums in breeze.[min|debug].js, manually (DataType.fromEdmDataType function, dt = DataType.String; for enum) or replace default function during app init:
breeze.DataType.fromEdmDataType = function (typeName) {
var dt = null;
var parts = typeName.split(".");
if (parts.length > 1) {
var simpleName = parts[1];
if (simpleName === "image") {
// hack
dt = DataType.Byte;
} else if (parts.length == 2) {
dt = DataType.fromName(simpleName);
if (!dt) {
if (simpleName === "DateTimeOffset") {
dt = DataType.DateTime;
} else {
dt = DataType.Undefined;
}
}
} else {
// enum
dt = DataType.String; // THIS IS A FIX!
}
}
return dt;
};
Dirty, dirty hacks, I know... But that's the solution I found
There will be a new release out in the next few days where we "change" breeze's enum behavior ( i.e. break existing code with regards to enums). In the new release enums are serialized and queried by their .NET names instead of as integers. I will post back here when the new release is out.

Resources