I use nest 7.0.0 and asp.net core 2.2. i want to create indxe from poco class. in elasticsearch index create but without any mapping. create index method is :
public async Task CreateIndex()
{
try
{
var getIndexResponse =await ElasticClient.Indices.GetAsync("myindex");
if (getIndexResponse.Indices == null || !getIndexResponse.Indices.Any())
{
var createIndexResponse = await ElasticClient.Indices.CreateAsync("myindex", c => c
.Map(p => p.AutoMap<MyModel>()));
}
}
catch (Exception)
{
}
}
and MyModel is like this:
[ElasticsearchType(IdProperty = nameof(Id), RelationName = "MyModelMessage")]
public class MyModel
{
[Number(NumberType.Long, Index = true, DocValues = false)]
public long UserId { get; set; }
[Date(Index = true)]
public DateTime CreatedAt { get; set; }
[Text(Index = false)]
public string ObjectName { get; set; }
[Date(Index = true)]
public DateTime UpdateAt { get; set; }
}
I checked your code against NEST 7.0.1 and elasticsearch 7.2.0 and mapping was created for the type:
class Program
{
[ElasticsearchType(IdProperty = nameof(Id), RelationName = "MyModelMessage")]
public class MyModel
{
[Number(NumberType.Long, Index = true, DocValues = false)]
public long UserId { get; set; }
[Date(Index = true)] public DateTime CreatedAt { get; set; }
[Text(Index = false)] public string ObjectName { get; set; }
[Date(Index = true)] public DateTime UpdateAt { get; set; }
}
static async Task Main(string[] args)
{
var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
var connectionSettings = new ConnectionSettings(pool);
connectionSettings.DefaultIndex("documents");
var client = new ElasticClient(connectionSettings);
var getIndexResponse = await client.Indices.GetAsync("myindex");
if (getIndexResponse.Indices == null || !getIndexResponse.Indices.Any())
{
var createIndexResponse = await client.Indices.CreateAsync("myindex", c => c
.Map(p => p.AutoMap<MyModel>()));
Console.WriteLine("index created");
}
}
}
and http://localhost:9200/myindex/_mapping returns:
{
"myindex": {
"mappings": {
"properties": {
"createdAt": {
"type": "date"
},
"objectName": {
"type": "text",
"index": false
},
"updateAt": {
"type": "date"
},
"userId": {
"type": "long",
"doc_values": false
}
}
}
}
}
Is something missing?
Related
I'm using the example on https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/custom-serialization.html#_serializing_type_information to get $type information for the documents in elasticsearch.
However as mentioned on the page this only returns type information for the outer document:
the type information is serialized for the outer MyDocument instance, but not for each MySubDocument instance in the SubDocuments collection.
So my question now is if anyone knows how to also get type information for sub documents?
I've tried using the same JsonSerializerSettings as in their example separate from Elasticsearch (using LinqPad) and there I get type information also for sub documents:
void Main()
{
var temp = new ListBlock
{
Id = 1,
Title = "Titel",
Blocks = new List<BlockContent> {
new InnerBlock {
Id = 11,
MyProperty ="Inner Property"
},
new InnerBlock2 {
Id = 12,
MyProperty2 = "Inner property 2"
}
}
};
var serializeOptions = new Newtonsoft.Json.JsonSerializerSettings
{
TypeNameHandling = Newtonsoft.Json.TypeNameHandling.All,
NullValueHandling = Newtonsoft.Json.NullValueHandling.Ignore,
TypeNameAssemblyFormatHandling = Newtonsoft.Json.TypeNameAssemblyFormatHandling.Simple,
Formatting = Newtonsoft.Json.Formatting.Indented
};
var serialized = Newtonsoft.Json.JsonConvert.SerializeObject(temp, serializeOptions);
serialized.Dump();
}
public class BlockContent
{
public int Id { get; set; }
}
public class ListBlock : BlockContent
{
public string Title { get; set; }
public List<BlockContent> Blocks { get; set; }
}
public class ListBlock2 : BlockContent
{
public string Title2 { get; set; }
public List<BlockContent> Blocks { get; set; }
}
public class InnerBlock : BlockContent
{
public string MyProperty { get; set; }
}
public class InnerBlock2 : BlockContent
{
public string MyProperty2 { get; set; }
}
This code results in the following json:
{
"$type": "UserQuery+ListBlock, LINQPadQuery",
"Title": "Titel",
"Blocks": {
"$type": "System.Collections.Generic.List`1[[UserQuery+BlockContent, LINQPadQuery]], System.Private.CoreLib",
"$values": [
{
"$type": "UserQuery+InnerBlock, LINQPadQuery",
"MyProperty": "Inner Property",
"Id": 11
},
{
"$type": "UserQuery+InnerBlock2, LINQPadQuery",
"MyProperty2": "Inner property 2",
"Id": 12
}
]
},
"Id": 1
}
Using these versions at the moment:
Elasticsearch 7.4.2
Nest 7.4.2
Update:
The solution provided by Russ Cam below works like a charm for the data model included in the response, however I've put together an example below based on how we create indices (using automap) and bulk index the initial list of documents. This works fine if we exclude the list of Guids (CategoryIds) in the model but if we include it the following exception is thrown:
{
"took": 8,
"errors": true,
"items": [{
"index": {
"_index": "testindex",
"_type": "_doc",
"_id": "1",
"status": 400,
"error": {
"type": "mapper_parsing_exception",
"reason": "failed to parse field [categoryIds] of type [keyword] in document with id '1'. Preview of field's value: '{$values=[], $type=System.Collections.Generic.List`1[[System.Guid, System.Private.CoreLib]], System.Private.CoreLib}'",
"caused_by": {
"type": "illegal_state_exception",
"reason": "Can't get text on a START_OBJECT at 1:140"
}
}
}
}, {
"index": {
"_index": "testindex",
"_type": "_doc",
"_id": "2",
"status": 400,
"error": {
"type": "mapper_parsing_exception",
"reason": "failed to parse field [categoryIds] of type [keyword] in document with id '2'. Preview of field's value: '{$values=[], $type=System.Collections.Generic.List`1[[System.Guid, System.Private.CoreLib]], System.Private.CoreLib}'",
"caused_by": {
"type": "illegal_state_exception",
"reason": "Can't get text on a START_OBJECT at 1:141"
}
}
}
}
]
}
Here is a simple (.Net 5) console application where this behaviour hopefully can be reproduced by others also:
using System;
using System.Collections.Generic;
using System.Linq;
using Elasticsearch.Net;
using Nest;
using Nest.JsonNetSerializer;
using Newtonsoft.Json;
namespace ElasticsearchTypeSerializer
{
internal class Program
{
private const string IndexName = "testindex";
private static void Main(string[] args)
{
var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
var settings = new ConnectionSettings(pool,
(builtin, settings) => new MySecondCustomJsonNetSerializer(builtin, settings));
settings.DisableDirectStreaming();
var client = new ElasticClient(settings);
CreateIndex(client);
IndexDocuments(client);
var documents = GetDocuments(client);
}
private static void CreateIndex(IElasticClient client)
{
var createIndexResponse = client.Indices.Create(IndexName, x => x.Map<MyDocument>(m => m.AutoMap()));
}
private static void IndexDocuments(IElasticClient client)
{
var documents = new List<MyDocument>
{
new()
{
Id = 1,
Name = "My first document",
OwnerId = 2,
SubDocuments = new List<SubDocument>
{
new MySubDocument {Id = 11, Name = "my first sub document"},
new MySubDocument2 {Id = 12, Description = "my second sub document"}
}
},
new()
{
Id = 2,
Name = "My second document",
OwnerId = 3,
SubDocuments = new List<SubDocument>
{
new MySubDocument {Id = 21, Name = "My third sub document"}
}
}
};
var bulkIndexResponse = client.Bulk(b => b.Index(IndexName).IndexMany(documents).Refresh(Refresh.True));
}
private static IEnumerable<MyDocument> GetDocuments(IElasticClient client)
{
var searchResponse = client.Search<MyDocument>(s => s.Index(IndexName).Query(q => q.MatchAll()));
var documents = searchResponse.Documents.ToList();
return documents;
}
}
public class MyDocument
{
public int Id { get; set; }
public string Name { get; set; }
public string FilePath { get; set; }
public int OwnerId { get; set; }
public List<Guid> CategoryIds { get; set; } = new();
public List<SubDocument> SubDocuments { get; set; }
}
public class SubDocument
{
public int Id { get; set; }
}
public class MySubDocument : SubDocument
{
public string Name { get; set; }
}
public class MySubDocument2 : SubDocument
{
public string Description { get; set; }
}
public class MySecondCustomJsonNetSerializer : ConnectionSettingsAwareSerializerBase
{
public MySecondCustomJsonNetSerializer(IElasticsearchSerializer builtinSerializer,
IConnectionSettingsValues connectionSettings)
: base(builtinSerializer, connectionSettings)
{
}
protected override JsonSerializerSettings CreateJsonSerializerSettings()
{
return new()
{
TypeNameHandling = TypeNameHandling.All,
NullValueHandling = NullValueHandling.Ignore,
TypeNameAssemblyFormatHandling = TypeNameAssemblyFormatHandling.Simple
};
}
}
}
Any help regarding this issue is very much appreciated!
If you want type information to be included for typed collections on a document, the derived contract resolver can be omitted, which supresses the type handling for collection item types
private static void Main()
{
var pool = new SingleNodeConnectionPool(new Uri($"http://localhost:9200"));
var settings = new ConnectionSettings(pool,
(builtin, settings) => new MySecondCustomJsonNetSerializer(builtin, settings));
var client = new ElasticClient(settings);
var document = new MyDocument
{
Id = 1,
Name = "My first document",
OwnerId = 2,
SubDocuments = new[]
{
new MySubDocument { Name = "my first sub document" },
new MySubDocument { Name = "my second sub document" },
}
};
var indexResponse = client.IndexDocument(document);
}
public class MyDocument
{
public int Id { get; set; }
public string Name { get; set; }
public string FilePath { get; set; }
public int OwnerId { get; set; }
public IEnumerable<MySubDocument> SubDocuments { get; set; }
}
public class MySubDocument
{
public string Name { get; set; }
}
public class MySecondCustomJsonNetSerializer : ConnectionSettingsAwareSerializerBase
{
public MySecondCustomJsonNetSerializer(IElasticsearchSerializer builtinSerializer, IConnectionSettingsValues connectionSettings)
: base(builtinSerializer, connectionSettings) { }
protected override JsonSerializerSettings CreateJsonSerializerSettings() =>
new JsonSerializerSettings
{
TypeNameHandling = TypeNameHandling.All,
NullValueHandling = NullValueHandling.Ignore,
TypeNameAssemblyFormatHandling = TypeNameAssemblyFormatHandling.Simple
};
}
I have a table student and a table classes and a third table studentclasses having studentid and classic as foreign keys. I want to write a query that allows me to display the following:
{
"code": "10001",
"Name": "Lola",
"classes": [
{
"id": 1,
"name": "class1"
},
{
"id": 2,
"name": "class2"
},
{
"id": 3,
"name": "class3"
}
]
},
using System;
using System.Collections.Generic;
using System.Linq;
namespace ConsoleApp3
{
class Student
{
public int Id { get; set; }
public string Name { get; set; }
public List<StudentClasses> StudentClasses { get; set; }
}
class Classes
{
public int Id { get; set; }
public string Name { get; set; }
public List<StudentClasses> StudentClasses { get; set; }
}
class StudentClasses
{
public int ClassesId { get; set; }
public int StudentId { get; set; }
public Classes Classes { get; set; }
public Student Student { get; set; }
}
class Program
{
static void Main(string[] args)
{
var mathClass = new Classes()
{
Id = 1,
Name = "Math"
};
var englishClass = new Classes()
{
Id = 1,
Name = "English"
};
var nermine = new Student()
{
Id = 1,
Name = "Nermine"
};
var nermineMath = new StudentClasses()
{
Classes = mathClass,
Student = nermine
};
var nermineEnglish = new StudentClasses()
{
Classes = englishClass,
Student = nermine
};
var data = new List<StudentClasses>() { nermineEnglish, nermineMath };
var result = data.Select(x =>
new
{
Code = x.Student.Id,
Name = x.Student.Name,
Classes = new { Id = x.Classes.Id, Name = x.Classes.Name }
}).ToList();
result.ForEach(x => Console.WriteLine(x));
//returns
//{ Code = 1, Name = Nermine, Classes = { Id = 1, Name = English } }
//{ Code = 1, Name = Nermine, Classes = { Id = 1, Name = Math } }
}
}
}
[System.Web.Http.Route("api/data/StudentClasses")]
public IEnumerable<Student> GetStudentClasses() {
var db = new School_DBEntities();
var xx = db.Students.ToList().Select(q=>new Student()
{
Student_ID=q.Student_ID,
First_Name=q.First_Name,
Classes=q.StudentsClasses.ToList().Select(e=>new Class()
{
Class_code=e.Class,
Class_Name=db.Classes.FirstOrDefault(v=>v.Class_code==e.Class).Class_Name
}).ToList()
});
return xx;
}
I have retrieved json data from facebook graph api requesting GET from
https://graph.facebook.com/YOUR_PAGE_ID/albums?fields=name,photos{picture}&access_token=YOUR_ACCESS_TOKEN/
Json data is as follows:
{
"data": [
{
"name": "School Kids",
"photos": {
"data": [
{
"picture": "https://scontent.xx.fbcdn.net/v/t1.0-0/s130x130/15267641_1789056354665765_6384898034258459703_n.jpg?oh=44daa7be0ac1878e769bc16df444bd0a&oe=58B29329",
"id": "1789056354665765"
},
{
"picture": "https://scontent.xx.fbcdn.net/v/t1.0-0/s130x130/15356660_1789056361332431_834718824553815513_n.jpg?oh=69b3f1b1697808b87eed1e3053a67aaf&oe=58B735FB",
"id": "1789056361332431"
},
{
"picture": "https://scontent.xx.fbcdn.net/v/t1.0-0/s130x130/15356635_1789056401332427_1231396155404307815_n.jpg?oh=3de32d320ac6762adc0dbf8b1ef64e0e&oe=58F69648",
"id": "1789056401332427"
},
{
"picture": "https://scontent.xx.fbcdn.net/v/t1.0-0/s130x130/15380574_1789337074637693_1697389498501034556_n.jpg?oh=ddd57d119882b47172af689abde20cfb&oe=58F15477",
"id": "1789337074637693"
}
],
"paging": {
"cursors": {
"before": "MTc4OTA1NjM1NDY2NTc2NQZDZD",
"after": "MTc4OTMzNzA3NDYzNzY5MwZDZD"
}
}
},
"id": "1789056284665772"
},
{
"name": "Cover Photos",
"photos": {
"data": [
{
"picture": "https://scontent.xx.fbcdn.net/v/t1.0-0/s130x130/14519945_1762073987364002_4539899568406717011_n.jpg?oh=fc8c02e9ced0839eea22d08859b964d0&oe=58BC80D4",
"id": "1762073987364002"
}
],
"paging": {
"cursors": {
"before": "MTc2MjA3Mzk4NzM2NDAwMgZDZD",
"after": "MTc2MjA3Mzk4NzM2NDAwMgZDZD"
}
}
},
"id": "1762074137363987"
},
{
"name": "Profile Pictures",
"photos": {
"data": [
{
"picture": "https://scontent.xx.fbcdn.net/v/t1.0-0/p130x130/14495327_1762072887364112_1611299743258720903_n.jpg?oh=ae87944069fd154e817468a38d9cb4a1&oe=58AE8D02",
"id": "1762072887364112"
}
],
"paging": {
"cursors": {
"before": "MTc2MjA3Mjg4NzM2NDExMgZDZD",
"after": "MTc2MjA3Mjg4NzM2NDExMgZDZD"
}
}
},
"id": "1762072884030779"
}
],
"paging": {
"cursors": {
"before": "MTc4OTA1NjI4NDY2NTc3MgZDZD",
"after": "MTc2MjA3Mjg4NDAzMDc3OQZDZD"
}
}
}
What I want from these data is to get image and album name and display in GridView in android.
My attempt parsing these data is as json object from this method
public async void downloadJsonFeedAsync(string url) {
var httpClient = new HttpClient();
Task < string > contentsTask = httpClient.GetStringAsync(url);
// await! control returns to the caller and the task continues to run on another thread
string content = await contentsTask;
Console.Out.WriteLine("Response Body: \r\n {0}", content);
//Convert string to JSON object
mObject = Newtonsoft.Json.JsonConvert.DeserializeObject < ImageGridItemRootObject > (content);
//Update listview
Activity.RunOnUiThread(() => {
mGridView.Adapter = new PhotoGalleryGridViewAdapter(this.Activity, Resource.Layout.PhotoGalleryGridItemView, mObject.data);
mProgressBar.Visibility = ViewStates.Gone;
});
}
// root object
public class ImageGridItemRootObject
{
public string name { get; set; }
public string photos { get; set; }
public List<ImageGridItem> data { get; set; }
}
//ImageGridItem
public class ImageGridItem
{
private string picture;
private string id;
public ImageGridItem():base()
{
}
public string Picture
{
get { return picture; }
set { picture = value; }
}
public string Id
{
get { return id; }
set { id = value; }
}
}
The problem is I get Picture Null. I wonder which is jSon Object and which is jSon Array so that I can format rootObject in-order to retrieve jSon Array .
Thank you in advance.
If you take the returned JSON and use Edit > Paste Special > Paste JSON as classes in Visual Studio you get the following classes:
public class Rootobject
{
public Datum[] data { get; set; }
public Paging paging { get; set; }
}
public class Paging
{
public Cursors cursors { get; set; }
}
public class Cursors
{
public string before { get; set; }
public string after { get; set; }
}
public class Datum
{
public string name { get; set; }
public Photos photos { get; set; }
public string id { get; set; }
}
public class Photos
{
public Datum1[] data { get; set; }
public Paging1 paging { get; set; }
}
public class Paging1
{
public Cursors1 cursors { get; set; }
}
public class Cursors1
{
public string before { get; set; }
public string after { get; set; }
}
public class Datum1
{
public string picture { get; set; }
public string id { get; set; }
}
It looks like your classes are not matching the JSON, which means the Deserializer does not know what to do with the key/values that do not match in your contract.
Adjust your classes to better match the classes above, and it will deserialize with the image urls.
Is it possible in odata4 to create a model such as:
public class PuppyDogs
{
public string Name { get; set; }
public virtual IList<Bone> Bones { get; set; }
}
public class Bone
{
public string ChewType { get; set; }
public int Numberofchews { get; set; }
}
And the controller class looks like
public class PuppyDogController : ODataController
{
List<PuppysDog> mydogs = new List<PuppysDog>();
private PuppyDogController()
{
if (mydogs.Count == 0)
{
PuppysDog mydog = new PuppysDog();
mydog.Name = "Fido";
mydog.Bones = new List<Bone>()
{
new Bone{ ChewType = "Soft", Numberofchews=1 },
new Bone{ ChewType = "Hard", Numberofchews=2 }
};
mydogs.Add(mydog);
}
}
[EnableQuery]
public IQueryable<PuppysDog> Get()
{
return mydogs.AsQueryable();
}
}
Can I include the Bones property of PuppyDogs without using expand? By default Bones is not returned to the client.
There are several things nor clear in your code, for example, the entity set PuppyDogs don't have a key, the naming convention in the controller is a little wired and etc. With the following code, it can work perfectly, please take a look
PuppyDog.cs
public class PuppyDog
{
[Key]
public string Name { get; set; }
public virtual IList<Bone> Bones { get; set; }
}
Bone.cs
public class Bone
{
public string ChewType { get; set; }
public int Numberofchews { get; set; }
}
PupyyDogsController.cs
public class PuppyDogsController : ODataController
{
List<PuppyDog> mydogs = new List<PuppyDog>();
private PuppyDogsController()
{
if (mydogs.Count == 0)
{
PuppyDog mydog = new PuppyDog();
mydog.Name = "Fido";
mydog.Bones = new List<Bone>()
{
new Bone {ChewType = "Soft", Numberofchews = 1},
new Bone {ChewType = "Hard", Numberofchews = 2}
};
mydogs.Add(mydog);
}
}
[EnableQuery]
public IQueryable<PuppyDog> Get()
{
return mydogs.AsQueryable();
}
}
WebApiConfig.cs
public static class WebApiConfig
{
public static void Register(HttpConfiguration config)
{
ODataConventionModelBuilder builder = new ODataConventionModelBuilder();
builder.EntitySet<PuppyDog>("PuppyDogs");
config.MapODataServiceRoute("odata", null, builder.GetEdmModel(), new DefaultODataBatchHandler(GlobalConfiguration.DefaultServer));
config.EnsureInitialized();
}
}
Then when try http://localhost:21830/PuppyDogs, I can successfully got the payload as
{
"#odata.context": "http://localhost:21830/$metadata#PuppyDogs",
"value": [
{
"Name": "Fido",
"Bones": [
{
"ChewType": "Soft",
"Numberofchews": 1
},
{
"ChewType": "Hard",
"Numberofchews": 2
}
]
}
]
}
I've been able to figure out how to call my web server, load JSON, loop through it and output values. But how can I populate this data into a cell. For example, I would like to output the 3 years returned into a table. How would I go about doing that?
async Task GetTowInfo()
{
loadingIndicator.StartAnimating ();
HttpClient client = new HttpClient ();
HttpResponseMessage response = await client.GetAsync ("myurl.com");
HttpContent content = response.Content;
var result = await content.ReadAsStringAsync ();
try
{
var parsed = JsonConvert.DeserializeObject<RootObject>(result);
foreach (var year in parsed.DATA.YEARMFG)
{
Console.WriteLine("Year: {0}", year);
}
}
catch (Exception e) {
Console.WriteLine (e);
}
loadingIndicator.StopAnimating ();
}
public class DATA
{
public List<int> YEARMFG { get; set; }
public List<string> MAKE { get; set; }
public List<string> MODEL { get; set; }
public List<string> ENGINE { get; set; }
public List<int> TOWLIMIT { get; set; }
public List<string> NOTE1 { get; set; }
public List<string> NOTE2 { get; set; }
}
public class RootObject
{
public int ROWCOUNT { get; set; }
public List<string> COLUMNS { get; set; }
public DATA DATA { get; set; }
}
<!----JSON------>
{
"ROWCOUNT": 3,
"COLUMNS": [
"YEARMFG",
"MAKE",
"MODEL",
"ENGINE",
"TOWLIMIT",
"NOTE1",
"NOTE2"
],
"DATA": {
"YEARMFG": [
2012,
2012,
2012
],
"MAKE": [
"Chevrolet/GMC",
"Chevrolet/GMC",
"Chevrolet/GMC"
],
"MODEL": [
"Avalanche 1500 4WD",
"Avalanche 1500 4WD",
"Avalanche 1500 4WD"
],
"ENGINE": [
"5.3L V-8",
"5.3L V-8",
"5.3L V-8"
],
"TOWLIMIT": [
5000,
5500,
8000
],
"NOTE1": [
"3.08 axle ratio",
"3.42 axle ratio",
"3.42 axle ratio"
],
"NOTE2": [
"",
"",
"Cooling or other accessory package required "
]
}
}