ml.net model works with console application but not web api - asp.net-web-api

I have created an MLModel with ml.net model builder and it works with a 'consume' console application and hard coded values for input. I wanted to go to the next step and feed values with web api. I am using a modified version of their tutorial example on using a mlmodel in a web api. When I use postman with the webapi I get the following error;
System.InvalidOperationException: 'Could not apply a map over type 'Single' to column 'PG' since it has type 'String''
The mlmodel builds successfully from model builder and the default code is generated from the model builder. Running a consume console app with hard coded values on the model works.
Here is the relevant code for modelinput
//*****************************************************************************************
//* *
//* This is an auto-generated file by Microsoft ML.NET CLI (Command-Line Interface) tool. *
//* *
//*****************************************************************************************
using Microsoft.ML.Data;
namespace JobClassMLModelML.Model.DataModels
{
public class ModelInput
{
[ColumnName("EducationLevel"), LoadColumn(0)]
public float EducationLevel { get; set; }
[ColumnName("Experience"), LoadColumn(1)]
public float Experience { get; set; }
[ColumnName("OrgImpact"), LoadColumn(2)]
public float OrgImpact { get; set; }
[ColumnName("ProblemSolving"), LoadColumn(3)]
public float ProblemSolving { get; set; }
[ColumnName("Supervision"), LoadColumn(4)]
public float Supervision { get; set; }
[ColumnName("ContactLevel"), LoadColumn(5)]
public float ContactLevel { get; set; }
[ColumnName("FinancialBudget"), LoadColumn(6)]
public float FinancialBudget { get; set; }
[ColumnName("PG"), LoadColumn(7)]
public string PG { get; set; }
}
}
And here it is for modeloutput
//*****************************************************************************************
//* *
//* This is an auto-generated file by Microsoft ML.NET CLI (Command-Line Interface) tool. *
//* *
//*****************************************************************************************
using System;
using Microsoft.ML.Data;
namespace JobClassMLModelML.Model.DataModels
{
public class ModelOutput
{
// ColumnName attribute is used to change the column name from
// its default value, which is the name of the field.
[ColumnName("PredictedLabel")]
public String Prediction { get; set; }
public float[] Score { get; set; }
}
}
I cloned the modelinput and modeloutput classes in equivalent webapi classes of jobclassdata and jobclassprediction respectively
The console application that works:
using System;
using Microsoft.ML;
using JobClassMLModelML;
using JobClassMLModelML.Model.DataModels;
namespace ConsumeApp
{
class Program
{
static void Main(string[] args)
{ // Load the model
MLContext mlContext = new MLContext();
ITransformer mlModel = mlContext.Model.Load("MLModel.zip", out
var modelInputSchema);
var predEngine =
mlContext.Model.CreatePredictionEngine<ModelInput, ModelOutput>(mlModel);
// Use the code below to add input data
var input = new ModelInput();
// input.
input.ContactLevel = 5;
input.EducationLevel = 5;
input.Experience = 5;
input.FinancialBudget = 5;
input.OrgImpact = 5;
input.ProblemSolving = 5;
input.Supervision =5;
//input.PG = "PG01";
// Try model on sample data
ModelOutput result = predEngine.Predict(input);
Console.WriteLine($"Predicted value: {result.Prediction} ");
}
}
}
Here is the code for the predict controller
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.ML;
using JobClassMLModelWebAPI.DataModels;
namespace JobClassMLModelWebAPI.Controllers
{
[Route("api/[controller]")]
[ApiController]
public class PredictController : ControllerBase
{
private readonly PredictionEnginePool<JobClassData,
JobClassPrediction> _predictionEnginePool;
public PredictController(PredictionEnginePool<JobClassData,
JobClassPrediction> predictionEnginePool)
{
_predictionEnginePool = predictionEnginePool;
}
[HttpGet]
public JobClassData Get(JobClassData jobClassData)
{
return new JobClassData
{
// insert test data here
};
}
[HttpPost]
public ActionResult<string> Post([FromBody] JobClassData input)
{
if (!ModelState.IsValid)
{
return BadRequest();
}
JobClassPrediction prediction =
_predictionEnginePool.Predict(input);
string PredictedJobGrade = prediction.Prediction;
return Ok(PredictedJobGrade);
}
}
}
Here is json body code posted
{
"educationLevel": 3,
"experience": 3,
"orgImpact": 3,
"problemSolving": 3,
"supervision": 3,
"contactLevel": 4,
"financialBudget": 4,
"pg" : "PG01"
}
The expected result would be predicted paygrade such as PG03. Instead the error is
System.InvalidOperationException: 'Could not apply a map over type 'Single' to column 'PG' since it has type 'String''
at the line
JobClassPrediction prediction =
_predictionEnginePool.Predict(input);
with no compile error and with console app working is the format of the json input?
Thanks

Related

Convert Expression<Func<T, bool>> to another Predicate Expression Expression<Func<U, bool>>

I want to convert One predicate Expression(Expression<Func<Item, bool>>) to another Predicate Expression (Expression<Func<ItemEntity, bool>>) but after converting I am not able to query through LINQ.
I Already try this and this approach but nothing work properly
can anyone tells me how to do it properly, My approach for this problems.
using System;
using System.Linq;
using System.Collections.Generic;
using System.Linq.Expressions;
using System.Threading.Tasks;
using System.Reflection;
public class ItemEntity {
public int ItemId {set;get;}
public int ItemParentId {set;get;}
public int ItemName {set;get;}
}
public class Item {
public int Id{ set;get;}
public int ParentId{set;get;}
public int Name {set;get;}
}
public class Program
{
public static Item Convert(ItemEntity itemChild)
{
return new Item()
{
Id = itemChild.ItemId,
ParentId = itemChild.ItemParentId,
Name = itemChild.ItemName
};
}
public async Task<IList<ItemEntity>> SelectAsync(Expression<Func<ItemEntity, bool>> predicate)
{
// using Microsoft.EntityFrameworkCore;
// private readonly DbContext _context; // Injected globally by using Service.AddScoped<ItemContext>();
// return await _context.Set<ItemEntity>().AsNoTracking().Where(predicate).ToListAsync();
return await Task.Run(() => new List<ItemEntity>()); // actually return the result with matching predicate
}
public async Task<List<Item>> GetItems(Expression<Func<Item, bool>> expression)
{
MethodInfo convertMethod = ((Func<ItemEntity, Item>)Convert).Method;
var p = Expression.Parameter(typeof(ItemEntity));
var converted = Expression.Lambda<Func<ItemEntity, bool>>(
Expression.Invoke(expression, Expression.Convert(p, typeof(Item), convertMethod)), p);
IList<ItemEntity> res = await SelectAsync(converted);
var t = res.Select(x => Convert(x)).ToList();
return t;
}
public static void Main()
{
Program pr = new Program();
Func<Expression<Func<Item, bool>>, Task<List<Item>>> getItem = pr.GetItems;
var res = getItem.Invoke(x => x.Id.Equals(1));
Console.WriteLine("Hello World");
}
}
but I am getting error
The LINQ expression 'DbSet()\r\n .Where(i => ((Item)i).Id.Equals(__Id_0))' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'. See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.
I don't able to understand it properly, as per my understanding I am using ToList() for client evaluation, but and also provide method to convert ItemEntity to Item.
I any other way to create fresh Expression Tree based on ItemEntity and then query on DBSet?
Any Help is appreciated
version used:
dot-net 5.0
Microsoft.EntityFrameworkCore 5.0.6
EntityFramework 6.4.4
Database SQL Server
Try the following approach. Main idea to use filter exactly on the projected DTO before materialization.
using System;
using System.Collections.Generic;
using System.Linq.Expressions;
using System.Threading.Tasks;
public class ItemEntity
{
public int ItemId { set; get; }
public int ItemParentId { set; get; }
public int ItemName { set; get; }
}
public class Item
{
public int Id { set; get; }
public int ParentId { set; get; }
public int Name { set; get; }
}
public class Program
{
public static Expression<Func<ItemEntity, Item>> ToItem()
{
return itemChild => new Item
{
Id = itemChild.ItemId,
ParentId = itemChild.ItemParentId,
Name = itemChild.ItemName
};
}
public Task<IList<Item>> SelectAsync(Expression<Func<Item, bool>> predicate)
{
return _context.Set<ItemEntity>()
.AsNoTracking()
.Select(ToItem())
.Where(predicate)
.ToListAsync();
//return Task.Run(() => new List<ItemEntity>().AsQueryable().Select(ToItem()).Where(predicate).ToList());
}
public async Task<IList<Item>> GetItems(Expression<Func<Item, bool>> expression)
{
var res = await SelectAsync(expression);
return res;
}
public static void Main()
{
var pr = new Program();
var res = pr.GetItems(x => x.Id == 1);
Console.WriteLine("Hello World");
}
}

csvHelper Sort field while writing

I am trying to use CsvHelper library to write CSV stream to a file. All is well however I am stuck with 2 things
How do I add additional columns to the csv file. The additional column value will be a constant but will only be known at runtime.
Is there a way to sort the CSV file on a particular field if I am writing one record at a time and also when I am writing all at once
static void Main(string[] args)
{
var records = new List<MyTest>
{
new MyTest { Id = 1, Name = "John", Extra = "hello" },
new MyTest { Id = 2, Name = "Jack", Extra = "hi" },
};
using (var writer = new StreamWriter("C:\\Users\\machina\\AppData\\Local\\Temp\\file.csv"))
using (var csv = new CsvWriter(writer, CultureInfo.InvariantCulture))
{
csv.Context.RegisterClassMap<TestMap<MyTest>>();
csv.WriteRecords<MyTest>(records);
}
}
}
public interface IData
{
public int Id { get; set; }
public string Name { get; set; }
}
public abstract class AbTest : IData
{
public abstract int Id { get; set; }
public abstract string Name { get; set; }
public abstract string Extra { get; set; }
}
public class MyTest : AbTest
{
public override int Id { get; set; }
public override string Name { get; set; }
public override string Extra { get; set; }
}
public class TestMap<T> : ClassMap<MyTest>
{
public TestMap()
{
AutoMap(CultureInfo.InvariantCulture);
}
}
The above works as I have prepared the data records however what I want to achieve is to dynamically add Extra column values to my existing stream of csv data which does not have it.
Also I saw DynamicPropertySort under csvconfiguration and wanted to check if this can be used in any way to sort the fields in my CSV stream before its written even when its writing one record at a time?
I am a novoice developer so let me know if there is a better way to achieve this.
The easiest way to dynamically add columns would likely be to manually write out each row.
DynamicPropertySort is just used for ordering the columns of a dynamic object. You can sort on a particular field using the linq OrderBy() method.
static void Main(string[] args)
{
var records = new List<MyTest>
{
new MyTest { Id = 1, Name = "John", Extra = "hello" },
new MyTest { Id = 2, Name = "Jack", Extra = "hi" },
};
var extraField1 = 10;
var extraField2 = "Testing";
using (var writer = new StreamWriter("C:\\Users\\machina\\AppData\\Local\\Temp\\file.csv"))
using (var csv = new CsvWriter(writer, CultureInfo.InvariantCulture))
{
csv.Context.RegisterClassMap<TestMap>();
var orderedRecords = records.OrderBy(r => r.Name);
csv.WriteHeader<MyTest>();
csv.WriteField("ExtraField1");
csv.WriteField("ExtraField2");
csv.NextRecord();
foreach (var record in orderedRecords)
{
csv.WriteRecord(record);
csv.WriteField(extraField1);
csv.WriteField(extraField2);
csv.NextRecord();
}
}
}

Execute or compile runtime c# code on textbox after changes event - "PlatformNotSupportedException"

I am wondering to provide an option in my application to users to be able to customize buttons in ASP.NET CORE with Blazor.
I mean, C# code can be stored in database, and compiled/executed in runtime.
How can I "provide" an object from razor to be manipulated in this "code stored in the database" and after that, make those changes in runtime, after code been executed?
Please share your ideas.
I already tried some things, like this:
using Blazored.Toast.Services;
using Microsoft.AspNetCore.Components;
using Microsoft.CSharp;using Newtonsoft.Json;
using System;
using System.CodeDom.Compiler;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using System.Threading.Tasks;
namespace Projeto.Web.Pages.Components {
public class McwEditModel : CustomComponentBase
{
[Inject]
public IToastService ToastService { get; set; }
#region Parameters
[Parameter]
public string nome { get; set; }
#endregion
public void OnTextChanged(string newValue)
{
string source =
#"public class SomeClass {
public int OnTextChanged (string newValue) {
if (!string.IsNullOrEmpty(newValue)) {
ToastService.ShowWarning(newValue);
}
}
} ";
var compParms = new CompilerParameters
{
GenerateExecutable = false,
GenerateInMemory = true
};
var csProvider = new CSharpCodeProvider();
CompilerResults compilerResults =
csProvider.CompileAssemblyFromSource(compParms, source);
object typeInstance =
compilerResults.CompiledAssembly.CreateInstance("SomeClass");
MethodInfo mi = typeInstance.GetType().GetMethod("OnTextChanged");
int methodOutput =
(int)mi.Invoke(typeInstance, new object[] { newValue });
InvokeAsync(StateHasChanged);
}
}}
The problem is that does not work, particularly I am getting the error "System.PlatformNotSupportedException: 'Operation is not supported on this platform.'" on line
" CompilerResults compilerResults = csProvider.CompileAssemblyFromSource(compParms, source); "
I am using Blazor .NET Core.
Is there any chance or way to achieve what I want?
I solved it using using Microsoft.CodeAnalysis package.
I needed to create a
public class Globals
{
[Inject]
public IToastService ToastService { get; set; }
public string newValue { get; set; }
public dynamic objeto { get; set; }
}
and inside my method I did:
public void OnTextChanged(string newValue){
Aql aql = new Aql { EaqcCodigo = "1", EaqcDescricao = "AQL" };
var globals = new Globals { ToastService = ToastService, newValue = newValue, objeto= aql };
CSharpScript.RunAsync("ToastService.ShowWarning(newValue + \"- desc:\" + objeto.EaqcDescricao + \"- cod:\" + objeto.EaqcCodigo);", ScriptOptions.Default.WithReferences("Microsoft.CSharp"), globals).Wait();
}
with this, now I can pass a dynamic object and also make any codes in a string field on database and retrieve this wherever I want, to customize a lot of things, ie: change of fields (in a custom generic and dynamic component)

SQLite-net-pcl not working properly on Android 7.0 (Xamarin)

SQLite-net-pcl seem's to work on all platforms except Android 7.0.
So far on Android 7.0 from a fresh install the application will crash due to an SQLException with the issue that it can't create a PK. If I uninstall the applications storage I.e. wipe the app from application settings, the app then works perfectly.
All other platforms seem to install instantly. See for youself https://play.google.com/store/apps/details?id=com.purewowstudio.fivestartips
I cannot seem to fix this. Worst of all, my iOS which shares a DB PCL doesn't load at all.
Does anyone know how to fix this?
Code example:-
Object
using System;
namespace App.LocalObjects
{
[Preserve(AllMembers = true)]
public class LocalArticle
{
[SQLite.PrimaryKey, SQLite.Column("articleObjectId")]
public string objectId { get; set; }
public DateTime createdAt { get; set; }
public DateTime updatedAt { get; set; }
public string Title { get; set; }
public string Description { get; set; }
public string Url { get; set; }
public int Status { get; set; }
public LocalArticle()
{
this.objectId = " ";
this.Title = " ";
this.Description = " ";
this.Url = " ";
}
}
}
Database
using System;
using System.Linq;
using System.Collections.Generic;
using SQLite;
using App.LocalObjects;
namespace App.DataLayer
{
public class Database
{
static object locker = new object();
public SQLiteConnection database;
public string path;
public Database(SQLiteConnection conn)
{
database = conn;
database.CreateTable<LocalArticle>();
}
public int SaveLocalArticleItem(LocalArticle item)
{
lock (locker)
{
LocalArticle article = database.Table<LocalArticle>().FirstOrDefault(x => x.objectId == item.objectId);
if (article != null && article.objectId.Equals(item.objectId) && !article.updatedAt.Equals(item.updatedAt))
{
database.Update(item);
return item.ID;
} else {
return database.Insert(item);
}
}
}
Code to initialize DB:
using System;
using System.IO;
using Android.App;
using App.BusinessLayer.Managers;
using App.BusinessLayer.ParseObjects;
using App.Utils;
using Android.Content;
using Com.Nostra13.Universalimageloader.Core;
using Com.Nostra13.Universalimageloader.Core.Assist;
using Android.Graphics;
using Com.Nostra13.Universalimageloader.Cache.Memory.Impl;
using Com.OneSignal;
using Parse;
using SQLite;
namespace App.Droid
{
[Application(LargeHeap = true)]
public class App : Application
{
public static App Current { get; private set; }
public ModelManager modelManager { get; set; }
private SQLiteConnection conn;
private ImageLoader imageLoader;
public App(IntPtr handle, global::Android.Runtime.JniHandleOwnership transfer)
: base(handle, transfer)
{
Current = this;
}
public override void OnCreate()
{
base.OnCreate();
SetupParse();
SetupDB();
}
private void SetupParse()
{
ParseObject.RegisterSubclass<ParseArticle>();
ParseClient.Initialize(new ParseClient.Configuration
{
ApplicationId = Constants.ParseApplicationID,
Server = Constants.ParseServer
});
}
private void SetupDB()
{
var sqliteFilename = Constants.DBName;
string libraryPath = Environment.GetFolderPath(Environment.SpecialFolder.Personal);
var path = System.IO.Path.Combine(libraryPath, sqliteFilename);
conn = new SQLiteConnection(path);
modelManager = new ModelManager(conn);
}
Here's what my project is currently referencing:-
Android N began actually enforcing rules around accessing the SQLite libraries that come with Android. Therefore any app (or app library) that was previously directly accessing SQLite without going through the special Java wrapper was already breaking the rules technically, the reason it was not a problem was because Android never really enforced those rules. Read more about that here
Now that Android N does enforce those rules, it causes the crash to occur. I am guessing that you are using the SQLite.Net-PCL library for accessing your SQLite DB. If you look here there is an open issue for that exact problem that the library creators never fixed.
We ran into the same issue and have since switched to the SQLite-Net-PCL library (notice the dash instead of the period) which works the exact same way and has an almost identical API. A link for that library can be found here.
i have same issue. i install this package for sqlite and set targeted version to android 8(oreo) my application works again

Can SolrNet client run at a different machine from solr server?

I set up a solr server running in Tomcat in machine 192.168.0.113(Centos 5.5).
And I deploy a website in matchine 192.168.0.114(Windows server 2003).
I use solrnet in matchine 192.168.0.114.
The full code like bellow(which have been edited thanks to #Paige Cook):
using System;
using System.Collections.Generic;
using System.Text;
using SolrNet;
using NUnit.Framework;
using SolrNet.Attributes;
using SolrNet.Commands.Parameters;
using Microsoft.Practices.ServiceLocation;
namespace MySolrNet
{
public class Video
{
private string videoid;
[SolrField("videoid")]
public string Videoid
{
get { return videoid; }
set { videoid = value; }
}
private string videoname;
[SolrField("videoname")]
public string Videoname
{
get { return videoname; }
set { videoname = value; }
}
private string videoorigin;
[SolrField("videoorigin")]
public string Videoorigin
{
get { return videoorigin; }
set { videoorigin = value; }
}
public Video(string id, string name, string origin)
{
this.Videoid = id;
this.Videoname = name;
this.Videoorigin = origin;
}
public Video()
{
}
public void FixtureSetup()
{
Startup.Init<Video>("http://192.168.0.113:8070/solr");
}
public void Add()
{
Video p = new Video("1", "test video", "Solr Test");
ISolrOperations<Video> solr = ServiceLocator.Current.GetInstance<ISolrOperations<Video>>();
solr.Add(p);
solr.Commit();
}
}
[TestFixture]
public class VideoTests
{
[TestFixtureSetUp]
public void FixtureSetup()
{
Startup.Init<Video>("http://192.168.0.113:8070/solr");
}
[Test]
public void Add()
{
Video p = new Video("1", "test video", "Solr Test");
ISolrOperations<Video> solr = ServiceLocator.Current.GetInstance<ISolrOperations<Video>>();
solr.Add(p);
solr.Commit();
}
[Test]
public void Query()
{
ISolrOperations<Video> solr = ServiceLocator.Current.GetInstance<ISolrOperations<Video>>();
SolrQueryResults<Video> results = solr.Query(new SolrQueryByField("videoid", "33013"));
Assert.AreEqual(1, results.Count);
Console.WriteLine(results[0].Videoname);
}
}
}
However,both Add and Query test fail.
It complains: TestFixture failed: SetUp : System.IO.FileLoadException : Could not load file or assembly“SolrNet, Version=0.4.0.2002, Culture=neutral, PublicKeyToken=bc21753e8aa334cb” Or one of its dependencies.
But I have add reference to Microsoft.Practices.ServiceLocation.dll and SolrNet.dll in my projoect,is there any other dll files I just miss?
By the way,,I can access my solr sever in browser with this url:http://192.168.0.113:8070/solr.
Can anyone tell me:
Can I run solrnet and solr in different machines?
How to do it.
Thanks a lot!
I don't understand why you're using two different URLs when you Init Solr. Try changing the URL in Paige's application to the one you posted in your original question: http://192.168.0.113:8070/solr
Thanks for posting the code. The first thing I see is that you are using the test class as the class type to pass data to Solr. Split those out, that might be causing some issues. I would suggest the following:
public class Video
{
private string videoid;
[SolrField("videoid")]
public string Videoid
{
get { return videoid; }
set { videoid = value; }
}
private string videoname;
[SolrField("videoname")]
public string Videoname
{
get { return videoname; }
set { videoname = value; }
}
private string videoorigin;
[SolrField("videoorigin")]
public string Videoorigin
{
get { return videoorigin; }
set { videoorigin = value; }
}
}
[TestFixture]
public class VideoTests
{
[TestFixtureSetUp]
public void FixtureSetup()
{
Startup.Init<Video>("http://192.168.0.113/solr");
}
[Test]
public void Add() {
Video p = new Video("1","test video","Solr Test");
ISolrOperations<Video> solr = ServiceLocator.Current.GetInstance<ISolrOperations<Video>>();
solr.Add(p);
solr.Commit();
}
[Test]
public void Query()
{
ISolrOperations<Video> solr = ServiceLocator.Current.GetInstance<ISolrOperations<Video>>();
SolrQueryResults<Video> results = solr.Query(new SolrQueryByField("videoid", "33013"));
Assert.AreEqual(1, results.Count);
Console.WriteLine(results[0].Videoname);
}
}
Update:
Try this in a console application and see if it works...
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using SolrNet;
using Microsoft.Practices.ServiceLocation;
namespace Test1
{
class Program
{
static void Main(string[] args)
{
Startup.Init<Video>("http://192.168.0.113:8070/solr");
var solr = ServiceLocator.Current.GetInstance<ISolrOperations<Video>>();
var video = new Video("1", "test", "test");
solr.Add(video);
solr.Commit();
var results = solr.Query(SolrQuery.All);
Console.WriteLine("{0} - {1} - {2}",
results[0].Videoid, results[0].Videoname, results[0].Videoorigin);
}
}
public class Video
{
public Video(string id, string name, string origin)
{
Videoid = id;
Videoname = name;
Videoorigin = origin;
}
public string Videoid { get; set;
public string Videoname { get; set; }
public string Videoorigin { get; set; }
}
}
Can you check your project setup as suggested by Paige?
Do you have SolrNet source code added as project reference in your solution?
If you are using the dll, can you paste your .csproj file contents? If it is urgent, download the source code and add as project reference in your solution until you figure out the issue.

Resources