Optimal detection of circular reference in object (currently brute forcing) - algorithm

I have a Product object, which can depend on other products.
This dependency information is stored in a ServiceTemplate object.
These dependencies can be chained (i use the arrow to indicate that 3 depends on 2, and 2 depends on 1):
1 <= 2 <= 3
I need to prevent circular references, where with the above 1 could be set to depend on 3 this causing a loop.
This is hashed up from how i think it may work but is brute forcing the solution - what would be the optimal algorithm approach or is this already the best way?
class Program
{
class Product
{
public Product(int id)
{
this.id = id;
}
private readonly int id;
public int Id { get { return this.id; } }
}
class ServiceTemplate
{
// stores the list of other products that a product depends on
Dictionary<int, List<int>> relationships = new Dictionary<int, List<int>>();
public void SetRequires(Product on, Product requires)
{
if (WouldCauseCircularDepdndency(on.Id, requires.Id) == true)
throw new ArgumentException("circular dependencies not allowed");
List<int> list = null;
this.relationships.TryGetValue(on.Id, out list);
if(list==null)
{
list = new List<int>();
this.relationships.Add(on.Id, list);
}
list.Add(requires.Id);
}
private bool WouldCauseCircularDepdndency(int on, int requires)
{
// get relationships of product we will depend on
List<int> list = null;
this.relationships.TryGetValue(requires, out list);
if (list == null)
{
return false;
}
else if (list.Contains(on))
{
return true;
}
else
{
foreach (var id in list)
{
// traverse each child recursively
if (WouldCauseCircularDepdndency(on, id))
return true;
}
}
return false; // if we got this far, no circular references detected
}
}
static void Main(string[] args)
{
var windows = new Product(1);
var linux = new Product(2);
var mySql = new Product(3);
var ms_sql = new Product(4);
var cluster = new Product(5);
var other = new Product(6);
var config = new ServiceTemplate();
config.SetRequires(mySql, windows); // mySql requires windows
config.SetRequires(mySql, linux); // mySql requires linux
config.SetRequires(ms_sql, windows); // microsoft sql requires windows
config.SetRequires(cluster, ms_sql); // cluster requires microsoft sql
config.SetRequires(other, cluster);
// this should throw an exception due to circular dependency
config.SetRequires(windows, other);
/* at this point the config relationships dictionary is as follows:
3 => 1,2
4 => 1
5 => 4
5 => 6
1 => 6
*/
}
}

You could try topological sorting. If it can be constructed, you have no circular dependency. Otherwise, you have a cycle.

I ended up using the QuickGraph nuget package and an AdjacencyGraph<int,Edge<int>>, once the relationship is added i try and to a TopologicalSort() as advised by #Vesi:
class Program
{
class Product
{
public Product(int id)
{
this.id = id;
}
private readonly int id;
public int Id { get { return this.id; } }
}
class ServiceTemplate
{
// stores the list of other products that a product depends on
AdjacencyGraph<int, Edge<int>> relationshipGraph = new AdjacencyGraph<int, Edge<int>>();
public void SetRequires(Product on, Product requires)
{
var toAdd = new Edge<int>(on.Id, requires.Id);
this.relationshipGraph.AddVerticesAndEdge(toAdd);
try
{
var list = this.relationshipGraph.TopologicalSort();
}
catch (NonAcyclicGraphException)
{
this.relationshipGraph.RemoveEdge(toAdd);
throw new ArgumentException("Circular dependencies not allowed");
}
}
}
static void Main(string[] args)
{
var windows = new Product(1);
var linux = new Product(2);
var mySql = new Product(3);
var ms_sql = new Product(4);
var cluster = new Product(5);
var other = new Product(6);
var config = new ServiceTemplate();
config.SetRequires(mySql, windows); // mySql requires windows
config.SetRequires(mySql, linux); // mySql requires linux
config.SetRequires(ms_sql, windows); // microsoft sql requires windows
config.SetRequires(cluster, ms_sql); // cluster requires microsoft sql
config.SetRequires(other, cluster);
// this should throw an exception due to circular dependency
config.SetRequires(windows, other);
}
}

Related

WCF Service to display on a Winform its current data

I'm making a 2-stage "reservation management" program that other Software interacts with via WCF messages. Clients can claim a reservation, and release a reservation they previously had laid claim to. I'd like all this reservation info to be viewable on a WinForm for the host.
What I'm not sure how to accomplish is getting data from the Service into my WinForm. Im still very green when it comes to WCF stuff, I followed the MSDN guide to get that skeleton, but they didnt go into much detail tying to GUIs, or having any stored info.
My Service:
namespace ReservationServiceLib
{
public class reservation
{
public string location;
public bool claimed;
public string currentHolder;
public reservation(string l)
{
location = l;
claimed = false;
currentHolder = "host";
}
}
public class ReservationService : IReservationService
{
public bool reservationManagerLocked= false;
public List<reservation> keys = new List<reservation>();
private static Mutex managerLock = new Mutex();
public bool GetAccess(string n1)
{
//get acceess lock for using the reservationManager
return true;
}
public bool ClaimReservation(string rez, string clientN)
{
//Client requests/retrieves key
if(reservationManagerLocked!=true)
{
int i = 999;
i = keys.IndexOf(keys.Find(delegate (reservation x) { return x.location == rez; }));
if((keys[i].claimed == false) && (i < 999))
{
i = 999;
keys[i].claimed = true;
keys[i].currentHolder = clientN;
return true;
}
}
return false;
}
public bool ReleaseReservation(string n1)
{
//Client relenquishes access to key
return true;
}
And my "host" Winform:
namespace ReservationManager
{
public partial class ReservationManager : Form
{
public void populateComboBox()
{
for (int x = 0; x < dataGridView_IZone.Rows.Count-1; x++)
{
//comboBox_keyNames.Items.Add(dataGridView_IZone.Rows[x].Cells[0].Value);
}
}
public IZoneManager()
{
//create Keys and add to service's store of keys
//setup GUI
InitializeComponent();
// Step 1: Create a URI to serve as the base address.
Uri baseAddress = new Uri("http://localhost:8000/ReservationServiceLib/");
CalculatorService CSs = new CalculatorService();
// Step 2: Create a ServiceHost instance.
ServiceHost selfHost = new ServiceHost(typeof(ReservationService), baseAddress);
try
{
// Step 3: Add a service endpoint.
selfHost.AddServiceEndpoint(typeof(IReservation), new WSHttpBinding(), "ReservationService");
// Step 4: Enable metadata exchange.
ServiceMetadataBehavior smb = new ServiceMetadataBehavior();
smb.HttpGetEnabled = true;
selfHost.Description.Behaviors.Add(smb);
// Step 5: Start the service.
selfHost.Open();
Console.WriteLine("The service is ready.");
}
catch (CommunicationException ce)
{
Console.WriteLine("An exception occurred: {0}", ce.Message);
selfHost.Abort();
}
}
}
}

Programmatically cloning Octopus Deploy Process steps and modifyng the cloned steps

We are developing a Pipeline for which we have to add over 100 steps and modify two things for each step: Step Name and PackageID.
Rather than going through the pain of doing this via the UI, we’d like to do this programmatically.
Below is some C# I’ve sketched out for this (I’m a C# developer with extremely limited PowerShell skills, that’s why I did this in C#).
The lines above the comment “From here on is where I'm fuzzy” are working code, but the lines below the comment are just pseudocode.
Can someone explain to me how to write the lines below the comment (or, the PowerShell equivalent)?
I wasn’t able to find API calls for this.
Thanks
namespace ODClientExample
{
class Program
{
static void Main(string[] args)
{
List<string> ListOfWindowsServices = new List<string>();
ListOfWindowsServices.Add("svc1");
ListOfWindowsServices.Add("svc2");
ListOfFWindowsServices.Add("svc3");
var server = "https://mysite.whatever/";
var apiKey = "API-xxxxxxxxxxxxxxxxxx"; // I generated this via the Octopus UI
var endpoint = new OctopusServerEndpoint(server, apiKey);
var repository = new OctopusRepository(endpoint);
var project = repository.Projects.FindByName("Windows Services");
// From here on is where I'm fuzzy:
//
var procesSteps = GetProcessSteps(project);
var processStepToClone = GetProcesStepByName(processSteps, "SomeProcessStep");
foreach (string svcName in ListofSvcNames)
{
processStepToClone.StepName = svcName;
processStepToClone.PackageID = svcName;
}
}
}
}
I've made a little more progress. I'm now able to access the Steps in the Process, and add a Step. However, when my code calls repository.DeploymentProcesses.Modify, I get this exception:
Please provide a value for the package ID.
Please select the feed that this package will be downloaded from.
Please select one or more roles that 'svc1' step will apply to.
Here's my latest code:
static void Main(string[] args)
{
List<string> ListOfFexWindowsServices = new List<string>();
ListOfFexWindowsServices.Add("svc2");
ListOfFexWindowsServices.Add("svc3");
ListOfFexWindowsServices.Add("svc4");
string server = "https://mysite.stuff/";
string apiKey = "API-xxxxxxxxxxxxxxxxxxxxxxxx"; // I generated this via the Octopus UI
OctopusServerEndpoint endpoint = new OctopusServerEndpoint(server, apiKey);
OctopusRepository repository = new OctopusRepository(endpoint);
ProjectResource projectResource = repository.Projects.FindByName("MyProject");
DeploymentProcessResource deploymentProcess = repository.DeploymentProcesses.Get(projectResource.DeploymentProcessId);
var projectSteps = deploymentProcess.Steps;
DeploymentStepResource stepToClone = new DeploymentStepResource();
foreach (DeploymentStepResource step in projectSteps)
{
if (step.Name == "svc1")
{
stepToClone = step;
break;
}
}
foreach (string serviceName in ListOfFexWindowsServices)
{
DeploymentStepResource newStep = new DeploymentStepResource();
PopulateNewStep(newStep, stepToClone, serviceName);
deploymentProcess.Steps.Add(newStep);
repository.DeploymentProcesses.Modify(deploymentProcess);
}
}
static void PopulateNewStep(DeploymentStepResource newStep, DeploymentStepResource stepToClone, string serviceName)
{
newStep.Name = serviceName;
newStep.Id = Guid.NewGuid().ToString();
newStep.StartTrigger = stepToClone.StartTrigger;
newStep.Condition = stepToClone.Condition;
DeploymentActionResource action = new DeploymentActionResource
{
Name = newStep.Name,
ActionType = "Octopus.TentaclePackage",
Id = Guid.NewGuid().ToString(),
};
PopulateActionProperties(action);
newStep.Actions.Add(action);
// ISSUE: Anything else to do (eg, any other things from stepToClone to copy, or other stuff to create)?
newStep.PackageRequirement = stepToClone.PackageRequirement;
}
static void PopulateActionProperties(DeploymentActionResource action)
{
action.Properties.Add(new KeyValuePair<string, PropertyValueResource>("Octopus.Action.WindowsService.CustomAccountPassword", "#{WindowsService.Password}"));
// TODO: Repeat this sort of thing for each Action Property you see in stepToClone.
}
void Main()
{
var sourceProjectName = "<source project name>";
var targetProjectName = "<target project name>";
var stepToCopyName = "<step name to copy>";
var repo = GetOctopusRepository();
var sourceProject = repo.Projects.FindByName(sourceProjectName);
var targetProject = repo.Projects.FindByName(targetProjectName);
if (sourceProject != null && targetProject != null)
{
var sourceDeploymentProcess = repo.DeploymentProcesses.Get(sourceProject.DeploymentProcessId);
var targetDeploymentProcess = repo.DeploymentProcesses.Get(targetProject.DeploymentProcessId);
if (sourceDeploymentProcess != null && targetDeploymentProcess != null)
{
Console.WriteLine($"Start copy from project '{sourceProjectName}' to project '{targetProjectName}'");
CopyStepToTarget(sourceDeploymentProcess, targetDeploymentProcess, stepToCopyName);
// Update or add the target deployment process
repo.DeploymentProcesses.Modify(targetDeploymentProcess);
Console.WriteLine($"End copy from project '{sourceProjectName}' to project '{targetProjectName}'");
}
}
}
private OctopusRepository GetOctopusRepository()
{
var octopusServer = Environment.GetEnvironmentVariable("OCTOPUS_CLI_SERVER");
var octopusApiKey = Environment.GetEnvironmentVariable("OCTOPUS_CLI_API_KEY");
var endPoint = new OctopusServerEndpoint(octopusServer, octopusApiKey);
return new OctopusRepository(endPoint);
}
private void CopyStepToTarget(DeploymentProcessResource sourceProcess, DeploymentProcessResource targetProcess, string sourceStepName, bool includeChannels = false, bool includeEnvironments = false)
{
var sourceStep = sourceProcess.FindStep(sourceStepName);
if (sourceStep == null)
{
Console.WriteLine($"{sourceStepName} not found in {sourceProcess.ProjectId}");
return;
}
Console.WriteLine($"-> copy step '{sourceStep.Name}'");
var stepToAdd = targetProcess.AddOrUpdateStep(sourceStep.Name);
stepToAdd.RequirePackagesToBeAcquired(sourceStep.RequiresPackagesToBeAcquired);
stepToAdd.WithCondition(sourceStep.Condition);
stepToAdd.WithStartTrigger(sourceStep.StartTrigger);
foreach (var property in sourceStep.Properties)
{
if (stepToAdd.Properties.ContainsKey(property.Key))
{
stepToAdd.Properties[property.Key] = property.Value;
}
else
{
stepToAdd.Properties.Add(property.Key, property.Value);
}
}
foreach (var sourceAction in sourceStep.Actions)
{
Console.WriteLine($"-> copy action '{sourceAction.Name}'");
var targetAction = stepToAdd.AddOrUpdateAction(sourceAction.Name);
targetAction.ActionType = sourceAction.ActionType;
targetAction.IsDisabled = sourceAction.IsDisabled;
if (includeChannels)
{
foreach (var sourceChannel in sourceAction.Channels)
{
targetAction.Channels.Add(sourceChannel);
}
}
if (includeEnvironments)
{
foreach (var sourceEnvironment in sourceAction.Environments)
{
targetAction.Environments.Add(sourceEnvironment);
}
}
foreach (var actionProperty in sourceAction.Properties)
{
if (targetAction.Properties.ContainsKey(actionProperty.Key))
{
targetAction.Properties[actionProperty.Key] = actionProperty.Value;
}
else
{
targetAction.Properties.Add(actionProperty.Key, actionProperty.Value);
}
}
}
}
The above code sample is available in the Octopus Client Api Samples

ASP.NET MVC5 add 2 Sync Times to the Application

I Added the 2 Sync Times to my DB as 2 new columns and inserted values as below:
USE [DB]
ALTER TABLE [dbo].[TableName]
ADD ColumnName2 time, ColumnName3 time
This was for adding the columns.
For inserting the row values I did:
USE DB
INSERT INTO TableName (ColumnName2, ColumnName3)
VALUES ('20:30:00', '23:30:00')
This was the data for the fixed times in the rows of those columns.
I also went through all the layers of the application such as (controller, models, views, queries, services, interfaces, and so forth. NOW when I try to update any of the new times added they default to the first time that already existed on the table as a COLUMN with time in the row.
I could not post the image for the time fields from the application because it is not permitted. However, the image is in a little panel and consists of 3 fields (textboxfor) with a time picker for each one.
Any help would be greatly appreciated.
Thanks
Now I thought I would post some of the example code to see if this helps
// My controller method for those sync times
[HttpPost]
[Page(PageName.UpdateSystemConfigTime)]
public ActionResult UpdateTime(SystemMaintenanceViewModel model)
{
var dateTime = DateTime.ParseExact(model.SystemConfiguration.SynchronizationTime, "h:mm tt", CultureInfo.InvariantCulture);
var dateTime2 = DateTime.ParseExact(model.SystemConfiguration.SynchronizationTime2, "h:mm tt", CultureInfo.InvariantCulture);
var dateTime3 = DateTime.ParseExact(model.SystemConfiguration.SynchronizationTime3, "h:mm tt", CultureInfo.InvariantCulture);
//model.ProcessTime
if (model.SystemConfiguration.SynchronizationTime != null &&
model.SystemConfiguration.SynchronizationTime2 != null &&
model.SystemConfiguration.SynchronizationTime3 != null);
{
var sysConfig = new DTO.SystemSync.SystemConfiguration
{
SyncTime = dateTime.TimeOfDay,
SyncTime2 = dateTime2.TimeOfDay,
SyncTime3 = dateTime3.TimeOfDay
};
configService.UpdateSyncTime(sysConfig);
configService.UpdateSyncTime2(sysConfig);
configService.UpdateSyncTime3(sysConfig);
}
return RedirectToAction("Index");
}
////My Private method
private SystemConfiguration GetSystemConfig()
{
var model = new SystemConfiguration();
var config = configService.GetSyncTime();
configService.GetSyncTime2();
configService.GetSyncTime3();
if (config == null) return model;
var ts = config.SyncTime;
if (ts != null)
{
model.SynchronizationTime = ts.ToString();
}
var ts2 = config.SyncTime2;
if (ts2 != null)
{
model.SynchronizationTime2 = ts2.ToString();
}
var ts3 = config.SyncTime3;
if (ts3 != null)
{
model.SynchronizationTime3 = ts3.ToString();
}
return model;
============================================================================
/// My configuration command
namespace --.--.Commands
{
public class ConfigurationCommand : CommandBase, IConfigurationCommand
{
static ConfigurationCommand()
{
ConfigureAutoMapper();
}
private static void ConfigureAutoMapper()
{
Mapper.CreateMap<SystemConfiguration, entity.SystemConfiguration>()
.ForMember(dest => dest.SyncTime, opt => opt.ResolveUsing<TimeSpanToSqlTimeResolver>())
.ForMember(dest => dest.SyncTime2, opt => opt.ResolveUsing<TimeSpanToSqlTimeResolver>())
.ForMember(dest => dest.SyncTime3, opt => opt.ResolveUsing<TimeSpanToSqlTimeResolver>());
}
public void UpdateSyncTime(SystemConfiguration timeOfDay)
{
Guard.NotNull(timeOfDay);
var mapped = Mapper.Map<entity.SystemConfiguration>(timeOfDay);
var config = Context.SystemConfigurations.SingleOrDefault();
//if this is the first time, then we need to insert
if (config == null)
{
var newConfig = new entity.SystemConfiguration
{
SyncTime = mapped.SyncTime
};
Context.SystemConfigurations.Add(newConfig);
}
else
{
config.SyncTime = mapped.SyncTime;
}
SaveChanges();
}
public void UpdateSyncTime2(SystemConfiguration timeOfDay)
{
Guard.NotNull(timeOfDay);
var mapped = Mapper.Map<entity.SystemConfiguration>(timeOfDay);
var config = Context.SystemConfigurations.SingleOrDefault();
if (config == null)
{
var newConfig = new entity.SystemConfiguration
{
SyncTime2 = mapped.SyncTime2
};
Context.SystemConfigurations.Add(newConfig);
}
else
{
config.SyncTime2 = mapped.SyncTime2;
}
SaveChanges();
}
public void UpdateSyncTime3(SystemConfiguration timeOfDay)
{
Guard.NotNull(timeOfDay);
var mapped = Mapper.Map<entity.SystemConfiguration>(timeOfDay);
var config = Context.SystemConfigurations.SingleOrDefault();
if (config == null)
{
var newConfig = new entity.SystemConfiguration
{
SyncTime3 = mapped.SyncTime3
};
Context.SystemConfigurations.Add(newConfig);
}
else
{
config.SyncTime3 = mapped.SyncTime3;
}
SaveChanges();
}
}
}
=========================================================================================================
// My configuration service
namespace --.--.--.SystemSync
{
public class ConfigurationService : IConfigurationService
{
private IConfigurationQuery query;
private IConfigurationCommand command;
public ConfigurationService(IConfigurationQuery query,IConfigurationCommand command)
{
this.query = query;
this.command = command;
}
public void UpdateSyncTime(SystemConfiguration timeOfDay)
{
command.UpdateSyncTime(timeOfDay);
}
public void UpdateSyncTime2(SystemConfiguration timeOfDay)
{
command.UpdateSyncTime2(timeOfDay);
}
public void UpdateSyncTime3(SystemConfiguration timeOfDay)
{
command.UpdateSyncTime3(timeOfDay);
}
public SystemConfiguration GetSyncTime()
{
return query.GetSyncTime();
}
public SystemConfiguration GetSyncTime2()
{
return query.GetSyncTime2();
}
public SystemConfiguration GetSyncTime3()
{
return query.GetSyncTime3();
}
public List<PageResource> GetPages()
{
return query.GetPages().ToList();
}
}
}
You made a comment about fixed times, Are you looking for something like this?
CREATE TABLE [dbo].[Zamen](
[Id] [int] IDENTITY(1,1) NOT NULL,
[time1] [time](3) NOT NULL,
[time2] [time](3) NOT NULL,
[Content] [varchar](100) NULL,
CONSTRAINT [PK_Zamen] PRIMARY KEY CLUSTERED
(
[Id] ASC
))
GO
ALTER TABLE [dbo].[Zamen] ADD CONSTRAINT [DF_Zamen_time1] DEFAULT (getdate()) FOR [time1]
GO
ALTER TABLE [dbo].[Zamen] ADD CONSTRAINT [DF_Zamen_time2] DEFAULT (getdate()) FOR [time2]
GO
Those alter table statements allow the time to be automatically inserted. So when you do this:
INSERT INTO Zamen (Content) VALUES ('demo')
The current times are placed into the values.
*After seeing the code you added, some input:
In your UpdateTime Action Method, a problem that stands out is you are calling UpdateTimeSync three times, but passing it all three variables each time. I would suggest to refactor your update method -- instead of three update methods, use one update method for all time variables.

Entity framework 6 code first Many to many insert slow

I am trying to find a way to improve insert performances with the following code (please, read my questions after the code block):
//Domain classes
[Table("Products")]
public class Product
{
[Key]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int Id { get; set; }
public string Sku { get; set; }
[ForeignKey("Orders")]
public virtual ICollection<Order> Orders { get; set; }
public Product()
{
Orders = new List<Order>();
}
}
[Table("Orders")]
public class Order
{
[Key]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int Id { get; set; }
public string Title { get; set; }
public decimal Total { get; set; }
[ForeignKey("Products")]
public virtual ICollection<Product> Products { get; set; }
public Order()
{
Products = new List<Product>();
}
}
//Data access
public class MyDataContext : DbContext
{
public MyDataContext()
: base("MyDataContext")
{
Configuration.LazyLoadingEnabled = true;
Configuration.ProxyCreationEnabled = true;
Database.SetInitializer(new CreateDatabaseIfNotExists<MyDataContext>());
}
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Entity<Product>().ToTable("Products");
modelBuilder.Entity<Order>().ToTable("Orders");
}
}
//Service layer
public interface IServices<T, K>
{
T Create(T item);
T Read(K key);
IEnumerable<T> ReadAll(Expression<Func<IEnumerable<T>, IEnumerable<T>>> pre);
T Update(T item);
void Delete(K key);
void Save();
void Dispose();
void BatchSave(IEnumerable<T> list);
void BatchUpdate(IEnumerable<T> list, Action<UpdateSpecification<T>> spec);
}
public class BaseServices<T, K> : IDisposable, IServices<T, K> where T : class
{
protected MyDataContext Context;
public BaseServices()
{
Context = new MyDataContext();
}
public T Create(T item)
{
T created;
created = Context.Set<T>().Add(item);
return created;
}
public void Delete(K key)
{
var item = Read(key);
if (item == null)
return;
Context.Set<T>().Attach(item);
Context.Set<T>().Remove(item);
}
public T Read(K key)
{
T read;
read = Context.Set<T>().Find(key);
return read;
}
public IEnumerable<T> ReadAll(Expression<Func<IEnumerable<T>, IEnumerable<T>>> pre)
{
IEnumerable<T> read;
read = Context.Set<T>().ToList();
read = pre.Compile().Invoke(read);
return read;
}
public T Update(T item)
{
Context.Set<T>().Attach(item);
Context.Entry<T>(item).CurrentValues.SetValues(item);
Context.Entry<T>(item).State = System.Data.Entity.EntityState.Modified;
return item;
}
public void Save()
{
Context.SaveChanges();
}
}
public interface IOrderServices : IServices<Order, int>
{
//custom logic goes here
}
public interface IProductServices : IServices<Product, int>
{
//custom logic goes here
}
//Web project's controller
public ActionResult TestCreateProducts()
{
//Create 100 new rest products
for (int i = 0; i < 100; i++)
{
_productServices.Create(new Product
{
Sku = i.ToString()
});
}
_productServices.Save();
var products = _productServices.ReadAll(r => r); //get a list of saved products to add them to orders
var random = new Random();
var orders = new List<Order>();
var count = 0;
//Create 3000 orders
for (int i = 1; i <= 3000; i++)
{
//Generate a random list of products to attach to the current order
var productIds = new List<int>();
var x = random.Next(1, products.Count() - 1);
for (int j = 0; j < x; j++)
{
productIds.Add(random.Next(products.Min(r => r.Id), products.Max(r => r.Id)));
}
//Create the order
var order = new Order
{
Title = "Order" + i,
Total = i,
Products = products.Where(p => productIds.Contains(p.Id))
};
orders.Add(order);
}
_orderServices.CreateRange(orders);
_orderServices.Save();
return RedirectToAction("Index");
}
This code works fine but is very VERY slow when the SaveChanges is executed.
Behind the scene, the annotations on the domain objects creates all the relationships needed: a OrderProducts table with the proper foreign keys are automatically created and the inserts are being done by EF properly.
I've tried many things with bulk inserts using EntityFramework.Utilities, SqlBulkCopy, etc... but none worked.
Is there a way to achieve this?
Understand this is only for testing purposes and my goal is to optimize the best I can any operations in our softwares using EF.
Thanks!
Just before you do your inserts disable your context's AutoDetectChangesEnabled (by setting it to false). Do your inserts and then set the AutoDetectChangesEnabled back to true e.g.;
try
{
MyContext.Configuration.AutoDetectChangesEnabled = false;
// do your inserts updates etc..
}
finally
{
MyContext.Configuration.AutoDetectChangesEnabled = true;
}
You can find more information on what this is doing here
I see two reasons why your code is slow.
Add vs. AddRange
You add entity one by one using the Create method.
You should always use AddRange over Add. The Add method will try to DetectChanges every time the add method is invoked while AddRange only once.
You should add a "CreateRange" method in your code.
public IEnumerable<T> CreateRange(IEnumerable<T> list)
{
return Context.Set<T>().AddRange(list);
}
var products = new List<Product>();
//Create 100 new rest products
for (int i = 0; i < 100; i++)
{
products.Add(new Product { Sku = i.ToString() });
}
_productServices.CreateRange(list);
_productServices.Save();
Disabling / Enabling the property AutoDetectChanges also work as #mark_h proposed, however personally I don't like this kind of solution.
Database Round Trip
A database round trip is required for every record to add, modify or delete. So if you insert 3,000 records, then 3,000 database round trip will be required which is VERY slow.
You already tried EntityFramework.BulkInsert or SqlBulkCopy, which is great. I recommend you first to try them again using the "AddRange" fix to see the newly performance.
Here is a biased comparison of library supporting BulkInsert for EF:
Entity Framework - Bulk Insert Library Reviews & Comparisons
Disclaimer: I'm the owner of the project Entity Framework Extensions
This library allows you to BulkSaveChanges, BulkInsert, BulkUpdate, BulkDelete and BulkMerge within your Database.
It supports all inheritances and associations.
// Easy to use
public void Save()
{
// Context.SaveChanges();
Context.BulkSaveChanges();
}
// Easy to customize
public void Save()
{
// Context.SaveChanges();
Context.BulkSaveChanges(bulk => bulk.BatchSize = 100);
}
EDIT: Added answer to sub question
An entity object cannot be referenced by multiple instances of
IEntityChangeTracker
The issue happens because you use two different DbContext. One for the product and one for order.
You may find a better answer than mine in a different thread like this answer.
The Add method successfully attach the product, subsequent call of the same product doesn't throw an error because it's the same product.
The AddRange method, however, attach the product multiple time since it's not come from the same context, so when Detect Changes is called, he doesn't know how to handle it.
One way to fix it is by re-using the same context
var _productServices = new BaseServices<Product, int>();
var _orderServices = new BaseServices<Order, int>(_productServices.Context);
While it may not be elegant, the performance will be improved.

Updating a datatable in Wicket

We are developing a new application in Wicket and have run into a small problem.
What we do:
1) create a new SortableDataProvider
2) create a new DefaultDataTablePagingInBottom
3) create a new WebMarkupContainer
4) add the DefaultDataTablePagingInBottom to the WebMarkupContainer
5) create a new AjaxCheckBox
6) in the onUpdate of the AjaxCheckBox, add the WebMarkupContainer to the AjaxRequestTarget
7) set the SortableDataProvider to a new SortableDataProvider (with the updated query)
8) DefaultDataTablePagingInBottom.replaceWith(new DefaultDataTablePagingInBottom - with the new provider).
What happends:
1) Click the checkbox -> nothing happends.
2) Click it again -> crash: "Last cause: This method can only be called on a component that has already been added to its parent.
WicketMessage: Method onRequest of interface org.apache.wicket.behavior.IBehaviorListener targeted at org.apache.wicket.ajax.markup.html.form.AjaxCheckBox$1#1a2fefd on component [ [Component id = checkBox]] threw an exception"
3) Click back in the browser -> the list i filtered with the new provider.
Any ideas?
---EDIT---
Here's some code.
1) In the constructor of the WebPage:
model = new Model(projectPlannerService);
provider = new SortableProjectDataProvider(model, (WebSession) getSession(), isChecked);
table = new DefaultDataTablePagingInBottom("table", columns, provider, 50);
listContainer = new WebMarkupContainer("wmc");
listContainer.add(table);
add(listContainer.setOutputMarkupId(true));
/*
* checkbox för filtrering
*/
AjaxCheckBox checkBox = new AjaxCheckBox("checkBox", new Model()) {
#Override
protected void onUpdate(AjaxRequestTarget target) {
target.add(listContainer, "wmc");
isChecked = !isChecked;
provider = new SortableProjectDataProvider(model, (WebSession) getSession(), isChecked);
updateTable();
}
};
add(checkBox);
2) In updateTable():
table.replaceWith(new DefaultDataTablePagingInBottom("table", columns, provider, 50));
3) The SortableProjectDataProvider:
// Constructor
public SortableProjectDataProvider(IModel<?> model, WebSession webSession, boolean isChecked) {
this.model = model;
this.projectPlannerService = (ProjectPlannerService) model.getObject();
this.webSession = webSession;
setSort("customer", SortOrder.ASCENDING);
System.out.println("ischecked:" + isChecked);
if(!isChecked)
list = ((ProjectPlannerService) model.getObject()).findAllProjects();
else
list = ((ProjectPlannerService) model.getObject()).findAllActiveProjects();
System.out.println("size: " + list.size());
comparator = new ProjectComparator();
}
public Iterator<Project> iterator(int first, int count) {
Collections.sort(list, comparator);
if (first > list.size()) {
first = 0;
}
if (first + count > list.size()) {
return list.subList(first, list.size()).iterator();
} else {
return list.subList(first, first + count).iterator();
}
}
public IModel<Project> model(Project object) {
return new DetachableProjectModel((Project) object);
}
public int size() {
return list.size();
}
private class DetachableProjectModel extends LoadableDetachableModel {
private Long id;
#SpringBean
ProjectPlannerService projectPlannerService;
public DetachableProjectModel(Long id) {
Injector.get().inject(this);
if (id == null) {
throw new IllegalArgumentException();
}
this.id = id;
}
public DetachableProjectModel(Project project) {
this(project.getPk());
Injector.get().inject(this);
}
public int hashCode() {
return id.hashCode();
}
public boolean equals(final Object obj) {
if (obj == this) {
return true;
} else if (obj == null) {
return false;
} else if (obj instanceof DetachableProjectModel) {
DetachableProjectModel other = (DetachableProjectModel) obj;
return other.id == this.id;
}
return false;
}
protected Object load() {
return ((ProjectPlannerService) model.getObject()).findProjectById(id);
}
}
}
wicket:extend
-input wicket:id="checkBox" type="checkbox"- Show active -/input-
-div wicket:id="wmc"-
-table wicket:id="table"--/table-
-/div-
-/wicket:extend-
Thanks in advance!
/Andreas
By replacing the instance of your SortableProjectDataProvider with a new one you are making your life difficult. Instead of using the boolean isChecked in the constructor you could use an IModel<Boolean>. Assign the same instance of that model to your data provider and the check-box and you are done. No need to replace anything in onUpdate, add your listContainer to the AjaxRequestTarget and everything should just work...
e.g.
...
private IModel<Boolean> isCheckedModel = new Model<Boolean>(Boolean.FALSE);
...
provider = new SortableProjectDataProvider(model, (WebSession) getSession(), isCheckedModel);
...
AjaxCheckBox checkBox = new AjaxCheckBox("checkBox", isCheckedModel) {
#Override
protected void onUpdate(AjaxRequestTarget target) {
target.add(listContainer);
}
};
...
It is almost never a good idea to replace such things with new ones in Wicket. Encapsulate what changes in a model and change / replace the model's object. Every object that has a reference to that model can see the updated value or change it as needed.
Hope this helps.
Try this:
Wrong: target.add(listContainer, "wmc");
Right: target.add(listContainer);
Wrong; table.replaceWith(new DefaultDataTablePagingInBottom("table", columns, provider, 50));
Right: DefaultDataTablePagingInBottom tmp = new DefaultDataTablePagingInBottom("table", columns, provider, 50);
table.replaceWith(tmp);
table = tmp;
(You replace the DefaultDataTablePagingInBottom but not your reference.)
//olle

Resources