private string FindTaxItemLocation(string taxItemDescription)
{
if (!templateDS.Tables.Contains(cityStateTaxesTable.TableName))
throw new Exception("The schema dos not include city state employee/employer taxes table");
var cityStateTaxes =
templateDS.Tables[cityStateTaxesTable.TableName].AsEnumerable().FirstOrDefault(
x => x.Field<string>(Fields.Description.Name) == taxItemDescription);//[x.Field<string>(Fields.SteStateCodeKey.Name)]);
if (cityStateTaxes != null)
return cityStateTaxes[Fields.SteStateCodeKey.Name].ToString();
return null;
}
cityStateTaxes is a DataRow, why/how I cannot get the column value inside FirstOrDefault()?
Thanks,
FirstOrDefault() selects the first item in the collection (optionally that satisfies a predicate) or returns null in the case there it is empty (or nothing satisfies the predicate). It will not do projections for you. So if you use it, it can be awkward to access a field of the item since you must include default value checks.
My suggestion is to always project to your desired field(s) first before using FirstOrDefault(), that way you get your field straight without needing to perform the check.
var cityStateTaxes = templateDS.Tables[cityStateTaxesTable.TableName]
.AsEnumerable()
.Where(row => row.Field<string>(Fields.Description.Name) == taxItemDescription) // filter the rows
.Select(row => row.Field<string>(Fields.SteStateCodeKey.Name)) // project to your field
.FirstOrDefault(); // you now have your property (or the default value)
return cityStateTaxes;
Related
I'm pulling data from a third party api. The api runs multiple times in a day. So, if the same data is present in the table it should ignore that record, else if there are any changes it should update that record or insert a new record if anything new shows up in the json received.
I'm using the below code for inserting any new data.
var input = JsonConvert.DeserializeObject<List<DeserializeLookup>>(resultJson).ToList();
var entryset = input.Select(y => new Lookup
{
lookupType = "JOBCODE",
code = y.Code,
description = y.Description,
isNew = true,
lastUpdatedDate = DateTime.UtcNow
}).ToList();
await _context.Lookup.AddRangeAsync(entryset);
await _context.SaveChangesAsync();
But, after the first run, when the api runs again it's again inserting the same data in the table. As a result, duplicate entries are getting into table. To handle the same, I used a foreach loop as below before inserting data to the table.
foreach (var item in input)
{
if (!_context.Lookup.Any(r =>
r.code== item.Code))
{
//above insert code
}
}
But, the same doesn't work as expected. Also, the api takes a lot of time to run when I put a foreach loop. Is there a solution to this in .net core 3.1
List<DeserializeLookup> newList=new();
foreach (var item in input)
{
if (!_context.Lookup.Any(r =>
r.code== item.Code))
{
newList.add(item);
//above insert code
}
}
await _context.Lookup.AddRangeAsync(newList);
await _context.SaveChangesAsync();
It will be better if you try this way
I’m on my phone so forgive me for not being able to format the code in my response. The solution to your problem is something I actually just encountered myself while syncing data from an azure function and third party app and into a sql database.
Depending on your table schema, you would need one column with a unique identifier. Make this column a primary key (first step to preventing duplicates). Here’s a resource for that: https://www.w3schools.com/sql/sql_primarykey.ASP
The next step you want to take care of is your stored procedure. You’ll need to perform what’s commonly referred to as an UPSERT. To do this you’ll need to merge a table with the incoming data...on a specified column (whichever is your primary key).
That would look something like this:
MERGE
Table_1 AS T1
USING
Incoming_Data AS source
ON
T1.column1 = source.column1
/// you can use an AND / OR operator in here for matching on additional values or combinations
WHEN MATCHED THEN
UPDATE SET T1.column2= source.column2
//// etc for more columns
WHEN NOT MATCHED THEN
INSERT (column1, column2, column3) VALUES (source.column1, source.column2, source.column3);
First of all, you should decouple the format in which you get your data from your actual data handling. In your case: get rid of the JSon before you actually interpret the data.
Alas, I haven't got a clue what your data represents, so Let's assume your data is a sequence of Customer Orders. When you get new data, you want to Add all new orders, and you want to update changed orders.
So somewhere you have a method with input your json data, and as output a sequence of Orders:
IEnumerable<Order> InterpretJsonData(string jsonData)
{
...
}
You know Json better than I do, besides this conversion is a bit beside your question.
You wrote:
So, if the same data is present in the table it should ignore that record, else if there are any changes it should update that record or insert a new record
You need an Equality Comparer
To detect whether there are Added or Changed Customer Orders, you need something to detect whether Order A equals Order B. There must be at least one unique field by which you can identify an Order, even if all other values are of the Order are changed.
This unique value is usually called the primary key, or the Id. I assume your Orders have an Id.
So if your new Order data contains an Id that was not available before, then you are certain that the Order was Added.
If your new Order data has an Id that was already in previously processed Orders, then you have to check the other values to detect whether it was changed.
For this you need Equality comparers: one that says that two Orders are equal if they have the same Id, and one that says checks all values for equality.
A standard pattern is to derive your comparer from class EqualityComparer<Order>
class OrderComparer : EqualityComparer<Order>
{
public static IEqualityComparer<Order> ByValue = new OrderComparer();
... // TODO implement
}
Fist I'll show you how to use this to detect additions and changes, then I'll show you how to implement it.
Somewhere you have access to the already processed Orders:
IEnumerable<Order> GetProcessedOrders() {...}
var jsondata = FetchNewJsonOrderData();
// convert the jsonData into a sequence of Orders
IEnumerable<Order> orders = this.InterpretJsonData(jsondata);
To detect which Orders are added or changed, you could make a Dictonary of the already Processed orders and check the orders one-by-one if they are changed:
IEqualityComparer<Order> comparer = OrderComparer.ByValue;
Dictionary<int, Order> processedOrders = this.GetProcessedOrders()
.ToDictionary(order => order.Id);
foreach (Order order in Orders)
{
if(processedOrders.TryGetValue(order.Id, out Order originalOrder)
{
// order already existed. Is it changed?
if(!comparer.Equals(order, originalOrder))
{
// unequal!
this.ProcessChangedOrder(order);
// remember the changed values of this Order
processedOrder[order.Id] = Order;
}
// else: no changes, nothing to do
}
else
{
// Added!
this.ProcessAddedOrder(order);
processedOrder.Add(order.Id, order);
}
}
Immediately after Processing the changed / added order, I remember the new value, because the same Order might be changed again.
If you want this in a LINQ fashion, you have to GroupJoin the Orders with the ProcessedOrders, to get "Orders with their zero or more Previously processed Orders" (there will probably be zero or one Previously processed order).
var ordersWithTPreviouslyProcessedOrder = orders.GroupJoin(this.GetProcessedOrders(),
order => order.Id, // from every Order take the Id
processedOrder => processedOrder.Id, // from every previously processed Order take the Id
// parameter resultSelector: from every Order, with its zero or more previously
// processed Orders make one new:
(order, previouslyProcessedOrders) => new
{
Order = order,
ProcessedOrder = previouslyProcessedOrders.FirstOrDefault(),
})
.ToList();
I use GroupJoin instead of Join, because this way I also get the "Orders that have no previously processed orders" (= new orders). If you would use a simple Join, you would not get them.
I do a ToList, so that in the next statements the group join is not done twice:
var addedOrders = ordersWithTPreviouslyProcessedOrder
.Where(orderCombi => orderCombi.ProcessedOrder == null);
var changedOrders = ordersWithTPreviouslyProcessedOrder
.Where(orderCombi => !comparer.Equals(orderCombi.Order, orderCombi.PreviousOrder);
Implementation of "Compare by Value"
// equal if all values equal
protected override bool Equals(bool x, bool y)
{
if (x == null) return y == null; // true if both null, false if x null but y not null
if (y == null) return false; // because x not null
if (Object.ReferenceEquals(x, y) return true;
if (x.GetType() != y.GetType()) return false;
// compare all properties one by one:
return x.Id == y.Id
&& x.Date == y.Date
&& ...
}
For GetHashCode is one rule: if X equals Y then they must have the same hash code. If not equal, then there is no rule, but it is more efficient for lookups if they have different hash codes. Make a tradeoff between calculation speed and hash code uniqueness.
In this case: If two Orders are equal, then I am certain that they have the same Id. For speed I don't check the other properties.
protected override int GetHashCode(Order x)
{
if (x == null)
return 34339d98; // just a hash code for all null Orders
else
return x.Id.GetHashCode();
}
I'm trying to update a record given the customer Id, the row Id, and a dynamic column name.
Thus far I have the following, with the trouble spot marked by ***:
public void UpdateRecord(int Id, string rval, string column, string value)
{
var rId = GetRvalId(rval);
var entry = _context.Customers
.Where(x => x.Id == Id && x.RVals.Id == rId && x.***column?*** == column).First();
entry = value;
}
I haven't been able to find a good example of how to do this.
Addition after comments at the end
The reason you couldn't find examples is because it is not a good design.
Your method is very error prone, difficult to test and horrible to maintain. What if someone types the incorrect column name? What if you try to assign a string to the customer's birthday? And even if you would implement some string checking for column names and proposed values, then your program wouldn't work anymore after someone changes the names or the types of the columns.
So let's redesign!
Apparently you have a Customer with an Id and a property Rvals. This property Rvals also has a property Id.
You also have a function GetRValId that can convert a string rval to an int rvalId.
What you want, is given an Id and a string rval, you want to update one of the columns of the first Customer with this Idand rValId.
Side questions: Can there be more than one Customer with Id? In that case: are you sure Id is an ID? What do you want if there are more matching Customers? Update all customers or update only the first one? Which customer do you define as the first customer?
Leaving the side questions aside. We want a function signature that reports errors at compile time if you use non-existing customer properties, or if you try to assign a string to a Birthday. Something like this perhaps?
Update the name of the customer:
int customerId = ...
string rval = ...
string proposedName = "John Doe";
UpdateCustomerRecord(id, rval, customer => customer.Name = proposedName);
Update the Birthday of the customer:
DateTime proposedBirthday = ...
UpdateCustomerRecord(id, rval, customer => customer.Birthday = proposedBirthday)
This way you can't use any column that does not exist, and you can't assign a string to a DateTime.
You want to change two values in one call? Go ahead:
UpdateCustomerRecord(id, rval, customer =>
{
customer.Name = ...;
customer.Birthday = ...;
});
Convinced? Let's write the function:
public void UpdateCustomerRecord(int customerId, string rval, Action<Customer> action)
{
// the beginning is as in your function:
var rId = GetRvalId(rval);
// get the customer that you want to update:
using (var _Context = ...)
{
// get the customer you want to update:
var customerToUpdate = _Context.Customers
.Where(customer => customer.Id == Id
&& customer.RVals.Id == rId)
.FirstOrDefault();
// TODO: exception if there is no customerToUpdate
// perform the action and save the changes
action(customerToUpdate);
_context.SaveChanges();
}
Simple comme bonjour!
Addition after comments
So what does this function do? As long as you don't call it, it does nothing. But when you call it, it fetches a customer, performs the Action on the Customer you provided in the call, and finally calls SaveChanges.
It doesn't do this with every Customer, no it does this only with the Customer with Id equal to the provided Id and customer.RVals.Id == ... (are you still certain there is more than one customer with this Id? If there is only one, why check for RVals.Id?)
So the caller not only has to provide the Id, and the RVal, which define the Customer to update, but he also has to define what must be done with this customer.
This definition takes the form of:
customer =>
{
customer.Name = X;
customer.BirthDay = Y;
}
Well if you want, you can use other identifiers than customer, but it means the same:
x => {x.Name = X; x.BirthDay = Y;}
Because you put it on the place of the Action parameter in the call to UpdateCustomerRecord, I know that x is of type Customer.
The Acton statement means: given a customer that must be updated, what must we do with the customer? You can read it as if it was a Function:
void Action(Customer customer)
{
customer.Name = ...
customer.BirthDay = ...
}
In the end it will do something like:
Customer customerToUpdate = ...
customerToUpdate.Name = X;
customerToUpdate.BirthDay = Y;
SaveChanges();
So in the third parameter, called Action you can type anything you want, even call functions that have nothing to do with Customers (probably not wise). You have an input parameter of which you are certain that it is a Customer.
See my earlier examples of calling UpdateCustomerRecord, one final example:
UpdateCustomerRecord( GetCustomerId(), GetCustomerRVal,
// 3rd parameter: the actions to perform once we got the customerToUpdate:
customer =>
{
DateTime minDate = GetEarliestBirthDay();
if (customer.BirthDay < minDate)
{ // this Customer is old
customer.DoThingsThatOldPeopleDo();
}
else
{ // this Customer is young
customer.DoThingsThatYoungPeopleDo();
}
}
}
So the Action parameter is just a simpler way to say: "once you've got the Customer that must be updated, please perform this function with the Customer
So if you only want to update a given property of the customer write something like:
UpdateCustomerRecord(... , customer =>
{
Customer.PropertyThatMustBeUpdated = NewValueOfProperty;
}
Of course this only works if you know which property must be updated. But since you wrote "I am trying to update a specific cell." I assume you know which property the cells in this column represent.
It is not possible to pass the column name as the string value in LINQ. Alternate way to do it, if you have the limited number of the column name which can be passed then it can be achieved as below:
public void UpdateRecord(int Id, string rval, string column, string value)
{
var rId = GetRvalId(rval);
var entry = _context.Customers
.Where(x => x.Id == Id &&
x.RVals.Id == rId &&
(x.column1 == value || column == column1) &&
(x.column2 == value || column == column2) &&
(x.column3 == value || column == column3) &&
(x.column4 == value || column == column4) &&
(x.column5 == value || column == column5) &&
)).First();
entry = value;
}
UpdateRecord(5, "rval", "column1", "value");
UpdateRecord(5, "rval", "column2", "value");
UpdateRecord(5, "rval", "column3", "value");
Here, suppose you have the 5 columns that can be passed while calling the funcion UpdateRecord then you can add the 5 clauses in the WHERE as above.
Other way to do it dynamic LINQ
var entry = db.Customers.Where(column + " = " + value).Select(...);
I am trying to access a user object in a collection with the id = to users101 and set this to another users.
Controller.MyObject.SingleOrDefault(x => x.Id == "user101") = OtherUser();
Thanks in advance.
You can't do it with one LINQ expression.
Usually LINQ extensions works on enumerables, if MyObject is a collection you first have to find the required item and then overwrite it with the new object (moreover SingleOrDefault() will simply return null if condition is not satisfied).
You should write something like this (exact code depends on what MyObject is):
var item = Controller.MyObject.SingleOrDefault(x => x.Id == "user101");
if (item != null)
Controller.MyObject[Controller.MyObject.IndexOf(item)] = new OtherUser();
Please note that if you do not really need the check performed by SingleOrDefault() you can simplify the code (and avoid the double search performed in SingleOrDefault() and IndexOf()).
If this is "performance critical" maybe it is better to write an ad-hoc implementation that does this task in one single pass.
Try it in two lines:
var objectWithId = Controller.MyObject.SingleOrDefault(x => x.Id == "user101");
(objectWithId as WhateverTypeOfObjectOtherUserIs) = OtherUser();
I have method where I need to retrieve using EF the last ten entries in the database and check to see if there is a match between the value and the current term. Here is what I have thus far
public static int ValidatePassword(string username, string password, int securityUserId)
{
int validResult = 0;
/*Need to pass to client a value based upon success or failure of validation
* 0 - success
* 1 - password has already been used in the last 10 entries
* 2 - password does not meet CJIS requirements
*/
IEnumerable<string> oldpassword = null;
// Create a Regular Expression to determine whether or not special characters are present.
Regex regularExpression = new Regex("[^a-z0-9]");
//if id exists pull last ten passwords
if (securityUserId > 0)
{
long id = Convert.ToInt64(securityUserId);
using (var context = new SecurityEntities(string.Empty))
{
try
{
oldpassword = (from p in context.SecurityAudits
where p.SecurityUserId == id &&
p.OldPassword == password
orderby p.ActionDate descending
select p.OldPassword.Take(10).ToString()).ToList();
}
catch (Exception ex)
{
string err = string.Format("ValidateCJISPassword() was unable to return an object msg:{0}", ex.Message);
throw new Exception(err, ex.InnerException);
}
finally
{
context.Dispose();
}
}
}
else if (oldpassword == null)
{
//no matching record found now check other requirements
if ((password.Length >= DEFAULT_CJIS_PASSWORD_MIN_LENGTH) && regularExpression.IsMatch(password) && (password != username))
{
//success
validResult = 0;
}
else
{
//password does not meet standard CJIS requirements
validResult = 2;
}
}
else
{
//matching record was found
validResult = 1;
}
return validResult;
}
}
Where I am currently hung up is the query throws an exception on the ToString() method
LINQ to Entities does not recognize the method 'System.String ToString()' method, and this method cannot be translated into a store expression.
I'm still learning EF and how linq works so I'm not sure what the best approach here is. Should I try to set the result to something other than IEnumerable like an array or List or is there another approach I should consider?
Thanks in advance,
Cheers,
Change this
oldpassword = (from p in context.SecurityAudits
where p.SecurityUserId == id &&
p.OldPassword == password
orderby p.ActionDate descending
select p.OldPassword.Take(10).ToString()).ToList();
To this
oldpassword = (from p in context.SecurityAudits
where p.SecurityUserId == id &&
p.OldPassword == password
orderby p.ActionDate descending
select p.OldPassword).Take(10).ToList();
The problem was that your Take(10) clause was not part of the whole result but inside the actual linq statement.. it goes on the outside of it to take the top 10 of the entire resultset.. then you do the ToList() which turns the whole thing into an array
The next problem is that you just created an array and assigned it to oldpassword
I don't see anything here that does anything with the array...
You need to do something like:
declare your array of strings
assign the array to the return of the linq query
evaluate the return for > 0 results
if > 0 then the password has been used in the last 10
if = 0 then new password should be ok, correct?
Now that I have an understanding of what I needed in the query I was able to also update the linq statement as follows:
var lastTenPassword = (from p in context.SecurityAudits.Take(10)
orderby p.ActionDate descending
where p.SecurityUserId == id
select p.OldPassword).ToList();
string oldpassword = lastTenPassword.Where(a => a == password).FirstOrDefault();
Testing is further down the line but now by moving the .Take() method inside the query I am explicitly grabbing the top ten where as my first attempt would have retrieved all the records and then grabbed the top ten.
For testing you can also see where I broke out the initial where() to first grab all records by id and then perform a filter on that set by looking for a matching password within that set.
Thanks again for your help
I need to update all rows of my table when user visits one specific page.
All fields need to be updated to "false".
I tried this code:
var history = db.UserHistory.Where(m => m.UserID == id);
TryUpdateModel(history);
history.IsActive = false;
db.SaveChanges();
But it throws me an error message.
Anyone could help me with this update?
I couldn't get at first that LINQ didn`t update many rows at once, I have to make a loop and update each one. My final code is this one:
var history = db.UserHistory.Where(m => m.UserID == id).ToList();
TryUpdateModel(history);
history.ForEach(m => m.IsActive = false);
db.SaveChanges();
Since it seems like you're just selecting a single entry you want to probably want to append .FirstOrDefault() to your first line, which should appear with intellisense.
var history = db.UserHistory.Where(m => m.UserID == id).FirstOrDefault();
TryUpdateModel(history);
history.IsActive = false;
db.SaveChanges();
Right now you're returning an IQueryable as opposed to a single entry. FirstOrDefault will return a single UserHistory entity, so at least you'd be passing a single item to TryUpdateModel.