I have a service that pulls statistics for a sales region. The service computes the stats for ALL regions and then caches that collection, then returns only the region requested.
public object Any(RegionTotals request)
{
string cacheKey = "urn:RegionTotals";
//make sure master list is in the cache...
base.Request.ToOptimizedResultUsingCache<RegionTotals>(
base.Cache, cacheKey, CacheExpiryTime.DailyLoad(), () =>
{
return RegionTotalsFactory.GetObject();
});
//then retrieve them. This is all teams
RegionTotals tots = base.Cache.Get<RegionTotals>(cacheKey);
//remove all except requested
tots.Records.RemoveAll(o => o.RegionID != request.RegionID);
return tots;
}
What I'm finding is that when I use a MemoryCacheClient (as part of a StaticAppHost that I use for Unit Tests), the line tots.Records.RemoveAll(...) actually affects the object in the cache. This means that I get the cached object, delete rows, and then the cache no longer contains all regions. Therefore, subsequent calls to this service for any other region return no records. If I use my normal Cache, of course the Cache.Get() makes a new copy of the object in the cache, and removing records from that object doesn't affect the cache.
This is because an In Memory cache doesn't add any serialization overhead and just stores your object instances in memory. Whereas when you use any of the other Caching Providers your values are serialized first then sent to the remote Caching Provider then when it's retrieved it's deserialized back so it's never reusing the same object instances.
If you plan on mutating cached values you'll need to clone the instances before mutating them, if you don't want to manually implement ICloneable you can serialize and deserialize them with:
var clone = TypeSerializer.Clone(obj);
Related
I have a service that indexes documents.
The service receives two following requests - the first is to insert a document and the second to delete it.
When there is some time between them it works fine but when they are sent one after another the document is not deleted.
The response from I get from Nest looks successful.
My function is kinds long so I will only write the insert and delete inside. If more info is needed I will add it (For example, in case of insertion in also deletes it from all other available indices and inserts some mapping if needed).
Insert code:
IBulkResponse res = await _client.IndexManyAsync(entities, index, type);
Delete Code:
var termFilter = new List<Func<QueryContainerDescriptor<JObject>, QueryContainer>>
{
c => c.Terms(t => t.Field(ID_FIELD).Terms(ids))
};
await _client.DeleteByQueryAsync<JObject>(indices, types, d => d.Query(q => q.Bool(b => b.Must(termFilter))));
For example, this integration test doesn't work:
var indices = new { "some_index_1", "some_index_2" };
var entity = new Entity { Action = ReplicationAction.INSERT, ... };
await elasticDal.Insert(new List { entity }, "some_index_1", "666", indices);
entity.Action = ReplicationAction.DELETE;
await elasticDal.Insert(new List { entity }, "some_index_1", "666", indices);
Versions: ElasticSearch 2.3.5, .Net 4.6, Nest 2.4.6
When you are inserting any document following steps happens:
Document is added to the in-memory buffer and appended to the translog.
Refresh
The docs in the in-memory buffer are written to a new segment, without
a Fsync.
a. The segment is opened to make it visible to search.
b. The in-memory buffer is cleared.
The segment is opened to make it visible to search.
Every so often—such as when the translog is getting too big—the index is
flushed; a new translog is created, and a full commit is performed :
a. Any docs in the in-memory buffer are written to a new segment.
b. The buffer is cleared.
c. A commit point is written to disk.
d. The filesystem cache is flushed with a Fsync.
e. The old translog is deleted.
Elasticsearch doesn't delete the document. It marks the document as a deleted document, and while merging the index segments, ES leaves deleted document if it is in memory.
So my guess is you are missing refresh API after deleting.
If your DELETE API is not so frequent, then you can refresh your ES after calling DELETE API by calling REFRESH API.
If you want to learn more about how indexing is happing behind the picture, you can refer to this link (https://www.elastic.co/guide/en/elasticsearch/guide/current/translog.html)
I have an application which performs the following steps:
Places object in session:
def product = Product.get(1)
session["product"] = product
Performs and Ajax call to update a 1-m relationship, and then renders a partial template which displays the new benefits. These should not yet be saved as the user may change their mind, so discard is called:
def product = session["product"]
if ( !product.isAttached() ) {
product.attach()
}
product.addToBenefits( new Benefit( title: "xx" ) )
product.discard()
session["product"] = product
Attempts to save the object in a save action.
def product = session["product"]
if ( !product.isAttached() ) {
product.attach()
}
product.save()
At this point we get the following exception:
org.springframework.orm.hibernate3.HibernateSystemException: reassociated object has dirty collection; nested exception is org.hibernate.HibernateException: reassociated object has dirty collection
Is there anyway to stop this happening, so that I can re-Attach the object, and save it, thus persisting the changes to the products benefits collection?
Don't store the object in the session, store the id, and reload it instead. You're incurring that cost anyway with attach, so you're not saving anything, and causing this problem, plus wasting server memory, which will affect scalability.
Be sure the instance doesn't have unsaved state before calling attach. If you have changed a dettached entity instance and left it in unsaved state, calling attach will throw HibernateSystemException: reassociated object has dirty collection because to reassociate a transient instance with a session Grails uses Session.LockRequest.lock in the attach call chain:
session.buildLockRequest(new LockOptions(lockMode)).lock(entity);
To reassociate a dettached instace with unsaved state call Session.update, Session.merge or Session.saveOrUpdate:
if ( !product.attached ) {
product.withSession { session -> session.saveOrUpdate(product) }
}
Still, storing only the id and reloading is the best option in OP's case as already mentioned. Most of the times that will be the best choice. For instance, you can use load or get:
def product = Product.load(session["productId"])
Why isn't the exception triggered? Linq's "Any()" is not considering the new entries?
MyContext db = new MyContext();
foreach (string email in {"asdf#gmail.com", "asdf#gmail.com"})
{
Person person = new Person();
person.Email = email;
if (db.Persons.Any(p => p.Email.Equals(email))
{
throw new Exception("Email already used!");
}
db.Persons.Add(person);
}
db.SaveChanges()
Shouldn't the exception be triggered on the second iteration?
The previous code is adapted for the question, but the real scenario is the following:
I receive an excel of persons and I iterate over it adding every row as a person to db.Persons, checking their emails aren't already used in the db. The problem is when there are repeated emails in the worksheet itself (two rows with the same email)
Yes - queries (by design) are only computed against the data source. If you want to query in-memory items you can also query the Local store:
if (db.Persons.Any(p => p.Email.Equals(email) ||
db.Persons.Local.Any(p => p.Email.Equals(email) )
However - since YOU are in control of what's added to the store wouldn't it make sense to check for duplicates in your code instead of in EF? Or is this just a contrived example?
Also, throwing an exception for an already existing item seems like a poor design as well - exceptions can be expensive, and if the client does not know to catch them (and in this case compare the message of the exception) they can cause the entire program to terminate unexpectedly.
A call to db.Persons will always trigger a database query, but those new Persons are not yet persisted to the database.
I imagine if you look at the data in debug, you'll see that the new person isn't there on the second iteration. If you were to set MyContext db = new MyContext() again, it would be, but you wouldn't do that in a real situation.
What is the actual use case you need to solve? This example doesn't seem like it would happen in a real situation.
If you're comparing against the db, your code should work. If you need to prevent dups being entered, it should happen elsewhere - on the client or checking the C# collection before you start writing it to the db.
How to use Cache.getOrElse(java.lang.String key, java.util.concurrent.Callable block, int expiration)
Could someone give me a example？
My point is how to use “expiration",I know it means expire time.
By the way:
I want save some object to cache,and set a expire time.
when the expire time,I can reset the object to the cache.
Thanks.
Let's assume that, you want to set User object on cache, for that you set userId as key and user object as value. If need set expiration time, for sample i set it as 30secs.
cache.set(userId, userObject, 30);
At some point of time, if you want to get user object from cache, which you set earlier using userId as key, you might try the following way to get the user object from cache.
User user = cache.get(userId);
Above will return you the user object, if you access within 30secs, otherwise it will return NULL. This will be perfect for case like validating the session.
In some case, you frequently need to retrieve value from cache, for that following is the best approach.
User user = cache.getOrElse(userId, () -> User.get(userId), 30);
cache will check, whether it has given userId as key, if available then straight away return the user object and update the expiration time to 30secs further.
If given userId not available, then callable block gets invoked and set userId as key, user object fetched from db as value and expiration time as 30secs.
Expiration is the number of seconds that the Object would be hold in the Cache. If you pass 0 as expiration the Cache doesn't expire and you would have to control it by hand.
What getOrElse does is check the Cache, if the Object is not there then call the callable block that you are passing and adds the result to the cache for the number of seconds that you are passing as expiration time.
I based my comment in the Play Framework Cache Javadoc.
I use getOrElse in controllers when I have dynamic and static content to display. Cache the static and then render it together with the dynamic part:
try {
Html staticHtml = Cache.getOrElse("static-content", () -> staticView.render(), 60 * 60);
Html rendered = dynamicPage.render(arg1, arg2, staticHtml);
return ok(rendered);
} catch (Exception e) {
e.printStackTrace();
return internalServerError();
}
staticView.render() returns some html from a view. This view should not call any other pages which are dynamic or you stash something you do not really want to stash.
60*60 means I want to store it for one hour (60 seconds times 60 minutes... ok you can write 3600 if you want)
I should add that getOrElse gets the Object from the cache with the specified key (in this example the key is static-content) but if it cannot find it, then it calls the function which returns an object which is then stored for the specified amount of time in the cache with that key. Pretty neat.
Then you can call some other (dynamic) page and pass the html to it.
The dynamic stuff will stay dynamic :)
I have a chicken and egg problem, it's trivial, so I thought I would ask what's the normal pattern to save an aggregate root where all the primary keys are identity fields.
I have a typical contact entity:
Customer {
HomeAddress {
}
WorkAddress {
}
}
where both addresses are stored in the address table and the only primary key is the identity field. We check all fields against each other to keep unique address records.
Here's the problem:
I need to retrieve the Address identity field to hookup the foreign keys, so I save the Address record prior to saving the Customer record only if it's unique, otherwise I load that existing Address.
If Address is in the same DC as Customer, then customer saves too prematurely (not all records are set.)
If Address is in a separate DC, then it doesn't hookup to the Customer record that has it's own DC because you cannot have an entity associated with two DCs (can't open in one, then save in another.)
So my thinking is that I would need a separate repository for every Address, then separately load the address again in the other DC, making redundant calls to the database for the same information.
Is there a way to partially save records in a data context / container in Entity Framework 4.1? For example, to save Address by itself while still being in the same DC?
The answer to your bold question is "No" as far as I can tell. The context is a unit of work and SaveChanges commits every new, changed or deleted object to the database in a single transaction. You cannot selectively say: Save only this or that object or save only entities in state Added and don't commit entities in state Modified or Deleted or something.
As a workaround you could try that:
using (var context1 = new MyContext())
{
Address address = context1.Addresses.Where(predicate).FirstOrDefault();
// if address != null it is attached now to context1
if (address == null)
{
// ... otherwise create new address in another context and save
using (var context2 = new MyContext())
{
address = new Address { Name = name, ... }
context2.Addresses.Add(address);
context2.SaveChanges();
} // context2 destroyed now and address is not attached to it anymore
// ... and attach to context1
context1.Addresses.Attach(address);
}
customer.HomeAddress = address;
// ...
context1.SaveChanges();
}
This way address is never attached to the two contexts at the same time. I am not sure though if this works.
Edit
I must add (because my code above looks so weird) that "normally" you could do all this in context1 alone. But I understood your point 2 this way that there is something happening in // ... (which I don't understand) before SaveChanges which prevents you to save the new address and the customer at the same time.