I have a weird case where I'm using NInject and I'm not sure how to proceed. Our repositories in this instance are custom-written sql generators, and not using linq to sql; nHibernate, etc.
In order to utilize code within the system, we're, within the implementation of the repositories, injecting the required repositories to build child objects (e.g., an "order" object needs to get its "orderdetails" objects and assign it to the order before returning it to the calling area in the system, so in our OrderRepository, we have an [Inject]IOrderDetailRepository OrderDetailRepo { get; set; }).
What the hangup comes in is up until this point, we've been able to keep everything configured InRequestScope(). Now we are using a Parallel ForEach loop and after an iteration completes, we're firing an event over to a singleton scoped event handler to update the database. We'd do the update within the loop, but have been trying to avoid tying the loop with a particular area in the system, since there could be many areas that could use this loop.
So, we need to figure out when we call into this singleton scoped event handler, how to have ninject configured that all IRepositories (IOrderRepository and IOrderDetailRepository) used within that singleton event handler die off and init immediately when they are used.
Any Hints?
So, I have "A" resolution, but I'm not happy with the resolution for this... what I did was to build-up a ChildKernel and then set the bindings the way that I wanted to have them. Although this works as expected, it feels like some serious "code smell". Would love to see a better way of handling this scenario, if one would exist.
Related
We are using the following frameworks and versions:
jOOQ 3.11.1
Spring Boot 2.3.1.RELEASE
Spring 5.2.7.RELEASE
I have an issue where some of our business logic is divided into logical units that look as follows:
Request containing a user transaction is received
This request contains various information, such as the type of transaction, which products are part of this transaction, what kind of payments were done, etc.
These attributes are then stored individually in the database.
In code, this looks approximately as follows:
TransactionRecord transaction = transactionRepository.create();
transaction.create(creationCommand);`
In Transaction#create (which runs transactionally), something like the following occurs:
storeTransaction();
storePayments();
storeProducts();
// ... other relevant information
A given transaction can have many different types of products and attributes, all of which are stored. Many of these attributes result in UPDATE statements, while some may result in INSERT statements - it is difficult to fully know in advance.
For example, the storeProducts method looks approximately as follows:
products.forEach(product -> {
ProductRecord record = productRepository.findProductByX(...);
if (record == null) {
record = productRepository.create();
record.setX(...);
record.store();
} else {
// do something else
}
});
If the products are new, they are INSERTed. Otherwise, other calculations may take place. Depending on the size of the transaction, this single user transaction could obviously result in up to O(n) database calls/roundtrips, and even more depending on what other attributes are present. In transactions where a large number of attributes are present, this may result in upwards of hundreds of database calls for a single request (!). I would like to bring this down as close as possible to O(1) so as to have more predictable load on our database.
Naturally, batch and bulk inserts/updates come to mind here. What I would like to do is to batch all of these statements into a single batch using jOOQ, and execute after successful method invocation prior to commit. I have found several (SO Post, jOOQ API, jOOQ GitHub Feature Request) posts where this topic is implicitly mentioned, and one user groups post that seemed explicitly related to my issue.
Since I am using Spring together with jOOQ, I believe my ideal solution (preferably declarative) would look something like the following:
#Batched(100) // batch size as parameter, potentially
#Transactional
public void createTransaction(CreationCommand creationCommand) {
// all inserts/updates above are added to a batch and executed on successful invocation
}
For this to work, I imagine I'd need to manage a scoped (ThreadLocal/Transactional/Session scope) resource which can keep track of the current batch such that:
Prior to entering the method, an empty batch is created if the method is #Batched,
A custom DSLContext (perhaps extending DefaultDSLContext) that is made available via DI has a ThreadLocal flag which keeps track of whether any current statements should be batched or not, and if so
Intercept the calls and add them to the current batch instead of executing them immediatelly.
However, step 3 would necessitate having to rewrite a large portion of our code from the (IMO) relatively readable:
records.forEach(record -> {
record.setX(...);
// ...
record.store();
}
to:
userObjects.forEach(userObject -> {
dslContext.insertInto(...).values(userObject.getX(), ...).execute();
}
which would defeat the purpose of having this abstraction in the first place, since the second form can also be rewritten using DSLContext#batchStore or DSLContext#batchInsert. IMO however, batching and bulk insertion should not be up to the individual developer and should be able to be handled transparently at a higher level (e.g. by the framework).
I find the readability of the jOOQ API to be an amazing benefit of using it, however it seems that it does not lend itself (as far as I can tell) to interception/extension very well for cases such as these. Is it possible, with the jOOQ 3.11.1 (or even current) API, to get behaviour similar to the former with transparent batch/bulk handling? What would this entail?
EDIT:
One possible but extremely hacky solution that comes to mind for enabling transparent batching of stores would be something like the following:
Create a RecordListener and add it as a default to the Configuration whenever batching is enabled.
In RecordListener#storeStart, add the query to the current Transaction's batch (e.g. in a ThreadLocal<List>)
The AbstractRecord has a changed flag which is checked (org.jooq.impl.UpdatableRecordImpl#store0, org.jooq.impl.TableRecordImpl#addChangedValues) prior to storing. Resetting this (and saving it for later use) makes the store operation a no-op.
Lastly, upon successful method invocation but prior to commit:
Reset the changes flags of the respective records to the correct values
Invoke org.jooq.UpdatableRecord#store, this time without the RecordListener or while skipping the storeStart method (perhaps using another ThreadLocal flag to check whether batching has already been performed).
As far as I can tell, this approach should work, in theory. Obviously, it's extremely hacky and prone to breaking as the library internals may change at any time if the code depends on Reflection to work.
Does anyone know of a better way, using only the public jOOQ API?
jOOQ 3.14 solution
You've already discovered the relevant feature request #3419, which will solve this on the JDBC level starting from jOOQ 3.14. You can either use the BatchedConnection directly, wrapping your own connection to implement the below, or use this API:
ctx.batched(c -> {
// Make sure all records are attached to c, not ctx, e.g. by fetching from c.dsl()
records.forEach(record -> {
record.setX(...);
// ...
record.store();
}
});
jOOQ 3.13 and before solution
For the time being, until #3419 is implemented (it will be, in jOOQ 3.14), you can implement this yourself as a workaround. You'd have to proxy a JDBC Connection and PreparedStatement and ...
... intercept all:
Calls to Connection.prepareStatement(String), returning a cached proxy statement if the SQL string is the same as for the last prepared statement, or batch execute the last prepared statement and create a new one.
Calls to PreparedStatement.executeUpdate() and execute(), and replace those by calls to PreparedStatement.addBatch()
... delegate all:
Calls to other API, such as e.g. Connection.createStatement(), which should flush the above buffered batches, and then call the delegate API instead.
I wouldn't recommend hacking your way around jOOQ's RecordListener and other SPIs, I think that's the wrong abstraction level to buffer database interactions. Also, you will want to batch other statement types as well.
Do note that by default, jOOQ's UpdatableRecord tries to fetch generated identity values (see Settings.returnIdentityOnUpdatableRecord), which is something that prevents batching. Such store() calls must be executed immediately, because you might expect the identity value to be available.
I'm currently developing a multiplayer cardgame for android, with libgdx as the game engine. My question is more generel tho.
I'm not sure whats the best practice for handling callbacks in this architecture. My controller is a big statemachine, that checks inputs over and over while beeing called from the render() method of the gameengine.
I have two main callbacks, userinput from the gui, and network callbacks from the android google play services part.
Currently these callback methode/ inputListeners just set member variables, which are check by getter methods from the controller/statemachine, for example i call this from the controller over and over, check if its != null and proceed if it is.
#Override
public Boolean allPlayersConnected() {
Boolean allConnected = null;
if (startGame != null) {
allConnected = startGame;
startGame = null;
}
return allConnected;
}
The startGame "flag" beeing set by callbacks from the google play services api.
I dont know if this is good practice, doesnt look like.
I could call controller methods from the google play services callbacks that set a controller member variable, and check this in each render loop, but thats just moving the variable.
I could also design the controller as an observer of those events, but what am i going to do in the update method inside the controller thats beeing called if an event happens. i dont think i want change stats in these, even if i can access the currrent state. Im spreading state code all over the place with this, some in different parts of a huge update method and some in the actual state machine code. Just setting a member variable in the update method is quite similar to the think i did above.
Another thing would be, to directly change controller state from the callback methods. That would be less code, less variables and a little faster, but i think i'd destroy the MVP concept, cause i take away the control from the controller and let i.e. the gui change the state of the controller.
Any input on this ?
Edit:
The more i think about it, the more i think a combination of observer and command pattern is the way to go.
So i could indeed cut big part of the current state machine and pack it into the observer update() method. Instead of sending the commands through a big command enum, i could create command object with the information available, and pass them to the observer(controller), where i check the command as viable, and call the execute with the information needed to be excecuted, eg the model interface.
First, I think whether your commands are enums or command objects is independent of the main problem here -- which is how to connect user and network input to state management.
The most common game architecture I've seen is an update loop that checks input, iterates the game simulation, and then renders a frame. In the MVC world, this structure just synchronizes those steps; you still have an encapsulated view and data model, with the controller (the game loop) serving as a a bridge between those two worlds.
Input, whether from the local user or one over the net, is generally treated as a request to modify game state. That is, the controller (as the first part of its loop) reads in pending input messages and processes them, modifying state as it goes. This way, the code that changes state is in one place: that controller. You are right, spreading state-modification code throughout the app is a bad practice; basically, it's not MVC.
In other words, all of your callbacks should convert the input to commands and stick them into a queue. You don't want to synchronize the controller -- whose job it is to modify state -- with those input callbacks. You have no idea when input will occur relative to the game loop, so it's best to decouple them. Serializing input processing with game simulation should also make your logic simpler.
You have some choice in how to connect the callbacks to the controller; a shared queue (where one side writes into it and the other reads out from it) is a strong pattern and easy to make thread-safe.
When you develop an ASP.NET application using the repository pattern, do each of your methods create a new entity container instance (context) with a using block for each method, or do you create a class-level/private instance of the container for use by any of the repository methods until the repository itself is disposed? Other than what I note below, what are the advantages/disadvantages? Is there a way to combine the benefits of each of these that I'm just not seeing? Does your repository implement IDisposable, allowing you to create using blocks for instances of your repo?
Multiple containers (vs. single)
Advantages:
Preventing connections from being auto-closed/disposed (will be closed at the end of the using block).
Helps force you to only pull into memory what you need for a particular view/viewmodel, and in less round-trips (you will get a connection error for anything you attempt to lazy load).
Disadvantages:
Access of child entities within the Controller/View is limited to what you called with Include()
For pages like a dashboard index that shows information gathered from many tables (many different repository method calls), we will add the overhead of creating and disposing many entity containers.
If you are instantiating your context in your repository, then you should always do it locally, and wrap it in a using statement.
If you're using Dependency Injection to inject the context, then let your DI container handle calling dispose on the context when the request is done.
Don't instantiate your context directly as a class member, since this will not dispose of the contexts resources until garbage collection occurs. If you do, then you will need to implement IDipsosable to dispose the context, and make sure that whatever is using your repository properly disposes of your repository.
I, personally, put my context at the class level in my repository. My primary reason for doing so is because a distinct advantage of the repository pattern is that I can easily swap repositories and take advantage of a different backend. Remember - the purpose of the repository pattern is that you provide an interface that provides back data to some client. If you ever switch your data source, or just want to provide a new data source on the fly (via dependency injection), you've created a much more difficult problem if you do this on a per-method level.
Microsoft's MSDN site has good information the repository pattern. Hopefully this helps clarify some things.
I disagree with all four points:
Preventing connections from being auto-closed/disposed (will be closed
at the end of the using block).
In my opinion it doesn't matter if you dispose the context on method level, repository instance level or request level. (You have to dispose the context of course at the end of a single request - either by wrapping the repository method in a using statement or by implementing IDisposable on the repository class (as you proposed) and wrapping the repository instance in a using statement in the controller action or by instantiating the repository in the controller constructor and dispose it in the Dispose override of the controller class - or by instantiating the context when the request begins and diposing it when the request ends (some Dependency Injection containers will help to do this work).) Why should the context be "auto-disposed"? In desktop application it is possible and common to have a context per window/view which might be open for hours.
Helps force you to only pull into memory what you need for a
particular view/viewmodel, and in less round-trips (you will get a
connection error for anything you attempt to lazy load).
Honestly I would enforce this by disabling lazy loading altogether. I don't see any benefit of lazy loading in a web application where the client is disconnected from the server anyway. In your controller actions you always know what you need to load and can use eager or explicit loading. To avoid memory overhead and improve performance, you can always disable change tracking for GET requests because EF can't track changes on a client's web page anyway.
Access of child entities within the Controller/View is limited to what
you called with Include()
Which is rather an advantage than a disadvantage because you don't have the unwished surprises of lazy loading. If you need to populate child entities later in the controller actions, depending on some condition, you could load them through additional repository methods (LoadNavigationProperty or something) with the same or even a new context.
For pages like a dashboard index that shows information gathered from
many tables (many different repository method calls), we will add the
overhead of creating and disposing many entity containers.
Creating contexts - and I don't think we are talking about hundreds or thousands of instances - is a cheap operation. I would call this a very theoretical overhead which doesn't play a role in practice.
I've used both approaches you mentioned in web applications and also the third option, namely to create a single context per request and inject this same context into every repository/service I need in a controller action. They all three worked for me.
Of course if you use multiple contexts you have to be careful to do all the work in the same unit of work to avoid attaching entities to multiple contexts which will lead to well know exceptions. It's usually not a problem to avoid this situations but requires a bit more attention, especially when processing POST requests.
I lately use contexts per request, because it is easier and I just don't see the benefit of having very narrow contexts and I see no reason to use more than one single unit of work for the whole request processing. If I would need multiple contexts - for whatever reason - I could always create specialized methods which act with their own context instead of the "default context" of the request.
The question is a bit long since it's conceptual. I hope it's not a bad read :)
I'm working in a performance critical Spring MVC/Tiles web-app (10,000 users typical load). We load an update employee screen, where we load an employee details screen (bound to an employee business object) for updates via a MultiActionController. There are multiple tabs on this screen, but only tab1 has the updatabale data. Rest of the tabs are read-only stuff, for reference basically.
Needless to say, we've decided to load these read-only tabs in a lazy manner, i.e., when each tab is activated, we fire an ajax call (one-time) for fetch the data from the server. We don't load everything via the update view loading method. Remember: this is one time, read-only data.
Now, I'm in a dilemma. I've made another multiaction controller, named "AjaxController" for handling these ajax calls. Now, my questions:
What should be the best scope for this controller?
Thoughts: If I make it request scoped, then 10,000 users together can create 10,000 instances of this bean: memory problem there. If I make it session scoped, then one will be created per user session. That means, when 10,000 users log in to the app, regardless of whether they hit the AjaxController methods, they will each have a bean in possession.
Then, is singleton the best scope for this controller?
Thoughts: A singleton bean will be created when spring boots, and this very instance will be provided throughout. Sounds good.
Should the handler methods (like fetchTab7DataInJsonFormat) be static and attached to the class?
Thoughts: In this case, can havign static methods semantically conflict with the scope? For example: does scope="session"/"request" + static methods make sense? I ask because even though each user session has its own AjaxController bean, the handler methods are actually attached to the class, and not the instances. Also, does scope="singleton" + static handler methods make sense?
Can I implement the singleton design pattern into AjaxController manually?
Thoughts: What if I control the creation: do the GoF singleton basically. Then what can the scope specification do? Scope session/request surely can't create multiple instances can they?
If, by whatever mechanism (bean specification/design pattern/static methods), I do manage to have one single instance of AjaxController: Will these STATIC methods need to be synchronized? I think not, because even if STATIC handler methods can talk to services (which talk to DB/WS/MQ etc.) which take time, I think each request thread entering the static methods will be returned by their thread Id's right? It's not like user1 enters the static method, and then user2 enters the static method before user1 has been returned, and then they both get some garbled data? This is probably silly, but I want to be sure.
I'm confused. I basically want exactly one single instance of the controller bean servicing all requests for all clients.
Critical Note: The AjaxController bean is not INJECTED anywhere else, it exists isolated. It's methods are hit via ajax calls.
If I were doing this, I would definitely make the LazyLoadController singleton without having static methods in it and without any state in it.
Also, you definitely shouldn't instantiate singletons manually, it's better to use Spring's common mechanism and let the framework control everything.
The overall idea is to avoid using any static methods and/or persistent data in controllers. The right mechanism would be use some service bean for generating data for request, so controller acts as request parameter dispatcher to fetch the data out into the view. No mutable state or concurrently unsafe stuff should be allowed in controller. If some components are user-specific, Spring's AOP system provides injection of the components based on session/request.
That's about good practice in doing thing like that. There's something to clarify to give more specific answer for your case. Did I understand it right that typical use case for will be that AjaxController will pass some of requests to LazyLoadController to get tab data? Please provide details about that in comment or your question, so I may update my answer.
The thing that is wrong with having static methods in controller is that you have to manage concurrent safety by yourself which is not just error-prone but will also reduce overall performance. Spring runs every request in its own thread, so if two concurrent calls need to use some static method and there are shared resources (so you need to use synchronize statement or locks), one of threads will have to wait for another one to complete working in protected block. From the other hand, if you use stateless services and avoid having data that may be shared for multiple calls, you get increased performance and no need to deal with concurrent data access.
I have a method that returns lot of data, should I use #TransactionAttribute(TransactionAttributeType.NOT_SUPPORTED) for this method. The method perform a JPA query an loads the full content of a table (about 1000 rows).
The client to this method - is that already in a transaction? When you use NotSupported the caller transaction will be suspended. If not I would say, just put Never as the transaction type. Never is better since callers know they are not supposed to call this method from inside a transaction. A more straight forward contract.
We always use Never for methods that do more processing so that developers are aware right off the bat not to call if they are involved in a transaction already. Hope it helps.
I would care to disagree as it seldom happens that user is not in a transaction in almost all the systems. The best approach is to use NOT SUPPORTED so that the transaction is suspended if the caller is in any transaction already. NEVER is troublesome unless you have a series of calls which are all in NO TRANSACTION scope. In short, NOT SUPPORTED is the type one should use.
As far as I know (at least this is the case with Hibernate), you cannot use JPA outside of a transaction as the entity manager's lifecycle is linked to the transaction's lifecycle. So the actual method that does the query must be transactional.
However, you can set it to TransactionAttributeType.REQUIRES_NEW; this would suspend any existing transaction, start a new one, and stop it when the method returns. That means all your entities would be detached by the time they reach the caller, which is what it sounds like you're trying to achieve.
In more complex systems, it pays to completely separate your data layer from your business layer and create a new set of object. Your method will then call the JPA query, then use the entities returned to populate objects from your business layer, and return those. That way the caller can never get their hands on the actual JPA entities and you are free to do in your data layer what you want, since now it's just an implementation detail. (Heck, you could change the database call to a remote API call and your caller wouldn't have to know.)