I am invoking a asynchronous PageMethod call from the client-side. The backend code is
[WebMethod(EnableSession = true)]
public static string BeginMethodCall()
{ //Session Accessible here
string g = Guid.NewGuid().ToString();
Func<object> f = () => MethodCall();
IAsyncResult asyncCall = f.BeginInvoke(null, f);
lock (AsyncThreadPool)
AsyncThreadPool[g] = asyncCall;
return g;
}
[WebMethod(EnableSession=true)]
public static object EndMethodCall(string guId)
{
IAsyncResult callResult;
lock (AsyncThreadPool)
{
callResult = AsyncThreadPool[guId];
AsyncThreadPool.Remove(guId);
}
Func<object> f = (Func<object>)callResult.AsyncState;
callResult.AsyncWaitHandle.WaitOne();
return f.EndInvoke(callResult);
}
[WebMethod(EnableSession = true)]
public static object MethodCall()
{
//Session not accessible here
}
Session state is accessible from BeginMethodCall() and EndMethodCall() but not from the MethodCall().
Can anyone tell me why I lose my session state here?
Do threads lose the Session context because asyn calls are not thread safe?
Is there a way to access session here?
In case someone may find this useful
BeginXXX does create a new theard and threads are not session safe for obvious reasons
In general,
Session + Async = fail
You're safe within callbacks and before you branch off into async code, but while you're in parallel, you don't get full session state.
However, there are a few workarounds. The easiest of which is
[SessionState(System.Web.SessionStateBehavior.ReadOnly)]
at the controller level, which allows, obviously, read-only access to session values. It'll work in parallel as long as you don't try to write to session. You can also implement your own HttpHandler, but that ones a little tougher.
Related
We are using java 8 parallel stream to process a task, and we are submitting the task through ForkJoinPool#submit. We are not using jvm wide ForkJoinPool.commonPool, instead we are creating our own custom pool to specify the parallelism and storing it as static variable.
We have validation framework, where we subject a list of tables to a List of Validators, and we submit this job through the custom ForkJoinPool as follows:
static ForkJoinPool forkJoinPool = new ForkJoinPool(4);
List<Table> tables = tableDAO.findAll();
ModelValidator<Table, ValidationResult> validator = ValidatorFactory
.getInstance().getTableValidator();
List<ValidationResult> result = forkJoinPool.submit(
() -> tables.stream()
.parallel()
.map(validator)
.filter(result -> result.getValidationMessages().size() > 0)
.collect(Collectors.toList())).get();
The problem we are having is, in the downstream components, the individual validators which run on separate threads from our static ForkJoinPool rely on tenant_id, which is different for every request and is stored in an InheritableThreadLocal variable. Since we are creating a static ForkJoinPool, the threads pooled by the ForkJoinPool will only inherit the value of the parent thread, when it is created first time. But these pooled threads will not know the new tenant_id for the current request. So for subsequent execution these pooled threads are using old tenant_id.
I tried creating a custom ForkJoinPool and specifying ForkJoinWorkerThreadFactory in the constructor and overriding the onStart method to feed the new tenant_id. But that doesnt work, since the onStart method is called only once at creation time and not during individual execution time.
Seems like we need something like the ThreadPoolExecutor#beforeExecute which is not available in case of ForkJoinPool. So what alternative do we have if we want to pass the current thread local value to the statically pooled threads?
One workaround would be to create the ForkJoinPool for each request, rather than make it static but we wouldn't want to do it, to avoid the expensive nature of thread creation.
What alternatives do we have?
I found the following solution that works without changing any underlying code. Basically, the map method takes a functional interface which I am representing as a lambda expression. This expression adds a preExecution hook to set the new tenantId in the current ThreadLocal and cleaning it up in postExecution.
forkJoinPool.submit(tables.stream()
.parallel()
.map((item) -> {
preExecution(tenantId);
try {
return validator.apply(item);
} finally {
postExecution();
}
}
)
.filter(validationResult ->
validationResult.getValidationMessages()
.size() > 0)
.collect(Collectors.toList())).get();
The best option in my view would be to get rid of the thread local and pass it as an argument instead. I understand that this could be a massive undertaking though. Another option would be to use a wrapper.
Assuming that your validator has a validate method you could do something like:
public class WrappingModelValidator implements ModelValidator<Table. ValidationResult> {
private final ModelValidator<Table. ValidationResult> v;
private final String tenantId;
public WrappingModelValidator(ModelValidator<Table. ValidationResult> v, String tenantId) {
this.v = v;
this.tenantId = tenantId;
}
public ValidationResult validate(Table t) {
String oldValue = YourThreadLocal.get();
YourThreadLocal.set(tenantId);
try {
return v.validate(t);
} finally {
YourThreadLocal.set(oldValue);
}
}
}
Then you simply wrap your old validator and it will set the thread local on entry and restore it when done.
There is some posts explain how to tackle, but couldnt help me much..
Logging Request/Response in middleware, it works when use 'await' with Task.Run() but since its awaited current operation to complete there is performance issue.
When I remove await as below, it runs fast but not logging anything, since HttpContext instance not available to use inside parallel thread
public class LoggingHandlerMiddleware
{
private readonly RequestDelegate next;
private readonly ILoggerManager _loggerManager;
public LoggingHandlerMiddleware(RequestDelegate next, ILoggerManager loggerManager)
{
this.next = next;
_loggerManager = loggerManager;
}
public async Task Invoke(HttpContext context, ILoggerManager loggerManager, IWebHostEnvironment environment)
{
_ = Task.Run(() =>
{
AdvanceLoggingAsync(context, _loggerManager, environment);
});
...
}
private void AdvanceLoggingAsync(HttpContext context, ILoggerManager loggerManager, IWebHostEnvironment environment, bool IsResponse = false)
{
{
context.Request.EnableBuffering(); // Throws ExecutionContext.cs not found
result += $"ContentType:{context.Request.ContentType},";
using (StreamReader reader = new StreamReader(context.Request.Body, Encoding.UTF8, true, 1024, true))
{
result += $"Body:{await reader.ReadToEndAsync()}";
context.Request.Body.Position = 0;
}
loggerManager.LogInfo($"Advance Logging Content(Request)-> {result}");
}
How can I leverage Task.Run() performance with accessing HttpContext?
Well, you can extract what you need from the context, build your string you want to log, and then pass that string to the task you run.
However, firing and forgetting a task is not good. If it throws an exception, you risk of bringing down the server, or at least you will have very hard time getting information about the error.
If you are concerned about the logging performance, better add what you need to log to a message queue, and have a process that responds to new messages in the queue and logs the message to the log file.
I guess there is not built-in way to achieve that:
I have some cached data, that need to be always up to date (interval of few 10s of minutes). Its generation takes around 1-2 minutes, therefore it leads sometimes to timeout requests.
For performances optimisation, I put it into memory cache, using Cache.GetOrCreateAsync, so I am sure to have fast access to the data during 40 minutes. However it still takes time when the cache expires.
I would like to have a mechanism that auto-refreshes the data before its expiration, so the users are not impacted from this refresh and can still access the "old data" during the refresh.
It would actually be adding a "pre-expiration" process, that would avoid data expiration to arrive at its term.
I feel that is not the functioning of the default IMemoryCache cache, but I might be wrong?
Does it exist? If not, how would you develop this feature?
I am thinking of using PostEvictionCallbacks, with an entry set to be removed after 35 minutes and that would trigger the update method (it involves a DbContext).
This is how I solve it:
The part called by the web request (the "Create" method should be called only the first time).
var allPlaces = await Cache.GetOrCreateAsync(CACHE_KEY_PLACES
, (k) =>
{
k.AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(40);
UpdateReset();
return GetAllPlacesFromDb();
});
And then the magic (This could have been implemented through a timer, but didn't want to handle timers there)
// This method adds a trigger to refresh the data from background
private void UpdateReset()
{
var mo = new MemoryCacheEntryOptions();
mo.RegisterPostEvictionCallback(RefreshAllPlacessCache_PostEvictionCallback);
mo.AddExpirationToken(new CancellationChangeToken(new CancellationTokenSource(TimeSpan.FromMinutes(35)).Token));
Cache.Set(CACHE_KEY_PLACES_RESET, DateTime.Now, mo);
}
// Method triggered by the cancellation token that triggers the PostEvictionCallBack
private async void RefreshAllPlacesCache_PostEvictionCallback(object key, object value, EvictionReason reason, object state)
{
// Regenerate a set of updated data
var places = await GetLongGeneratingData();
Cache.Set(CACHE_KEY_PLACES, places, TimeSpan.FromMinutes(40));
// Re-set the cache to be reloaded in 35min
UpdateReset();
}
So the cache gets two entries, the first one with the data, expiring after 40 minutes, the second one expiring after 35min via a cancellation token that triggers the post eviction method.
This callback refreshes the data before it expires.
Keep in mind that this will keep the website awake and using memory even if not used.
** * UPDATE USING TIMERS * **
The following class is registered as a singleton. DbContextOptions is passed instead of DbContext to create a DbContext with the right scope.
public class SearchService
{
const string CACHE_KEY_ALLPLACES = "ALL_PLACES";
protected readonly IMemoryCache Cache;
private readonly DbContextOptions<AppDbContext> AppDbOptions;
public SearchService(
DbContextOptions<AppDbContext> appDbOptions,
IMemoryCache cache)
{
this.AppDbOptions = appDbOptions;
this.Cache = cache;
InitTimer();
}
private void InitTimer()
{
Cache.Set<AllEventsResult>(CACHE_KEY_ALLPLACESS, new AllPlacesResult() { Result = new List<SearchPlacesResultItem>(), IsBusy = true });
Timer = new Timer(TimerTickAsync, null, 1000, RefreshIntervalMinutes * 60 * 1000);
}
public Task LoadingTask = Task.CompletedTask;
public Timer Timer { get; set; }
public long RefreshIntervalMinutes = 10;
public bool LoadingBusy = false;
private async void TimerTickAsync(object state)
{
if (LoadingBusy) return;
try
{
LoadingBusy = true;
LoadingTask = LoadCaches();
await LoadingTask;
}
catch
{
// do not crash the app
}
finally
{
LoadingBusy = false;
}
}
private async Task LoadCaches()
{
try
{
var places = await GetAllPlacesFromDb();
Cache.Set<AllPlacesResult>(CACHE_KEY_ALLPLACES, new AllPlacesResult() { Result = places, IsBusy = false });
}
catch{}
}
private async Task<List<SearchPlacesResultItem>> GetAllPlacesFromDb()
{
// blablabla
}
}
Note:
DbContext options require to be registered as singleton, default options are now Scoped (I believe to allow simpler multi-tenancy configurations)
services.AddDbContext<AppDbContext>(o =>
{
o.UseQueryTrackingBehavior(QueryTrackingBehavior.NoTracking);
o.UseSqlServer(connectionString);
},
contextLifetime: ServiceLifetime.Scoped,
optionsLifetime: ServiceLifetime.Singleton);
I have a big action that takes 50 seconds to process.
But, at the same time, I have another action that could be processed on the server (by clicking on a link).
However, if my second action try to access session's attributes put by my first action, they are note available until the end of the first action.
This is my big action:
public String bigAction() {
HttpSession session = request.getSession();
synchronized (session) {
for(int i = 0 ; i < 100000 ; ++i)
session.setAttribute("foo_"+i, "bar");
}
return SUCCESS;
}
And this is my smaller action:
public String smallAction() {
HttpSession session = request.getSession();
synchronized (session) {
session.getAttribute("foo_1", "bar");
}
return SUCCESS;
}
First action: -----------------------------------------------
Second action: --- -- --- - ---
So, in this example, my second action needs session's attributes created by the first action, but, actually, they don't exist.
How may I synchronize my session?
As per Servlet spec:
Multiple servlets executing request threads may have active access to the same session object at the same time. The container must ensure that manipulation of internal data structures representing the session attributes is performed in a threadsafe manner. The Developer has the responsibility for threadsafe access to the attribute objects themselves. This will protect the attribute collection inside the HttpSession object from concurrent access, eliminating the opportunity for an application to cause that collection to become corrupted.
This is safe:
request.getSession().setAttribute("bar", "foo");
This is not guaranteed to be safe:
HttpSession session = request.getSession();
synchronized (session) {
String value = (String) session.getAttribute("bar");
}
Moreover , the locks will work if on the same object , don't rely on request.getSession() returning same object. There is nothing in the Servlet specification that says a HttpServletSession instance can't be recreated as a facade object every time it is requested.
Read Java theory and practice: Are all stateful Web applications broken? and How HttpSession is not thread safe.
One of the approach is defined here , Java-synchronizing-on-transient-id.
Changes for today:
I am using Struts 2 so I implemented SessionAware because I read it could be a good solution. But this is the same.
Boot Pros,
I recently started to program in spring-boot and I stumbled upon a question where I would like to get your opinion on.
What I try to achieve:
I created a Controller that exposes a GET endpoint, named nonBlockingEndpoint. This nonBlockingEndpoint executes a pretty long operation that is resource heavy and can run between 20 and 40 seconds.(in the attached code, it is mocked by a Thread.sleep())
Whenever the nonBlockingEndpoint is called, the spring application should register that call and immediatelly return an Operation ID to the caller.
The caller can then use this ID to query on another endpoint queryOpStatus the status of this operation. At the beginning it will be started, and once the controller is done serving the reuqest it will be to a code such as SERVICE_OK. The caller then knows that his request was successfully completed on the server.
The solution that I found:
I have the following controller (note that it is explicitely not tagged with #Async)
It uses an APIOperationsManager to register that a new operation was started
I use the CompletableFuture java construct to supply the long running code as a new asynch process by using CompletableFuture.supplyAsync(() -> {}
I immdiatelly return a response to the caller, telling that the operation is in progress
Once the Async Task has finished, i use cf.thenRun() to update the Operation status via the API Operations Manager
Here is the code:
#GetMapping(path="/nonBlockingEndpoint")
public #ResponseBody ResponseOperation nonBlocking() {
// Register a new operation
APIOperationsManager apiOpsManager = APIOperationsManager.getInstance();
final int operationID = apiOpsManager.registerNewOperation(Constants.OpStatus.PROCESSING);
ResponseOperation response = new ResponseOperation();
response.setMessage("Triggered non-blocking call, use the operation id to check status");
response.setOperationID(operationID);
response.setOpRes(Constants.OpStatus.PROCESSING);
CompletableFuture<Boolean> cf = CompletableFuture.supplyAsync(() -> {
try {
// Here we will
Thread.sleep(10000L);
} catch (InterruptedException e) {}
// whatever the return value was
return true;
});
cf.thenRun(() ->{
// We are done with the super long process, so update our Operations Manager
APIOperationsManager a = APIOperationsManager.getInstance();
boolean asyncSuccess = false;
try {asyncSuccess = cf.get();}
catch (Exception e) {}
if(true == asyncSuccess) {
a.updateOperationStatus(operationID, Constants.OpStatus.OK);
a.updateOperationMessage(operationID, "success: The long running process has finished and this is your result: SOME RESULT" );
}
else {
a.updateOperationStatus(operationID, Constants.OpStatus.INTERNAL_ERROR);
a.updateOperationMessage(operationID, "error: The long running process has failed.");
}
});
return response;
}
Here is also the APIOperationsManager.java for completness:
public class APIOperationsManager {
private static APIOperationsManager instance = null;
private Vector<Operation> operations;
private int currentOperationId;
private static final Logger log = LoggerFactory.getLogger(Application.class);
protected APIOperationsManager() {}
public static APIOperationsManager getInstance() {
if(instance == null) {
synchronized(APIOperationsManager.class) {
if(instance == null) {
instance = new APIOperationsManager();
instance.operations = new Vector<Operation>();
instance.currentOperationId = 1;
}
}
}
return instance;
}
public synchronized int registerNewOperation(OpStatus status) {
cleanOperationsList();
currentOperationId = currentOperationId + 1;
Operation newOperation = new Operation(currentOperationId, status);
operations.add(newOperation);
log.info("Registered new Operation to watch: " + newOperation.toString());
return newOperation.getId();
}
public synchronized Operation getOperation(int id) {
for(Iterator<Operation> iterator = operations.iterator(); iterator.hasNext();) {
Operation op = iterator.next();
if(op.getId() == id) {
return op;
}
}
Operation notFound = new Operation(-1, OpStatus.INTERNAL_ERROR);
notFound.setCrated(null);
return notFound;
}
public synchronized void updateOperationStatus (int id, OpStatus newStatus) {
iteration : for(Iterator<Operation> iterator = operations.iterator(); iterator.hasNext();) {
Operation op = iterator.next();
if(op.getId() == id) {
op.setStatus(newStatus);
log.info("Updated Operation status: " + op.toString());
break iteration;
}
}
}
public synchronized void updateOperationMessage (int id, String message) {
iteration : for(Iterator<Operation> iterator = operations.iterator(); iterator.hasNext();) {
Operation op = iterator.next();
if(op.getId() == id) {
op.setMessage(message);
log.info("Updated Operation status: " + op.toString());
break iteration;
}
}
}
private synchronized void cleanOperationsList() {
Date now = new Date();
for(Iterator<Operation> iterator = operations.iterator(); iterator.hasNext();) {
Operation op = iterator.next();
if((now.getTime() - op.getCrated().getTime()) >= Constants.MIN_HOLD_DURATION_OPERATIONS ) {
log.info("Removed operation from watchlist: " + op.toString());
iterator.remove();
}
}
}
}
The questions that I have
Is that concept a valid one that also scales? What could be improved?
Will i run into concurrency issues / race conditions?
Is there a better way to achieve the same in boot spring, but I just didn't find that yet? (maybe with the #Async directive?)
I would be very happy to get your feedback.
Thank you so much,
Peter P
It is a valid pattern to submit a long running task with one request, returning an id that allows the client to ask for the result later.
But there are some things I would suggest to reconsider :
do not use an Integer as id, as it allows an attacker to guess ids and to get the results for those ids. Instead use a random UUID.
if you need to restart your application, all ids and their results will be lost. You should persist them to a database.
Your solution will not work in a cluster with many instances of your application, as each instance would only know its 'own' ids and results. This could also be solved by persisting them to a database or Reddis store.
The way you are using CompletableFuture gives you no control over the number of threads used for the asynchronous operation. It is possible to do this with standard Java, but I would suggest to use Spring to configure the thread pool
Annotating the controller method with #Async is not an option, this does not work no way. Instead put all asynchronous operations into a simple service and annotate this with #Async. This has some advantages :
You can use this service also synchronously, which makes testing a lot easier
You can configure the thread pool with Spring
The /nonBlockingEndpoint should not return the id, but a complete link to the queryOpStatus, including id. The client than can directly use this link without any additional information.
Additionally there are some low level implementation issues which you may also want to change :
Do not use Vector, it synchronizes on every operation. Use a List instead. Iterating over a List is also much easier, you can use for-loops or streams.
If you need to lookup a value, do not iterate over a Vector or List, use a Map instead.
APIOperationsManager is a singleton. That makes no sense in a Spring application. Make it a normal PoJo and create a bean of it, get it autowired into the controller. Spring beans by default are singletons.
You should avoid to do complicated operations in a controller method. Instead move anything into a service (which may be annotated with #Async). This makes testing easier, as you can test this service without a web context
Hope this helps.
Do I need to make database access transactional ?
As long as you write/update only one row, there is no need to make this transactional as this is indeed 'atomic'.
If you write/update many rows at once you should make it transactional to guarantee, that either all rows are updated or none.
However, if two operations (may be from two clients) update the same row, always the last one will win.