I want to implement some sort of "batch signing" scenario using CryptSignMessage.
Each call to CryptSignMessage must use the same certificate. I plan to obtain CERT_CONTEXT once from store, and then reuse it in parallel CryptSignMessage calls.
Assuming, that each call (thread) will operate its own instance of CRYPT_SIGN_MESSAGE_PARA and data to be signed, but the same pointer to CERT_CONTEXT, will this be correct usage?
In other words, is such CryptSignMessage usage thread-safe?
Related
Is there a tool that can estimate how much gas a contract call will make before submitting to the NEAR network?
Currently the best estimation is to use runtime-standalone, which can process transactions without having to worry about consensus/networking. This means you can create accounts, deploy contracts, and invoke them and the outcome returned includes how much gas was burnt and used. The difference being burnt gas is used to execute the function call and used gas is how much was used by contract promise calls.
However, it's currently a MVP prototype and has only been used to test our core contract, here is it being used to test the lockup contract.
If your contract method doesn't invoke any batch promises and only normal promises,the mock runtime in near-sdk-as provides a way to create accounts and "deploy" contracts. It does this by internally using the binary of near-vm-runner-standalone, which is a rust crate. The binary provides a CLI to invoke a single transaction, which takes as input the current state of the contract being called, the contract's binary, the config file that defines the current context (who is calling the contract, how much gas is prepaid, etc), and a config for the cost of different fees. It then returns the updated state, the outcome of the transaction (e.g. how much gas was used and any receipts of transactions queued by promise calls).
The near-vm-runner-standalone is also published to npm with the package name: near-vm, which is what the mock runtime uses.
This is still an active area of development and we hope to turn runtime standalone into a useful easy to use tool for testing and gas estimation.
The easiest way to do it is to submit sample transaction with more than needed gas attached and then check in explorer how much gas was used, e.g. see
https://explorer.testnet.near.org/transactions/23dgV15pydiVhirWJ4He7TMoyRJM2DUXtcWb7VXFSy2G
300 Tgas was attached and 47 Tgas used for that given transaction.
We have a monitoring agent written in Go that uses a number of goroutines to gather system metrics from WMI. We recently discovered the program was leaking memory when the go binary is run on Server 2016 or Windows 10 (also possibly on other OS using WMF 5.1). After creating a minimal test case to reproduce the issue it seems that the leak only occurs if you make a large number of calls to the ole.CoInitializeEx method (possibly something changed in WMF 5.1 but we could not reproduce the issue using the python comtypes package on the same system).
We are using COINIT_MULTITHREADED for multithread apartment (MTA) in our application, and my question is this: Because we are issuing OLE/WbemScripting calls from various goroutines, do we need to call ole.CoInitializeEx just once on startup or once in each goroutine? Our query code already uses runtime.LockOSThread to prevent the scheduler from running the method on different OS threads, but the MSDN remarks on CoInitializeEx seem to indicate it must be called at least once on each thread. I am not aware of any way to make sure new goroutines run on an already initialized OS thread, so multiple calls to CoInitializeEx seemed like the correct approach (and worked fine for the last few years).
We have already refactored the code to do all the WMI calls on a dedicated background worker, but I am curious to know if our original code should work using only one CoInitializeEx at startup instead of once for every goroutine.
AFAIK, since Win32 API is defined only in terms of native OS threads, a call to CoInitialize[Ex]() only ever affects the thread it completed on.
Since the Go runtime uses free M×N scheduling of the goroutines to OS threads, and these threads are created / deleted as needed at runtime in a manner completely transparent to the goroutines, the only way to make sure the CoInitialize[Ex]() call has any lasting effect on the goroutine it was performed on is to first bind that goroutine to its current OS thread by calling runtime.LockOSThread() and doing this for every goroutine intended to do COM calls.
Please note that this basically creates an 1×1 mapping between goroutines and OS threads which defeats much of the purpose of goroutines to begin with. So supposedly you might want to consider having just a single goroutine calling into COM and listening for requests on a channel, or having
a pool of such worker goroutines hidden behing another one which would dispatch the clients' requests onto the workers.
Update regarding COINIT_MULTITHREADED.
To cite the docs:
Multi-threading (also called free-threading) allows calls to methods
of objects created by this thread to be run on any thread. There is no
serialization of calls — many calls may occur to the same method or
to the same object or simultaneously. Multi-threaded object
concurrency offers the highest performance and takes the best
advantage of multiprocessor hardware for cross-thread, cross-process,
and cross-machine calling, since calls to objects are not serialized
in any way. This means, however, that the code for objects must
enforce its own concurrency model, typically through the use of
synchronization primitives, such as critical sections, semaphores, or
mutexes. In addition, because the object doesn't control the lifetime
of the threads that are accessing it, no thread-specific state may be
stored in the object (in Thread Local Storage).
So basically the COM threading model has nothing to do with initialization of the threads theirselves—but rather with how the COM subsystem is allowed to call the methods of the COM objects you create on the COM-initialized threads.
IIUC, if you will COM-initialize a thread as COINIT_MULTITHREADED and create some COM object on it, and then pass its reference to some outside client of that object so that it is able to call that object's methods, those methods can be called by the OS on any thread in your process.
I really have no idea how this is supposed to interact with Go runtime,
so I'd start small and would use a single thread with STA model and then
maybe try to make it more complicated if needed.
On the other hand, if you only instantiate external COM objects and not
transfer their descriptors outside (and it appears that's the case),
the threading model should not be relevant. That is, only unless some
code in the WUA API would call some "event-like" method on a COM object you
have instantiated.
I am using Radio Layer Interface (RIL) Native APIs in Windows Mobile application. In this API, the return values / results of most functions are not returned immediately but are passed through a callback function which is passed to the RIL API.
Some usage examples are found at XDA Develompent Tools and Google Gears Geolocation API.
My question is, in these two examples, a mutex is used to guard the data instead of other synchronization objects.
Now, will Critical Section do fine here in the use cases described by both examples? Which thread or process will actually call the callback functions?
Edit:
My data is accessed by my codes only from inside my process but which thread/process is calling the callback functions in RIL API? I mean, I passed a function callback to the RIL API, but are the callbacks called from other process? in that case, it will give another explanation why the samples are using Mutex. If the RIL API actually creates a thread inside my process and it calls my callback functions, then I think Critical Section would be fine (and it's faster than a mutex).
Update:
I have data which is (1) accessed by my codes from within my own process and is also (2) modified from a function callback. The callback is done by RIL API.
My Question: Which thread/process is calling the callback functions in RIL API?
The Story so far:
Me: Hi Mr RIL, please put some data into my office (a.k.a variables).
RIL: OK Sir. I will put the data later and I will signal you when it is done (I used an event here).
An access card is required to enter my office. If Mr RIL is from the same company as me, Mr RIL can use his own access card to enter my office (in my case, it means a Critical Section). If he is from other companies, I will need to set up an access card/visitor card for him (in my case, I need a mutex here).
If Mr RIL uses his own access card, it means I don't need to set up an access card/visitor card for him and that means less trouble for me. (i.e. Critical Section is faster than a Mutex)
The problem is, I just met this Mr RIL a few days ago and I don't know much about him. I don't know if he is from the same company as me. One option as mentioned by nobugz is to set up an access card for Mr RIL regardless whether Mr RIL is from the same company as me. This way, Mr RIL is guaranteed to be able to enter my office. (my data/variables are guaranteed to be safe)
Right now I use mutex in my code (set up a possibly redundant access card for Mr RIL).
Aha! Just got an idea when writing this. I think I will just ask Mr RIL from which company he is. That way, I don't have to set up access card for him in the future if he turns out to be in the same company as me. (i.e. put GetCurrentProcessId() and GetCurrentThreadId() in the callback function)
The Windows Mobile RIL normally resides in device.exe (for WM6.x). However, when your process invokes the RIL, your call passes via the RIL Proxy.
The RIL proxy is linked with, and resides in your process, and handles all of the issues associated with process boundaries for you (as an aside, this is at least part of the reason why all RIL data structures need to be packed into a single block of memory of known size). Internally the RIL Proxy creates a thread on which your callback is executed.
This means that your code can use a CRITICAL_SECTION object to provide the necessary synchronization/protection.
The point of using the mutex is that you don't know what thread might make the callback. Yes, a critical section would work too. Careful, getting it wrong causes random and very hard to diagnose failure.
A critical section is a mutex. A critical section is different from a normal mutex (at least primarily) in one way: it's specific to one process, where a mutex can be used across processes.
So, in this case, the basic question is exactly what you're protecting -- if it's the data inside your program, that won't be accessible to another process, then a critical section should do the job nicely. If you're protecting something that would be shared by the two processes if the user were to run two instances of your program at once, then you probably need a mutex.
Edit: As far as having to use a critical section to protect what RIL itself does, no, that isn't (or at least definitely shouldn't) be needed. With a mutex, you're counting on all the processes cooperate by opening a mutex with the same name to control access to the shared resource(s). You can't count on that, so if it is needed the interface is completely broken.
Update: unless they're doing something really unusual in RIL, the callback will happen within your process, so a critical should be adequate. If it's modifying your data, that means your data is mapped and visible to that code -- which means the data in the data in the critical section will also be mapped and visible, and it'll work. The time a critical section doesn't work is when you're dealing with separate processes, so the data in one isn't mapped/visible to the other.
Well, one other difference between a mutex and a critical section (Windows implementations, of course) is that a critical section is re-entrant - i.e. the same thread can acquire the critical section twice without having to release it.
I am working on a program which is essentially single-threaded, and its only thread is the main event-loop thread. Consequently, all its data structures are basically not protected by anything like critical region.
Things work fine until it recently integrates some new functions based on DirectShow API. Some DirectShow APIs open a second event-loop and within that second loop it dispatch messages (i.e. invoke other event-handling callbacks unpredictably). So when a second event-handling function is invoked, it might damage the data struct which is being accessed by the function that invokes the DirectShow API.
I have some experience in kernel programming. And what comes in my mind is that, for a single-threaded program, how it should deal with its data structure is very like how kernel should deal with per-CPU data structure. And in kernel, when a function accesses per-CPU data, it must disable the interrupt (very like the message-dispatching in a second event-loop). However, I find there is no easy way to either avoid invoke DirectShow API or to prevent the create of a second event-loop within them, is there any way?
mutexes. semaphores. locking. whatever name you want to call it, that's what you need.
There are several possible solutions that come to mind, depending on exactly what's going wrong and your code:
Make sure your data structures are in a consistent state before calling any APIs that run a modal loop.
If that's not possible, you can use a simple boolean variable to protect the structure. If it's set, then simply abort any attempt to update it or queue the update for later. Another option is to abort the previous operation.
If the problem is user generated events, then disable the problematic menus or buttons while the operation is in progress. Alternatively, you could display a modal dialog.
I normally work on single threaded applications and have generally never really bothered with dealing with threads. My understanding of how things work - which certainly, may be wrong - is that as long as we're always dealing with single threaded code (i.e. no forks or anything like that) it will always be executed in the same thread.
Is this assumption correct? I have a fuzzy idea that UI libraries/frameworks may spawn off threads of their own to handle GUI stuff (which accounts for the fact that the Windows task manager tells me that my 'single threaded' application is actually running on 10 threads) but I'm guessing that this shouldn't affect me?
How does this apply to COM? For instance, if I were to create an instance of a COM component in my code; and that COM component writes some information to a thread-based location (using System.Threading.Thread.GetData for instance) will my application be able to get hold of that information?
So in summary:
In single threaded code, can I be sure that whatever I store in a thread-based location can be retrievable from anywhere else in the code?
If that single threaded code were to create an instance of a COM component which stores some information in a thread-based location, can that be similarly retrievable from anywhere else?
UI usually has the opposite constraint (sadly): it's single threaded and everything must happen on that thread.
The easiest way to check if you are always in the same thread (for, say, a function) is to have an integer variable set at -1, and have a check function like (say you are in C#):
void AssertSingleThread()
{
if (m_ThreadId < 0) m_ThreadId = Thread.CurrentThread.ManagedThreadId;
Debug.Assert(m_ThreadId == Thread.CurrentThread.ManagedThreadId);
}
That said:
I don't understand the question #1, really. Why store in a thread-based location if your purpose is to have a global scope ?
About the second question, most COM code runs on a single thread and, most often, on the thread where your UI message processing lives - this is because most COM code is designed to be compatible with VB6 which is single-thread.
The reason your program has about 10 threads is because both Windows (if you use some of its features like completion ports, or some kind of timers) and the CLR (for example for the GC or, again, some types of timers) may create threads in your process space (technically any program with enough priviledges, can too).
Think about having the model of having a single dataStore class running in your mainThread that all threads can read and write their instance variables to. This will avoid a lot of problems that might arise accessing threads all over the shop.
Simple idea, until you reach the fun part of threading. Concurrency and synchronization; simply, if you have two threads that want to read and write to the same variable inside dataStore at the same time, you have a problem.
Java handles this by allowing you to declare a variable or method synchronized, allowing only one thread access at a time.
I believe some .NET objects have Lock and Synchronized methods defined on them, but I know no more than this.