It seems like there are two styles for writing NEAR smart contracts in assembly script
Bag of functions like Meme Museum
Singleton style like Lottery.
I was wondering under what circumstance one style is recommended over the other.
When should you use one over the other? What are the advantages/disadvantages of each style?
The big differences is in initialization of the contract.
Bag of Functions (BoF)
For the first style, usually a persistent collection is declared at the top level. All top level code in each file is placed into a function and one start function calls each. The Wasm binary format allows to specify the start function so that anytime the binary is instantiated (e.i. loaded by a runtime) that function is called before any exported functions can be called.
For persistent collections this means allocating the objects, but no data is required from storage, until something is read, including the length or size of the collection.
Singleton
A top level instance of the contract is declared. Then storage is checked for the "STATE" key, which contains the state of the instance. If it is present storage is read and the instance is deserialized from storage. Then each "method" of the contract is an exported function that uses the global instance, e.g. instance.method(...), passing the arguments to the method. If the method is decorated with #mutateState then the instance is written back to storage after the method call.
Which To Use
Singleton's provide a nice interface for the contract and is generally easier to understand. However, since you are reading from storage at every method call, it can be more expensive.
The other advantage of the BoF is it's easier to extend as you can export more functions from a dependency.
However, it's also possible to use a combination of both as well.
So really it's whatever makes sense to you.
I was reading about reflect.MakeFunc and was wondering if there's also a way to create a Method (a function with a receiver) at runtime.
No, this is not possible because the receiver's type method set would change on runtime if you did this. As you may know, Go in its current implementations is type checked at compile time. You would require runtime interface-implementation checks on every function-call that takes an interface argument if a type could suddenly acquire (or lose) methods at runtime.
a way to create a Method (a function with a receiver) at runtime
Technically, though, you could build a value representing a method attached to an arbitrary type by forking the reflect package. This would not, however, change said type's method set because it'd be essentially a hack around Go's type system.
What about swapping method pointers on an object?
Go, unlike e.g. Java does not embed a virtual method dispatch table in concrete values, only in interface values. If you're willing to get your hands dirty you could get a hold of a reflect.nonEmptyInterface and modify its itable (itab field).
I know the general philosophy behind std::unique_ptr<T> and std::shared_ptr<T>. A unique pointer should be used in cases where no other pointer variable will ask for access to the object / primitive data pointed to by the unique pointer. A shared pointer, however, exists for shared/concurrent access to a single resource such as a file.
This all remains true for class data members of pointer types as well of course. However, in particular regards at the implementation level, does this general rule cover all cases?
Assume that you have a class with 3 member variables, all of pointer type. If none of your member functions return a pointer to one of these variables (such as a getter function) then you should declare the member variable to be of type std::unique_ptr<T>. If however, you return one these member variables then you could potentially have a situation where another pointer tries to bind to the same resource. In which case, you should declare that particular member to be of type std::shared_ptr<T>.
This seems to be a logical rule to follow. I guess, what I'm trying to figure out is how to deal with smart pointers when declared as member variables because the decision is more difficult.
A unique pointer should be used in cases where no other pointer variable will ask for access to the object / primitive data pointed to by the unique pointer. A shared pointer, however, exists for shared/concurrent access to a single resource such as a file.
I have a different understanding. The difference between unique and shared ptr is not the access. It is the lifecycle.
unique_ptr doesn't support copy semantics so it always has a single owner.
So I think for a class member variable, it depends on whether you want the class instance to be the only owner of the lifecycle.
Another benefit of unique_ptr is that, in most cases, it has the same size as a raw ptr. So it is smaller and faster than shared_ptr.
I've been reading the FAQ re using pointer or value method receivers and it says:
Next is consistency. If some of the methods of the type must have
pointer receivers, the rest should too, so the method set is
consistent regardless of how the type is used.
Why is this important? Surely if I have some methods that are purely for reading data I want to use value receivers so as not to risk making destructive changes to the receiver. This advice suggests that if I then create a single method that should modify data on the receiver I should change all my methods to use pointer receivers.
Can someone explain the reasoning behind this advice? What's wrong with using the right tool for the job?
The key point in this quote is method set consistency. Types T and *T are different types in Go and may have different method sets (explanation is also in the FAQ).
One of the reasons it's important, is because interface satisfaction is implicit in Go. So if some of your type's methods have a pointer receiver, while others don't, this can result in a situation, when you expect that your object satisfies some interface, but now that depends on whether you are using it's pointer or value.
So it's good practice to avoid this kind of confusion.
I'm trying to marshal an interface to another thread.
Windows provides the convenient CoMarshalInterThreadInterfaceInStream helper function to take care of the boilerplate code associated with using CoMarshalInterface directly.
const Guid CLSID_Widget = "{F8383852-FCD3-11d1-A6B9-006097DF5BD4}";
const Guid IID_IWidget = "{EBBC7C04-315E-11D2-B62F-006097DF5BD4}";
//Create our widget
HRESULT hr = CoCreateInstance(CLSID_Widget, null,
CLSCTX_INPROC_SERVER | CLSCTX_LOCAL_SERVER,
IID_IWidget, out widget);
OleCheck(hr);
//Marshall the interface into an IStream
IStream stm;
hr = CoMarshalInterThreadInterfaceInStream(IID_IWidget, widget, out stm);
OleCheck(hr);
Except that the call to CoMarshalThreadInterfaceInStream fails with:
REGDB_E_IIDNOTREG (0x80040155)
Interface not registered
Go directly to CoMarshalInterface
The COM API function CoMarshalInterThreadInterfaceInStream provides a simple wrapper around CreateStreamOnHGlobal and CoMarshalInterface, as shown here:
// from OLE32.DLL (approx.)
HRESULT CoMarsha1InterThreadInterfaceInStream(
REFIID riid, IUnknown *pItf, IStream **ppStm)
{
HRESULT hr = CreateStreamOnHGlobal(0, TRUE, ppStm);
if (SUCCEEDED(hr))
hr = CoMarshalInterface(*ppStm, riid, pItf,
MSHCTX_INPROC, 0, MSHLFLAGS_NORMAL);
return hr;
}
So we can try ourselves.
IStream stm = new Stream()
hr = CoMarshallInterface(stm, IID_IWidget, widget,
MSHCTX_INPROC, // destination context is in-process/same host
NULL, // reserved, must be null
MSHLFLAGS_NORMAL // marshal once, unmarshal once
);
OleCheck(hr);
But that fails with:
REGDB_E_IIDNOTREG (0x80040155)
Interface not registered
Use standard marshaling
My class does not implement IMarhsal interface. This is right and normal.
By default, when CoMarshalInterface is first called on an object, the object is asked whether it wishes to handle its own cross-apartment communications. This question comes in the form of a QueryInterface request for the IMarshal interface. Most objects do not implement the IMarshal interface and fail this
QueryInterface request, indicating that they are perfectly happy to let COM
handle all communications via ORPC calls. Objects that do implement the
IMarshal interface are indicating that ORPC is inappropriate and that the object implementor would prefer to handle all cross-apartment communications
via a custom proxy. When an object implements the IMarshalinterface all references to the object will be custom marshaled.
When an object does not implement the IMarshal interface, all references to the object will be standard marshaled. Most objects elect to use standard marshaling.
So the the question becomes why is the standard COM marshaler having so much problems? What is the source of the error Interface not registered?
The interface is, in fact, not registered
The requires for COM are undocumented, but i can tell you that my interface GUID does not exist in:
HKEY_CLASSES_ROOT\Interface\{EBBC7C04-315E-11D2-B62F-006097DF5BD4}
The reason for this will be explained at the end.
I know that Windows provides you the CoRegisterPSClsid function that allows you to register an interface inside your process, so that the standard marshaler will be able to marshal it:
Enables a downloaded DLL to register its custom interfaces within its running process so that the marshaling code will be able to marshal those interfaces.
HRESULT CoRegisterPSClsid(
_In_ REFIID riid,
_In_ REFCLSID rclsid
);
Parameters:
riid [in]: A pointer to the IID of the interface to be registered.
rclsid [in]: A pointer to the CLSID of the DLL that contains the proxy/stub code for the custom interface specified by riid.
Which i can try calling, but what clsid do i use?
CoRegisterPSClsid(IID_IWidget, ???);
What is the CLSID of the DLL that contains the proxy/stub code for the custom interface specified by riid? Do i use my class itself?
CoRegisterPSClsid(IID_IWidget, CLSID_Widget);
That doesn't sound right; but i don't understand the COM standard marshaler well enough. Doesn't that CLSID have to be one of the standard COM marsharling classes; implementing IPSFactoryBuffer?
Either way, it doesn't work. I still get the error "Interface not registered".
Register the interface
I of course can register my interface in the registry:
HKEY_CURRENT_USER\Software\Classes\Interface\{EBBC7C04-315E-11D2-B62F-006097DF5BD4}
(default) = "IWidget"
But that doesn't fix it. Spelunking through the Interface registry keys, i notice that many specify a ProxyStubClsid32 entry.
When a new interface is requested on an object, the proxy and stub managers
must resolve the requested IID onto the CLSID of the interface marshaler.
Under Windows NT 5.0, the class store maintains these mappings in the NT
directory, and they are cached at each host machine in the local registry. The
machine-wide IID-to-CLSID mappings are cached at
HKEY_CLASSES_ROOT\Interface
and the per-user mappings are cached at
HKEY_CURRENT_USER\Software\Classes\Interface
One or both of these keys will contain a subkey for each known interface. If the interface has an interface marshaler installed, there will be an additional
subkey (ProxyStubClsid32) that indicates the CLSID of the interface
marshaler.
Except what class implements marshaling? I don't have a marshaler.
Can COM automatically marshal based on a TypeLibrary
Is it possible that if i register a Type Library with my Interface, that COM's standard marshaler will be able to bootstrap a proxy class on the fly?
I registered my interface above. Now i manually include the TypeLibrary:
HKEY_CURRENT_USER\Software\Classes\Interface\{EBBC7C04-315E-11D2-B62F-006097DF5BD4}\TypeLib
(default) = "{38D528BD-4948-4F28-8E5E-141A51090580}"
And if i monitor the registry during the call to CoMarshalInterface i see that it attempts, and does find, my Interface IID:
Operation: RegOpenKey
Path: HKCR\WOW6432Node\Interface\{EBBC7C04-315E-11D2-B62F-006097DF5BD4}
Result: SUCCESS
It then tries to look for a ProxyStubClsid32, and fails:
Operation: RegOpenKey
Path: HKCR\WOW6432Node\Interface\{668790E3-83CC-47E0-907F-A44BA9A99C8D}\ProxyStubClsid32
Result: NAME NOT FOUND
My hope would then be that the standard COM marshaler attempts to look for:
Operation: RegOpenKey
Path: HKCR\WOW6432Node\Interface\{668790E3-83CC-47E0-907F-A44BA9A99C8D}\TypeLib
But it doesn't.
The OLE Automation marshaler
According to Don Box, the Ole Automation marshaler (PSOAInterface - {00020424-0000-0000-C000-000000000046}) is able to build a stub/proxy out of a type library:
The Type Library Marshaler
When these specially annotated interfaces are encountered by RegisterTypeLib (or LoadTypeLib in legacy mode), COM adds ProxyStubClsid32 entries for the interface with the value {00020424-0000-0000-C0000-000000000046}. This GUID corresponds to the class PSOAInterface that is registered as living in OLEAUT32.DLL, the OLE automation DLL. Because of the DLL that it lives in, this marshaler is sometimes called the [oleautomation] marshaler, although it is also called the type library marshaler or the universal marshaler. I'll refer to it as the type library marshaler, since it really has very little to do with IDispatch. (In fact, it is common to use the [oleautomation] attribute on interfaces that don't derive from IDispatch directly or indirectly.)
The class factory for the type library marshaler does something very tricky in its CreateProxy and CreateStub routines. Rather than return a statically compiled vtable (which is impossible given the fact that the requested interface didn't exist when OLEAUT32.DLL was built as part of the OS), the type library marshaler actually builds an /Oicf-style proxy and stub based on the type library for the interface. Because there is no efficient way to find the ITypeInfo for an arbitrary interface, the LIBID and version of the interface's type library must be stored under the:
HKCR\Interface\{XXX}\TypeLib
registry key.
I tried to set PSOAInterface as my interface's ProxyStub class:
Register standard COM class PSOAInterface as our proxy stub clsid
HKEY_CURRENT_USER\Software\Classes\Interface\{EBBC7C04-315E-11D2-B62F-006097DF5BD4}\ProxyStubClsid32
(default) = "{00020424-0000-0000-C0000-000000000046}"
Register our type library for our interface
HKEY_CURRENT_USER\Software\Classes\Interface\{EBBC7C04-315E-11D2-B62F-006097DF5BD4}\TypeLib
(default) = "{38D528BD-4948-4F28-8E5E-141A51090580}"
Version = "1.0"
The type library itself is already registered:
HKEY_CURRENT_USER\Software\TypeLib\{38D528BD-4948-4F28-8E5E-141A51090580}\1.0\0\win32
(default) = "D:\Junk\ComLibraryProxyTest\Win32\Project1.dll"
But it still fails.
it reads HKCR\Interface\[IID_IWidget]
it reads HKCR\Interface\[IID_IWidget]\ProxyStubClsid32
But it:
never reads: HKCR\Interface\[IID_IWidget]\TypeLib
So it fails to proxy the object.
Questions
Is it possible for the standard COM "Ole Automation" marshaler to build a proxy class out of a Type Library at runtime?
Is it possible for me to build a proxy class out of a Type Library at runtime?
Background
CoMarshalInterface takes an interface pointer on input and writes the
serialized representation of the pointer to a caller-provided byte stream. This byte stream can then be passed to another apartment, where the
CoUnmarshalInterface API function uses the byte stream to return an interface
pointer that is semantically equivalent to the original object yet can be
legally accessed in the apartment that executes the CoUnmarshalInterface
call. When calling CoMarshalInterface, the caller must indicate how far away
the importing apartment is expected to be. COM defines an enumeration for
expressing this distance:
enum MSHCTX {
MSHCTX_INPROC = 4, // in-process/same host
MSHCTX_LOCAL = 0, // out-of-process/same host
MSHCTX_NOSHAREDMEM = 1, //16/32 bit/same host
MSHCTX_DIFFERENTMACHINE = 2, // off-host
};
It is legal to specify a greater distance than required, but it is more efficient
to use the correct MSHCTX when possible. CoMarshalInterface also allows
the caller to specify the marshaling semantics using the following marshal flags:
enum MSHLFLAGS {
MSHLFLAGS_NORMAL, // marshal once, unmarshal once
MSHLFLAGS_TABLESTRONG, // marshal once, unmarshal many
MSHLFLAGS_TABLEWEAK, // marshal once, unmarshal many
MSHLFlAGS_NOPING = 4 // suppress dist. garbage collection
Normal marshaling (sometimes called call marshaling) indicates that the
marshaled object reference must be unmarshaled only once, and if additional
proxies are needed, additional calls to CoMarshalInterface are required. Table
marshaling indicates that the marshaled object reference may be unmarshaled
zero or more times without requiring additional calls to CoMarshalInterface.
I think that all COM object type libraries, when compiled by MIDL, can automatically create a proxy/stub factory. But in my case:
if the COM standard marshaler can't find a proxy/stub factory for an interface, it returns the error REGDB_E_IIDNOTREG.
It may be the case that i have to either:
use CreateProxyFromTypeInfo and CreateStubFromTypeInfo to create my own proxy
let the standard COM marshaler automatically create a proxy/stub if there's a typeinfo chunk associated with the interface GUID.
Bonus Reading
Old New Thing: What are the rules for CoMarshalInterThreadInterfaceInStream and CoGetInterfaceAndReleaseStream?
Old New Thing: Why do I get the error REGDB_E_IIDNOTREG when I call a method that returns an interface?
Old New Thing: Why do I get E_NOINTERFACE when creating an object that supports that interface?
Don Box - MSJ: Standard Marshalling in COM (archive)
Mysterious disappearing acts by samples from Microsoft SDKs
What CoMarshalInterface actually needs is IMarshal implementation (pointer) for the given interface, so that API could request the marshaling magic from it and, in particular, request IMarshal::GetUnmarshalClass in order to obtain information who is going to do the reverse magic afterwards:
This method is called indirectly, in a call to CoMarshalInterface, by whatever code in the server process is responsible for marshaling a pointer to an interface on an object. This marshaling code is usually a stub generated by COM for one of several interfaces that can marshal a pointer to an interface implemented on an entirely different object.
You don't have IMarshal implemented on your widget, so you are going to get it somewhere from.
As you started your question mentioning that you "want to marshal an interface to another thread" and the code comment says "Create our widget" there is a chance that you can utilize IMarhsal implementation of free threaded marshaler. The question does not provide information to tell whether it is possible and/or acceptable solution.
Back to the challenge of obtaining a marshaler, you are trying to work this around by utilizing a "standard" marshaler by "registering the interface":
why is the standard COM marshaler having so much problems
[...]
I of course can register my interface in the registry
Well, this is not how things actually work.
Interface registration is not just a registry key that "okay, this IID has its own key in the registry". The purpose of such registration is to point where to look for proxy/stub pair for this interface. Your manual creating of registry entries cannot help here if you don't have a proxy/stub DLL for the interface in question. If you had it, you would just regsvr32 it the usual way to create the registry keys.
So called standard marshaler, your next try, is not supposed to marshal any interface and your IWidget in particular. OLE supplies so called "PSOAInterface" marshaler, which is capable of supplying proxy/stub pairs for OLE Automation compatible interfaces. Just for them! The marshaler does not have so much problems, it actually has just one: your IWidget is unlikely to be compatible or you would not have the problem in first place.
If your IWidget was compatible, had an associated type library, where it was marked as [oleautomation], the type library registration process would have automatically created the registry keys referencing PSOAInterface and supplying ProxyStubClsid32. Then marshaling API would have picked PSOAInterface for your widget, it would have picked up the registered type library, load details of the interface, then provided standard proxy/stub pair for it and that's it. Standard marshaler works just within these contraints and not just for any interface pointed to.
That is, your options are:
implement IMarshal on the widget server
make IWidget OLE Automation compatible, with type library, so that type library registration process activates "standard marshaler" PSOAInterface for your interface
build, if possible and applicable, proxy/stub implementation for your interface automatically generated by MIDL complier (you can check standard ATL DLL project, created from Visual Studio template on how it can be done - it creates additional project with "PS" suffix, for a supplementary proxy/stub DLL)
implement separately IMarshal in a standalone DLL, and register it with the registry and your IWidget interface, so that marshaling API would pick it up when it attempts to marshal the interface pointer (I suppose it's the only option here if your IWidget is OLE incompatible and you have reasons to not alter the original implementation)
use free threaded marshaler in the widget server if its restrictions are acceptable
Oh wait, hold on - there is another weird one. If you don't want or cannot afford, or you are reluctant to change the COM server (widget, that is) but you can modify client side code as you like, you can create a thin wrapper that implements two interfaces IWidget (all methods forward calls to real server) and IMarshal on client side, and pass its IWidget to the CoMarshalInterThreadInterfaceInStream API. This will force COM to use your marshaling without altering the original server. You are on your own, of course, to do the actual marshaling afterwards. It is unlikely that it matches your actual need, and it is just an abstract note to the discussion (which mostly consists of attempts to do impossible without details on interface itself and available options on modification of server implementation).
TL;DR: Actual questions:
Is it possible for the standard COM "Ole Automation" marshaler to build a proxy class out of a Type Library at runtime?
Short answer: yes.
Is it possible for me to build a proxy class out of a Type Library at runtime?
Short answer: yes, with the type library marshaler and IPSFactoryBuffer. Or if you're willing to use the undocumented CreateProxyFromTypeInfo and CreateStubFromTypeInfo.
I wonder why you'd want to this.
This question is riddled with red herrings.
I'm trying to marshal an interface to another thread.
(...) the call to CoMarshalThreadInterfaceInStream fails with:
REGDB_E_IIDNOTREG (0x80040155)
Interface not registered
A useful error code, in this case.
Go directly to CoMarshalInterface
(...) that fails with:
REGDB_E_IIDNOTREG (0x80040155)
Interface not registered
It's not by switching API calls that do essentially the same thing that you could solve your issue.
Use standard marshaling
My class does not implement IMarhsal interface.
Can you confirm it doesn't implement INoMarshal, IStdMarshalInfo and IAgileObject either, just to be exhaustive?
The interface is, in fact, not registered
This was expected.
I know that Windows provides you the CoRegisterPSClsid
(...)
Which i can try calling, but what clsid do i use?
CoRegisterPSClsid(IID_IWidget, ???);
If you don't have a proxy/stub, why would you go through this?
But to answer this question, a standard marshalers' CLSID is usually the same GUID as the first IID found in an IDL file.
Register the interface
I of course can register my interface in the registry:
HKEY_CURRENT_USER\Software\Classes\Interface\{EBBC7C04-315E-11D2-B62F-006097DF5BD4}
(default) = "IWidget"
But that doesn't fix it. Spelunking through the Interface registry keys, i notice that many specify a ProxyStubClsid32 entry.
You should read the documentation, instead of relying on what other registry entries contain.
(...) I don't have a marshaler.
This should be the actual problem. If this object is yours, how do you intend to marshal it? Please see the bottom section of my answer.
Can COM automatically marshal based on a TypeLibrary
This is clearly rhetoric. Why would you think of a type library just now, instead of to begin with? The rest of your question corroborates.
Now i manually include the TypeLibrary:
HKEY_CURRENT_USER\Software\Classes\Interface\{EBBC7C04-315E-11D2-B62F-006097DF5BD4}\TypeLib
(default) = "{38D528BD-4948-4F28-8E5E-141A51090580}"
You mention this twice.
But it:
never reads: HKCR\Interface[IID_IWidget]\TypeLib
The type library marshaler has its own cache of interface->type library. To clear the cache, try recreating the registry entries, logging off and logging on, or ultimately, reboot.
The object may implement IProvideClassInfo, I don't know if the type library marshaler actually cares to QueryInterface for this to fetch a runtime ITypeInfo to work with.
These are the main types of marshalers:
Standard marshalers, which are usually compiled into a DLL from C/C++ source code generated by MIDL (which consists mainly of declarations on how to marshal types)
The type library marshaler, which can marshal types at runtime based on automation-compatible type information
The free-threaded marshaler aggregated with CoCreateFreeThreadedMarshaler, which avoids marshaling between threads within the same process
Custom marshalers, which do whatever the developer wants, most commonly to implement marshal-by-value, or a free-threaded marshaler back when CoCreateFreeThreadedMarshaler didn't exist
The standard marshalers generated by MIDL consist mainly of declarations on how to marshal types in and out, so the way they work is in the same vein as the type library marshaler. According to Don Box, the result is very similar.
However, the actual declarations are very different. The type library marshaler works on type information that is meant to work in VB6 and (excluding certain things, such as user-defined types) scripting languages (mainly VBScript and JScript) and is intended to be consumed by IDEs (e.g. VB6, VBA, Delphi, OleView) and interop code generators (e.g. VC++ #import, tlbimp.exe, Delphi's "Import Type Library", Lisp 1 2), etc.