Below code is the serialize method of org.springframework.util.SerializationUtils(spring-core.3.2.1.jar).
As you can see, there is no oos.close method. Is it OK not to call close method?
In this particular case, it's okay - the two streams involved (the ObjectOutputStream and ByteArrayOutputStream) don't use any unmanaged resources (file handles, network handles) etc so it's okay for the garbage collector to just take care of it.
Personally I usually close even streams like that, just to be in the right habits and in case someone changes the type of the stream to one that does have unmanaged resources, but I don't think it really counts as a bug in this case.
close should be called by most sane objects when they are destroyed (garbage collected). But since you don't know exactly when that will happen, you should make it a point to call close yourself, so that the cleanup (file buffers flushed, network kissoff performed) happens when you expect it to.
Related
I've allocated some GPU global memory with cudaMalloc(), say, in the constructor of some class. Now it's time to destruct the instance I've constructed, and I have my instance's data pointer. The thing is, I'm worried maybe some mischievous code elsewhere has called cudaDeviceReset(), after which my cudaFree() will probably fail (I'll get an invalid device pointer error). So, how can can I tell whether my pointer is elligible for cudaFree()ing?
I don't believe you can do much about that.
About the best you can do is try and engineer the lifespan of objects which will call the CUDA APIs in their destructors to do so before context destruction. In practice, that means having them fall of of scope in a well defined fashion before the context is automatically or manually torn down.
For a call like cudaFree(), which is somewhat "fire and forget" anyway, the best thing to do might be to write your own wrapper for the call and explicitly catch and tastefully ignore any obvious error conditions which would arise if the call was made after context destruction-
Given what talonmies says, one might consider doing the converse:
wrap your cudaDeviceReset() calls to also regard a 'generation counter'.
Counter increases will be protected by a lock.
While you lock, you reset and increment the generation counter.
Wrap cudaMalloc() to also keep the generation index (you might need a class/struct for that) - obtained during allocation (which also locks).
Wrap cudaFree() to lock and only really cudaFree() if the reset generation has not changed.
... now, you might say "Is all that locking worth it? At worst, you'll get an error, it's not such a big deal." And, to be honest - I'm not sure it's worth it. You could make this somewhat less painful by using a Reader-Writer lock instead of a simple lock, where the allocate and free are just readers that can all access concurrently.
I know there are no destructors in Go since technically there are no classes. As such, I use initClass to perform the same functions as a constructor. However, is there any way to create something to mimic a destructor in the event of a termination, for the use of, say, closing files? Right now I just call defer deinitClass, but this is rather hackish and I think a poor design. What would be the proper way?
In the Go ecosystem, there exists a ubiquitous idiom for dealing with objects which wrap precious (and/or external) resources: a special method designated for freeing that resource, called explicitly — typically via the defer mechanism.
This special method is typically named Close(), and the user of the object has to call it explicitly when they're done with the resource the object represents. The io standard package does even have a special interface, io.Closer, declaring that single method. Objects implementing I/O on various resources such as TCP sockets, UDP endpoints and files all satisfy io.Closer, and are expected to be explicitly Closed after use.
Calling such a cleanup method is typically done via the defer mechanism which guarantees the method will run no matter if some code which executes after resource acquisition will panic() or not.
You might also notice that not having implicit "destructors" quite balances not having implicit "constructors" in Go. This actually has nothing to do with not having "classes" in Go: the language designers just avoid magic as much as practically possible.
Note that Go's approach to this problem might appear to be somewhat low-tech but in fact it's the only workable solution for the runtime featuring garbage-collection. In a language with objects but without GC, say C++, destructing an object is a well-defined operation because an object is destroyed either when it goes out of scope or when delete is called on its memory block. In a runtime with GC, the object will be destroyed at some mostly indeterminate point in the future by the GC scan, and may not be destroyed at all. So if the object wraps some precious resource, that resource might get reclaimed way past the moment in time the last live reference to the enclosing object was lost, and it might even not get reclaimed at all—as has been well explained by #twotwotwo in their respective answer.
Another interesting aspect to consider is that the Go's GC is fully concurrent (with the regular program execution). This means a GC thread which is about to collect a dead object might (and usually will) be not the thread(s) which executed that object's code when it was alive. In turn, this means that if the Go types could have destructors then the programmer would need to make sure whatever code the destructor executes is properly synchronized with the rest of the program—if the object's state affects some data structures external to it. This actually might force the programmer to add such synchronization even if the object does not need it for its normal operation (and most objects fall into such category). And think about what happens of those exernal data strucrures happened to be destroyed before the object's destructor was called (the GC collects dead objects in a non-deterministic way). In other words, it's much easier to control — and to reason about — object destruction when it is explicitly coded into the program's flow: both for specifying when the object has to be destroyed, and for guaranteeing proper ordering of its destruction with regard to destroying of the data structures external to it.
If you're familiar with .NET, it deals with resource cleanup in a way which resembles that of Go quite closely: your objects which wrap some precious resource have to implement the IDisposable interface, and a method, Dispose(), exported by that interface, must be called explicitly when you're done with such an object. C# provides some syntactic sugar for this use case via the using statement which makes the compiler arrange for calling Dispose() on the object when it goes out of the scope declared by the said statement. In Go, you'll typically defer calls to cleanup methods.
One more note of caution. Go wants you to treat errors very seriously (unlike most mainstream programming language with their "just throw an exception and don't give a fsck about what happens due to it elsewhere and what state the program will be in" attitude) and so you might consider checking error returns of at least some calls to cleanup methods.
A good example is instances of the os.File type representing files on a filesystem. The fun stuff is that calling Close() on an open file might fail due to legitimate reasons, and if you were writing to that file this might indicate that not all the data you wrote to that file had actually landed in it on the file system. For an explanation, please read the "Notes" section in the close(2) manual.
In other words, just doing something like
fd, err := os.Open("foo.txt")
defer fd.Close()
is okay for read-only files in the 99.9% of cases, but for files opening for writing, you might want to implement more involved error checking and some strategy for dealing with them (mere reporting, wait-then-retry, ask-then-maybe-retry or whatever).
runtime.SetFinalizer(ptr, finalizerFunc) sets a finalizer--not a destructor but another mechanism to maybe eventually free up resources. Read the documentation there for details, including downsides. They might not run until long after the object is actually unreachable, and they might not run at all if the program exits first. They also postpone freeing memory for another GC cycle.
If you're acquiring some limited resource that doesn't already have a finalizer, and the program would eventually be unable to continue if it kept leaking, you should consider setting a finalizer. It can mitigate leaks. Unreachable files and network connections are already cleaned up by finalizers in the stdlib, so it's only other sorts of resources where custom ones can be useful. The most obvious class is system resources you acquire through syscall or cgo, but I can imagine others.
Finalizers can help get a resource freed eventually even if the code using it omits a Close() or similar cleanup, but they're too unpredictable to be the main way to free resources. They don't run until GC does. Because the program could exit before next GC, you can't rely on them for things that must be done, like flushing buffered output to the filesystem. If GC does happen, it might not happen soon enough: if a finalizer is responsible for closing network connections, maybe a remote host hits its limit on open connections to you before GC, or your process hits its file-descriptor limit, or you run out of ephemeral ports, or something else. So it's much better to defer and do cleanup right when it's necessary than to use a finalizer and hope it's done soon enough.
You don't see many SetFinalizer calls in everyday Go programming, partly because the most important ones are in the standard library and mostly because of their limited range of applicability in general.
In short, finalizers can help by freeing forgotten resources in long-running programs, but because not much about their behavior is guaranteed, they aren't fit to be your main resource-management mechanism.
There are Finalizers in Go. I wrote a little blog post about it. They are even used for closing files in the standard library as you can see here.
However, I think using defer is more preferable because it's more readable and less magical.
So I was doing some research on the File class in Ruby. As I was digging I learned that File was a subclass of IO. To my understanding when you create an IO object (or File object), a buffer is opened to that file that allows you to read and write to that file. I don't completely understand what a buffer is, but apparently it stays open until you call the #close method on the object. To my understanding this buffer is opened whether you call File.new or File.open (please correct me if I'm wrong on any of this).
So say you like to use the File class for paths and stuff like this:
f = File.new('spec/tmp/testfile.md')
File.basename(f)
But you never call f.close. Does leaving this buffer open leak memory? If I called this several hundred times for a tree in a filesystem would I be in deep trouble?
Thanks for your replies!
PS I know you can just use File.basename('spec/tmp/testfile.md') instead, I'm just using this as an example
Yes
Except for the sys* family of operations, Ruby's IO ops ultimately allocate both file descriptors and buffers.
If you don't close the IO object then you are correct ... you most likely leak both the fd and the buffer.
Now, if you allocate it in such a way as to overwrite or otherwise end the lifetime of the old reference, then Ruby can g/c the entire object. This will definitely free the buffer, and it will eventually free the FD as well.
In all languages, however, it's considered quite bad practice to rely upon a g/c-triggered finalizer as it's unpredictable how long it will take and how many outstanding OS-level resources will exist at one time. You may exceed some local limit before the g/c machinery even starts up.
The general rule is to allocate and free OS resources synchronously.
And as long as I'm beating the subject to death, there is an exception. If you are allocating a fixed number of descriptors or something, and they all must exist at once anyway, and the program is going to exit after finishing its work, then it's OK to just leave them. The OS cleans up everything. For example, it's best not to free memory right before exit. The processing needed to manage the heap is completely wasted if the program is about to exit. The OS is just going to put every single page of the program on its free list. And there is an exception to the exception. If it's homework, I would free everything.
Related to asynchronous IO using a (bidirectional) NetworkStream, MSDN says that "EndRead must be called once for every call to BeginRead."
Is this true even for cases where the EndRead() will throw an exception, such as in the case that the NetworkStream has been closed after the BeginRead() has been issued?
I don't want the overhead of the throwing of the exception, but neither do I want to leak OS precious resources reserved by BeginRead().
I also know that the stream could be closed between a test of the stream's state and the conditional EndRead(), but if EndRead() can be omitted when we know the stream is closed, that will save on the exception handling in the majority case.
Am I doing it wrong?
Thanks!
GCHandles pinning your buffers and some other unmanaged resources are released by the completion port callback. The unmanaged OVERLAPPED structure will hang around until the IAsyncResult gets finalized. This may be tolerable if the network load in your application is not large, but may become a problem if your application handles many connections per second, because finalization happens only after a full GC collection and on a separate thread.
NB: these are implementation details, obtained with Reflector. Caveat emptor.
What are the implications of calling CloseHandle more than once?
The docs say "you shouldn't" but I think I have a realistic case with named pipes where a handle might be closed externally (See end of post).
CloseHandle throws an exception in debug mode in this case, which suggests to me the developers think this is serious, but the docs aren't exactly clear.
(Polite request: Please avoid the answer "just don't!" :-). Of course one should avoid closing a handke more than once, and of course there are good techniques to help with this: I'm just interested in what happens if you don't).
I've heard some people suggest that if the handle was quickly reused by the OS you might end up closing another, different handle.
Is this likely?
How does Windows choose handle IDs?
Is there any guarantee about how regularly a handle value will be reused?
(e.g. TCP ensures that a port number cannot be reused within a certain timeframe).
Can you close handles accross handle types? E.g., could I be thinking I'm closing a pipe but end up closing an Event?
Thanks!
John
(Context to this: I'm using named pipes in a client/server model. It seems to me very difficult to ensure that exactly one party is guaranteed to close the handle, e.g. in process crash/killed case. Perhaps I'm wrong, but certainly the MSDN sample code seems to my mind to allow the client to close the shared handle, and then when the server tries to close it, it is already closed).
Simple enough to check:
HANDLE h = 0;
h = CreateMutex(NULL, TRUE, NULL);
printf("%X\n", h);
CloseHandle(h);
h = 0;
h = CreateMutex(NULL, TRUE, NULL);
printf("%X\n", h);
In my WinXP x64 this produced:
2E8
2E8
So there you have it.
Unlike TCP ports, handles are recycled immediately.
Repeat this experiment with your favorite API or any mix thereof.
You probably have the wrong mental image of a pipe. It has two ends, each represented by a different handle. Yes, CloseHandle has to be called twice to make the pipe instance disappear. But since they are different handles, that can never cause any problem. Also note that handle instances are process specific. Even if they have the same value in both processes, they do not reference the same pipe endpoint.
There are two things that could happen:
You close a handle opened by some other code. That probably doesn't affect your code but it's likely to be catastrophic for the other code.
If you're running with a debugger attached, you crash your application because the OS will raise an exception when it detects an invalid handle being closed.
Neither of these is particularly attractive IMHO.