Golang-like defer in Rust - go

In Go, you can use the defer keyword to execute a function when the current function returns, similar to the traditional finally keyword in other languages. This is useful for cleaning up state regardless of what happens throughout the function body. Here's an example from the Go blog:
func CopyFile(dstName, srcName string) (written int64, err error) {
src, err := os.Open(srcName)
if err != nil {
return
}
defer src.Close()
dst, err := os.Create(dstName)
if err != nil {
return
}
defer dst.Close()
return io.Copy(dst, src)
}
How can this functionality be achieved in Rust? I know about RAII, but in my specific case the state is in an external system. I'm writing a test that writes a key to a key-value store, and I need to make sure it's deleted at the end of the test regardless of whether or not the assertions in the test cause a panic.
I found this Gist but I don't know if this is a recommended approach. The unsafe destructor is worrisome.
There is also this issue on the Rust GitHub repository, but it's three years old and clearly not very relevant anymore.

(e: don't miss bluss's answer and their scopedguard crate, below.)
The correct way to achieve this by having code that runs in a destructor, like the defer! macro you link to does. For anything more than ad-hoc testing I would recommend writing a handle type with a proper destructor, e.g. one interacts with std::sync::Mutex via its MutexGuard type (returned by lock): there's no need to call unlock on the mutex itself. (The explicit handle-with-destructor approach is more flexible too: it has mutable access to the data, whereas the deferred approach may not be able to, due to Rust's strong aliasing controls.)
In any case, that macro is now (much!) improved due to recent changes, in particular, pnkfelix's sound generic drop work, which removes the necessity for #[unsafe_destructor]. The direct update would be:
struct ScopeCall<F: FnMut()> {
c: F
}
impl<F: FnMut()> Drop for ScopeCall<F> {
fn drop(&mut self) {
(self.c)();
}
}
macro_rules! defer {
($e:expr) => (
let _scope_call = ScopeCall { c: || -> () { $e; } };
)
}
fn main() {
let x = 42u8;
defer!(println!("defer 1"));
defer!({
println!("defer 2");
println!("inside defer {}", x)
});
println!("normal execution {}", x);
}
Output:
normal execution 42
defer 2
inside defer 42
defer 1
Although, it would be syntactically nicer as:
macro_rules! expr { ($e: expr) => { $e } } // tt hack
macro_rules! defer {
($($data: tt)*) => (
let _scope_call = ScopeCall {
c: || -> () { expr!({ $($data)* }) }
};
)
}
(The tt hack is necessary due to #5846.)
The use of the generic tt ("token tree") allows one to invoke it without the inner { ... } when there are multiple statements (i.e. it behaves more like a "normal" control flow structure):
defer! {
println!("defer 2");
println!("inside defer {}", x)
}
Also, for maximum flexibility about what the deferred code can do with captured variables, one could use FnOnce instead of FnMut:
struct ScopeCall<F: FnOnce()> {
c: Option<F>
}
impl<F: FnOnce()> Drop for ScopeCall<F> {
fn drop(&mut self) {
self.c.take().unwrap()()
}
}
That will also require constructing the ScopeCall with a Some around the value for c. The Option dance is required because calling a FnOnce moves ownership, which isn't possible from behind self: &mut ScopeCall<F> without it. (Doing this is OK, since the destructor only executes once.)
All in all:
struct ScopeCall<F: FnOnce()> {
c: Option<F>
}
impl<F: FnOnce()> Drop for ScopeCall<F> {
fn drop(&mut self) {
self.c.take().unwrap()()
}
}
macro_rules! expr { ($e: expr) => { $e } } // tt hack
macro_rules! defer {
($($data: tt)*) => (
let _scope_call = ScopeCall {
c: Some(|| -> () { expr!({ $($data)* }) })
};
)
}
fn main() {
let x = 42u8;
defer!(println!("defer 1"));
defer! {
println!("defer 2");
println!("inside defer {}", x)
}
println!("normal execution {}", x);
}
(Same output as the original.)

I use the following for a scope guard. It uses the Deref traits to provide shared & mutable access to the guarded value, without moving it out (that would invalidate the guard!)
My use case is correctly resetting the terminal when the program exits, even if panicking:
extern crate scopeguard;
use scopeguard::guard;
// ... terminal setup omitted ...
// Use a scope guard to restore terminal settings on quit/panic
let mut tty = guard(tty, |tty| {
// ... I use tty.write() here too ...
ts::tcsetattr(tty.as_raw_fd(), ts::TCSANOW, &old_attr).ok();
});
game_main(&mut tty).unwrap(); // Deref coercion magic hands off the inner &mut TTY pointer here.
Module scopeguard.rs:
use std::ops::{Deref, DerefMut};
pub struct Guard<T, F> where
F: FnMut(&mut T)
{
__dropfn: F,
__value: T,
}
pub fn guard<T, F>(v: T, dropfn: F) -> Guard<T, F> where
F: FnMut(&mut T)
{
Guard{__value: v, __dropfn: dropfn}
}
impl<T, F> Deref for Guard<T, F> where
F: FnMut(&mut T)
{
type Target = T;
fn deref(&self) -> &T
{
&self.__value
}
}
impl<T, F> DerefMut for Guard<T, F> where
F: FnMut(&mut T)
{
fn deref_mut(&mut self) -> &mut T
{
&mut self.__value
}
}
impl<T, F> Drop for Guard<T, F> where
F: FnMut(&mut T)
{
fn drop(&mut self) {
(self.__dropfn)(&mut self.__value)
}
}
This is now the crate scopeguard on crates.io.

Related

Approximating structural typing in a Golang function

A library is providing me a series of types like this:
type T1 struct {
n int
}
type T2 struct {
n int
}
type T3 struct {
n int
}
there is a marker interface like this
type S interface {
isS()
}
func (T1) isS() {}
func (T2) isS() {}
func (T3) isS() {}
I am trying to write a function:
func getN(s S) int {
return s.n
}
Obviously, that won’t compile, and it shouldn’t.
What I am doing now is
func getN(s S) int {
if t1, ok := s.(T1); ok {
return t1
}
if t2, ok := s.(T2); ok {
return t2
}
...
}
That works, but it unbearably awful. The library adds new T structures fairly often and it’s a disastrous violation of open/closed. Any tolerable language has a facility for this kind of situation; what is Golang’s?
Create a method on the interface that returns the type you are looking for. That will get it to compile correctly.
type S interface {
GetN()
}
func (T1) GetN() int {
// impl
}
func (T2) GetN() int {
// impl
}
func (T3) GetN() int {
// impl
}
then if you still need your interesting function that takes an interface, you can do this:
func getN(s S) int {
return s.GetN()
}

How to implement a generic Either type in Go?

With the new generics in Go 1.18, I thought it might be possible to create a 'Either[A,B]' type that can be used to express that something could be either of type A or type B.
A situation where you might use this is in situations where a function might return one of two possible values as a result (e.g. one for 'normal' result and one for an error).
I know the 'idiomatic' Go for errors would be to return both a 'normal' value and an error value, returning a nil for either the error or the value. But... it sort of bothers me that we are essentially saying 'this returns A and B' in the type, where what we really mean to say is 'this returns A or B'.
So I thought maybe we can do better here, and I thought this might also be a good exercise to see/test the boundaries of what we can do with these new generics.
Sadly,try as I might, so far I have not been able solve the exercise and get anything working/compiling. From one of my failed attempts, here is an interface I'd like to implement somehow:
//A value of type `Either[A,B]` holds one value which can be either of type A or type B.
type Either[A any, B any] interface {
// Call either one of two functions depending on whether the value is an A or B
// and return the result.
Switch[R any]( // <=== ERROR: interface methods must have no type parameters
onA func(a A) R),
onB func(b B) R),
) R
}
Unfortunately, this fails rather quickly because declaring this interface isn't allowed by Go. Apparantly because 'interface methods must have no type parameters'.
How do we work around this restriction? Or is there simply no way to create a 'type' in Go that accurately expresses the idea that 'this thing is/returns either A or B' (as opposed to a tuple of both A and B).
If I had to do this, I would look up a functional programming language(like OCaml) and knock-off their solution of the either type..
package main
import (
"errors"
"fmt"
"os"
)
type Optional[T any] interface {
get() (T, error)
}
type None[T any] struct {
}
func (None[T]) get() (T, error) {
var data T
return data, errors.New("No data present in None")
}
type Some[T any] struct {
data T
}
func (s Some[T]) get() (T, error) {
return s.data, nil
}
func CreateNone[T any]() Optional[T] {
return None[T]{}
}
func CreateSome[T any](data T) Optional[T] {
return Some[T]{data}
}
type Either[A, B any] interface {
is_left() bool
is_right() bool
find_left() Optional[A]
find_right() Optional[B]
}
type Left[A, B any] struct {
data A
}
func (l Left[A, B]) is_left() bool {
return true
}
func (l Left[A, B]) is_right() bool {
return false
}
func left[A, B any](data A) Either[A, B] {
return Left[A, B]{data}
}
func (l Left[A, B]) find_left() Optional[A] {
return CreateSome(l.data)
}
func (l Left[A, B]) find_right() Optional[B] {
return CreateNone[B]()
}
type Right[A, B any] struct {
data B
}
func (r Right[A, B]) is_left() bool {
return false
}
func (r Right[A, B]) is_right() bool {
return true
}
func right[A, B any](data B) Either[A, B] {
return Right[A, B]{data}
}
func (r Right[A, B]) find_left() Optional[A] {
return CreateNone[A]()
}
func (r Right[A, B]) find_right() Optional[B] {
return CreateSome(r.data)
}
func main() {
var e1 Either[int, string] = left[int, string](4143)
var e2 Either[int, string] = right[int, string]("G4143")
fmt.Println(e1)
fmt.Println(e2)
if e1.is_left() {
if l, err := e1.find_left().get(); err == nil {
fmt.Printf("The int is: %d\n", l)
} else {
fmt.Fprintln(os.Stderr, err)
}
}
if e2.is_right() {
if r, err := e2.find_right().get(); err == nil {
fmt.Printf("The string is: %s\n", r)
} else {
fmt.Fprintln(os.Stderr, err)
}
}
}
The Either could be modeled as a struct type with one unexported field of type any/interface{}. The type parameters would be used to ensure some degree of compile-time type safety:
type Either[A, B any] struct {
value any
}
func (e *Either[A,B]) SetA(a A) {
e.value = a
}
func (e *Either[A,B]) SetB(b B) {
e.value = b
}
func (e *Either[A,B]) IsA() bool {
_, ok := e.value.(A)
return ok
}
func (e *Either[A,B]) IsB() bool {
_, ok := e.value.(B)
return ok
}
If Switch has to be declared as a method, it can't be parametrized in R by itself. The additional type parameter must be declared on the type definition, however this might make usage a bit cumbersome because then R must be chosen upon instantiation.
A standalone function seems better — in the same package, to access the unexported field:
func Switch[A,B,R any](e *Either[A,B], onA func(A) R, onB func(B) R) R {
switch v := e.value.(type) {
case A:
return onA(v)
case B:
return onB(v)
}
}
A playground with some code and usage: https://go.dev/play/p/g-NmE4KZVq2
You can use the https://github.com/samber/mo library (disclaimer: I'm the project author).
Either signature is:
type Either[L any, R any] struct {}
Some examples:
import "github.com/samber/mo"
left := lo.Left[string, int]("hello")
left.LeftOrElse("world")
// hello
left.RightOrElse(1234)
// 1234
left.IsLeft()
// true
left.IsRight()
// false
Your question about a Switch pattern can be implemented this way:
import "github.com/samber/mo"
left := lo.Left[string, int]("hello")
result := left.Match(
func(s string) Either[string, int] {
// <-- should enter here
return lo.Right[string, int](1234)
},
func(i int) Either[string, int] {
// <-- should not enter here
return lo.Right[string, int](i * 42)
},
)
result.LeftOrElse("world")
// world
result.RightOrElse(42)
// 1234
A solution finally came to me. The key was defining the 'Either' type as a 'struct' instead of an interface.
type Either[A any, B any] struct {
isA bool
a A
b B
}
func Switch[A any, B any, R any](either Either[A, B],
onA func(a A) R,
onB func(b B) R,
) R {
if either.isA {
return onA(either.a)
} else {
return onB(either.b)
}
}
func MakeA[A any, B any](a A) Either[A, B] {
var result Either[A, B]
result.isA = true
result.a = a
return result
}
func MakeB[A any, B any](b B) Either[A, B] {
... similar to MakeA...
}
That works, but at the 'price' of really still using a 'tuple-like' implementation under the hood were we store both an A and a B but ensure it is only possible to use one of them via the public API.
I suspect this is the best we can do given the restrictions Go puts on us.
If someone has a 'workaround' that doesn't essentially use 'tuples' to represent 'unions'. I would consider that a better answer.

How to eliminate Data Race even with Mutex Locks around pointer variable

Some starter code goes here,
func (chm *ConcurrentHashMap) NFetchWorker() {
for {
key := <-NFetchWorkerPipe
chm.mu.RLock()
data := chm.data[string(key)]
chm.mu.RUnlock()
if data.IsUsingNFetch {
chm.mu.Lock()
*(chm.data[string(key)].NFetch)--
chm.mu.Unlock()
}
}
}
go NFetchWorker()
Struct ConcurrentHashMap looks like this,
type ConcurrentHashMap struct {
data map[string]DataBlock
mu sync.RWMutex
}
Struct DataBlock looks like this,
type DataBlock struct {
...
NFetch *int32
IsUsingNFetch bool
...
}
Now when I try to run tests with race flag enabled. I get,
Write at 0x00c00012e310 by goroutine 8:
(*ConcurrentHashMap).NFetchWorker()
The line numbers point to this line.
*(chm.data[string(key)].NFetch)--
However I don't face this issue when I do not use pointer in DataBlock.NFetch.
But if I do that I lose the ability to make changes to NFetch directly from map without reassigning a whole new struct object to that hash in map, which would be relatively computationally expensive. I want to change the value of NFetch without reassigning the whole struct again for one small change while being free from DATA RACE. Any solutions??
Fairly New to Golang, I could be doing something really stupid here, feel free to point it out.
UPDATE: Adding Read and Write Op function
func (chm *ConcurrentHashMap) Get(key Key) Value {
chm.mu.RLock()
fetch, ok := chm.data[string(key)]
chm.mu.RUnlock()
if ok {
if CheckValueValidity(&fetch) {
NFetchWorkerPipe <- key
return fetch.Value
} else {
chm.mu.Lock()
delete(chm.data, string(key))
chm.mu.Unlock()
}
}
return constants.NIL
}
Write Op
func (chm *ConcurrentHashMap) Insert(key Key, value Value, options Options) {
...
chm.mu.Lock()
chm.data[string(key)] = Block{
Value: value,
NFetch: nFetchOld,
Expiry: time.Now().Add(delay),
IsUsingNFetch: foundNFetch,
IsUsingExpiry: foundTTL,
mu: mutex,
}
chm.mu.Unlock()
}
The problem is in the CheckValueValidity where you are accessing NFetch without locking it. Pointers should never be accessed without locking them.
func CheckValueValidity(value *Block) bool {
if value.IsUsingExpiry && !value.Expiry.After(time.Now()) {
return false
}
if value.IsUsingNFetch && *(value.NFetch) <= 0 {
return false
}
return true
}
This code should work
func (chm *ConcurrentHashMap) Get(key Key) Value {
chm.mu.RLock()
fetch, ok := chm.data[string(key)]
isValid := CheckValueValidity(&fetch)
chm.mu.RUnlock()
if ok {
if isValid {
NFetchWorkerPipe <- key
return fetch.Value
} else {
chm.mu.Lock()
delete(chm.data, string(key))
chm.mu.Unlock()
}
}
return constants.NIL
}

function with mutex.Lock that returns before unlocking

I need to use mutex to read a variable and if the variable is 0, return from the function. This would prevent the mutex from Unlocking though.
I know that I could simply put a mutex.Unlock just before the return but it does not seem nice / correct.
I can't even do a defer mutex.Unlock() at the beginning of the function because the the code after requires a lot of time to run.
Is there a correct way to do so?
This is the example:
func mutexfunc() {
mutex.Lock()
if variable == 0 {
return
}
mutex.Unlock()
// long execution time (mutex must be unlocked)
}
UPDATE:
this is the solution I prefer:
var mutex = &sync.Mutex{}
var mutexSensibleVar = 0
func main() {
if withLock(func() bool { return mutexSensibleVar == 1 }) {
fmt.Println("it's true")
} else {
fmt.Println("it's false")
}
fmt.Println("end")
}
func withLock(f func() bool) bool {
mutex.Lock()
defer mutex.Unlock()
return f()
}
If you can't use defer, which is something you can't do here, you have to do the obvious:
func mutexfunc() {
mutex.Lock()
if variable == 0 {
mutex.Unlock()
return
}
mutex.Unlock()
// long execution time (mutex must be unlocked)
}
If the mutex is there only to protect that variable (that is, there isn't other code you're not showing us), you can also use sync/atomic:
func f() {
if atomic.LoadInt64(&variable) ==0 {
return
}
...
}
You can separate the locked part into its own function.
func varIsZero() bool {
mutex.Lock()
defer mutex.Unlock()
return variable == 0
}
func mutexfunc() {
if varIsZero() { return }
...
}
An alternative would be to use an anonymous function inside mutexfunc rather than a completely independent function, but it's a matter of taste here.
Also consider the (clumsy but readable) variant with a "need to unlock" boolean:
func f(arg1 argtype1, arg2 argtype2) ret returntype {
var needToUnlock bool
defer func() {
if needToUnlock {
lock.Unlock()
}
}()
// arbitrary amount of code here that runs unlocked
lock.Lock()
needToUnlock = true
// arbitrary amount of code here that runs locked
lock.Unlock()
needToUnlock = false
// arbitrary amount of code here that runs unlocked
// repeat as desired
}
You can wrap such a thing up in a type:
type DeferableLock struct {
L Locker
isLocked bool
}
func (d *DeferableLock) Lock() {
d.L.Lock()
d.isLocked = true
}
func (d *DeferableLock) Unlock() {
d.L.Unlock()
d.isLocked = false
}
func (d *DeferableLock) EnsureUnlocked() {
if d.isLocked {
d.Unlock()
}
}
func NewDeferableLock(l Locker) *DeferableLock() {
return &DeferableLock{L: l}
}
You can now wrap any sync.Locker with a DeferableLock. At functions like f, use the deferable wrapper to wrap the lock, and call defer d.EnsureUnlock.
(Any resemblance to sync.Cond is entirely deliberate.)

How to use builder pattern to construct a struct which dynamically implements interfaces

I'm trying to use builder patterns (borrowed from Java) to allow structs to implement interfaces. For example, I would ideally like this code pattern:
package main
import "fmt"
type Oner interface {
One() int
}
type Twoer interface {
Two() int
}
func main() {
s := NewObject().
WithOne(1).
Build()
_, ok := s.(Oner)
fmt.Println(ok) // Prints true
_, ok = s.(Twoer)
fmt.Println(ok) // Prints false
t := NewObject().
WithOne(1).
WithTwo(2).
Build()
_, ok = t.(Oner)
fmt.Println(ok) // Prints true
_, ok = t.(Twoer)
fmt.Println(ok) // Prints true
}
As you could see, the definition of the builder determines what interfaces s and t implement.
How would one write the function definition of the builder NewObject() so the Build() method returns a struct which can (possibly) implement a Oner and Twoer?
Edit:
Here's some clarification on how it's going to be used. I'm constructing a library barring certain structs from being passed into functions if they violate the type safety. For example:
type Oner interface {
One() int
}
type OneAndTwoer interface {
Oner
Two() int
}
type Library interface {
DoSomethingWithOner(Oner)
DoSomethingWithOneAndTwoer(Twoer)
}
Though we can define a function which always constructs a OneAndTwoer, my constraints are whenever we construct a OneAndTwoer, this takes a lot longer time than just constructing a Oner
func NewOneAndTwoer() OneAndTwoer {
// Do some really really complicated logic which takes a lot of time
}
func NewOner() Oner {
// Do simple logic
}
You could imagine how if we have a Threer, Fourer, etc, this becomes extremely unwieldly, and we have to construct constructors for all possible permutations of attributes.
This is where builder patterns come in handy. Assuming the calculations for One, Two, etc are independent of each other, we can pick and choose which interface we want to create.
Here is a way to do it, though it feels very clunky.
package main
import (
"fmt"
)
type FieldOner interface {
FieldOne() int
}
type FieldTwoer interface {
FieldTwo() int
}
Set up structs One and Two implementing FieldOner and FieldTwoer respectively.
type One struct {
one int
}
func (f One) FieldOne() int {
return f.one
}
type Two struct {
two int
}
func (f Two) FieldTwo() int {
return f.two
}
Create the FieldBuilder which can store both values and whether it has been given each value, plus WithFieldOne and WithFieldTwo.
type FieldBuilder struct {
one int
has_one bool
two int
has_two bool
}
func NewObject() FieldBuilder {
return FieldBuilder{ has_one: false, has_two: false }
}
func (f FieldBuilder) WithFieldOne(one int) FieldBuilder {
f.one = one
f.has_one = true
return f
}
func (f FieldBuilder) WithFieldTwo(two int) FieldBuilder {
f.two = two
f.has_two = true
return f
}
Build might return One, Two, or a combination of One and Two. Since it can return multiple things which have nothing in common between them (a red flag) it returns an interface{}.
func (f FieldBuilder) Build() interface{} {
switch {
case f.has_one && f.has_two:
return struct {
One
Two
}{
One{one: f.one}, Two{two: f.two},
}
case f.has_one:
return One{ one: f.one }
case f.has_two:
return Two{ two: f.two }
}
panic("Should never be here")
}
Because Build returns an interface{} it's necessary to typecast the result in order to actually use it possibly defeating the whole point of the exercise.
func main() {
s := NewObject().
WithFieldOne(1).
Build()
s1, ok := s.(FieldOner)
fmt.Println(s1.FieldOne())
_, ok = s.(FieldTwoer)
fmt.Println(ok) // Prints false
t := NewObject().
WithFieldOne(1).
WithFieldTwo(2).
Build()
t1, ok := t.(FieldOner)
fmt.Println(t1.FieldOne())
t2, ok := t.(FieldTwoer)
fmt.Println(t2.FieldTwo())
}
This does not scale particularly well. Two interfaces require three cases. Three will require six. Four will require ten. Five will need fifteen...

Resources