Is it possible to use the atomic directive for a single for loop
For example:
#pragma omp atomic
for (i=0;i<5;i++)
A[i]++;
No, this is not possible.
Technically speaking, the atomic construct applies to an access to a specific storage location. The following statements are valid expressions for omp atomic (for C/C++) as per 2.13.6 of the standard:
If atomic-clause is read:
v = x;
If atomic-clause is write:
x = expr;
If atomic-clause is update or not present:
x++; (and -- as well as prefix forms)
x binop= expr;
x = x binop expr;
x = expr binop x;
If atomic-clause is capture:
v = x++; (and -- as well as prefix forms)
v = x binop= expr;
v = x = x binop= expr;
v = x = expr binop= x;
In this case, some specific structured blocks are allowed:
{v = x; x++;}
{x++; v = x;}
And all combinations of the update forms with v = x
The standard further specifically limits x and v to scalar types, so vector / array types are not allowed, even if you were to implement an operator=.
If you want to make this update to appear atomically, you must protect the entire update and all accesses to the A with a critical region, a lock, or ensure otherwise that no concurrent access is performed and memory visibility is guaranteed. I cannot give a more specific recommendation without more specific code.
Related
I'm just starting to learn Rust and I'm coming from an OCaml background.
One thing that I often use is nested let expressions, in order to group related code together.
let x =
let y = 42 in
y + 1
in
...
My naive translation of this to rust would be:
let x =
let y = 42;
y + 1
;
However this fails to compile.
Is this type of syntax possible, or is it heavily frowned upon and thus is not supported?
Rust uses curly braces to delimit scopes. This works fine:
let x = {
let y = 42;
y + 1
};
In Ocaml you can introduce new type variables inside a constraint, which is useful to enforce type-identities in the type-checker:
let f g n = (g (n:'n):'n) ;;
val f : ('n -> 'n) -> 'n -> 'n = <fun>
It is obviously possible to re-use these type variables (otherwise it would be a rather pointless exercise). However, since they are not introduced by some special statement, I wonder what there scope is? Is it the enclosing function, let-binding or top-level statement?
Is there a way to limit the scope of such an implicitly introduced type-variable?
A scope of any type variable used in a type constraint is the body of the enclosing let-expression. If an expression is mutually recursive, then the scope is extended to the whole set of mutual recursive expressions. The scope cannot be reduced. Let-expression is a typing primitive. It is not possible to hide or override a type variable.
Whenever a new type variable is introduced, it is looked up in a current typing context. If it was already introduced, then it is unified. Otherwise a new type variable is added to the context. (That can be later used for unification).
An example to clarify the idea:
let rec f g h x y = g (x : 'a) + h (y : 'a) and e (x : 'a) = x + 1;;
Here, 'a used to constraint x in e is the same 'a that was used to contraint x and y in the body of function f. Since, x in e is unified with int type the unification extends to function f, constraining function g and h to type int -> int.
In some programming languages, I see (ex.):
x := y
What is this := operator generally called and what does it do?
In all languages that support an operator := it means assignment.
In languages that support an operator :=, the = operator usually means an equality comparison.
In languages where = means assignment, == is typically used for equality comparison.
does := mean =?
I can't recall any languages where := means the same as =.
In MySQL := and = are both used for assignment, however they are not interchangeable and selecting the correct one depends on the context. To make matters more confusing the = operator is also used for comparison. The interpretation of = as either assignment or comparison also depends on context.
This is a new operator that is coming to Python 3.8 and actually had a role in BDFL Guido van Rossum's early retirement.
Formally, the operator allows what's called an "assignment expression". Informally, the operator is referred to as the "walrus operator".
It allows assignment while also evaluating an expression.
So this:
env_base = os.environ.get("PYTHONUSERBASE", None)
if env_base:
return env_base
can be shortened to:
if env_base := os.environ.get("PYTHONUSERBASE", None):
return env_base
https://www.python.org/dev/peps/pep-0572/#examples-from-the-python-standard-library
I usually see it more in pseudocode where it means an assignment. Thus x := y means 'set the value of x to the value of y' whereas x = y means 'does the value of x equal the value of y?'
The symbol is called "becomes" and was introduced with IAL (later called Algol 58) and Algol 60. It is the symbol for assigning a value to a variable. One reads x := y; as "x becomes y".
Using ":=" rather than "=" for assignment is mathematical fastidiousness; to such a viewpoint, "x = x + 1" is nonsensical. Other contemporary languages might have used a left arrow for assignment, but that was not common (as a single character) in many character sets.
Algol 68 further distinguished identification and assignment; INT the answer = 42; says that "the answer" is declared identically equal to 42 (i.e., is a constant value). In INT the answer := 42; "the answer" is declared as a variable and is initially assigned the value 42.
There are other assigning symbols, like +:=, pronounced plus-and-becomes; x +:= y adds y to the current value of x, storing the result in x.
(Spaces have no significance, so can be inserted "into" identifiers rather than having to mess with underscores)
A lot of languages use common operators. Generally the = is reserved for variable assignment and should not be viewed in a mathematical context if it is alone. Equality in some languages like Java and Bash is tested though ==
PL/I has (had?) both = and :=. = is used for both assignment and comparison -- the compiler tries to figure out which you meant based on context. When/if it decides to do comparison where you really meant assignment, you can use := to force assignment.
For example, consider x=y=0; In C (for one example) this would mean "assign 0 to y, then the result of that (also 0) to x."
In PL/I, it means compare y to 0, and then assign the Boolean result of that comparison to x (i.e., equivalent to x = y == 0; in something like C). If you (being sane, unlike the designers of PL/I) intended that to mean "assign 0 to x and y", you'd use x = y := 0; (or x := y := 0;).
:= it means "set equal to"
An assignment with syntax
v := expr
sets the value of variable «v» to the value obtained from expression «expr».
Example:
X := B Sets the Definition of X to the value of B
There is a basic monad question in here, unrelated to Repa, plus several Repa-specific questions.
I am working on a library using Repa3. I am having trouble getting efficient parallel code. If I make my functions return delayed arrays, I get excruciatingly slow code that scales very well up to 8 cores. This code takes over 20GB of memory per the GHC profiler, and runs several orders of magnitude slower than the basic Haskell unboxed vectors.
Alternatively, if I make all of my functions return Unboxed manifest arrays (still attempting to use fusion within the functions, for example when I do a 'map'), I get MUCH faster code (still slower than using Haskell unboxed vectors) that doesn't scale at all, and in fact tends to get slightly slower with more cores.
Based on the FFT example code in Repa-Algorithms, it seems the correct approach is to always return manifest arrays. Is there ever a case where I should be returning delayed arrays?
The FFT code also makes plentiful use of the 'now' function. However, I get a type error when I try to use it in my code:
type Arr t r = Array t DIM1 r
data CycRingRepa m r = CRTBasis (Arr U r)
| PowBasis (Arr U r)
fromArray :: forall m r t. (BaseRing m r, Unbox r, Repr t r) => Arr t r -> CycRingRepa m r
fromArray =
let mval = reflectNum (Proxy::Proxy m)
in \x ->
let sh:.n = extent x
in assert (mval == 2*n) PowBasis $ now $ computeUnboxedP $ bitrev x
The code compiles fine without the 'now'. With the 'now', I get the following error:
Couldn't match type r' withArray U (Z :. Int) r'
`r' is a rigid type variable bound by
the type signature for
fromArray :: (BaseRing m r, Unbox r, Repr t r) =>
Arr t r -> CycRingRepa m r
at C:\Users\crockeea\Documents\Code\LatticeLib\CycRingRepa.hs:50:1
Expected type: CycRingRepa m r
Actual type: CycRingRepa m (Array U DIM1 r)
I don't think this is my problem. It would be helpful if someone could explain the how the Monad works in 'now'. By my best estimation, the monad seems to be creating a 'Arr U (Arr U r)'. I'm expecting a 'Arr U r', which would then match the data constructor pattern. What is going on and how do I fix this?
The type signatures are:
computeUnboxedP :: Fill r1 U sh e => Array r1 sh e -> Array U sh e
now :: (Shape sh, Repr r e, Monad m) => Array r sh e -> m (Array r sh e)
It would be helpful to have a better idea of when it is appropriate to use 'now'.
A couple other Repa questions:
Should I explicitly call computeUnboxedP (as in the FFT example code), or should I use the more general computeP (because the unbox part is inferred by my data type)?
Should I store delayed or manifest arrays in the data type CycRingRepa?
Eventually I would also like this code to work with Haskell Integers. Will this require me to write new code that uses something other than U arrays, or could I write polymorphic code that creates U arrays for unbox types and some other array for Integers/boxed types?
I realize there are a lot of questions in here, and I appreciate any/all answers!
Here's the source code for now:
now arr = do
arr `deepSeqArray` return ()
return arr
So it's really just a monadic version of deepSeqArray. You can use either of these to force evaluation, rather than hanging on to a thunk. This "evalulation" is different than the "computation" forced when computeP is called.
In your code, now doesn't apply, since you're not in a monad. But in this context deepSeqArray wouldn't help either. Consider this situation:
x :: Array U Int Double
x = ...
y :: Array U Int Double
y = computeUnboxedP $ map f x
Since y refers to x, we'd like to be sure x is computed before starting to compute y. If not, the available work won't be distributed correctly among the gang of threads. To get this to work out, it's better to write y as
y = deepSeqArray x . computeUnboxedP $ map f x
Now, for a delayed array, we have
deepSeqArray (ADelayed sh f) y = sh `deepSeq` f `seq` y
Rather than computing all the elements, this just makes sure the shape is computed, and reduces f to weak-head normal form.
As for manifest vs delayed arrays, there are certainly time delayed arrays are preferable.
multiplyMM arr brr
= [arr, brr] `deepSeqArrays`
A.sumP (A.zipWith (*) arrRepl brrRepl)
where trr = computeUnboxedP $ transpose2D brr
arrRepl = trr `deepSeqArray` A.extend (Z :. All :. colsB :. All) arr
brrRepl = trr `deepSeqArray` A.extend (Z :. rowsA :. All :. All) trr
(Z :. _ :. rowsA) = extent arr
(Z :. colsB :. _ ) = extent brr
Here "extend" generates a new array by copying the values across some set of new dimensions. In particular, this means that
arrRepl ! (Z :. i :. j :. k) == arrRepl ! (Z :. i :. j' :. k)
Thankfully, extend produces a delayed array, since it would be a waste to go through the trouble of all this copying.
Delayed arrays also allow the possiblity of fusion, which is impossible if the array is manifest.
Finally, computeUnboxedP is just computeP with a specialized type. Giving computeUnboxedP explicitly might allow GHC to optimize better, and makes the code a little clearer.
Repa 3.1 no longer requires the explict use of now. The parallel computation functions are all monadic, and automatically apply deepSeqArray to their results. The repa-examples package also contains a new implementation of matrix multiply that demonstrates their use.
I have a recursive immutable data structure in ocaml which can be simplified to something like this:
type expr =
{
eexpr : expr_expr;
some_other_complex_field : a_complex_type;
}
and expr_expr =
| TInt of int
| TSum of (expr * expr)
| TMul of (expr * expr)
It's an AST, and sometimes it gets pretty complex (it's very deep).
there is a recursive function that evaluates an expression. For example, let's say,
let rec result expr =
match expr.eexpr with
| TInt i -> i
| TSum (e1, e2) -> result e1 + result e2
| TMul (e1, e2) -> result e1 * result e2
Now suppose I am mapping an expression to another expression, and I need to constantly check the result of an expr, sometimes more than once for the same expr, and sometimes for expressions that were recently mapped by using the pattern
{ someExpr with eexpr = TSum(someExpr, otherExpr) }
Now, the result function is very lightweight, but running it many times for a deep AST will not be very optimized. I know I could cache the value using a Hashtbl, but AFAIK the Hashtbl will only do structural equality, so it will need to traverse my long AST anyway.
I know the best option would be to include a probably immutable "result" field in the expr type. But I can't.
So is there any way in Ocaml to cache a value to an immutable type, so I don't have to calculate it eagerly every time I need it ?
Thanks!
Hash-cons the values of expr_expr. By doing this structurally equal values in your program will share exactly the same memory representation and you can substitute structural equality (=) by physical equality (==).
This paper should get you quickly started on hash-consing in OCaml.
You can use the functorial interface to control the kind of equality used by the hash table. I believe the semantics of (==) are legitimate for your purposes; i.e., if A == B then f A = f B for any pure function f. So you can cache the results of f A. Then if you find a B that's physically equal to A, the cached value is correct for B.
The downside of using (==) for hashing is that the hash function will send all structurally equal objects to the same hash bucket, where they will be treated as distinct objects. If you have a lot of structurally equal objects in the table, you get no benefit from the hashing. The behavior degenerates to a linear search.
You can't define the hash function to work with physical addresses, because the physical addresses can be changed at any time by the garbage collector.
However, if you know your table will only contain relatively few large-ish values, using physical equality might work for you.
I think you can merge the two ideas above : use hash-consing-like techniques to get the hash of the "pure expression" part of your data, and use this hash as key in the memoization table for the eval function.
Of course this only works when your eval function indeed only depends on the "pure expression" part of the function, as in the example you gave. I believe that is a relatively general case, at least if you restrict yourself to storing the successful evaluations (that won't, for example, return an error including some location information).
Edit: a small proof of concept:
type 'a _expr =
| Int of int
| Add of 'a * 'a
(* a constructor to avoid needing -rectypes *)
type pure_expr = Pure of pure_expr _expr
type loc = int
type loc_expr = {
loc : loc;
expr : loc_expr _expr;
pure : pure_expr (* or any hash_consing of it for efficiency *)
}
(* this is where you could hash-cons *)
let pure x = Pure x
let int loc n =
{ loc; expr = Int n; pure = pure (Int n) }
let add loc a b =
{ loc; expr = Add (a, b); pure = pure (Add(a.pure, b.pure)) }
let eval =
let cache = Hashtbl.create 251 in
let rec eval term =
(* for debug and checking memoization *)
Printf.printf "log: %d\n" term.loc;
try Hashtbl.find cache term.pure with Not_found ->
let result =
match term.expr with
| Int n -> n
| Add(a, b) -> eval a + eval b in
Hashtbl.add cache term.pure result;
result
in eval
let test = add 3 (int 1 1) (int 2 2)
# eval test;;
log: 3
log: 2
log: 1
- : int = 3
# eval test;;
log: 3
- : int = 3