compilation error: record replacement of macro in Erlang - compilation

This is the directory structure.
src/
animal.hrl
people.hrl
data_animal.erl
data_people.erl
test.erl
test_macro.erl
animal.hrl
%% The record definition of animal.
-ifndef(ANIMAL).
-define(ANIMAL,true).
-record(animal,{
id,
animal_name,
age
}).
-endif.
people.hrl
%% The record definition of people.
-ifndef(PEOPLE).
-define(PEOPLE,true).
-record(people,{
id,
people_name,
age
}).
-endif.
data_animal.erl
%% The data file of animal.
-module(data_animal).
-include("animal.hrl").
%% API
-export([get/1,get_ids/0]).
get(1)->
#animal{
id=1,
animal_name="cat",
age=23
};
get(2)->
#animal{
id=2,
animal_name="dog",
age=19
};
get(3)->
#animal{
id=3,
animal_name="tiger",
age=23
};
get(4)->
#animal{
id=4,
animal_name="pig",
age=19
};
get(_)->
undefined.
get_ids()->
[1,2,3,4].
data_people.erl
%% The data file of people.
-module(data_people).
-include("people.hrl").
%% API
-export([get/1,get_ids/0]).
get(1)->
#people{
id=1,
people_name="John",
age=23
};
get(2)->
#people{
id=2,
people_name="Ken",
age=19
};
get(3)->
#people{
id=3,
people_name="Tom",
age=23
};
get(4)->
#people{
id=4,
people_name="Healthy",
age=19
};
get(_)->
undefined.
get_ids()->
[1,2,3,4].
Notice that, for data_animal.erl and data_people.erl, the parameter of get/1is the record's id of the return value, and the return value of get_ids/0 is a list of get/1's parameters.
test.erl
-module(test).
%% API
-export([get_animal_list/1,get_people_list/1]).
-include("animal.hrl").
-include("people.hrl").
get_animal_list(Age)->
Fun=fun(Id,Acc)->
case data_animal:get(Id) of
#animal{age=Age}=Conf->
[Conf|Acc];
_->
Acc
end
end,
lists:foldl(Fun,[],data_animal:get_ids()).
get_people_list(Age)->
Fun=fun(Id,Acc)->
case data_people:get(Id) of
#people{age=Age}=Conf->
[Conf|Acc];
_->
Acc
end
end,
lists:foldl(Fun,[],data_people:get_ids()).
I want to get the data of animal and people, whose ages are 23, so I write 2 functions, get_animal_list/1, get_people_list/1.
I run
1> c(data_animal),c(data_people),c(test).
{ok,test}
2> test:get_people_list(23).
[{people,3,"Tom",23},{people,1,"John",23}]
3> test:get_animal_list(23).
[{animal,3,"tiger",23},{animal,1,"cat",23}]
Suddenly, I find that the 2 functions share the same pattern. Then I attempt to write a macro get_list, and make 2 calls instead.
test_macro.erl
-module(test_macro).
%% API
-export([get_animal_list/1,get_people_list/1]).
-include("animal.hrl").
-include("people.hrl").
-define(get_list(DataMod,Record,Age),(
Fun=fun(Id,Acc)->
case DataMod:get(Id) of
#Record{age=Age}=Conf->
[Conf|Acc];
_->
Acc
end
end,
lists:foldl(Fun,[],DataMod:get_ids())
)).
get_animal_list(Age)->
?get_list(data_animal,animal,Age).
get_people_list(Age)->
?get_list(data_people,people,Age).
But I got the compile error:
4> c(test_macro).
test_macro.erl:22: syntax error before: ','
test_macro.erl:25: syntax error before: ','
test_macro.erl:4: function get_animal_list/1 undefined
test_macro.erl:4: function get_people_list/1 undefined
error
Tell me why~y~y~
Thank you all!
I have 3 questions now.
Is my code really not Erlang-like? It's extracted from my company's project. Am I still thinking in OOP? Or so do the programming guys in my company?
Thanks to #mlambrichs 's advice. It works, but I still wonder why my code get the compilation error? Is it because Erlang preprocessor is a one-pass scanner, so it fails to recognize #Record{age=Age}?
According to #mlambrichs 's suggestion, I try to change the macro
-define(get_list(DataMod, Record, Age),
[P || P <- lists:map(fun(Id) -> DataMod:get(Id) end,
DataMod:get_ids()),
P#Record.age =:= Age]
).
into a function
get_list(DataMod, Record, Age)->
[P || P <- lists:map(fun(Id) -> DataMod:get(Id) end,
DataMod:get_ids()),
P#Record.age =:= Age].
Then I get the compilation error:
syntax error before: Record

The cause of the error is a misplaced '(' which should be removed:
-define(get_list(DataMod,Record,Age), (
^^^
Fun=fun(Id,Acc)->
case DataMod:get(Id) of
#Record{age=Age}=Conf->
[Conf|Acc];
_->
Acc
end
end,
lists:foldl(Fun,[],DataMod:get_ids())
).
EDIT
You added some questions which I would like to start answering now.
Is my code really not Erlang-like?
Usage of macros. There's no real need to use macros in your situation.
In general: you would like to hide the fact what kind of records are used in people and animals. That's implementation and should be shielded by your interface. You can just define a getter function that takes care of that in the right module. Pls. read my rewrite suggestions.
It works, but I still wonder why my code get the compilation error? See top of answer.
....I try to change the macro .... You're right, that function doesn't compile. Apparently a rewrite is needed.
Like this:
get_list(DataMod, Age) ->
[ P || P = {_,_,_,A} <- lists:map(fun(Id) -> DataMod:get(Id) end,
DataMod:get_ids()),
A =:= Age].
EDIT
Taking up a rewrite. What you want is a concatenation of two list in function test (yours test/0, mine test/1). Using a comma doesn't do that for you. ;-)
test(X)->
?get_list(data_animal,X) ++
?get_list(data_people,X).
Let's fix that get_list macro as well. Your macro definition get_list has 3 parameters, where it only needs 2. Why use Record as a parameter when you already use that in get_people_list/1 and get_animal_list/1? For example, try this:
-define(get_list(DataMod, Y),
case DataMod of
data_animal -> get_animal_list(Y);
data_people -> get_people_list(Y);
_Else -> []
end
)
Overall, there is a lot of code replication in your test module. As a follow up to #yjcdll's advice, move the interface functions to animal and people to their own modules.
Let's have a look at your data modules and their get functions, as well.
I would suggest putting all people records in an array, in your case in the data_people module.
people() -> [
#people{ id=1, people_name="John", age=23 },
#people{ id=2, people_name="Ken", age=19 },
#people{ id=3, people_name="Tom", age=23 },
#people{ id=4, people_name="Healthy", age=19 } ].
Next, you would need a getter function to get only the people with a certain age:
get(Age) ->
[X || X <- people(), is_age( X, Age )].
And the is_age/2 function would be:
is_age( Person, Age ) ->
Person#people.age =:= Age.
So in module test your get_people_list/1 would get a lot simpler.
get_people_list(Age) ->
data_people:get(Age).
And so on. Always be on the lookout for code that looks pretty much the same like code you've already used somewhere. Just try to behave as a sane, lazy programmer. Lazy = good. ;-)
EDIT: OP has to stick to modules given. So a rewrite of the macro is:
-define(get_list(DataMod, Record, Age),
[P || P <- lists:map(fun(Id) -> DataMod:get(Id) end,
DataMod:get_ids()),
P#Record.age =:= Age]
).

Related

With ruamel.yaml how can I conditionally convert flow maps to block maps based on line length?

I'm working on a ruamel.yaml (v0.17.4) based YAML reformatter (using the RoundTrip variant to preserve comments).
I want to allow a mix of block- and flow-style maps, but in some cases, I want to convert a flow-style map to use block-style.
In particular, if the flow-style map would be longer than the max line length^, I want to convert that to a block-style map instead of wrapping the line somewhere in the middle of the flow-style map.
^ By "max line length" I mean the best_width that I configure by setting something like yaml.width = 120 where yaml is a ruamel.yaml.YAML instance.
What should I extend to achieve this? The emitter is where the line-length gets calculated so wrapping can occur, but I suspect that is too late to convert between block- and flow-style. I'm also concerned about losing comments when I switch the styles. Here are some possible extension points, can you give me a pointer on where I'm most likely to have success with this?
Emitter.expect_flow_mapping() probably too late for converting flow->block
Serializer.serialize_node() probably too late as it consults node.flow_style
RoundTripRepresenter.represent_mapping() maybe? but this has no idea about line length
I could also walk the data before calling yaml.dump(), but this has no idea about line length.
So, where should I and where can I adjust the flow_style whether a flow-style map would trigger line wrapping?
What I think the most accurate approach is when you encounter a flow-style mapping in the dumping process is to first try to emit it to a buffer and then get the length of the buffer and if that combined with the column that you are in, actually emit block-style.
Any attempt to guesstimate the length of the output without actually trying to write that part of a tree is going to be hard, if not impossible to do without doing the actual emit. Among other things the dumping process actually dumps scalars and reads them back to make sure no quoting needs to be forced (e.g. when you dump a string that reads back like a date). It also handles single key-value pairs in a list in a special way ( [1, a: 42, 3] instead of the more verbose [1, {a: 42}, 3]. So a simple calculation of the length of the scalars that are the keys and values and separating comma, colon and spaces is not going to be precise.
A different approach is to dump your data with a large line width and parse the output and make a set of line numbers for which the line is too long according to the width that you actually want to use. After loading that output back you can walk over the data structure recursively, inspect the .lc attribute to determine the line number on which a flow style mapping (or sequence) started and if that line number is in the set you built beforehand change the mapping to block style. If you have nested flow-style collections, you might have to repeat this process.
If you run the following, the initial dumped value for quote will be on one line.
The change_to_block method as presented changes all mappings/sequences that are too long
that are on one line.
import sys
import ruamel.yaml
yaml_str = """\
movie: bladerunner
quote: {[Batty, Roy]: [
I have seen things you people wouldn't believe.,
Attack ships on fire off the shoulder of Orion.,
I watched C-beams glitter in the dark near the Tannhäuser Gate.,
]}
"""
class Blockify:
def __init__(self, width, only_first=False, verbose=0):
self._width = width
self._yaml = None
self._only_first = only_first
self._verbose = verbose
#property
def yaml(self):
if self._yaml is None:
self._yaml = y = ruamel.yaml.YAML(typ=['rt', 'string'])
y.preserve_quotes = True
y.width = 2**16
return self._yaml
def __call__(self, d):
pass_nr = 0
changed = [True]
while changed[0]:
changed[0] = False
try:
s = self.yaml.dumps(d)
except AttributeError:
print("use 'pip install ruamel.yaml.string' to install plugin that gives 'dumps' to string")
sys.exit(1)
if self._verbose > 1:
print(s)
too_long = set()
max_ll = -1
for line_nr, line in enumerate(s.splitlines()):
if len(line) > self._width:
too_long.add(line_nr)
if len(line) > max_ll:
max_ll = len(line)
if self._verbose > 0:
print(f'pass: {pass_nr}, lines: {sorted(too_long)}, longest: {max_ll}')
sys.stdout.flush()
new_d = self.yaml.load(s)
self.change_to_block(new_d, too_long, changed, only_first=self._only_first)
d = new_d
pass_nr += 1
return d, s
#staticmethod
def change_to_block(d, too_long, changed, only_first):
if isinstance(d, dict):
if d.fa.flow_style() and d.lc.line in too_long:
d.fa.set_block_style()
changed[0] = True
return # don't convert nested flow styles, might not be necessary
# don't change keys if any value is changed
for v in d.values():
Blockify.change_to_block(v, too_long, changed, only_first)
if only_first and changed[0]:
return
if changed[0]: # don't change keys if value has changed
return
for k in d:
Blockify.change_to_block(k, too_long, changed, only_first)
if only_first and changed[0]:
return
if isinstance(d, (list, tuple)):
if d.fa.flow_style() and d.lc.line in too_long:
d.fa.set_block_style()
changed[0] = True
return # don't convert nested flow styles, might not be necessary
for elem in d:
Blockify.change_to_block(elem, too_long, changed, only_first)
if only_first and changed[0]:
return
blockify = Blockify(96, verbose=2) # set verbose to 0, to suppress progress output
yaml = ruamel.yaml.YAML(typ=['rt', 'string'])
data = yaml.load(yaml_str)
blockified_data, string_output = blockify(data)
print('-'*32, 'result:', '-'*32)
print(string_output) # string_output has no final newline
which gives:
movie: bladerunner
quote: {[Batty, Roy]: [I have seen things you people wouldn't believe., Attack ships on fire off the shoulder of Orion., I watched C-beams glitter in the dark near the Tannhäuser Gate.]}
pass: 0, lines: [1], longest: 186
movie: bladerunner
quote:
[Batty, Roy]: [I have seen things you people wouldn't believe., Attack ships on fire off the shoulder of Orion., I watched C-beams glitter in the dark near the Tannhäuser Gate.]
pass: 1, lines: [2], longest: 179
movie: bladerunner
quote:
[Batty, Roy]:
- I have seen things you people wouldn't believe.
- Attack ships on fire off the shoulder of Orion.
- I watched C-beams glitter in the dark near the Tannhäuser Gate.
pass: 2, lines: [], longest: 67
-------------------------------- result: --------------------------------
movie: bladerunner
quote:
[Batty, Roy]:
- I have seen things you people wouldn't believe.
- Attack ships on fire off the shoulder of Orion.
- I watched C-beams glitter in the dark near the Tannhäuser Gate.
Please note that when using ruamel.yaml<0.18 the sequence [Batty, Roy] never will be in block style
because the tuple subclass CommentedKeySeq does never get a line number attached.

Rcpp sample sugar function, how to use

I am trying to permute the order of elements in a CharacterVector. In R I would simply use:
sample(charvec)
I am trying the same thing in Rcpp using the sample sugar function, but it keeps throwing 'error: no matching function for call to 'sample(Rcpp::CharacterVector&)'. Other sugar functions I have tried, like intersect or sort_unique work fine with CharacterVector, but sample refuses to work. This is the minimal example I have been experimenting with:
cppFunction('CharacterVector samplefunc() {
CharacterVector v = {"Cat", "Dog", "Fox", "Fish", "Lion"} ;
CharacterVector v2 = sample(v) ;
return v2 ;
}')
What I am doing wrong when trying to use the sample sugar function?
You are just missing the size parameter, which is mandatory for Rcpp::sample:
set.seed(42)
Rcpp::cppFunction('CharacterVector samplefunc() {
CharacterVector v = {"Cat", "Dog", "Fox", "Fish", "Lion"} ;
CharacterVector v2 = sample(v, v.size()) ;
return v2 ;
}')
samplefunc()
#> [1] "Lion" "Fish" "Cat" "Dog" "Fox"
UPDATE (about debugging this kind of errors): Admittedly, the error you see when you do not provide the size argument is kind of obscure (at least with gcc), but you can see:
file1294a34f4734f.cpp: In function ‘Rcpp::CharacterVector samplefunc()’:
file1294a34f4734f.cpp:8:30: error: no matching function for call to ‘sample(Rcpp::CharacterVector&)’
8 | CharacterVector v2 = sample(v) ;
| ~~~~~~^~~
This is the error: no matching function. And then,
In file included from /***/Rcpp/include/Rcpp/sugar/functions/functions.h:89,
from /***/Rcpp/include/Rcpp/sugar/sugar.h:31,
from /***/Rcpp/include/Rcpp.h:78,
from file1294a34f4734f.cpp:1:
/***/Rcpp/include/Rcpp/sugar/functions/sample.h:437:1: note: candidate: ‘template<int RTYPE> Rcpp::Vector<RTYPE, Rcpp::PreserveStorage> Rcpp::sample(const Rcpp::Vector<RTYPE, Rcpp::PreserveStorage>&, int, bool, Rcpp::sugar::probs_t)’
437 | sample(const Vector<RTYPE>& x, int size, bool replace = false, sugar::probs_t probs = R_NilValue)
| ^~~~~~
where gcc is showing you a candidate, and you can see that this function accepts a constant Vector of any RTYPE (numeric, character...), and then it needs a size argument, because there is no default. The others (replace, probs) do have a default. R functions may have missing arguments, C++ functions cannot.

Xtext - Validating for duplicate names

I have the following grammer, but I want to do some validation on this. I want to make an error if there are duplicate names in the "players" list.
Grammer:
Football:
'Club' name=STRING playerList=PlayerList
footballObjects+=FootballObject
;
FootballObject:
Player | Coach
;
PlayerList:
players+=[Player] ( players+=[Player] )*
;
Player:
'Player' name=ID
;
I tried the following:
#Check
def checkGreetingStartsWithCapital(Football model) {
val names = newHashSet
for (g : model.playersList.players) {
if(!names.add(g.name))
error("duplicate" , g, FOOTBALLPACKAGE.Literals.FOOTBALL__PLAYERS_LIST)
}
}
But this does not work. Any ideas why?
The easiest is to mark the list entry by calling error not on the referenced player but on the playersList itself and call the error method that takes an index as well. e.g.
error("bad", playersList, MyDslPackage.Literals.PLAYERS_LIST__PLAYERS, index)

Lists garbage collection

If I do the following:
List2 = [V || V <- List1, ...]
It seems that the List2 refers to the List1 and erlang:garbage_collect() doesn't clear memory. How is it possible to create a new list without references and discard the old?
In any language with garbage collection you simply need to 'lose' all references to a piece of data before it can be garbage collected. Simply returning from the function that generates the original list, while not storing it in any other 'persistent' location (e.g. the process dictionary), should allow the memory to be reclaimed.
The VM is supposed to manage the garbage collecting. If you use a gen_server, or if you use a "home made" server_loop(State), you should have always the same pattern:
server_loop(State) ->
A = somefunc(State),
B = receive
mesg1 -> func1(...);
...
after Timeout ->
func2(...)
end,
NewState = func3(...),
server_loop(NewState).
As long as a process is alive, executing this loop, the VM will allocate and manage memory areas to store all needed information (variables, message queue...+ some margin) As far as I know, there is some spare memory allocated to the process, and if the VM does not try to recover the memory very fast after it has been released, but if you force a garbage collecting, using erlang:garbage_collect(Pid) you can verify that the memory is free - see example bellow.
startloop() -> spawn(?MODULE,loop,[{lists:seq(1,1000),infinity}]).
loop(endloop) -> ok;
loop({S,T}) ->
NewState = receive
biglist -> {lists:seq(1,5000000),T};
{timeout,V} -> {S,V};
sizelist -> io:format("Size of the list = ~p~n",[length(S)]),
{S,T};
endloop -> endloop
after T ->
L = length(S) div 2,
{lists:seq(1,L),T}
end,
loop(NewState).
%% Here, NewState is a copy of State or a totally new data, depending on the
%% received message. In general, for performance consideration it can be
%% interesting to take care of the function used to avoid big copies,
%% and allow the compiler optimize the beam code
%% [H|Q] rather than Q ++ [H] to add a term to a list for example
and the results in the VM:
2> P = lattice:startloop().
<0.57.0>
...
6> application:start(sasl).
....
ok
7> application:start(os_mon).
...
ok
...
11> P ! biglist.
biglist
...
% get_memory_data() -> {Total,Allocated,Worst}.
14> memsup:get_memory_data().
{8109199360,5346488320,{<0.57.0>,80244336}}
...
23> P ! {timeout,1000}.
{timeout,1000}
24> memsup:get_memory_data().
{8109199360,5367361536,{<0.57.0>,80244336}}
the worst case is the loop process: {<0.57.0>,80244336}
...
28> P ! sizelist.
Size of the list = 0
sizelist
...
31> P ! {timeout,infinity}.
{timeout,infinity}
32> P ! biglist.
biglist
33> P ! sizelist.
Size of the list = 5000000
sizelist
...
36> P ! {timeout,1000}.
{timeout,1000}
37> memsup:get_memory_data().
{8109199360,5314289664,{<0.57.0>,10770968}}
%% note the garbage collecting in the previous line: {<0.57.0>,10770968}
38> P ! sizelist.
sizelist
Size of the list = 156250
39> memsup:get_memory_data().
{8109199360,5314289664,{<0.57.0>,10770968}}
...
46> P ! sizelist.
Size of the list = 0
sizelist
47> memsup:get_memory_data().
{8109199360,5281882112,{<0.57.0>,10770968}}
...
50> erlang:garbage_collect(P).
true
51> memsup:get_memory_data().
{8109199360,5298778112,{<0.51.0>,688728}}
%% after GC, the process <0.57.0> is no more the worst case
If you create new list like this, the new list will have elements from the first one, some elements will be shared between both the lists. And if you throw the first list away, shared elements will still be reachable from the new list and won't count as garbage.
How do you check if the first list is garbage collected? Do you test this in erlang console? The console stores results of evaluation each expression that may be the cause you don't see the list garbage collected.

Simple debugging in Haskell

I am new to Haskell. Previously I have programmed in Python and Java. When I am debugging some code I have a habit of littering it with print statements in the middle of code. However doing so in Haskell will change semantics, and I will have to change my function signatures to those with IO stuff. How do Haskellers deal with this? I might be missing something obvious. Please enlighten.
Other answers link the official doco and the Haskell wiki but if you've made it to this answer let's assume you bounced off those for whatever reason. The wikibook also has an example using Fibonacci which I found more accessible. This is a deliberately basic example which might hopefully help.
Let's say we start with this very simple function, which for important business reasons, adds "bob" to a string, then reverses it.
bobreverse x = reverse ("bob" ++ x)
Output in GHCI:
> bobreverse "jill"
"llijbob"
We don't see how this could possibly be going wrong, but something near it is, so we add debug.
import Debug.Trace
bobreverse x = trace ("DEBUG: bobreverse" ++ show x) (reverse ("bob" ++ x))
Output:
> bobreverse "jill"
"DEBUG: bobreverse "jill"
llijbob"
We are using show just to ensure x is converted to a string correctly before output. We also added some parenthesis to make sure the arguments were grouped correctly.
In summary, the trace function is a decorator which prints the first argument and returns the second. It looks like a pure function, so you don't need to bring IO or other signatures into the functions to use it. It does this by cheating, which is explained further in the linked documentation above, if you are curious.
Read this. You can use Debug.Trace.trace in place of print statements.
I was able to create a dual personality IO / ST monad typeclass, which will print debug statements when a monadic computation is typed as IO, them when it's typed as ST. Demonstration and code here: Haskell -- dual personality IO / ST monad? .
Of course Debug.Trace is more of a swiss army knife, especially when wrapped with a useful special case,
trace2 :: Show a => [Char] -> a -> a
trace2 name x = trace (name ++ ": " ++ show x) x
which can be used like (trace2 "first arg" 3) + 4
edit
You can make this even fancier if you want source locations
{-# LANGUAGE TemplateHaskell #-}
import Language.Haskell.TH
import Language.Haskell.TH.Syntax as TH
import Debug.Trace
withLocation :: Q Exp -> Q Exp
withLocation f = do
let error = locationString =<< location
appE f error
where
locationString :: Loc -> Q Exp
locationString loc = do
litE $ stringL $ formatLoc loc
formatLoc :: Loc -> String
formatLoc loc = let file = loc_filename loc
(line, col) = loc_start loc
in concat [file, ":", show line, ":", show col]
trace3' (loc :: String) msg x =
trace2 ('[' : loc ++ "] " ++ msg) x
trace3 = withLocation [| trace3' |]
then, in a separate file [from the definition above], you can write
{-# LANGUAGE TemplateHaskell #-}
tr3 x = $trace3 "hello" x
and test it out
> tr3 4
[MyFile.hs:2:9] hello: 4
You can use Debug.Trace for that.
I really liked Dons short blog about it:
https://donsbot.wordpress.com/2007/11/14/no-more-exceptions-debugging-haskell-code-with-ghci/
In short: use ghci, example with a program with code called HsColour.hs
$ ghci HsColour.hs
*Main> :set -fbreak-on-exception
*Main> :set args "source.hs"
Now run your program with tracing on, and GHCi will stop your program at the call to error:
*Main> :trace main
Stopped at (exception thrown)
Ok, good. We had an exception… Let’s just back up a bit and see where we are. Watch now as we travel backwards in time through our program, using the (bizarre, I know) “:back” command:
[(exception thrown)] *Main> :back
Logged breakpoint at Language/Haskell/HsColour/Classify.hs:(19,0)-(31,46)
_result :: [String]
This tells us that immediately before hitting error, we were in the file Language/Haskell/HsColour/Classify.hs, at line 19. We’re in pretty good shape now. Let’s see where exactly:
[-1: Language/Haskell/HsColour/Classify.hs:(19,0)-(31,46)] *Main> :list
18 chunk :: String -> [String]
vv
19 chunk [] = head []
20 chunk ('\r':s) = chunk s -- get rid of DOS newline stuff
21 chunk ('\n':s) = "\n": chunk s
^^

Resources