I am currently working on optimizing the rust jpeg decoder crate using SIMD. In order to avoid long repetitions in the code, I would like to write a macro that generates the following matrix transposition code :
s = [
i32x8::new(s[0].extract(0),s[1].extract(0),s[2].extract(0),s[3].extract(0),s[4].extract(0),s[5].extract(0),s[6].extract(0),s[7].extract(0), ),
i32x8::new(s[0].extract(1),s[1].extract(1),s[2].extract(1),s[3].extract(1),s[4].extract(1),s[5].extract(1),s[6].extract(1),s[7].extract(1), ),
i32x8::new(s[0].extract(2),s[1].extract(2),s[2].extract(2),s[3].extract(2),s[4].extract(2),s[5].extract(2),s[6].extract(2),s[7].extract(2), ),
i32x8::new(s[0].extract(3),s[1].extract(3),s[2].extract(3),s[3].extract(3),s[4].extract(3),s[5].extract(3),s[6].extract(3),s[7].extract(3), ),
i32x8::new(s[0].extract(4),s[1].extract(4),s[2].extract(4),s[3].extract(4),s[4].extract(4),s[5].extract(4),s[6].extract(4),s[7].extract(4), ),
i32x8::new(s[0].extract(5),s[1].extract(5),s[2].extract(5),s[3].extract(5),s[4].extract(5),s[5].extract(5),s[6].extract(5),s[7].extract(5), ),
i32x8::new(s[0].extract(6),s[1].extract(6),s[2].extract(6),s[3].extract(6),s[4].extract(6),s[5].extract(6),s[6].extract(6),s[7].extract(6), ),
i32x8::new(s[0].extract(7),s[1].extract(7),s[2].extract(7),s[3].extract(7),s[4].extract(7),s[5].extract(7),s[6].extract(7),s[7].extract(7), ),
];
The macro should be able to generate the code for different matrix sizes (4 or 8).
I have tried several different approaches, but I never manage to get the macro to repeat n times an n-item pattern.
The most logical to me would be:
macro_rules! square {
(($($x:tt),*), ($($y:tt),*)) => {
[
$([
$( s[$x].extract($y) ),*
]),*
]
};
($($x:expr),*) => { square!( ($($x),*) , ($($x),*) ) };
}
but it fails with
error: attempted to repeat an expression containing no syntax variables matched as repeating at this depth
You can do it, but you will need to handle the outer repetition through recursion:
macro_rules! square {
(#row [$($acc:expr),*] [$($before:expr),*] $current:expr $(, $after:expr)*) => {
square!(#row
[ $($acc,)*
stringify!($(s[$current].extract ($before),)*
s[$current].extract ($current)
$(, s[$current].extract ($after))*) ]
[ $($before,)* $current ]
$($after),*)
};
(#row [$($acc:tt)*] [$($before:expr),*]) => { vec![ $($acc)* ] };
($($r:expr),*) => {
square!(#row [] [] $($r),*)
};
}
Playground
I've called stringify! so that the code would compile on the playground without the dependencies. You will need to replace them to fit your needs (probably just remove the stringify! invocation and replace s by an identifier you pass to the macro).
The idea of accumulating the values in $acc and outputting them all at once at the end of the recursion is called the push-down accumulation pattern. If you are not familiar with this concept, it is described in detail in the little book of Rust macros.
Related
I have a combo ReScript and TypeScript project. I want to use gentype to expose TypeScript friendly objects. Gentype works for string, bool, records, and other basic objects without any hassles. How can I work with list and char and other ReScript-specific types from TypeScript?
There is a shims feature but I don't know how to do that. I would assume there are built-in shims for the standard Belt library but can't find them.
I plan to do most of the calculations and heavy-lifting inside Rescript. So when the info is sent to TypeScript, I'll probably just be consuming the data, not modifying it. This might be iterating through a list to display the contents in a non-Rescript React project. I don't plan to manipulate the data.
One option is to convert lists to arrays inside ReScript before exporting the results. So any time I want to consume information from TypeScript, I'll create a function and/or type in ReScript that only has bool, number, string, array, and object. I could create an "interop" module with all the stuff I want to consume or use from TypeScript. This provides a clean separation between the ReScript and TypeScript world and is easy to get my head around but seems a little inefficient (like converting all lists to arrays) and extra work.
On the other hand, if using the Belt objects like list and map is cumbersome from TypeScript, even with the shims, then I'm probably better off creating my "interop" module anyway.
What is the recommended/simplest/best way to use list, map, char, and other ReScript specific objects from within TypeScript? Are there friendly Typescript definitions for the Belt standard library that I could use even if I wasn't using ReScript?
===
One additional note. I have experience trying to use F# (functional) from C#. It was painful. The best solution for me in that environment was to create an interface on the F# side that was easily consumable on the C# side that did not use the F# native objects.
As rescript compiles to JavaScript, and the output is very readable and (usually) straight-forward, you could just emulate what it generates.
For example, this rescript code:
let xs = list{1, 2, 3}
let _ = List.map(x => x + 1, xs)
is compiled into this (slightly simplified) JavaScript:
var List = require("./stdlib/list.js");
var xs = {
hd: 1,
tl: {
hd: 2,
tl: {
hd: 3,
tl: /* [] */0
}
}
};
List.map((x) => x + 1, xs);
There is a slight problem with the literal syntax of lists specifically, but that could be simplified a bit by using List.cons instead:
let xs = List.cons(1, List.cons(2, List.cons(3, list{})))
which becomes:
var xs = List.cons(1, List.cons(2, List.cons(3, /* [] */0)));
Pattern matching also isn't as convenient, obviously, but still pretty straight-forward for the simple things at least. For example:
let rec sum = xs => switch xs {
| list{} => 0
| list{x, ...rest} => x + sum(rest)
}
becomes:
function sum(xs) {
if (xs) {
return xs.hd + sum(xs.tl);
} else {
return 0;
}
}
Most other types don't have any special compiler support, and so becomes just plain function calls. Using Belt.Map for example, despite using some advanced language features, compiles to very straight-forward JavaScript:
module IntCmp = Belt.Id.MakeComparable({
type t = int
let cmp = (a, b) => a - b
})
let m = Belt.Map.make(~id=module(IntCmp))
let _ = Belt.Map.set(m, 0, "a")
becomes (more or less):
var Belt_Id = require("./stdlib/belt_Id.js");
var Belt_Map = require("./stdlib/belt_Map.js");
var IntCmp = Belt_Id.MakeComparable({
cmp: (a, b) => a - b
});
var m = Belt_Map.make(IntCmp);
Belt_Map.set(m, 0, "a");
I have seen the # symbol used in macros but I cannot find mention of it in the Rust Book or in any official documentation or blog posts. For example, in this Stack Overflow answer it is used like this:
macro_rules! instructions {
(enum $ename:ident {
$($vname:ident ( $($vty: ty),* )),*
}) => {
enum $ename {
$($vname ( $($vty),* )),*
}
impl $ename {
fn len(&self) -> usize {
match self {
$($ename::$vname(..) => instructions!(#count ($($vty),*))),*
}
}
}
};
(#count ()) => (0);
(#count ($a:ty)) => (1);
(#count ($a:ty, $b:ty)) => (2);
(#count ($a:ty, $b:ty, $c:ty)) => (3);
}
instructions! {
enum Instruction {
None(),
One(u8),
Two(u8, u8),
Three(u8, u8, u8)
}
}
fn main() {
println!("{}", Instruction::None().len());
println!("{}", Instruction::One(1).len());
println!("{}", Instruction::Two(1, 2).len());
println!("{}", Instruction::Three(1, 2, 3).len());
}
From the usage, it appears that it is used for declaring another macro that is local to the main one.
What does this symbol mean and why would you use it rather than just creating another top-level macro?
In the pattern-matching part of a macro, symbols can mean whatever the author desires them to mean. A leading symbol # is often used to denote an "implementation detail" of the macro — a part of the macro that an external user is not expected to use.
In this example, I used it to pattern-match the tuple parameters to get a count of the tuple parameters.
Outside of macros, the # symbol is used to match a pattern while also assigning a name to the entire pattern:
match age {
x # 0 => println!("0: {}", x),
y # 1 => println!("1: {}", y),
z => println!("{}", z),
}
With a bit of a stretch, this same logic can be applied to the use in the macro — we are pattern-matching the tuple, but also attaching a name to that specific pattern. I think I've even seen people use something even more parallel: (count # .... However, The Little Book of Rust Macros points out:
The reason for using # is that, as of Rust 1.2, the # token is not used in prefix position; as such, it cannot conflict with anything. Other symbols or unique prefixes may be used as desired, but use of # has started to become widespread, so using it may aid readers in understanding your code.
rather than just creating another top-level macro
Creating another macro is likely better practice, but only in modern Rust. Before recent changes to Rust that made it so you could import macros directly, having multiple macros could be tricky for end users who tried to selectively import macros.
See also:
Internal rules in The Little Book of Rust Macros
Why must I use macros only used by my dependencies
What does the '#' symbol do in Rust?
Appendix B: Operators and Symbols
I'm trying to dynamically create a list of values from an AJAX request (using Catalyst), like this:
my #listofvalues = #{$params->{parameter_with_many_values}};
Then I loop through the list to make database insertions (one for each value). Since I need to loop through various lists like the one above, I need to access the index of the list. I am currently doing it like this:
foreach my $key (0 .. $#listofvalues){
$table_model->new({
field1 => $listofvalues[$key],
field2 => $x,
field3 => $another_listofvalues[$key]
field4 => $yet_another_listofvalues[$key]
});
}
This seems to work fine when two or more elements are received in the request. Whenever a single element is received, I get an error like
[error] Caught exception in pbitdb::Controller::Subjects->add "Can't use string ("1") as an ARRAY ref while "strict refs" in use at /home/lioneluranl/svn/pbitdb/pbitdb/script/../lib/pbitdb/Controller/Subjects.pm line 119."
Where, in this case, 1 is the value received and line 119 is the line where #listofvalues is being declared.
Now I've tried several to workaround this issue, but haven't found anything that works both ways (for a single or for various values). Any suggestions?
Yes, single element lists are OK in Perl, as are arrays and references to such arrays.
#!/usr/bin/perl
use warnings;
use strict;
use Data::Dumper;
sub new {
print Dumper \#_;
}
my $table_model = 'main';
for my $values ( [ 'a' .. 'c' ],
[ 'd' ]
) {
my $params = { parameter_with_many_values => $values };
my #listofvalues = #{ $params->{parameter_with_many_values} };
my #another_listofvalues = map uc, #listofvalues;
for my $key (0 .. $#listofvalues) {
my $x = rand;
$table_model->new({
field1 => $listofvalues[$key],
field2 => $x,
field3 => $another_listofvalues[$key]
});
}
}
How do you populate $params->{parameter_with_many_values}?
Update
It seems Catalyst::Request should mention that their "safe" parameters should be handled as follows:
#!/usr/bin/perl
use warnings;
use strict;
use Data::Dumper;
for my $params ( { param_with_many_values => 'a' },
{ param_with_many_values => [ 'a' .. 'e' ] },
{ something => 'else' }
) {
my $value_or_values = $params->{param_with_many_values};
my #list_of_values = ref $value_or_values ? #$value_or_values
: defined $value_or_values ? $value_or_values
: ();
print Dumper \#list_of_values;
}
First of all, you are asking about arrays (a type of variable), not lists (a vague term which can take on numerous definitions, none of which are pertinent here).
Yes, you can have an array with only one element.
$ perl -e'my #a = "ele"; CORE::say 0+#a; CORE::say for #a;'
1
ele
That's not the problem. The problem is that
#{$params->{parameter_with_many_values}}
expects
$params->{parameter_with_many_values}
to contain a reference to an array, but it contains 1 instead. It was probably set using
$params->{parameter_with_many_values} = #a; # Assigns number of elements
instead of
$params->{parameter_with_many_values} = \#a;
my %hash =
two => 2,
three => 3,
one => 1,
;
for %hash.sort(*.key)>>.kv -> ($key, $value) {
say "'$key' => '$value'";
}
Is %hash.sort({.key})>>.kv equivalent to above sort?
Why this sort doesn't work without hyper >> hint?
The sort method is returning a List of Pairs.
Since calling .kv on the list returns a list of index, Pair lists, which you don't want; you can't just call .kv. So you have to pull out the key and value from the Pair objects in the list individually by calling the .kv method on each of them, which >>.kv does.
You could also have used .map(*.kv) instead.
The >>.kv syntax allows an implementation to spread the work over multiple threads if it makes sense to do so.
( Currently Rakudo just does the work in a semi-random order to prevent people from using the feature wrong )
There is an alternate way of writing the loop by extracting the attributes using adverbs in the sub-signature:
for %hash.sort -> (:$key, :$value) {
say "'$key' => '$value'";
}
for %hash.sort -> $pair (:$key, :$value) {
say $pair;
say $key === $pair.key and $value === $pair.value; # True
}
# :$key is short for :key($key)
for %hash.sort -> (:key($k), :value($v)) {
say "'$k' => '$v'";
}
This can be useful on other objects which don't have a method for creating a list of their public attributes
class C { has $.a; has $.b; has $.c; has $!private-value }
my $c = 5;
my $obj = C.new(:a<A>,:b(1),:$c);
given $obj -> ( :$a, :b($b), :$c) ) {
say "$a $b $c";
}
# ignore $.a by using an unnamed scalar
given $obj -> ( :a($), :$b, :$c ) { ... }
# places any unspecified public attributes in %others
given $obj -> ( :$a, :$b, *%others ) {
.say for keys %others; # c
}
# ignores any unspecified attributes
# useful to allow subclasses to add more attributes
# or to just drop any values you don't care about
given $obj -> ( :$a, :$b, *% ) { ... }
# fails because it doesn't handle the public c attribute
# in the sub-signature
given $obj -> ( :$a, :$b ) { ... }
That is only the beginning of what's possible with signatures.
All of the following is also allowed in subroutine and method signatures, optional, and completely overkill for this example.
It is really useful in multi subs and multi methods for restricting the possible candidates.
for 'one' => 1, 1/3
->
# Type is an alias to the object type
::Type Any $_ # Any is the default type requirement
# the public attributes of the object
(
::A-Type Any :key( :numerator( $a ) ),
::B-Type Any :value( :denominator( $b ) ) where $b >= 1,
)
{
my Type $obj = $_; # new variable declared as having the same type
my A-Type $new-a = $a;
my B-Type $new-b = $b;
# could have used $_.^name or .^name instead of Type.^name
# so you don't actually have to add the alias to the signature
# to get the name of the arguments type
say Type.^name, ' ', $_;
say ' ', A-Type.^name, ' ', $a;
say ' ', B-Type.^name, ' ', $b;
}
Pair one => 1
Str one
Int 1
Rat 0.333333
Int 1
Int 3
As to using .sort({.key}), yes that is basically the same thing, as sort accepts anything Callable there.
I would like to point out that you didn't even need to provide an argument to sort because it's default is even smarter than what you gave it.
Perl 6 has many ways of creating and accessing Callable things. So any of the following would have worked:
*.key
{ .key } # { $_.key }
-> $_ { .key } # basically what the previous line turns into
{ $^placeholder-var.key }
sub ($_) { .key }
&a-subroutine-reference # you would have to create the subroutine though
Also since all of the normal operators are actually subroutines, you could use them in other places where you need a Callable. ( I can't think of one that works in that spot though )
&infix:<+> # the subroutines responsible for the numeric addition operator
&[+] # ditto
&prefix:<++>
&postfix:<++>
# etc
As far as I can see, the ony difference between the two versions is use of a block with implicit $_ parameter instead of using a Whatever-Star, so they are indeed equivalent.
This is Perl, so There Is More Than One Way to Do It:
*.key
{ .key }
{ $^arg.key }
-> $arg { $arg.key }
Why this sort doesn't work without hyper >> hint?
sort coerces the hash to a list of pairs, and that's what you'll get:
say %hash.sort(*.key).perl;
# ("one" => "1", "three" => "3", "two" => "2")
To get rid of the pairs, you need to iterate over the list and call .kv on each one:
say %hash.sort(*.key)>>.kv.perl;
# (("one", "1"), ("three", "3"), ("two", "2"))
say %hash.sort(*.key).map(*.kv).perl;
# (("one", "1"), ("three", "3"), ("two", "2"))
You could coerce to Hash before calling .kv:
say %hash.sort(*.key).hash.kv.perl;
# ("one", "1", "three", "3", "two", "2")
but this would of course defeat the purpose of the excercise as hash ordering cannot be relied on.
You may have noticed that you'll get different results depending on how exactly you write the code. If there no trailing .list, what you get is actually a Parcel and not a List, but semantics have not been finalized.
Note that even though the returned objects all perlify with simple parentheses, some are parcels and some are lists, which you can check by calling .WHAT. This is still a work in progress.
Also note the inner parentheses in some of these variants, which you can get rid of with a call to .flat. If you do so, you can use -> $key, $value as signature of your for loop instead of -> ($key, $value) (or, more explicitly, -> $anon ($key, $value)) which uses signature binding to unpack the parcels.
Instead of using .kv, you could use the same approach to unpack the pair objects instead:
for %hash.sort(*.key) -> (:$key, :$value) {
say "'$key' => '$value'";
}
[It seems my explanations and expectations are not clear at all, so I added precision on how I'd like to use the feature at the end of the post]
I'm currently working on grammars using boost qi. I had a loop construction for a rule cause I needed to build it from the elements of a vector. I have re-written it with simple types, and it looks like:
#include <string>
// using boost 1.43.0
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/qi_eps.hpp>
#include <boost/spirit/include/phoenix.hpp>
namespace bqi = boost::spirit::qi;
typedef const char* Iterator;
// function that you can find [here][1]
template<typename P> void test_phrase_parser(char const* input, P const& p, bool full_match = true);
int main()
{
// my working rule type:
bqi::rule<Iterator, std::string()> myLoopBuiltRule;
std::vector<std::string> v;
std::vector<std::string>::const_iterator iv;
v.push_back("abc");
v.push_back("def");
v.push_back("ghi");
v.push_back("jkl");
myLoopBuiltRule = (! bqi::eps);
for(iv = v.begin() ; iv != v.end() ; iv++)
{
myLoopBuiltRule =
myLoopBuiltRule.copy() [ bqi::_val = bqi::_1 ]
| bqi::string(*iv) [ bqi::_val = bqi::_1 ]
;
}
debug(myLoopBuiltRule);
char s[] = " abc ";
test_phrase_parser(s, myLoopBuiltRule);
}
(Looks like here does not want to be replaced by corresponding hyperlink, so here is the address to find function test_phrase_parser(): http://www.boost.org/doc/libs/1_43_0/libs/spirit/doc/html/spirit/qi/reference/basics.html)
All was for the best in the best of all worlds... until I had to pass an argument to this rule. Here is the new rule type:
// my not-anymore-working rule type:
bqi::rule<Iterator, std::string(int*)> myLoopBuiltRule;
'int*' type is for example purpose only, my real pointer is adressing a much more complex class... but still a mere pointer.
I changed my 'for' loop accordingly, i.e.:
for(iv = v.begin() ; iv != v.end() ; iv++)
{
myLoopBuiltRule =
myLoopBuiltRule.copy()(bqi::_r1) [ bqi::_val = bqi::_1 ]
| bqi::string(*iv) [ bqi::_val = bqi::_1 ]
;
}
I had to add a new rule because test_phrase_parser() cannot guess which value is to be given to the int pointer:
bqi::rule<Iterator> myInitialRule;
And change everything that followed the for loop:
myInitialRule = myLoopBuiltRule((int*)NULL);
debug(myLoopBuiltRule);
char s[] = " abc ";
test_phrase_parser(s, myInitialRule);
Then everything crashed:
/home/sylvain.darras/software/repository/software/external/include/boost/boost_1_43_0/boost/spirit/home/qi/nonterminal/rule.hpp:199: error: no matching function for call to ‘assertion_failed(mpl_::failed************ (boost::spirit::qi::rule<Iterator, T1, T2, T3, T4>::operator=(const Expr&)
Then I got crazy and tried:
myLoopBuiltRule =
myLoopBuiltRule.copy(bqi::_r1) [ bqi::_val = bqi::_1 ]
| bqi::string(*iv) [ bqi::_val = bqi::_1 ]
-->
error: no matching function for call to ‘boost::spirit::qi::rule<const char*, std::string(int*), boost::fusion::unused_type, boost::fusion::unused_type, boost::fusion::unused_type>::copy(const boost::phoenix::actor<boost::spirit::attribute<1> >&)’
Then I got mad and wrote:
myLoopBuiltRule =
myLoopBuiltRule(bqi::_r1) [ bqi::_val = bqi::_1 ]
| bqi::string(*iv) [ bqi::_val = bqi::_1 ]
Which compiles since it is perfectly syntactically correct, but which magnificently stack overflows coz it happily, nicely, recursively, calls itself to death...
Then I lost my mind and typed:
myLoopBuiltRule =
jf jhsgf jshdg fjsdgh fjsg jhsdg jhg sjfg jsgh df
Which, as you probably expect, has failed to compile.
You imagine that before writing the above novel, I checked out on the web, but didn't find out anything related to copy() and argument passing in the same time. Has anyone already experienced this problem ? Have I missed something ?
Be assured that any help will be really really appreciated.
PS: Great thanks to hkaiser who has, without knowing it, answered a lot of my boost::qi problems through google (but this one).
Further information:
The purpose of my parser is to read files written in a given language L. The purpose of my post is to propagate my "context" (i.e.: variable definitions and especially constant values, so I can compute expressions).
The number of variable types I handle is small, but it's bound to grow, so I keep these types in a container class. I can loop on these managed types.
So, let's consider a pseudo-algorithm of what I would like to achive:
LTypeList myTypes;
LTypeList::const_iterator iTypes;
bqi::rule<Iterator, LType(LContext*)> myLoopBuiltRule;
myLoopBuiltRule = (! bqi::eps);
for(iTypes = myTypes.begin() ; iTypes != myTypes.end() ; iTypes++)
{
myLoopBuiltRule =
myLoopBuiltRule.copy()(bqi::_r1) [ bqi::_val = bqi::_1 ]
| iTypes->getRule()(bqi::_r1) [ bqi::_val = bqi::_1 ]
}
This is done during initialization and then myLoopBuiltRule is used and reused with different LContext*, parsing multiple types. And since some L types can have bounds, which are integer expressions, and that these integer expressions can exhibit constants, I (think that I) need my inherited attribute to take my LContext around and be able to compute expression value.
Hope I've been clearer in my intentions.
Note I just extended my answer with a few more informational links. In this particular case I have a hunch that you could just get away with the Nabialek trick and replacing the inherited attribute with a corresponding qi::locals<> instead. If I have enough time, I might work out a demonstration later.
Caveats, expositioning the problem
Please be advised that there are issues when copying proto expression trees and spirit parser expressions in particular - it will create dangling references as the internals are not supposed to live past the end of the containing full expressions. See BOOST_SPIRIT_AUTO on Zero to 60 MPH in 2 seconds!
Also see these answers which also concerns themselves with building/composing rules on the fly (at runtime):
Generating Spirit parser expressions from a variadic list of alternative parser expressions
Can Boost Spirit Rules be parameterized which demonstrates how to return rules from a function using boost::proto::deepcopy (like BOOST_SPIRIT_AUTO does, actually)
Nabialek Trick
In general, I'd very strongly advise against combining rules at runtime. Instead, if you're looking to 'add alternatives' to a rule at runtime, you can always use qi::symbols<> instead. The trick is to store a rule in the symbol-table and use qi::lazy to call the rule. In particular, this is known as the Nabialek Trick.
I have a toy command-line arguments parser here that demonstrates how you could use this idiom to match a runtime-defined set of command line arguments:
https://gist.github.com/sehe/2a556a8231606406fe36
Limitations of qi::lazy, what's next?
Unfortunately, qi::lazy does not support inherited arguments see e.g.
http://boost.2283326.n4.nabble.com/pass-inhertited-attributes-to-nabialek-trick-td2679066.html
You might be better off writing a custom parser component, as documented here:
http://boost-spirit.com/home/articles/qi-example/creating-your-own-parser-component-for-spirit-qi/
I'll try to find some time to work out a sample that replaces inherited arguments by qi::locals later.