I would like to define a comparison method for a C++ set from python. In particular, I have a struct together with a set
from libcpp.set cimport set as cppset
ctypedef packed struct MyObject:
int value
cdef cppset[MyObject*] myset
In standard C++, I can define a Compare object with which I can impose a particular order on the set (e.g. using the value property of the object). Can I achieve the same in cython?
Related
I trying to find a way to get the number of possible enumerations in an enum type at compile time. I need this for initializing a templated class that uses enumerated types.
I am curious if there is a utility function (or system task) that gives this. It would be similar to $size() but for enumerated types. However, I can't seem to find a function for that. After doing a lot of research, it doesn't seem to be possible.
Here is an example I am trying to do:
typedef enum {RANDOM, STICKY, SWEEP} bias_t;
// can be parameterized to pick another enum type at random
class enum_picker #(type T = bias_t); //type must be an enumerated type
local T current_type;
local const int weights[$size(T)]; //<--- How do I get the number of enumerated types?
function T pick_type();
... some code ...
endfunction
endclass
So for the variable weights, it is an array of weights in which its size is the number of enumerated types. Right now it is 32 because of the $size() call but that is wrong; in this particular code example, the array size should be 3.
Is there a way to do this? Or is this simply not allowed in SystemVerilog?
You probably don't want to set up weights as a const; you would not be able to set values into it. You can use the num() method to get the number of enumerations.
class enum_picker #(type T = bias_t); //type must be an enumerated type
local T current_type;
local int weights[];
function new;
weights = new[current_type.num()];
foreach (weights[i]) weights[i] = $urandom_range(10);
endfunction
function T pick_type();
endfunction
endclass
TypeScript has a bunch of different ways to define an enum:
enum Alpha { X, Y, Z }
const enum Beta { X, Y, Z }
declare enum Gamma { X, Y, Z }
declare const enum Delta { X, Y, Z }
If I try to use a value from Gamma at runtime, I get an error because Gamma is not defined, but that's not the case for Delta or Alpha? What does const or declare mean on the declarations here?
There's also a preserveConstEnums compiler flag -- how does this interact with these?
There are four different aspects to enums in TypeScript you need to be aware of. First, some definitions:
"lookup object"
If you write this enum:
enum Foo { X, Y }
TypeScript will emit the following object:
var Foo;
(function (Foo) {
Foo[Foo["X"] = 0] = "X";
Foo[Foo["Y"] = 1] = "Y";
})(Foo || (Foo = {}));
I'll refer to this as the lookup object. Its purpose is twofold: to serve as a mapping from strings to numbers, e.g. when writing Foo.X or Foo['X'], and to serve as a mapping from numbers to strings. That reverse mapping is useful for debugging or logging purposes -- you will often have the value 0 or 1 and want to get the corresponding string "X" or "Y".
"declare" or "ambient"
In TypeScript, you can "declare" things that the compiler should know about, but not actually emit code for. This is useful when you have libraries like jQuery that define some object (e.g. $) that you want type information about, but don't need any code created by the compiler. The spec and other documentation refers to declarations made this way as being in an "ambient" context; it is important to note that all declarations in a .d.ts file are "ambient" (either requiring an explicit declare modifier or having it implicitly, depending on the declaration type).
"inlining"
For performance and code size reasons, it's often preferable to have a reference to an enum member replaced by its numeric equivalent when compiled:
enum Foo { X = 4 }
var y = Foo.X; // emits "var y = 4";
The spec calls this substitution, I will call it inlining because it sounds cooler. Sometimes you will not want enum members to be inlined, for example because the enum value might change in a future version of the API.
Enums, how do they work?
Let's break this down by each aspect of an enum. Unfortunately, each of these four sections is going to reference terms from all of the others, so you'll probably need to read this whole thing more than once.
computed vs non-computed (constant)
Enum members can either be computed or not. The spec calls non-computed members constant, but I'll call them non-computed to avoid confusion with const.
A computed enum member is one whose value is not known at compile-time. References to computed members cannot be inlined, of course. Conversely, a non-computed enum member is once whose value is known at compile-time. References to non-computed members are always inlined.
Which enum members are computed and which are non-computed? First, all members of a const enum are constant (i.e. non-computed), as the name implies. For a non-const enum, it depends on whether you're looking at an ambient (declare) enum or a non-ambient enum.
A member of a declare enum (i.e. ambient enum) is constant if and only if it has an initializer. Otherwise, it is computed. Note that in a declare enum, only numeric initializers are allowed. Example:
declare enum Foo {
X, // Computed
Y = 2, // Non-computed
Z, // Computed! Not 3! Careful!
Q = 1 + 1 // Error
}
Finally, members of non-declare non-const enums are always considered to be computed. However, their initializing expressions are reduced down to constants if they're computable at compile-time. This means non-const enum members are never inlined (this behavior changed in TypeScript 1.5, see "Changes in TypeScript" at the bottom)
const vs non-const
const
An enum declaration can have the const modifier. If an enum is const, all references to its members inlined.
const enum Foo { A = 4 }
var x = Foo.A; // emitted as "var x = 4;", always
const enums do not produce a lookup object when compiled. For this reason, it is an error to reference Foo in the above code except as part of a member reference. No Foo object will be present at runtime.
non-const
If an enum declaration does not have the const modifier, references to its members are inlined only if the member is non-computed. A non-const, non-declare enum will produce a lookup object.
declare (ambient) vs non-declare
An important preface is that declare in TypeScript has a very specific meaning: This object exists somewhere else. It's for describing existing objects. Using declare to define objects that don't actually exist can have bad consequences; we'll explore those later.
declare
A declare enum will not emit a lookup object. References to its members are inlined if those members are computed (see above on computed vs non-computed).
It's important to note that other forms of reference to a declare enum are allowed, e.g. this code is not a compile error but will fail at runtime:
// Note: Assume no other file has actually created a Foo var at runtime
declare enum Foo { Bar }
var s = 'Bar';
var b = Foo[s]; // Fails
This error falls under the category of "Don't lie to the compiler". If you don't have an object named Foo at runtime, don't write declare enum Foo!
A declare const enum is not different from a const enum, except in the case of --preserveConstEnums (see below).
non-declare
A non-declare enum produces a lookup object if it is not const. Inlining is described above.
--preserveConstEnums flag
This flag has exactly one effect: non-declare const enums will emit a lookup object. Inlining is not affected. This is useful for debugging.
Common Errors
The most common mistake is to use a declare enum when a regular enum or const enum would be more appropriate. A common form is this:
module MyModule {
// Claiming this enum exists with 'declare', but it doesn't...
export declare enum Lies {
Foo = 0,
Bar = 1
}
var x = Lies.Foo; // Depend on inlining
}
module SomeOtherCode {
// x ends up as 'undefined' at runtime
import x = MyModule.Lies;
// Try to use lookup object, which ought to exist
// runtime error, canot read property 0 of undefined
console.log(x[x.Foo]);
}
Remember the golden rule: Never declare things that don't actually exist. Use const enum if you always want inlining, or enum if you want the lookup object.
Changes in TypeScript
Between TypeScript 1.4 and 1.5, there was a change in the behavior (see https://github.com/Microsoft/TypeScript/issues/2183) to make all members of non-declare non-const enums be treated as computed, even if they're explicitly initialized with a literal. This "unsplit the baby", so to speak, making the inlining behavior more predictable and more cleanly separating the concept of const enum from regular enum. Prior to this change, non-computed members of non-const enums were inlined more aggressively.
There are a few things going on here. Let's go case by case.
enum
enum Cheese { Brie, Cheddar }
First, a plain old enum. When compiled to JavaScript, this will emit a lookup table.
The lookup table looks like this:
var Cheese;
(function (Cheese) {
Cheese[Cheese["Brie"] = 0] = "Brie";
Cheese[Cheese["Cheddar"] = 1] = "Cheddar";
})(Cheese || (Cheese = {}));
Then when you have Cheese.Brie in TypeScript, it emits Cheese.Brie in JavaScript which evaluates to 0. Cheese[0] emits Cheese[0] and actually evaluates to "Brie".
const enum
const enum Bread { Rye, Wheat }
No code is actually emitted for this! Its values are inlined. The following emit the value 0 itself in JavaScript:
Bread.Rye
Bread['Rye']
const enums' inlining might be useful for performance reasons.
But what about Bread[0]? This will error out at runtime and your compiler should catch it. There's no lookup table and the compiler doesn't inline here.
Note that in the above case, the --preserveConstEnums flag will cause Bread to emit a lookup table. Its values will still be inlined though.
declare enum
As with other uses of declare, declare emits no code and expects you to have defined the actual code elsewhere. This emits no lookup table:
declare enum Wine { Red, Wine }
Wine.Red emits Wine.Red in JavaScript, but there won't be any Wine lookup table to reference so it's an error unless you've defined it elsewhere.
declare const enum
This emits no lookup table:
declare const enum Fruit { Apple, Pear }
But it does inline! Fruit.Apple emits 0. But again Fruit[0] will error out at runtime because it's not inlined and there's no lookup table.
I've written this up in this playground. I recommend playing there to understand which TypeScript emits which JavaScript.
I've got a function with the signature:
function(std::vector<double> vector);
And I've exposed it, but it doesn't take in Python lists. I've looked through the other SO answers, and most involve changing the function to take in boost::python::lists, but I don't want to change the function. I imagine I can use the vector_indexing_suite to write a simple wrapper around this function, but I have many functions of this form and would rather not write a wrapper for every single one. Is there a way to automatically make a Python list->std::vector mapping occur?
There are a few solutions to accomplish this without having to modify the original functions.
To accomplish this with a small amount of boilerplate code and transparency to python, consider registering a custom converter. Boost.Python uses registered converters when going between C++ and Python types. Some converters are implicitly created when creating bindings, such as when class_ exports a type.
The following complete example uses an iterable_converter type that allows for the registration of conversion functions from a python type supporting the python iterable protocol. The example enable conversions for:
Collection of built-in type: std::vector<double>
2-dimensional collection of strings: std::vector<std::vector<std::String> >
Collection of user type: std::list<foo>
#include <iostream>
#include <list>
#include <vector>
#include <boost/python.hpp>
#include <boost/python/stl_iterator.hpp>
/// #brief Mockup model.
class foo {};
// Test functions demonstrating capabilities.
void test1(std::vector<double> values)
{
for (auto&& value: values)
std::cout << value << std::endl;
}
void test2(std::vector<std::vector<std::string> > values)
{
for (auto&& inner: values)
for (auto&& value: inner)
std::cout << value << std::endl;
}
void test3(std::list<foo> values)
{
std::cout << values.size() << std::endl;
}
/// #brief Type that allows for registration of conversions from
/// python iterable types.
struct iterable_converter
{
/// #note Registers converter from a python interable type to the
/// provided type.
template <typename Container>
iterable_converter&
from_python()
{
boost::python::converter::registry::push_back(
&iterable_converter::convertible,
&iterable_converter::construct<Container>,
boost::python::type_id<Container>());
// Support chaining.
return *this;
}
/// #brief Check if PyObject is iterable.
static void* convertible(PyObject* object)
{
return PyObject_GetIter(object) ? object : NULL;
}
/// #brief Convert iterable PyObject to C++ container type.
///
/// Container Concept requirements:
///
/// * Container::value_type is CopyConstructable.
/// * Container can be constructed and populated with two iterators.
/// I.e. Container(begin, end)
template <typename Container>
static void construct(
PyObject* object,
boost::python::converter::rvalue_from_python_stage1_data* data)
{
namespace python = boost::python;
// Object is a borrowed reference, so create a handle indicting it is
// borrowed for proper reference counting.
python::handle<> handle(python::borrowed(object));
// Obtain a handle to the memory block that the converter has allocated
// for the C++ type.
typedef python::converter::rvalue_from_python_storage<Container>
storage_type;
void* storage = reinterpret_cast<storage_type*>(data)->storage.bytes;
typedef python::stl_input_iterator<typename Container::value_type>
iterator;
// Allocate the C++ type into the converter's memory block, and assign
// its handle to the converter's convertible variable. The C++
// container is populated by passing the begin and end iterators of
// the python object to the container's constructor.
new (storage) Container(
iterator(python::object(handle)), // begin
iterator()); // end
data->convertible = storage;
}
};
BOOST_PYTHON_MODULE(example)
{
namespace python = boost::python;
// Register interable conversions.
iterable_converter()
// Build-in type.
.from_python<std::vector<double> >()
// Each dimension needs to be convertable.
.from_python<std::vector<std::string> >()
.from_python<std::vector<std::vector<std::string> > >()
// User type.
.from_python<std::list<foo> >()
;
python::class_<foo>("Foo");
python::def("test1", &test1);
python::def("test2", &test2);
python::def("test3", &test3);
}
Interactive usage:
>>> import example
>>> example.test1([1, 2, 3])
1
2
3
>>> example.test1((4, 5, 6))
4
5
6
>>> example.test2([
... ['a', 'b', 'c'],
... ['d', 'e', 'f']
... ])
a
b
c
d
e
f
>>> example.test3([example.Foo(), example.Foo()])
2
A few comments on this approach:
The iterable_converter::convertible function could be changed to only allowing python list, rather than allowing any type that supports the iterable protocol. However, the extension may become slightly unpythonic as a result.
The conversions are registered based on C++ types. Thus, the registration only needs to be done once, as the same registered conversion will be selected on any number of exported functions that accept the C++ type as an argument.
It does not introduce unnecessary types into the example extension namespace.
Meta-programming could allow for multi-dimensional types to recursively register each dimension type. However, the example code is already complex enough, so I did not want to add an additional level of complexity.
Alternative approaches include:
Create a custom function or template function that accepts a boost::python::list for each function accepting a std::vector. This approach causes the bindings to scale based on the amount of functions being exported, rather than the amount of types needing converted.
Using the Boost.Python vector_indexing_suite. The *_indexing_suite classes export a type that is adapted to match some semantics of Python list or dictionaries. Thus, the python code now has to know the exact container type to provide, resulting in a less-pythonic extension. For example, if std::vector<double> is exported as VecDouble, then the resulting Python usage would be:
v = example.VecDouble()
v[:] = [1, 2, 3]
example.test1(v)
However, the following would not work because the exact types must match, as exporting the class only registers a conversion between VecDouble and std::vector<double>:
example.test1([4, 5, 6])
While this approach scales to types rather than functions, it results in a less pythonic extension and bloats the example namespace with unnecessary types.
I remember having read somewhere that it's possible to override the .init property of custom types in D. I'd like to do this for a struct I've created, but I'm not finding any way to do this, especially since default constructors aren't allowed. Is this actually possible, and if so, how can I do this?
you can specify the init values of the fields (with compile time vars only)
struct foo{
int a=0;
real b = 5.0;
}
foo.init will then be equal to foo(0,5.0)
the new object structure in 1.9 embeds some ivars into objects for faster access:
#define ROBJECT_EMBED_LEN_MAX 3
struct RObject {
struct RBasic basic;
union {
struct {
long numiv;
VALUE *ivptr;
struct st_table *iv_index_tbl;
} heap;
VALUE ary[ROBJECT_EMBED_LEN_MAX];
} as;
};
My question is are the first 3 ivars always embedded? or are they only embedded if the number of ivars is <=3 ?
I've tried reading the source but find it next to incomprehensible.
Thanks
The instance variable heap (called heap) and the embedded instance variables (called ary) are contained in a union. You'll also find some macros defined below the snippit you quoted that all look like:
#define ROBJECT_IVPTR(o) \
((RBASIC(o)->flags & ROBJECT_EMBED) ? \
ROBJECT(o)->as.ary : \
ROBJECT(o)->as.heap.ivptr)
Key in all these is RBASIC(o)->flags & ROBJECT_EMBED. The ROBJECT_EMBED flag indicates whether embedded instance variables are in use, or the heap is in use.
So embedded variables are only used when the number of instance variables is <= 3.