Procedural macros live in their own crates, which are compiled for the development machine (so that they can be executed when crates that use them are compiled). Any conditional compilation directives within the procedural macro crates will accordingly be based on their compilation environment, rather than that of the invoking crate.
Of course, such macros could expand to tokens that include conditional compilation directives that will then be evaluated in the context of the invoking crate's compilation—however, this is not always possible or desirable.
Where one wishes for the expanded tokens themselves to be some function of the invoking crate's compilation environment, there is a need for the macro to determine that environment during its run-time (which is of course the invoking crate's compilation-time). Clearly a perfect use-case for the std::env module.
However, rustc doesn't set any environment variables; and Cargo sets only a limited few. In particular, some key information (like target architecture/operating system etc) is not present at all.
I appreciate that a build script in the invoking crate could set environment variables for the macro then to read, but this places an unsatisfactory burden on the invoking crate author.
Is there any way that a proc macro author can obtain runtime information about the invoking crate's compilation environment (target architecture and operating system being of most interest to me)?
I've somewhat inelegantly solved this by recursing into a second proc macro invocation, where the first invocation adds #[cfg_attr] attributes with literal boolean parameters that can then be accessed within the second invocation:
#[cfg_attr(all(target_os = "linux"), my_macro(linux = true ))]
#[cfg_attr(not(target_os = "linux"), my_macro(linux = false))]
// ...
A hack, but it works.
I found another solution:
Instead of generating the code depending on a flag like that, you can generate the code for all the OS and use #[cfg(...)] inside the quoted code.
quote! {
#[cfg(linux)]
{
// linux specific stuff
}
#[cfg(not(linux))]
{
// not linux specific stuff
}
}
This is probably cleaner.
Related
I'm moderately new to common lisp, but have extended experience with other 'separate compilation' languages (think C/C++/FORTRAN and such)
I know how to do an ASDF system definition. I know how to separate stuff in packages. I'm using SBCL, by the way.
The question is this: what's the best practice for splitting code (large packages) between .lisp files? I mean, in C there are include files, while lisp lives with the current image state. So with multiple files I need to handle dependencies or serial order in the system definition. But without something like forward declarations it's painful.
Simple example on what I want to do: I have, for example, two defstructs that are part of the same bigger data structure (like struct1 is a parent of some set of struct2). Some functions works on one, some other works on the other and some other use both.
So I would have: a packages.lisp, a fun1.lisp (with the first defstruct and related functions), a fun2.lisp (with the other defstruct and functions) and a funmix.lisp (with functions that use both). In an ideal world everything is sealed and compiling these in this order would be fine. As most of you know, this in practice almost never happen.
If I need to use struct2 functions from the struct1 ones I would need to either reorder or add a dependency. But then if there's some kind of back call (that can't be done with a closure) I would have struct1.lisp depending on struct2.lisp and vice-versa which is obviously not valid. So what? I could break the loop putting the defstruct in a separate file (say, structs.lisp) but what if either of the struct's function need to access the common functions in the third file? I would like to avoid style notes.
What's the common way to solve this, i.e. keeping loosely related code in the same file but still be able to interface to other ones. Is the correct solution to seal everything in a compilation unit (a single file)? use a package for every file with exports?
Lisp dependencies are simple, because in many cases, a Lisp implementation doesn't need to process the definition of something in order to compile its use.
Some exceptions to the rule are:
Macros: macros must be loaded in order to be expanded. There is a compile-time dependency between a file which uses macro and the file which defines them.
Packages: a package foo must be defined in order to use symbols like foo:bar or foo::priv. If foo is defined by a defpackage form in some foo.lisp file, then that file has to be loaded (either in source or compiled form).
Constants: constants defined with defconstant should be seen before their use. Similar remarks apply to inline functions, compiler macros.
Any custom things in a "domain specific language" which enforces definition before use. E.g. if Whizbang Inference Engine needs rules to be defined when uses of the rules are compiled, you have to arrange for that.
For certain diagnostics to be suppressed like calls to undefined functions, the defining and using files must be taken to be as a single compilation unit. (See below.)
All the above remarks also have implications for incremental recompilation.
When there is dependency like the above between files so that one is a prerequisite of the other, when the prerequisite is touched, the dependent one must be recompiled.
How to split code into files is going to be influenced by all the usual things: cohesion, coupling and what have you. Common-Lisp-specific reasons to keep certain things together in one file is inlining. The call to a function which is in the same file as the caller may be inlined. If your program supports any in-service upgrade, the granularity of code loading is individual files. If some functions foo and bar should be independently redefinable, don't put them in the same file.
Now about compilation units. Suppose you have a file foo.lisp which defines a function called foo and bar.lisp which calls (foo). If you just compile bar.lisp, you will likely get a warning that an undefined function foo has been called. You could compile foo.lisp first and then load it, and then compile bar.lisp. But that will not work if there is a circular reference between the two: say foo.lisp also calls (bar) which bar.lisp defines.
In Common Lisp, you can defer such warnings to the end of a compilation unit, and what defines a compilation unit isn't a single file, but a dynamic scope established by a macro called with-compilation-unit. Simply put, if we do this:
(with-compilation-unit
(compile-file "foo.lisp") ;; contains (defun foo () (bar))
(compile-file "bar.lisp")) ;; contains (defun bar () (foo))
If a compile-file isn't surrounded by with-compilation-unit then there is a compilation unit spanning that file. Otherwise, the outermost nesting of the with-compilation-unit macro determines the scope of what is in the compilation unit.
Warnings about undefined functions (and such) are deferred to the end of the compilation unit. So by putting foo.lisp and bar.lisp compilation into one unit, we suppress the warnings about either foo or bar not being defined and we can compile the two in any order.
Build systems use with-compilation-unit under the hood, as appropriate.
The compilation unit isn't about dependencies but diagnostics. Above, we don't have a compile time dependency. If we touch foo.lisp, bar.lisp doesn't have to be recompiled or vice versa.
By and large, Lisp codebases don't have a lot of hard dependencies among the files. Incremental compilation often means that just the affected files that were changed have to be recompiled. The C or C++ problem that everything has to be rebuilt because a core header file was touched is essentially nonexistent.
but what if
No matter how you first organize your code, if you change it significantly you are going to have to refactor. IMO there is no ideal way of grouping dependencies in advance.
As a rule of thumb it is generally safe to define generic functions first, then types, then actual methods, for example. For non-generic functions, you can cut circular dependencies by adding forward declarations:
(declaim (ftype function ...))
Having too much circular dependency is a bit of a code smell.
Is the correct solution to seal everything in a compilation unit
Yes, if you group the definitions in the same compilation unit (the same file), the file compiler will be able to silence the style notes until it reaches the end of file: at this point it knows if there are still missing references or if all the cross-references are resolved.
But then if there's some kind of back call (that can't be done with a closure)
If you have a specific example in mind please share, but typically you can define struct1 and its functions in a way that can be self-contained; maybe it can accept a map that binds event names to callbacks:
(make-struct-1 :callbacks (list :on-empty one-is-empty
:on-full one-is-full))
Similarly, struct2 can accept callbacks too (Dependency Injection) and the main struct ties them using closures (?).
Alternatively, you can design your data-structures so that they signal conditions, and the in the caller code you intercept them to bind things together.
I am trying to understand the preprocessor directives (like #if , #ifdef, #ifndef) and following is the code I have tried out.
Note: The member functions are further wrapped and used in python and hence the results show the python like calls but this does not affect any of the c++ process.
Question: 1. As per my understanding the global variables have a scope of whole file from the point it is declared. Then in this case, why is the defined value not accepted inside another function?
Requirement: I want to do something like mentioned below:
void Somefunc(int val){
set variable x;
}
Based on the x value, I want to include functions. Now the condition is:
If x=1, only some functions should be compiled since others utilize headers which would throw errors with the compiler I am using.
Thanks in advance!
Preprocessing runs before compilation. It handles the source code as plain text, it doesn't care for C++ language semantics.
The reason why var is not defined is that a preprocessor definition is valid from the point of definition until the end of the file (preprocessed translation unit) or a corresponding #undef.
Several weeks ago, SBCL updated 2.0.2 and brought the Block compilation feature. I have read this article to understand what it is.
I have a question, what's the difference between (declaim (inline 'some-function)) and Block compilation? Block compilation is automatic by the compiler?
Thanks.
Inline compilation is a specific optimization technique. A function being called is directly integrated into the calling function - usually using its source code - and then compiled.
This means that the inlined function might not be inlined only in one function, but in multiple functions.
Advantage: the overhead of calling a function disappears.
Disadvantage: the code size increases and the calling function(s) needs to be recompiled, when the inlined function changed and we want this change to become visible. Macros have the same problem.
Block compilation means that a bunch of code gets compiled together with different semantic constraints and that this enables the compiler to do a bunch of new optimizations.
Common Lisp has in the standard support for block compilation of single files. It allows the file compiler to assume that a file is such a block of code.
Example from the Common Lisp standard:
3.2.2.3 Semantic Constraints
A call within a file to a named function that is defined in the same file refers to that function, unless that function has been declared notinline. The consequences are unspecified if functions are redefined individually at run time or multiply defined in the same file.
This allows the code to call a global function and not use the symbol's function cell for the call. Thus this disables late binding for global function calls - in this file and for functions in this file.
It's not said how this can be achieved, but the compiler might just allocate the code somewhere and the calls just jump there.
So this part of block compilation is defined in the standard and some compilers are doing that.
Block compilation for multiple files
If the file compiler can use block compilation for one file, then what about multiple files? A few compilers can also tell the file compiler that several files make a block for compilation. CMUCL does that. SBCL was derived and simplified from CMUCL and lacks it until now. I think Lucid Common Lisp (which is no longer actively sold) did support something like that, too.
Might be useful to add this to SBCL, too.
I'm struggling with the last pieces of logic to make our Ada builder work as expectedly with variantdir. The problem is caused by the fact that the inflexible tools gnatbind and gnatlink doesn't allow the binder files to be placed in a directory other than the current one. This leaves me with two options:
Let gnatbind write the the binder files to topdir and then let gnatlink pick it from there. This may however cause race conditions if we want to allow simulatenous builds for different architectures and compiler versions which we want.
Modify the calls to gnatbind and gnatlink to temporarily go down to the build directory, in our case build/$ARCH/src-path. I successfully fixed the gnatbind step as this is explicitly called using a env.Execute from within the Ada builder. To try to fix the linking step I've modified the Program env using
env["LINKCOM"] = SCons.Action.Action(ada_linkcom)
where ada_linkcom is defined as
def ada_linkcom(source, target,env ):
....
return ret
where ret is a string describing what should be done in the shell. I need this to be a function it contains a bit complicated logic to convert paths from being relative to top-level to just containing their basenames.
This however fails with an error in scons-2.3.1/SCons/Executor.py on line 347 in function do_execute. Isn't env["LINKCOM"] allowed to be a function with ada_linkcom's signature?
No, it's not. You seem to think that 'env["LINKCOM"]' is what actually calls/executes the final build command, and that's not quite correct. Instead, environment variables like LINKCOM get expanded by the Executor/Builder for each specified Action, and are then executed.
You can have Python functions as Actions, and also use a so-called "generator" to create your Action strings on-the-fly. But you have to assign this Action to a Builder, and can't set it as an environment variable directly.
Please also have a look at the UserGuide ( http://www.scons.org/doc/production/HTML/scons-user.html ), especially section 18.4 "Builders That Execute Python Functions". Our basic guide for writing Builders and Tools might also prove to be helpful: http://www.scons.org/wiki/ToolsForFools
I have a question regarding gcc. Why I get an error of unused variable when I define the variable locally in a function but not when the variable is global in a unique file?.
I can understand that it can be use for someone else, but to do that then I need to put the external word right?
Thanks in advance.
The compiler has no way of knowing if a global variable is used - it could be used from a compilation unit written in a completely different language, for example.
If by "global in a unique file", you mean "int x;" outside of any function, the it's not the compilers job to detect that, the variable needs to be available to the linker in case another compilation unit needs it (such as errno).
If you meant "static int x" where it's not made available to the linker, this is probably just a choice made by GCC. I don't believe compilers are required to notify of this and it does no real damage other than wasting a few bytes in your address space.
Because global variables can be used on any other place that the compiler cannot known. For instance on a external library o program.
Unused locals can be determined by the compiler. Unused globals can only be determined by the linker, since they can be shared across object files.
In general, the linker doesn't do warnings for code-gen.
When the variable is global, the compiler has not full visibility across all the compilation units in the project - the variable could be modified in another compilation unit. The linker is able to tell that it is unused, probably it will remove it from the object file.
Because if it's global it can be used by another module that gets linked in later.
It's a common idiom to have all your globals defined in a single file. That file may not even have any code, much less code that uses all the variables.
I have encountered the same question when I build the dalvikVM in android2.3 and I got the key of the point. It is because that the parameters of the compiler is too strict:
LOCAL_CFLAGS += -Werror.