I come from many years of experience with make. One thing that is baffling me is the use of task dependencies rather than file dependencies in Gradle. For example, if I have a C program that has these dependencies (in makefile format):
app : file1.o file2.o
file1.o : file1.c file1.h file2.h
file2.o : file2.c file2.h
So, both object files are dependent on file2.h and would be rebuilt if file2.h changed. However, if file1.h changed, only file1.o would be built.
How can I represent this in Gradle? I am mainly interested in how one can handle this in raw Gradle as opposed to using a plugin.
Thanks!
Blake McBride
The answer has to do with Gradle's incremental build support. It is documented at https://blog.gradle.org/introducing-incremental-build-support and https://docs.gradle.org/current/userguide/more_about_tasks.html#sec:up_to_date_checks
Basically, for built-in tasks, Gradle internally calculates what the input and output files associated with a task are. It then uses this information to avoid executing unneeded tasks thus eliminating unneeded steps. For custom tasks, Gradle has a method, documented in the above links, to tell Gradle what the input and output files are for a particular task.
Related
As Bitbake builds -dev and -debug for recipes is it possible for defining compilation definitions specific to debug build for a particular recipe. Lets say I have some source code under DEBUG_INFO for some recipe i.e.,
#ifdef DEBUG_INFO
........... do something
#endif /* DEBUG_INFO */
and uses cmake in bitbake environment.
I want this flag be enabled for the debug binaries generated in the .debug folder. Is this possible?
If I use EXTRA_OECMAKE = "-DDEBUG_INFO" it gets enabled to both dev and debug builds.
No, it is not possible. All packages of a recipe are built in one go, they're just the same files but split somehow.
The only difference is with "special flavors" of a recipe (native, nativesdk, target, multilib, toolchain-specific recipes, etc...), in that case, you can have different flags but still, all the packages resulting from the build of this "flavor" will be built with the same flag.
If you want to build another variant of a package where a certain CMake flag is set in the compilation, you can create a variant of the recipe. If the main recipe is named my-app_git.bb you can create another recipe file named my-app-tweak_git.bb and a common base, my-app.inc. In the bb files, include the inc file:
require my-app.inc
Move most of what's now in my-app_git.bb to my_app.inc, e.g. SRC_URI, but define different contents for EXTRA_OECMAKE in the .bb files.
Now you will have to decide which one of my-app and my-app-tweak goes into the image by specifying either my-app or my-app-tweak in an IMAGE_INSTALL definition.
This is not exactly what you asked for, but as has been stated by qschulz, you cannot change the contents of the -dev and -dbg sub-packages.
Also note that dbg and dev can be considered reserved words for variants of the package name, so if you want to use something other than tweak, as in my example, you cannot use any of them.
It seem to me that scons targets are being generated not in declaration sequence. My problem is, I need to generate some code first, I'm using protoc to process a my.proto file into .h and .cc file, I need some pseudo code like this(what should the working code look like?)
import os
env=Environment(ENV=os.environ,LIBPATH='/usr/local/lib')
env.ShellExecute('protoc', '--outdir=. --out-lang=cpp', 'my.proto')//produces my.cc
myObj=Object('my.cc')//should wait until 'my.cc' is generated by protoc
Dependency(myObj, 'my.cc')
mainObj=Object('main.cpp')
My question is:
How to specify this ShellExecution of protoc in SConstruct/SConscript?
How to make sure that the compilation of 'main.cpp' depends on the existence of 'my.cc', in another word, wait until 'my.cc' is generated and then execute?
Your observations and assumptions are correct, SCons will not execute the single build commands in the order that you list them in the SConstruct files. It will run them based on the dependencies of the targets and source files in your build, either defined implicitly (header includes in C++, for example) or explicitly (via the Depends() method).
So you have to define and setup your dependencies correctly, such that SCons delivers the output that you want. For the special protoc case in your example, a special Builder exists that will help you to get the dependency graph right. It is available in our ToolsIndex, where also support for a variety of other languages and dialects can be found.
These special builders will emit the correct target nodes, e.g. when given a *.proto input file, and SCons is then able to automatically detect the dependency between the protoc input file and your main program if you say something like:
env=Environment(tools=['default','protoc'])
env.Protoc([], "test.proto")
env.Program('main', ['main.cpp'] + Glob('*.cc'))
The Glob('*.cc') will detect your *.cc files, coming out of the protoc Tool, and include them as dependencies for your final target main.
You can always write your own Builders and Emitters in SCons, which is the canonical way of making new tools/toolchains known to SCons dependency analysis. In the UserGuide, sect. "18 Writing Your Own Builders", and especially our ToolsForFools Guide you can find more infos about this.
My makefile makes use of auto-generated dependencies. To do this, I have in my top-level makefile something similar to:
# Makefile
include target1.deps
include target2.deps
all: target2.deps
cat $^
target2.deps: target1.deps
target1.deps:
echo "target2.deps:" > $#
echo " touch target2.deps" >> $#
Initially, target1.deps and target2.deps do not exist. When make is first instantiated, it parses the entire Makefile and searches for a way to generate these include files. After building them, it reinvokes itself, causing the Makefile to be reparsed and the include files to be included this time. At least, that's my understanding.
The issue is that when I run the above Makefile, Make first builds target1.deps, then executes the body of the all rule, never having built or included target2.deps. This causes cat to error: cat: target2.deps: No such file or directory
This seems like a contradiction to me. I explicitly tell Make that all depends on target2.deps, but it attempts to execute the rule before satisfying its prerequisites!
The intended behavior is that target1.deps should be built and included, then target2.deps should be built and included using the rule contained within target1.deps, and then all should be run. How do I achieve this?
Context: Since this is weirdly abstract, here's my goal: I have a target index.html, which gets generated from a template index.html.in, but I don't know anything about its dependencies. I need to find out (a) which files I need to create before building index.html and (b) which files index.html will depend on at runtime.
For example index.html includes some inline css that's pulled out of global.css - I need to therefore build global.css before building index.html. On the other hand, index.html links to about.html, so after I build index.html I want to also build about.html. I call the former "build dependencies" and the latter "runtime dependencies". So my makefile looks something like this:
include index.html.build_deps
include index.html.runtime_deps
all: index.html $(runtime_deps_index.html)
%.build_deps: %.in
./extract_build_deps %< -o %#
%.runtime_deps: %
./extract_runtime_deps %< -o %#
%: %.in
./compile_template %< -o $#
What I want to happen is for Make to follow these steps:
Build index.html.build_deps
Include index.html.build_deps
Build global.css (now a known prerequisite of index.html)
Build index.html
Build index.html.runtime_deps
Include index.html.runtime_deps
Build about.html (contained inside $(runtime_deps_index.html) included from index.html.runtime_deps)
Target all is reached
What actually happens:
Make sees that index.html.build_deps can be directly build from index.html.in; does so.
Make sees that index.html.runtime_deps can be built from index.html, can be built from index.html.in.
Make builds index.html. It errors because global.css hasn't yet been built.
If Make had included index.html.build_deps after building that, then it would be aware of the global.css dependency. But because it tries to build all include files before expanding any of them, it's unaware of the dependency. I want to add a dependency "index.html.runtime_deps depends on index.html.build_deps having been included, but I'm not sure how to specify such a dependency.
#Dario is correct. To be a bit more specific, these are the steps make will follow here:
Read the makefile.
Try to build target1.deps.
Find a target target1.deps and execute the recipe.
The recipe succeeds, but make observes that the file target1.deps still does not exist, so make doesn't mark the target as updated.
Try to build target2.deps.
There's a target for it that depends on target1.deps, which make already built, but there's no recipe for it so make doesn't mark target2.deps as updated (since it was never updated, as far as make can tell--it didn't run any recipe to update it).
So, make decides none of the included makefiles were actually updated and it won't re-exec.
Then make wants to build all; it sees that all depends on target2.deps but make already considered that target and decided it didn't need to be rebuilt, so now make is done with all its work.
You can run make -d and follow along with the decisions make takes.
I'm creating a makefile to build an existing project. We're using GNU Make 4, but the compiler doesn't automatically generate dependency information so I'm trying to determine the best way to specify dependencies for code that includes header files which in turn may include header files. Due to the complexity of the codebase, I think it will be unrealistic to traverse the entire #include tree, gather up all the header files, and then specify them for each source file.
One solution is to create makefile definitions of each header file and its direct dependencies, then create a recipe for a header file that touches that header file to trigger the recompile of the source that includes it:
a123.o: a123.c header1.h
header1.h: header2.h
header2.h: header3.h
# Recipes
%.o:
# compile command for building a .o
%.h:
#touch $#
This seems to work; if header3.h changes, then the %.h recipe updates header2.h's timestamp, which causes header1.h's timestamp to be updated, which causes a123.o to be rebuilt.
However, it seems messy, with all the touching, plus it seems odd to create a recipe for a source file that isn't directly compiled. Is this the correct way to do this sort of thing, or is there a cleaner approach? Please note that I simplified my question; in reality several languages are in use with different file extensions, the compiler isn't GCC, and the output isn't a .o file. But the logic is exactly the same.
I have been tooling around with autotools for the past couple of days, and finally have made significant progress. One problem I am having is that I have two libraries that need to be compiled before the main application code. I'm not quite sure how to do this. My directory structure is below and a snippet from my configure.ac as well.
AC_CONFIG_FILES([Makefile
src/Makefile
gtkworkbook/Makefile
csv/Makefile])
AC_OUTPUT
I need the csv/Makefile and gtkworkbook/Makefile to both be compiled before src/Makefile; is there any way to specify this? Right now I am getting an error about the library (csv) not existing during the application compile process.
The order of items in AC_CONFIG_FILES() does not affect the build order. If you're using automake, which I assume you are, it will traverse your directory tree in the order that you list directories in each Makefile.am's SUBDIRS list.
That being said, you should have the order of items in AC_CONFIG_FILES() mirror the build order, for consistency/maintainability.
Example of how your toplevel Makefile.am's SUBDIRS to build in the desired order:
SUBDIRS = csv gtkworkbook src
Also, for this simple case you don't need both AC_CONFIG_FILES() and AC_OUTPUT(). You can pass your list directory to AC_OUTPUT():
AC_OUTPUT([
Makefile
src/Makefile
gtkworkbook/Makefile
csv/Makefile
])