I'm a beginner with bazel and I know I did not undertsand all and concepts too, I am really more familiar with makefile and cmake.
What I dont understand and need explanation please is the following behavior with the dependency I created
Myrepo
|
| WORKSPACE
|inc
| hello.h
| BUILD
| src
| hello.c
| BUILD
| deps
| third_part # not from git repo , can be deleted
| BUILD
| scripts
| install_third_part.sh
| BUILD
so my hello.c needs headers that script install_third_part.sh puts inside the deps/third_part
so that bazels files are:
src/BUILD
cc_library(
name = "hello",
visibility = ["//visibility:public"],
linkstatic = 1,
hdrs = glob(["inc/*.h"]),
srcs = glob(["src/*.c"]),
deps = ["//deps:third",
]
)
deps/BUILD
cc_library(
name = "third",
hdrs = glob(["third_part/usr/local/include/**/*"]),
data = ["//scripts:install_third_part"],
copts=["-Ithird_part/usr/local/include/"],
visibility = ["//visibility:public"],
)
scripts/BUILD
genrule(
name = "install_third_part",
srcs= ["//scripts:install_third_part.sh"],
outs = ["install_third_part.txt"],
cmd = "$(location //scripts:install_third_part.sh) -i > \"$#\"",
local =1,
visibility = ["//visibility:public"],
)
so when I called
bazel build //deps:third
bazel build //scripts:install_third_part
as I set local to true, I can see the folder deps/third_part/ created with all header files
BUT when I called
bazel build //src:hello
I can see the rule is executed as bazel-bin/scripts/install_third_part.txt is present but the deps/third_part is not created if I removed it before.
(bazel 5.2)
what's wrong with my way of doing that ?
maybe you will told me to create a macro to do such thing, I need a clear example to understand please
thank you
Karim
#ahumesky
Related
The problem
Suppose that I have written a lengthy script
in some language "lang",
and now want to convert this single-file script
into a directory tree with a project consisting of many files. I want to insert some kind of separators and file-paths into this file, and process it in some way so that in the end I obtain:
a proper project directory layout (sth. like this),
build-definition file,
readme's,
separate subdirectories for main/src and test/src etc.
For example, given the following script (pseudocode):
// required dependencies, should be moved
// into the build definition build.foo
require "org.foo" % "foo-core" % "1.2.3"
require "org.bar" % "bar-gui" % "3.2.1"
// A longer comment that should be converted
// into a text file and moved into a 'notes'
// subdirectory
/*
#README
Another lengthy comment that should go into
a readme.md
*/
/** A class that should
* go to src/main/lang/proj/A.lang
*/
class A {
def a = "foo"
}
/** Another class
* that should go to src/main/lang/proj/B.lang
*/
class B {
def b = "bar"
}
/** Some tests,
* should end up in
* src/test/lang/proj/MyTest.lang
#Test def testFoo() {
assert(2 + 2 == 5)
}
and assuming that I can insert arbitrary separators, commands, escape-sequences and file paths into this file, I would like to obtain the following project:
project/
|-- build.txt
|-- notes
| `-- note_01.txt
|-- readme.md
`-- src
|-- main
| `-- lang
| `-- proj
| |-- A.lang
| `-- B.lang
`-- test
`-- lang
`-- proj
`-- MySpec.lang
Edit:
What follows is a less-sophisticated version of my own answer below
What I've tried
Here is one naive way to do it:
Convert the original script into a bash script by prepending #!/bin/bash
split the source code into HEREDOCS
insert package declarations where necessary
add bunch of mkdir -p and cd between the HEREDOC-pieces
cat the HEREDOC pieces into appropriately named files
test the script on empty directories until it works as expected
For the above script, it might look somehow like this:
#!/bin/bash
mkdir project
cd project
cat <<'EOF' > build.txt
// required dependencies, should be moved
// into the build definition build.foo
require "org.foo" % "foo-core" % "1.2.3"
require "org.bar" % "bar-gui" % "3.2.1"
EOF
mkdir notes
cd notes
cat <<'EOF' > note_01.txt
// A longer comment that should be converted
// into a text file and moved into a 'notes'
// subdirectory
EOF
cd ..
cat <<'EOF' > readme.md
/*
#README
Another lengthy comment that should go into
a readme.md
*/
EOF
mkdir -p src/main/lang/proj
cd src/main/lang/proj
cat <<'EOF' > A.lang
package proj
/** A class
* that should go to src/main/lang/proj/A.lang
*/
class A {
def a = "foo"
}
EOF
cat <<'EOF' > B.lang
package proj
/** Another class
* that should go to src/main/lang/proj/B.lang
*/
class B {
def b = "bar"
}
EOF
cd ../../..
mkdir -p test/lang/proj
cd test/lang/proj
cat <<'EOF' > MySpec.lang
package proj
/** Some tests,
* should end up in
* src/test/lang/proj/MyTest.lang
#Test def testFoo() {
// this should end up in test
assert(2 + 2 == 5)
}
EOF
cd ../../..
What's wrong with this approach
It does generate the correct tree, but this approach seems rather error-prone:
it's too easy to cd ../../.. to the wrong nesting level
too easy to mkdir with a wrong name, and then fail to cd into it.
There is no way to handle the entire tree construction as a single
transaction, that is, if something fails later in the script,
there is no simple way to clean up the mess generated before
the error occurred.
I certainly could try to make it a bit less brittle by defining
special functions that mkdir and cd in one go, and
then wrap invocations of those functions together with cats into
(mkdirAndCd d ; cat) etc.
But it just doesn't feel quite right. Isn't there a much simpler
way to do it? Could one somehow combine the standard bash/linux utilities
into a tiny & very restricted domain specific language for
generating directory trees with text files? Maybe some newer version of split where one could specify where to split and where to put the pieces?
Related questions:
mkdir and touch in single command
The reverse of tree - reconstruct file and directory structure from text file contents?
Other interesting proposals that don't seem to work:
Use tar. That would mean that one would have to convert the text file manually into a valid tar-archive. While a tar archive indeed is a single plain-text file, its internal format does not look like the most comfortable DSL for such a simple task. It was never intended to be used by humans directly in that way.
Similar argument holds for shar. Since shar uses the bash itself to extract the archive, my above proposal is, in principle, a manually generated shar-archive in a very uncommon format, therefore shar seems to share all the drawbacks with the above proposal. I'd rather prefer something more restricted, that allows to do fewer things, but provides more guarantees about the quality of the outcome.
Maybe I should emphasize again that I don't have a tree to begin with, so there is nothing to compress. I have only the single script file and a rough idea of what the tree should look like in the end.
Seems to me that you are trying to write a custom parser. Provided that all blocks mentioned by you are ended by double line endings, this could help you
#!/bin/bash
gawk 'BEGIN{RS="\n\n([/][*]|[/]{2,2})"}
{
if ($0 ~ /#README/){
system("echo -e \"\nThis is a Readme.md\n--------\n" $0 "\"")
}else if ($0 ~ /class /){
system("echo -e \"\nThis is a class\n---------\n/*" $0 "\"")
}else if ($0 ~ /require /){
system("echo -e \"\nthis is a conf\n-----------\n" $0 "\"")
}else if($0 ~ /[/]{2,2}.*\n[/]{2,2}/){
system("echo -e \"\nthis is a note\n-----------\n" $0 "\"")
}
}' your_script.lang
The key part is the record separator RS that splits block of code that start with '\n\n//' or '\n\n/*'.
Instead of echo -e you could write custom scripts for each type of block.
Please note that the record separator will not be present on $0 so you have to add the missing characters, as in the /class / example above.
The output of the above code is
this is a conf
-----------
// required dependencies, should be moved
// into the build definition build.foo
require org.foo % foo-core % 1.2.3
require org.bar % bar-gui % 3.2.1
this is a note
-----------
A longer comment that should be converted
// into a text file and moved into a 'notes'
// subdirectory
This is a Readme.md
--------
#README
Another lengthy comment that should go into
a readme.md
*/
This is a class
---------
/** A class that should
* go to src/main/lang/proj/A.lang
*/
class A {
def a = foo
}
This is a class
---------
/** Another class
* that should go to src/main/lang/proj/B.lang
*/
class B {
def b = bar
}
About your concerns:
it's too easy to cd ../../.. to the wrong nesting level
-> define a variable with root path and cd to it.
too easy to mkdir with a wrong name, and then fail to cd into it.
-> define variables with directory names and check if they already exists.
path1=src/main/lang/some
if [ -d $path1 ]; then
do_something
fi
There is no way to handle the entire tree construction as a single transaction ...
-> write to file paths of every NEW directory/file that you create and use it to revert if necessary.
(my own answer)
Consider the following definition of a tiny embedded domain specific language for defining directory trees with text files:
#!/bin/bash
enter() {
local subdir="$1"
if [ ! -d "$subdir" ]
then
mkdir -p "$subdir"
fi
pushd "$subdir" > /dev/null
}
leave() {
popd > /dev/null
}
save() {
local fileName="$1"
cut -d'|' -f2- > "$fileName"
}
The enter command creates a directory if necessary, and cds into this directory, it works with arbitrary relative paths. The save command saves the text content of a here-document to file. The leave command changes to previous directory.
When a file is saved, the margin (empty space followed by '|') is stripped from each line. This is to ensure that the indentation of the script does not interfere with the indentation of the written files.
If these definitions are sourced, then the tree-generation script can be written as follows:
#!/bin/bash
source explode.sh
mkdir project
cd project
enter "src"
enter "main/lang/proj"
save "A.lang" <<'____EOF'
|package proj
|
|/** A totally useful class
| * that should go to src/main/lang/proj/A.lang
| */
|class A {
| def a = "foo"
|}
____EOF
save "B.lang" <<'____EOF'
|package proj
|/** Another very useful class
| * that should go to src/main/lang/proj/B.lang
| */
|class B {
| def b = "bar"
|}
____EOF
leave
enter "test/lang/proj"
save "MyTest.lang" <<'____EOF'
|package proj
|
|/** A test that should end up in
| * src/test/lang/proj/MyTest.lang
|#Test def testFoo() {
| assert(2 + 2 == 5)
|}
____EOF
leave
leave
save "build.txt" <<'EOF'
|require "org.foo" % "foo-core" % "1.2.3"
|require "org.bar" % "bar-gui" % "3.2.1"
EOF
enter "notes"
save "note_01.txt" <<'__EOF'
|A longer comment that should be converted
|into a text file and moved into a 'notes'
|subdirectory. This is a very long comment
|about the purpose of the project. Blah
|blah blah.
__EOF
leave
save "README.md" <<'EOF'
|#README
|
|This is a readme file for my awesome project.
|It ends with this line. Bye.
EOF
When executed, the script generates the following directory tree:
project/
├── build.txt
├── notes
│ └── note_01.txt
├── README.md
└── src
├── main
│ └── lang
│ └── proj
│ ├── A.lang
│ └── B.lang
└── test
└── lang
└── proj
└── MyTest.lang
The bash-script mirrors the tree structure very closely, and it's impossible to mess up the cd ../../../../../.. commands. It still lacks various desirable properties though (not transactional, no dry-run capability).
I am running a build on my Jenkins server and I am looking to dynamically populate the git_commit field with the commit number from the current build. The file has multiple functions in it and I want to use sed to match core-lambda-function1 name of the module and update the git_commit field with the commit number from the current build. Any help is appreciated. Thanks.
module "core-lambda-function1" {
source = "./lambda"
name = "core-lambda-function"
runtime = "nodejs6.10"
role = "${aws_iam_role.iam_role_for_lambda.arn}"
filename = "../Archive.zip"
source_code_hash = "${base64sha256(file("../Archive.zip"))}"
source_dir = "../"
git_commit = ""
}
module "core-lambda-function2" {
source = "./lambda"
name = "core-lambda-function"
runtime = "nodejs6.10"
role = "${aws_iam_role.iam_role_for_lambda.arn}"
filename = "../Archive.zip"
source_code_hash = "${base64sha256(file("../Archive.zip"))}"
source_dir = "../"
git_commit = ""
}
this is what i currently have.
#!/bin/bash
set -e
while read p; do
NAME=$p
GIT_COMMIT=`git rev-parse HEAD`
echo $NAME | grep `xargs` main.tf -A 7 | sed -ri '7s/git_commit = ""/git_commit\ = \"'$GIT_COMMIT'"/g'
done < build_name
Why not Input Variables in Terraform?
variable "git_commit" {}
module "core-lambda-function1" {
source = "./lambda"
name = "core-lambda-function"
runtime = "nodejs6.10"
role = "${aws_iam_role.iam_role_for_lambda.arn}"
filename = "../Archive.zip"
source_code_hash = "${base64sha256(file("../Archive.zip"))}"
source_dir = "../"
git_commit = "${var.git_commit}" #### Use variable here.
}
So in your wrapper script, you can update to:
#!/bin/bash
set -e
GIT_COMMIT=$(git rev-parse HEAD)
terraform plan -var 'git_commit=${GIT_COMMIT}' ...
On Ubuntu 14.04
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.3 LTS
Release: 14.04
Codename: trusty
Building a Yocto Poky image using the fido branch
inherit core-image
IMAGE_FEATURES += "x11-base x11-sato package-management ssh-server-dropbear"
IMAGE_INSTALL += "chromium \
lsb \
kernel-modules \
alsa-utils \
... and I am getting this sort of message
I look like it related to the Chromium recipe /meta-browser/recipes-browser/chromium/chromium_45.0.2454.85.bb which starts as such
include chromium.inc
DESCRIPTION = "Chromium browser"
DEPENDS += "libgnome-keyring"
and I get this message
ERROR: Logfile of failure stored in: /home/joel/yocto/build-fido/tmp/work/cortexa7hf-vfp-vfpv4-neon-poky-linux-gnueabi/chromium/45.0.2454.85-r0/temp/log.do_configure.28622
Log data follows:
| DEBUG: Executing python function sysroot_cleansstate
| DEBUG: Python function sysroot_cleansstate finished
| DEBUG: Executing shell function do_configure
| Updating projects from gyp files...
| Package xkbcommon was not found in the pkg-config search path.
| Perhaps you should add the directory containing `xkbcommon.pc'
| to the PKG_CONFIG_PATH environment variable
| No package 'xkbcommon' found
| gyp: Call to 'pkg-config --cflags xkbcommon' returned exit status 1.
| WARNING: exit code 1 from a shell command.
What I have tried
Installed the library
$ sudo apt-get install libxkbcommon-x11-dev
Search for xkbcommon.pc
$ apt-file search xkbcommon.pc
libxkbcommon-dev: /usr/lib/x86_64-linux-gnu/pkgconfig/xkbcommon.pc
pkg-config
joel#linux-Lenovo-G50-70:~/yocto/build-fido$ pkg-config --cflags xkbcommon
<=== Return is EMPTY (?)
joel#linux-Lenovo-G50-70:~/yocto/build-fido$ pkg-config --libs xkbcommon
-lxkbcommon <=== Looks correct
Added PKG_CONFIG_PATH
$ PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/lib/x86_64-linux-gnu/pkgconfig/
$ export PKG_CONFIG_PATH
$ env | grep PKG
PKG_CONFIG_PATH=:/usr/lib/x86_64-linux-gnu/pkgconfig/
but I am still getting the same message when running bitbake
Any suggestions?
Find xkbcommon
$ find /usr/lib/ -name *xkbcommon*
/usr/lib/x86_64-linux-gnu/libxkbcommon.so
/usr/lib/x86_64-linux-gnu/libxkbcommon.so.0.0.0
/usr/lib/x86_64-linux-gnu/libxkbcommon-x11.so.0.0.0
/usr/lib/x86_64-linux-gnu/libxkbcommon-x11.a
/usr/lib/x86_64-linux-gnu/libxkbcommon.a
/usr/lib/x86_64-linux-gnu/libxkbcommon-x11.so.0
/usr/lib/x86_64-linux-gnu/libxkbcommon-x11.so
/usr/lib/x86_64-linux-gnu/pkgconfig/xkbcommon.pc
/usr/lib/x86_64-linux-gnu/pkgconfig/xkbcommon-x11.pc
/usr/lib/x86_64-linux-gnu/libxkbcommon.so.0
In this case, it was the chromium recipe that failed to find libxkbcommon. As the error occurred when building a recipe for the target system, we need to tell the build system that the chromium recipe has a dependency on libxkbcommmon.
This can be done by adding
DEPENDS += "libxkbcommon"
to the chromium recipe.
It's worth noting, that libxkbcommon quite likely is an optional dependency, and in that case, it should be handled by a suitable PACKAGECONFIG. (See PACKAGECONFIG in ref.manual).
Note: I've never built chromium myself, thus I'd prefer to not suggest any suitable PACKAGECONFIG.
I think the Chromium_45 recipe is taken down since the last time I saw it (don't see it anymore).
Anyway, this is what I did to Chromium_40.
I have disabled Wayland (ozone-wayland in Chromium) so that it will only use x11.
In local.conf, I added
CHROMIUM_ENABLE_WAYLAND = "0"
By doing this, I will bypass CHROMIUM_WAYLAND_DEPENDS = "wayland libxkbcommon"
CHROMIUM_X11_DEPENDS = "xextproto gtk+ libxi libxss"
CHROMIUM_X11_GYP_DEFINES = ""
CHROMIUM_WAYLAND_DEPENDS = "wayland libxkbcommon"
CHROMIUM_WAYLAND_GYP_DEFINES = "use_ash=1 use_aura=1 chromeos=0 use_ozone=1"
python() {
if d.getVar('CHROMIUM_ENABLE_WAYLAND', True) == '1':
d.appendVar('DEPENDS', ' %s ' % d.getVar('CHROMIUM_WAYLAND_DEPENDS', True))
d.appendVar('GYP_DEFINES', ' %s ' % d.getVar('CHROMIUM_WAYLAND_GYP_DEFINES', True))
else:
d.appendVar('DEPENDS', ' %s ' % d.getVar('CHROMIUM_X11_DEPENDS', True))
d.appendVar('GYP_DEFINES', ' %s ' % d.getVar('CHROMIUM_X11_GYP_DEFINES', True))
}
P.S.: One more thing I found weird is use-egl.
PACKAGECONFIG[use-egl] = ",,virtual/egl virtual/libgles2" is overrided with PACKAGECONFIG[use-egl] = "" so I have removed PACKAGECONFIG[use-egl] = "" from chromium.inc
PACKAGECONFIG ??= "use-egl"
# this makes sure the dependencies for the EGL mode are present; otherwise, the configure scripts
# automatically and silently fall back to GLX
PACKAGECONFIG[use-egl] = ",,virtual/egl virtual/libgles2"
# Additional PACKAGECONFIG options - listed here to avoid warnings
PACKAGECONFIG[component-build] = ""
PACKAGECONFIG[disable-api-keys-info-bar] = ""
PACKAGECONFIG[ignore-lost-context] = ""
PACKAGECONFIG[impl-side-painting] = ""
PACKAGECONFIG[use-egl] = ""
PACKAGECONFIG[kiosk-mode] = ""
This is in my Makefile.core.def:
...
...
module = {
name = mymod;
common = net/mymod.c;
};
...
...
When I tried to build I get:
mv syminfo.lst.new syminfo.lst
cat syminfo.lst | sort | gawk -f /build/boot_project/src/grub/grub2/grub-core/genmoddep.awk > moddep.lst || (rm -f moddep.lst; exit 1)
grub_efi_get_variable in mymod is not defined
make[5]: *** [moddep.lst] Error 1
mymod.c has "#include <grub/efi/efi.h>" and tries to use "grub_efi_get_variable" function. I see that in syminfo.lst
> more syminfo.lst
...
undefined mymod grub_efi_get_variable
...
Can someone shed a light on the error and how to fix?
Thanks,
P.S I edited Makefile.core.def and Makefile.core.am in /build/boot_project/src/grub/grub2/grub-core/ to include my module and ran autogen.sh in /build/boot_project/src/grub/ to regenerate Makefile.in, then I ran dmake in /build/boot_project/src/grub/
Configuration was ran with --with-platform=efi. Anyhow, I noticed Makefile.core.am has all platform enabled for module mymod. So I edited Makefile.core.def to: module = { name = mymod; common = net/mymod.c; enable = efi}; After re-running autogen.sh, only platform with efi were added to Makefile.core.am and the build works.
I would like to generate a number of files using GNU Make using the following recipe.
ina_as%.dat: ina_driver.m ina_as$(word 1,$(subst _epsi, , %)).m
echo "modelType = '$(word 1,$(subst _epsi, , $*))'; ofile = '$#'; epsi = '$(word 2,$(subst _epsi, , $*))';" | cat - $< | nohup matlab -nodesktop -nosplash
The targets are in a format -- ina_as%d_epsi%.2f.dat (e.g. ina_as1_epsi0.50.dat) and the second prerequisite is ina_as%d.m (e.g. ina_as1.m) (notice, the second part _epsi%.2f missing in the prerequisite file name).
I have tried several combination for the implicit rule ($, $$, $(eval $*) etc.), but it still does not work. I think it could be because Make could not understand the functions ( '$(word 1,$(subst _epsi, , %))' ) in the dependency definition.
There is any way to overcome this problem?
Thanks.
Questions like this come up from time to time. The short answer is that Make simply can't do this in a clean way; the text manipulation statements expand before executing any rule (i.e. before % has any value), and Make doesn't handle wildcards (or regular expressions) very well.
The longer answer is that it can be done, but only by resorting to one kludge or another. If your version of Make supports SECONDEXPANSION, you can do it this way:
.SECONDEXPANSION:
ina_as%.dat: ina_as$$(word 1,$$(subst _, ,%)).m
#echo "modelType = '$(word 1,$(subst _epsi, , $*))'; ofile = '$#'; epsi\
= '$(word 2,$(subst _epsi, , $*))';" | cat - $< | nohup matlab -nodesktop\
-nosplash
If not, you can resort to recursive Make (useful sometimes, no matter what they say):
ina_as%.dat :
#$(MAKE) dummy MODELTYPE=`echo $* | sed "s/_.*//"` EPSI=`echo $* | sed \
"s/.*_epsi//"`
dummy: ina_as$(MODELTYPE).m
#echo "modelType = $(MODELTYPE); ofile = ina_as$(MODELTYPE)_epsi$(EPSI)\
; epsi = $(EPSI);" | cat - ina_as$(MODELTYPE).m | nohup matlab -nodesktop\
-nosplash