extconf.rb how to define the sources files to use - ruby

I have a little projet of a ruby extension which was organized like this :
./
Rakefile
ext/
mymodule/
extconf.rb
mymodule.rb
mymodule.cpp
source1.h
source1.cpp
source2.h
source2.cpp
Everything worked but I needed to put some of the source files in a directory (I create/modify/test all those source files in a git repos in order to just have to update this directory when my C++ code is tested). So the new organization looks like this :
./
Rakefile
ext/
mymodule/
extconf.rb
mymodule.rb
mymodule.cpp
somesourcesfiles/
source1.h
source1.cpp
source2.h
source2.cpp
My problem now is how to specify to extconf.rb where to find the source files (extconf.rb use source files in current directory by default).
I have tested this in the extconf.rb whitout success:
myfiles = %w{somesourcefiles/source1 somesourcefiles/source2 mymodule}
$srcs = myfiles.map{|i| i + ".cpp"}
I have had an error message saying that the linker couldn't find some ".o" files so I added this:
myfiles = %w{somesourcefiles/source1 somesourcefiles/source2 mymodule}
$srcs = myfiles.map{|i| i + ".cpp"}
$objs = myfiles.map{|i| i + ".o"}
But this didn't work.

See mkmf ignores files in sub-folders when it compiles the C extension
$INCFLAGS << " -I$(srcdir)/somesourcesfiles"
$VPATH << "$(srcdir)/somesourcesfiles"

Related

Yocto Transaction Test, two recipes install the same file

I am adding a custom recipe to my image. It is bases on DBCPPP. This project is built using cmake. My issue only comes to light when I include a new layer meta-swupdate. It seems meta-swupdate alters the kernel in a way that conflicts with dbcppp. My full error is:
Error: Transaction test error:
file /usr/lib/libxml2.so.2.9.10 conflicts between attempted installs of dbcppp-1.0+gitr0+fa8ce17468-r0.cortexa7t2hf_neon and libxml2-2.9.10-r0.cortexa7t2hf_neon
To build and include dbcppp I added the recipe:
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
SRC_URI = "gitsm://github.com/xR3b0rn/dbcppp.git;protocol=https;branch=master"
PV = "1.0+gitr${SRCPV}"
SRCREV = "${AUTOREV}"
DEPENDS += " boost"
S = "${WORKDIR}/git"
inherit pkgconfig cmake
FILES_${PN} += "/usr/lib/xml2Conf.sh /usr/lib/lib*.so.*"
SOLIBS = ".so"
FILES_SOLIBSDEV = ""
INSANE_SKIP_${PN} += "dev-so"
I have read a few other questions and attempted two solutions by adding a do_install_append step.
Change folder permission level install -d 0755 ${D}/usr/lib
Remove the folder rm -rf ${D}/usr/lib
Neither solution worked. I need both libraries in my application and I unsure how to proceed.
Edit: After some further reading I found a suggestion to delete the tmp, cache, and sstate-chace folders. I did this but I receive the same error.
Edit: My local.conf has PACKAGE_CLASSES ?= 'package_rpm' defined. if I remove this I still get a do_rootfs error, but the error message is not helpful.
Edit: In my recipe for dbcppp I have attempted to remove the file in question. This gives me the same error. Makes no difference.
do_install_append() {
rm -rf /usr/lib/libxml2.so.2.9.10
}

do_patch: Function failed: patch_do_patch -- running bitbake

I am using yocto build a linux image that integrates some layers of wlan and a specific wifi chip, but seems like patching of one of the files is failing (details below)
Steps that I did:
Created a working directory
initialized a repo for kernel 4.14.98 via:
repo init -u https://source.codeaurora.org/external/imx/imx-manifest -b imx-linux-sumo -m imx-4.14.98-2.3.3.xml
sync the repo via repo sync
created a tmp folder outside of the current working dir, and download the specific code for wifi via
repo init -u git://codeaurora.org/quic/le/le/manifest.git -b release -m CHSS.LNX_FSLS.1.0-01200-QCAAUTOHOSTHZ.xml –repo-url=git://codeaurora.org/tools/repo.git –repo-branch=caf-stable
repo the sync
copy the meta-qti-connectivity and wlan-opensource folders into the source folder of the working directory
Download the files for a specific wifi chip and copy over meta-qticonnectivity-prop and wlanproprietary into the source folder of the working directory
So now we have added additional 4 directories into the source folder of the working directory
Set up the build environment
Run bitbake core-image-minimal
and I see the following error:
ERROR: wpa-supplicant-git-r0 do_patch: Command Error: 'quilt --quiltrc <working_directory>/linux_image/build/tmp/work/imx8qxpmek-poky-linux/wpa-supplicant/git-r0/recipe-sysroot-native/etc/quiltrc push' exited with 0 Output:
Applying patch 0009-Sync-with-mac80211-next.git-include-uapi-linux-nl802.patch
patching file src/drivers/nl80211_copy.h
...
15 out of 20 hunks FAILED -- rejects in file src/drivers/nl80211_copy.h
Patch 0009-Sync-with-mac80211-next.git-include-uapi-linux-nl802.patch does not apply (enforce with -f)
ERROR: wpa-supplicant-git-r0 do_patch: Function failed: patch_do_patch
I'm new to yocto, and from the looks of it, it seems the patch 0009-Sync-with-mac80211-next.git-include-uapi-linux-nl802.patch didn't apply, but does that mean the issue is with the patch file or the way it's referenced.
This patch file resides in the meta-fsl-bsp-release layer under wpa-supplicant sub directory, which I DID NOT add manually. This sub-directory has a .bbappend file that refers to these patch files via SRC_URI variable, but it doesn't contain any .bb file.
One of the meta layers that I added has wpa-supplicant sub-directory as well but it only has .bb file and no .bbappend.
I'm confused as to how are these two subdirectories different or could they conflict in anyway? Also, shouldn't the subdirectory have both the .bbappend & a corresponding .bb file?
The SRC_URI variable is used to locate the applicable patch files and do_patch isn't needed in the respective .bb file of wpa-supplicant, right?
the respective bb file wpa-supplicant has the following:
SRC_URI = "file://wlan-opensource/wpa_supplicant_8/"
SRC_URI += "file://hostapdconf \
file://supplicantconf \
shouldn't patch files be defined with .patch at the end?
Also, I see in the yocto documentation that the path defined in file:// is relative to the FILESPATH variable, which in the bb file is defined to be "${BSPDIR}/sources:" - not certain of BSPDIR itself but I think it's referring to <top_dir>/sources, but does that mean hostapdconf is supposed to be at <top_dir>/sources? I don't see it there but in other sub directory
To me it looks like you have added a .bbappend for the wpa-supplicant recipe in a version that does not match the recipe. Likely the .bbappend is from one of the meta layers you copied into your source tree manually (meta-qti-connectivity or wlan-opensource).
I'm new to yocto, and from the looks of it, it seems the patch
0009-Sync-with-mac80211-next.git-include-uapi-linux-nl802.patch didn't
apply, but does that mean the issue is with the patch file or the way
it's referenced?
Likely the patch is targetet at a different version of the source file.
The SRC_URI variable is used to locate the applicable patch files and
do_patch isn't needed in the respective .bb file of wpa-supplicant,
right?
Right, if you don't see it explicitly ly in the .bb. The default do_patch task will be used.
the respective bb file wpa-supplicant has the following:
SRC_URI = "file://wlan-opensource/wpa_supplicant_8/" SRC_URI +=
"file://hostapdconf
file://supplicantconf \ shouldn't patch files be defined
with .patch at the end?
The .patch file is probably referenced by name in a .bbappend if you dont see it in the .bb file.

Build error while copying kernel config to the source tree

i am trying to compile the defconfig file bcm2711_defconfig, so i am trying to copy the bcm2711_defconfig file to the kernel source path, kernel-source/arch/arm/configs/
Please let me know how to overcome from the error. below is the recipe file content.
file name: linux-raspberrypi_4.14.bbappend
FILESEXTRAPATHS_prepend := "${THISDIR}/files:"
SRC_URI += "file://bcm2711_defconfig"
do_preconfigure_prepend(){
cp ${WORKDIR}/bcm2711_defconfig ${S}/arch/${ARCH}/configs/
}
KBUILD_DEFCONFIG_raspberrypi4 ?= "bcm2711_defconfig""
build error:
ERROR: linux-raspberrypi-1_4.14.68+gitAUTOINC+8c8666ff6c-r0 do_kernel_metadata: A KBUILD_DECONFIG '/bcm2711_defconfig' was specified, but not present in the source tree
do_kernel_metadata task is executed after do_unpack & before do_patch, so try to copy your defconfig at the end of do_unpack task as below,
do_unpack_append(){
cp ${WORKDIR}/bcm2711_defconfig ${S}/arch/${ARCH}/configs/
}

Automake/Autoconf scripts custom directory

My project's root folder is PROJECT, and there is only one script in PROJECT/scripts/abc.sh, which I want to install it into the directory /usr/share/xxx/abc.sh.
Because it is not install into /usr/bin/abc.sh, so bin_SCRIPTS = abc.sh is incorrect.
Could you please tell me how to write the Makefile.am?
You'll need to define the install directory:
xxxdir = $(datarootdir)/xxx
xxx_SCRIPTS = scripts/abc.sh
More typically as something like:
pkgdata_SCRIPTS = scripts/abc.sh
where automake has already setup pkgdatadir.

ruby native extension: undefined symbol

I'm attempting to create a ruby native extension, but when I run rake which uses ext/example_project/extconf.rb to build my project and run my tests under test/, I get the following error when the tests are run:
./home/jbuesking/.rbenv/versions/2.3.0/bin/ruby: symbol lookup error:
/home/jbuesking/repositories/example_project/lib/example_project/example_project.so: undefined symbol: some_function
I'm pretty sure my files are not being linked correctly and that I need to alter my extconf.rb and/or Rakefile in some way, but I'm not sure how.
I've created a simple repository that demonstrates the issue over on GitHub. It'll fail with the same error if you clone it and run rake from the projects root.
Some additional information:
I used the ruby gem hoe to create the project via sow example_project
The failing function is attempting to call a function defined in the subdirectory ext/example_project/c_example_project. My actual project uses a git submodule from the ext/example_project directory, which in turn sets up the submodule as a subdirectory. The submodule is a c project with a flattened structure (all files in the root directory). Note: That wording may be confusing, but the key point is that there's a nested c project defined at ext/example_project/c_example_project which has methods I'm trying to call.
Let me know if any clarification is needed, and I'll do my best to provide it.
So, there are some interesting issues you have here. By default, mkmf doesn't actually support specifying multiple directories for building sources.
There is a workaround, as seen here (Takehiro Kubo's comment about setting objs):
https://www.ruby-forum.com/topic/4224640
Basically, you construct the $objs global in your extconf.rb file yourself.
Using your github code, here's what I added to the extconf.rb and got to work
extconf.rb
globs = [".", "c_example_project"].map do |directory|
File.join(File.dirname(__FILE__), directory)
end.join(",")
$objs = Dir.glob("{#{globs}}/*.c").map do |file|
File.join(File.dirname(file), "#{File.basename(file, ".c")}.o")
end
Notice I'm actually constructing an absolute path to each of the c sources, for some reason rake-compiler was freaking out if we were just globbing with {.,c_example_project}/*.c, presumably since it's running the extconf.rb file from another directory.
In addition, your tests/c extensions have a few errors in them. Making the following change in example_project.c fixes the test failure:
static VALUE example_project_c_code_function()
{
- return some_function();
+ VALUE _string = rb_str_new2(some_function());
+ int _enc = rb_enc_find_index("UTF-8");
+ rb_enc_associate_index(_string, _enc);
+ return _string;
}
Explanation
Basically even though you're checking the c_example_project.h header in your extconf.rb, you're not actually generating the object file where some_function is defined. So, when linking the final dynamic library that ruby loads up, there's no definition for some_function and you get your error.
I don't have any experience with building native extensions, but from mkmf source code it looks like you can only specify one source directory. I moved both files from c_example_project to the parent directory and everything was linked properly. I think that is how you should do it. All common gems (like pg, nokogiri etc) have such code structure, all *.c and *.h files are in one directory.
You can always create Makefile yourself, but that would require too much work to maintain.
PS. Your project compiled successfully, but you there is segmentation fault, because you should return proper ruby string object in some_function and not pointer to a string.
photoionized has a great answer, but $obj can be an array instead of an Enumerator.It seems to be okay to simply use an absolute path.
$objs = Dir.glob([".c", "libfoobar/*.c"], base: __dir__)
.map { |f| File.expand_path(f, __dir__) }
.map { |f| f.sub(/\.c$/, ".o") }

Resources