CLion custom compiler: Makefile parser says "No compilation commands found" - macos

I'm trying to get a custom compiler working in CLion and having a bear of a time. Can anyone help me find out what I'm doing wrong? I have the full code on Github.
What I have
The command line tools are all behind the same executable named mpw. So the C compiler is behind mpw SC, the linker is behind mpw link. There's also a tool named Rez to add some metadata to the executable, but I'm fine if CLion just ignores that.
I'm using a make file to do the actual build.
I've created a custom compiler definition YAML and selected it in CLion's project settings. I tried to follow the Jetbrains docs [1] [2] but couldn't find out what code insight target name to use (It eventually compiles for 68000 CPU, old MacOS, anyone know where I can find a list of allowed clangd target names?).
The Makefile works when I call make clean or make all from command line.
Problem
When I open the folder in CLion, it tries to parse the Makefile and reports:
(x) Analysing makefile
(x) No compilation commands found
Goal
Get CLion to see all my code (including system headers at ~/mpw/Interfaces/CIncludes) so I can use its code navigation to auto-complete code. Refactoring would be nice too.
Get CLion to understand my Makefile so I can build using the "hammer" icon inside CLion.
Custom Compiler Definition
compilers:
- description: "MPW SC"
match-sources: ".*\\.c"
match-language: "C"
match-compiler-exe: "(.*/)?mpw SC"
code-insight-target-name: mpw
include-dirs:
- ${user-home}/mpw/Interfaces/CIncludes
defines-text: "
#define __MRC__ 0x0700
#define OLDROUTINENAMES 1
"
Makefile
SOURCES=SillyBalls.c
RFILES=SillyBalls.r Size.r
EXECUTABLE=SillyBalls
MPW=~/Programming/mpw/build/bin/mpw
RINCLUDES=~/mpw/Interfaces/RIncludes
LDFLAGS =-w -c 'SILB' -t APPL \
-sn STDIO=Main -sn INTENV=Main -sn %A5Init=Main
LIBRARIES={Libraries}Stubs.o \
{Libraries}MacRuntime.o \
{Libraries}IntEnv.o \
{Libraries}Interface.o \
{Libraries}ToolLibs.o \
{CLibraries}StdCLib.o
TOOLBOXFLAGS=-d OLDROUTINENAMES=1 -typecheck relaxed
OBJECTS=$(SOURCES:%.c=obj/%.o)
all: prepass bin/$(EXECUTABLE)
prepass:
mkdir -p obj bin
bin/$(EXECUTABLE): $(OBJECTS)
$(MPW) link $(LDFLAGS) $(OBJECTS) $(LIBRARIES) -o $#
Rez -rd $(RFILES) -o $# -i $(RINCLUDES) -append
obj/%.o : %.c
$(MPW) SC $(TOOLBOXFLAGS) $< -o $#
clean:
rm -rf bin obj

Thanks to #JohnBollinger for getting me on the right track:
CLion is apparently not smart enough to recognize $(MPW) SC as mpw SC. If I change it to
CC="~/Programming/mpw/build/bin/mpw SC"
CLion is happy, but of course there is no file named mpw SC.
So my solution was to create a shell script sc.sh:
#!/bin/zsh
~/Programming/mpw/build/bin/mpw SC $#
and then set
CC=./sc.sh
and
match-compiler-exe: "(.*/)?sc.sh"
and then use ./sc.sh everywhere where I used to have $(MPW) SC
CLion recognizes it, starts indexing the system headers, and the "hammer" icon triggers a build all just as desired.

Related

How to see exactly what make is doing

Ive got some large make files for a third party project that are not building due to linker issues.
From looking at the make files, I think it should be executing something like:
LIBS = -lm
CC = gcc
bin = bin
myapp: $(bin)/main.o $(bin)/other.o $(bin)/etc.o
$(CC) $(bin)/main.o $(bin)/other.o $(bin)/etc.o $(LIBS) -o myapp
gcc bin/main.o bin/other.o bin/etc.o -lm -o myapp
Instead from the error it seems to be failing on something like: It also didn't put any of the .o files in the expected bin/ location, but just left them in the source directory...
cc main.o -o myapp
But I cant locate anywhere that might come from. Is there some way to get some kind of stacktrace through the make files?
I am aware of -n and -d, but neither seems to tell me what target line and file yeilded that command, or which series of targets led there and the values of any $() expansions (The one im expecting is the only myapp: I can find in any of the makefiles...)
Check out the --debug option. From my manpage:
--debug[=FLAGS]
Print debugging information in addition to normal processing. If the
FLAGS are omitted, then the behavior is the same as if -d was specified.
FLAGS may be a for all debugging output (same as using -d), b for basic
debugging, v for more verbose basic debugging, i for showing implicit
rules, j for details on invocation of commands, and m for debugging
while remaking makefiles.
remake is a very good choice but in a pinch something like the following (saved as debug.mk) can be a good help too. It won't tell you as much as remake but it might tell you enough to start with.
# Use as: MAKEFILES=debug.mk make
OLD_SHELL := $(SHELL)
ifneq (undefined,$(origin X))
override X = -x
endif
SHELL = $(if $#,$(warning Running $#$(if $<, (from: $<))$(if $?, (newer: $?))))$(OLD_SHELL) $(X)
You can print out the other automatic variables there too if you wanted to see a bit more about what was going on.

make command with math library

I need to just run the make command for makefile. But when I run this make command I get the error that "undefined reference to `log'" because I know this fact that it doesn't include math Library and we have to include at runtime. I know that if I run this using gcc comiler then i can write -lm at the end, it will include math library. My problem is I need to run it using make command that is- make lu.
In this if I write make lu -lm it is not linking math library. Please help
Using this link How to use LDFLAGS in makefile
I updated my make file but still same problem persists. Please Help.
SHELL=/bin/sh
BENCHMARK=ep
BENCHMARKU=EP
include ../config/make.def
OBJS = ep.o ${COMMON}/c_print_results.o ${COMMON}/c_${RAND}.o \
${COMMON}/c_timers.o ${COMMON}/c_wtime.o
include ../sys/make.common
LDLIBS=-lm
LDFLAGS=-lm
${PROGRAM}: config ${OBJS}
${CLINK} ${CLINKFLAGS} -o ${PROGRAM} $(LDFLAGS) $(LOADLIBES) ${OBJS} ${C_LIB}
ep.o: ep.c npbparams.h
${CCOMPILE} ep.c
clean:
- rm -f *.o *~
- rm -f npbparams.h core
Why does your makefile refer to all sorts of variables that don't exist, like LOADLIBES, C_LIB? Why do you set variables that you never use, like LDLIBS?
The reason it doesn't work is that you're putting the library reference in the LDFLAGS variable, which comes early in your link command before any of your object files. So when the linker goes to link in the math library it thinks that it's not needed because nothing is using it yet.
You have to put libraries at the end of the link line.
Since you already have the C_LIB variable at the end which you are not using, if you add:
C_LIB = -lm
then it should work.

Library Linking

I am trying to learn to C programing using Zed Shaw's Learn C the hard way. I have been working on ex26 where we create a program "devpkg" used to install software. This exercise requires installing Apache Portable Runtime library. After writing the code for this exercise I could not get program to compile using the following makefile:
PREFIX?=/user/local
CFLAGS=-g -Wall -I${PREFIX}/apr/include/apr-1 -I{PREFIX}/apr/include/apr-util-1
LDFLAGS=-L${PREFIX}/apr/lib -lapr-1 -pthread -laprutil-1
all: devpkg
install: all
install -d${DESTDIR}/${PREFIX}/bin/
install devpkg ${DESTDIR}/${PREFIX}/bin/
clean:
rm -f *.o
rm -f devpkg
rm -f *.dSYM
This makefile did not seem to work as when I used "$make devpkg" not all of the APR library functions were declared. As a side note I am running this on a Ubuntu virtual machine. A solution given in the text says to alter a config file and then "run ldconfig" to help the linker find the appropriate library.
I do not understand the man page for ldconfig well enough to correctly utilize the function. How do run ldconfig correctly?
Also after some digging I found a reference that using "LDLIBS" instead of "LDFLAGS" in the makefile fixed the problem. I altered the makefile and the program compiled.
What is the difference between "LDFLAGS" and "LDLIBS" that allowed the C compiler to correctly link to the APR library? Is there a handy list of commands somewhere that can help me better understand how a makefile is correctly generated?
Thanks for your time.
From the GNU Make Manual, section 10.2 Catalogue of Implicit Rules:
Linking a single object file
n is made automatically from n.o by running the linker (usually called ld) via the C compiler. The precise recipe used is '$(CC) $(LDFLAGS) n.o $(LOADLIBES) $(LDLIBS)'.
As you can see, LDFLAGS comes before your object file and LDLIBS after. Sometimes that order can matter - apparently it does in your case.
Editorial note: While it might sometimes be convenient to use make's implicit rule support, it almost always ends up more confusing down the road. I'd urge you to write a complete makefile - it'll help you understand what's going on better and hopefully avoid this sort of problem in the future.
I just wanted to add this answer as an alternative to changing "LDFLAGS" to "LDLIBS". The above solution did work in my case but I found an alternative (though less direct) solution before I saw this thread which others may find useful or interesting. When compiling I was seeing lots of "undefined reference" errors e.g.:
/MyCode/LCTHW/devpkg/devpkg.c:18: undefined reference to `apr_pool_initialize'
After much trial and error, I changed the makefile thus (still using LDFLAGS):
CC=gcc
PREFIX?=/usr/local
CFLAGS=-g -Wall -I$(PREFIX)/apr/include/apr-1 -I$(PREFIX)/apr/include/apr-util-1
LDFLAGS=-L$(PREFIX)/apr/lib -lapr-1 -laprutil-1 -pthread
OBJECTS=bstrlib.o db.o shell.o commands.o devpkg.o
all: devpkg
devpkg: $(OBJECTS)
$(CC) $(CFLAGS) $(OBJECTS) -o devpkg $(LDFLAGS)
install: all
install -d $(DESTDIR)/$(PREFIX)/bin/
install devpkg $(DESTDIR)/$(PREFIX)/bin/
clean:
rm -f *.o
rm -f devpkg
rm -rf *.dSYM
I then had to add a .conf file to /etc/ld.so.conf.d containing the path to the apr libraries, namely
/usr/local/apr/lib
And then run
sudo ldconfig
so the system would pick up the new .conf file and so know where to find the library. From what I have read, it seems this last step was necessary because the library wasn't stored in /usr/local/lib. If I remove the .conf file and re-run ldconfig to update, the program compiles but then fails to find the libraries at run-time (whether compiled with my makefile or OP's).
While I don't fully understand my solution, it at least allowed me to compile and run the program with no errors. Hopefully this solution will be of interest to others, and maybe somebody more knowledgeable will be able to explain in more detail why it works.

Makefile - compiling back and forth

Following is the directory structure of my project:
expt-main
---------
Makefile_main
/ \
subdir-1 subdir-2
-------- --------
Makefile_1 Makefile_2
mod_codeA.f90 mod_code1.f90
mod_codeB.f90 mod_code2.f90
mod_codeC.f90 mod_code3.f90
Makefile_main:
export
SHELL = /bin/sh
F90 = mpxlf95
SRCDIRS = $(subdir-1) $(subdir-2)
all:
#for DIR in ${SRCDIRS} ;
do \
back=`pwd`; \
cd $$DIR ;\
$(MAKE) ; status=$$? ; \
if [ $$status != 0 ] ; then \
echo "Exit status fro make was $$status" ; exit $$status ; \
fi ; \
cd $$back ; \
done
-------------------------------------------------------------------------------
Makefile-1:
%.o: %.f90
$(F90) $(F90FLAGS) -I$(subdir-2) -c $<
mod_codeA.o: mod_codeC.o $(subdir-2)/mod_code2.o
-------------------------------------------------------------------------------
Makefile-2:
PROG = $(exec)
subdir-1_objs = $(subdir-1)/mod_codeA.o mod_codeB.o mod_codeC.o
all: $(PROG)
$(PROG): $(subdir-2_objs) $(subdir-1_objs) -o $# $(subdir-2_objs) $(subdir-1_objs)
---------------------------------------------------------------------------------
-
I've written the Makefile_main such that it compiles the codes (modules) in subdir-1 first and then the ones in subdir-2 and finally makes the executable. The issue: modules in subdir-1 uses modules from subdir-2 and in similar fashion, modules in subdir-2 uses those in subdir-1. My make is getting failed because the modules being used is in other directory. How to write a makefile which will take care of this issue that is, while compiling modules in subdir-1, whenever it encounters the need for an object file from subdir-2, it should switch to subdir-2 compile the necessary modules and return back to subdir-1 for further action?
If modules in different subdirectories need each other as you say, then this is not a good use of recursive Make.
Do away with Makefile-1 and Makefile-2, and let Makefile_main do all the work. (I can't tell you specifically how to change Makefile-main, since I don't do Fortran, I don't understand Makefile-2, and I don't see any dependency of modules in subdir-2 upon those in subdir-1).
If you want to stick to this directory layout and still keep three separated Makefiles, then you can use compiler flags to instruct the FORTRAN compiler to put module files into a common directory of your choice.
For instance using:
$ gfortran --version
GNU Fortran (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3
Copyright (C) 2011 Free Software Foundation, Inc.
you can use -I and -J flags to instruct the compiler on:
where to search for module files (.mod)
where to put generated module files
That said I think that the suggestion given by Beta to join the Makefiles makes a lot of sense. To know some of the reasons why you should do that you can read this paper.
Finally, as your project seems not to be very large at this stage, I also suggest to take into consideration CMake as a build system, as it possibly provides a more convenient way of specifying dependencies between targets (as well as many other things).

Invalid module format

$insmod helloworld module generates the error message "Invalid module format".
$dmesg outputs:
overflow in relocation type 10 val ffffffff88640070
'hello' likely not compiled with -mcmodel=kernel
The Makefile is a mix of tradition format (using (CC)) and module build system format "make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules".
The system is 1.6.18-194.el5 x86_64. The same Makefile works fine when it is used in i386 machine.
Any idea of what to look into? Thanks.
#Makefile, mix of (CC) and kernel module build system
CFLAGS+=-D"KBUILD_STR(s)=\#s" -D"KBUILD_BASENAME=KBUILD_STR(hello)"
CFLAGS+=-D__KERNEL__ -DMODULE -I$(KERNEL_BUILD_DIR)/include
KERNEL_BUILD_DIR=/lib/modules/2.6.18-194.el5/build
TARGETNAME=hello
BUILD_ALT_DIR=linux
# The main target (note that both library and driver are .ko files
#
all: $(BUILD_ALT_DIR)/$(TARGETNAME).ko
$(BUILD_ALT_DIR)/_$(TARGETNAME).o: hello.o
#echo Linking objects to loadable module
#mkdir -p $(BUILD_ALT_DIR)
#echo $(CURDIR)/$#
#$(LD) -Map=$#.map -r -o $# $^
#echo " LD_D [$#]"
$(BUILD_ALT_DIR)/$(TARGETNAME).ko: $(BUILD_ALT_DIR)/_$(TARGETNAME).o
#rm -f $(BUILD_ALT_DIR)/$(TARGETNAME).o
#echo create Makefile
#$(SHELL) -c 'echo "obj-m := $(TARGETNAME).o" > $(BUILD_ALT_DIR)/Makefile'
#$(SHELL) -c 'echo "$(TARGETNAME)-objs := _$(TARGETNAME).o" >> $(BUILD_ALT_DIR)/Makefile'
#$(SHELL) -c 'echo ".PHONY: `pwd`/$(BUILD_ALT_DIR)/_$(TARGETNAME).o" >> $(BUILD_ALT_DIR)/Makefile'
#$(SHELL) -c 'cd $(BUILD_ALT_DIR); $(MAKE) -C $(KERNEL_BUILD_DIR) M=`pwd`'
#echo " KO_D [$#]"
$(BUILD_ALT_DIR)/%.o: %.c
#echo Compiling C source to object file:
#mkdir -p $(BUILD_ALT_DIR)
# #echo $(CURDIR)/$#
#$(CC) -c -Wall $(CFLAGS) $(CFLAGS) $< -o $#
#echo " CC_D [$#]"
clean:
rm -f $(BUILD_ALT_DIR)/*.o $(BUILD_ALT_DIR)/*.d $(BUILD_ALT_DIR)/core $(BUILD_ALT_DIR)/*.map
hello.c
#include <linux/autoconf.h> // this is needed
#include <linux/init.h>
#include <linux/module.h>
static int hello_init(void)
{
printk(KERN_ALERT "Hello, world\n");
return 0;
}
static void hello_exit(void)
{
printk(KERN_ALERT "Goodbye, cruel world\n");
}
module_init(hello_init);
module_exit(hello_exit);
The dmesg output tells you what is wrong:
'hello' likely not compiled with -mcmodel=kernel
On the x86-64 architecture, all code running in the kernel (including modules) must be compiled with a special flag, which tells the compiler to generate code which will run in the top half of the virtual address space (user mode programs run on the bottom half of the address space).
If I am reading your Makefile correctly, you are compiling the C code outside the kernel build system, and calling into the kernel build system only for the final linking. Do not do this. It is not just the memory model, there are several other flags which must be added when compiling the C source code. These flags can change with the kernel version, or even with the kernel configuration — you do not know, and you are not supposed to have to know, since the kernel build system deals with it all for you.
It is not just -mcmodel=kernel. There are many others, and getting them wrong can and will cause problems.
The fact that you are missing the correct flags is quite visible in your code:
#include <linux/autoconf.h> // this is needed
No, this is not needed. If you need it, you are doing it wrong. Take a look at one of the flags the kernel build system passes to the C compiler:
-include include/linux/autoconf.h
This flags tells the C compiler to always implicitly include the linux/autoconf.h header. Since it is always implicitly included, you never have to include it manually. And since you never include it manually, its location is allowed to change — it moved to generated/autoconf.h and later to linux/kconfig.h, and who knows where else it will end up next.
The fact that it worked at all for you in 32-bit x86 is just luck. You should do it the correct way even on 32-bit.
Second iteration:
All right, you've confirmed that Make builds _helloworld.o and $(BUILD_ALT_DIR)/Makefile. Now could you post the makefile that's in $(KERNEL_BUILD_DIR) (on the x86_64)?
(I'll go out on a limb and suggest that maybe $(BUILD_ALT_DIR) and $(KERNEL_BUILD_DIR) are the same on the i386 but not on the x84_64.)
EDIT: (third iteration)
1400 lines? Most of that in one branch? Half a dozen include directives? Architecture-specific branches? Autoconf?
Are you allowed to modify this nightmare in place, or must you tinker with the higher-level tools like automake? (I ask only because in the latter case it won't do much good to figure out what the problem is, you'll probably have to start over from scratch.)
1) From $(KERNEL_BUILD_DIR), try make M=$(BUILD_ALT_DIR) (whatever $(BUILD_ALT_DIR) is). Verify that that gives you the same error message. If it doesn't then ignore what follows (in this iteration).
2) From $(BUILD_ALT_DIR), try make, and see if it gives the same error. If it does, try replacing the makefile (in $(BUILD_ALT_DIR)) with a trivial makefile like
all:
#echo trivial makefile working
and test it, then try step 1 again. If it just says "trivial makefile working" and doesn't give the error, then things look good.
EDIT: (fourth iteration)
All right, we can't change $(KERNEL_BUILD_DIR)/Makefile. That doesn't necessarily sink us.
"Your suggestion (1) would not work, since $(BUILD_ALT_DIR) is a output directory, no source is in it."
Make does not require the presence of source code, and whether or not this would work (whatever "work" means), it is what your makefile appears to be attempting:
#$(SHELL) -c 'cd $(BUILD_ALT_DIR); $(MAKE) -C $(KERNEL_BUILD_DIR) M=`pwd`'
Try it and see what happens.
"Is there a possibility that x86_64 doesn't support tradition format (CC) for the kernel module?"
I'm not entirely sure what you mean, but we can probably answer your question by experiment.

Resources