cmake & gcc compiles every file every time - gcc

I'm a learning c++ developer writing a game initially on the Mac platform using XCode, but now moving to cross platform by leveraging CMake. So far I can get it compiled on my ickle linux netbook and I'm putting together a dev environment on this machine for on the go coding. However I'm finding that gcc recompiles every file whenever I make a change. Clearly I need some additional configuration to the CMakeLists.txt . My current one is very simple. Like so;
cmake_minimum_required (VERSION 2.8)
set (source
Creature.cpp
DisplayManager.cpp
Engine.cpp
EngineState.cpp
Entity.cpp
GameWorld.cpp
GfxSFML.cpp
Item.cpp
Map.cpp
Position.cpp
Projectile.cpp
ScreenTile.cpp
SquadAI.cpp
Terrain.cpp
UIButton.cpp
UICharPanel.cpp
UIView.cpp
Utility.cpp
Weapon.cpp
fov.cpp
main.cpp
)
find_package (OpenAL)
find_package (OpenGL)
find_package (SFML)
set(CMAKE_CXX_FLAGS "-g -Wall -pg")
add_executable (tractionedge ${source})
target_link_libraries(tractionedge ${SFML_LIBRARY} ${OPENGL_LIBRARY} ${OPENAL_LIBRARY})
I've concentrated so far on C++ as a language rather than build systems by sticking with XCode for everything. My knowledge of Autotools (make?) and Gcc is very limited. How do I have gcc only recompile the changed source?

Are you rerunning cmake every time? If you just modify one source file, you should be able to simply rerun make, and it should rebuild just the one object file before linking. If you rerun cmake, it might mark all of the source files dirty and rebuild everything.
Only rerun cmake if you change the actual list of source files being used, or other major changes like that.

Rebuilding only the modified sources SHOULD be the default behavior. Of course if you change a central header included by nearly all dependent cpp files it'll trigger a nearly complete rebuild. Look at what happens if you only modify one cpp file (adding a comment or alike), if more than that compilation unit is compiling then I propose you to invest more time investigating it eventually giving you my EMail to have a deeper look at the configuration.
Another possibility is that you are compiling under windows and using a 2.8 cmake that has a bug regarding this. Look at a 2.9 version to see if that defect is away then: http://www.mail-archive.com/cmake#cmake.org/msg24876.html

I would rewrite your CMakeLists.txt using glob (maybe move the files in a "src" directory if you have other *.cpp files around) and give your project a name (this sets some important variables):
cmake_minimum_required (VERSION 2.8)
project(TRACTION)
file (GLOB TRACTION_SOURCES *.cpp)
find_package (OpenAL)
find_package (OpenGL)
find_package (SFML)
set(CMAKE_CXX_FLAGS "-g -Wall -pg")
add_executable (tractionedge ${TRACTION_SOURCES})
target_link_libraries(tractionedge ${SFML_LIBRARY} ${OPENGL_LIBRARY} ${OPENAL_LIBRARY})

I also experienced unnecessary rebuilds using cmake and visual studio. The problem is related to an inappropriate x64 configuration parameter: Visual Studio 2008 Unnecessary Project Building

A simple solution in many of these cases is to completely wipe the build tree and regenerate it (and I mean something along the lines of rm -rf build && mkdir build && cd build && cmake -G "Unix Makefiles" ../src, not just make clean)

Related

Understanding roles of CMake, make and GCC

1. cmake is a command from CMake software: preparation for build automation system; make and make install are commands from Make software: build automation system.
2. From reading this post, what I understand is that:
a. This "cmake and make" stuffs actually use g++ / gcc in its implementation. cmake and make stuffs are basically just tools in using g++ / gcc. Is that correct?
b. gcc / g++ are the compiler that do the actual work.
c. So I can just use gcc / g++ directly without using the make and CMake things?
3. According to this stackoverflow answer: CMake takes a CMakeList.txt file, and outputs it to a platform-specific build format, e.g., a Makefile, Visual Studio, etc.
However when I came across this openCV installation :
mkdir release
cd release
cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local ..
It executes cmake command in a directory where there is no CMakeLists.txt file. Can you explain and elaborate on this?
4. The usual steps that I've seen are: cmake, make, sudo make install.
I read this stackoverflow post, what I understand:
(i) make is for building the project.
(ii) make install is to copy the binary / executables to the installed directories.
a. So when we make, where are the result / binary files / executables stored at?
b. If we only run make without make install, does it mean that the files are not generated?
c. I came across this openCV tutorial on using openCV with GCC and CMake. It uses:
cd <DisplayImage_directory>
cmake .
make
Why doesn't it do make install as well?
5. In summary:
CMake takes CMakeList.txt file (which is cross platform) to generate a Makefile (which is specific to a platform).
I can just write Makefile manually and skip the CMake step. but it is better to do with the CMake step because it is cross platform, otherwise I have to rewrite the Makefile again if I change platform.
Make takes Makefile (which is generated by CMake or written manually) as a guide to compile and build. Make basically uses gcc / g++ or other compiler in its work. Make itself is just a tool for the compiler.
Make install put the result / executables into the install path
CMake generates files for other build systems. These can be Makefiles, Ninja files or projects files for IDEs like Visual Studio or Eclipse. The build files contain calls to compilers like GCC, Clang, or cl.exe. If you have several compilers installed, you can choose one.
All three parts are independent. The compiler, the build system and CMake.
It is easier to understand when you have the history. People used their compiler. Over time they added so many flags, that it was cumbersome to type them every time. So they put the calls in a script. From that the build systems (Make, Ninja) evolved.
The people wanted to support multiple platforms, compilers, scenarios and so on and the build system files became hard to maintain and their use was error-prone. That's the reason people invented meta build system that creates the files for the actual build system. Examples are Autotools or CMake.
Yes
CMake does not use your compiler, make does not implement it, but it calls (uses) the compiler.
The CMakeLists.txt file should be in the parent directory of release. The last argument of the CMake call indicates the path where the CMakeLists.txt file is located.
Right, make generates the file in the build directory. In your example from 3. release is the build directory. You can find all the generated files and use them. Installing is optional, especially if you want to develop the software, you are not installing it.
Try writing Makefiles for a large project and you will see how much work it is. But yes, everything in 5 is right.

Building libraries that have ExternalProjects Added using CMake and Xcode

I am trying to use CMake to build a library (which in turn links to other libraries built via CMake). I am having a couple of issues with this process and would love to have some guidance.
For the library, I have a CMakeLists.txt which essentially has
set(SRC srcfile1.cpp srcfile2.cpp)
set(HEADERS srcfile1.h srcfile2.h)
add_library(myLib ${SRC} ${HEADERS})
INSTALL(FILES ${HEADERS} DESTINATION “include/myLib”)
For a “Unix Makefiles” generator, with cmake command
cmake -DCMAKE_INSTALL_PREFIX=. -DCMAKE_BUILD_TYPE=debug -G “Unix Makefiles” ..
I get an appropriate debug library libMyLib.a in build/src. For a release build type, a release build of libMyLib.a is placed in build/src
However, for an Xcode generator, the library is placed in src/Debug. The issues I am facing here are
a. Doing an Archive in Xcode doesn’t seem to have any effect. I don’t see a corresponding Release library being created anywhere
b. I also have a myLibTests target (which uses googles gtest) which is defined in it’s CMakeLists.txt as
set(SRC myTest.cpp)
add_executable(myLibTests ${SRC})
add_dependencies(myLibTests myLib gtest)
set(gTestLib External/src/gTestLib)
target_link_libraries(myLibTests myLib gTestLib)
When the cmake generator is “Unix Makefiles” myLibTests build, links and runs fine. gTestLib is placed in External/src/gtest-build/. But, when it is “Xcode”, it can’t find the gTest libraries. because, the library is placed in External/src/gTestLib/Debug(or Release) as the case may be (and the above path set in CMakeLists.txt is no longer valid). I am not sure how to set it’s path appropriately in the above set statement.
Since, debug/release is controlled in Xcode (and the configure process in CMake is unaware of this), I assume there isn’t a way to separate the release/debug builds of the gTestLib and also have CMake configure Xcode to pick up the appropriate version while building myLib i.e if I do a debug build of myLib it also does a debug build of gTest and links to it (similar for release builds)? Alternatively, is it possible to configure Xcode to place both the Debug and Release builds in the same location (and then hardcode it’s path above)? What would be the best way of configuring this?
Thanks

Using local makefile for CLion instead of CMake

Is there a way to configure CLion to use a local makefile to compile code, rather than CMake? I can't seem to find the way to do it from the build options.
Update: If you are using CLion 2020.2, then it already supports Makefiles. If you are using an older version, read on.
Even though currently only CMake is supported, you can instruct CMake to call make with your custom Makefile. Edit your CMakeLists.txt adding one of these two commands:
add_custom_target
add_custom_command
When you tell CLion to run your program, it will try to find an executable with the same name of the target in the directory pointed by PROJECT_BINARY_DIR. So as long as your make generates the file where CLion expects, there will be no problem.
Here is a working example:
Tell CLion to pass its $(PROJECT_BINARY_DIR) to make
This is the sample CMakeLists.txt:
cmake_minimum_required(VERSION 2.8.4)
project(mytest)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")
add_custom_target(mytest COMMAND make -C ${mytest_SOURCE_DIR}
CLION_EXE_DIR=${PROJECT_BINARY_DIR})
Tell make to generate the executable in CLion's directory
This is the sample Makefile:
all:
echo Compiling $(CLION_EXE_DIR)/$# ...
g++ mytest.cpp -o $(CLION_EXE_DIR)/mytest
That is all, you may also want to change your program's working directory so it executes as it is when you run make from inside your directory. For this edit: Run -> Edit Configurations ... -> mytest -> Working directory
While this is one of the most voted feature requests, there is one plugin available, by Victor Kropp, that adds support to makefiles:
Makefile support plugin for IntelliJ IDEA
Install
You can install directly from the official repository:
Settings > Plugins > search for makefile > Search in repositories > Install > Restart
Use
There are at least three different ways to run:
Right click on a makefile and select Run
Have the makefile open in the editor, put the cursor over one target (anywhere on the line), hit alt + enter, then select make target
Hit ctrl/cmd + shift + F10 on a target (although this one didn't work for me on a mac).
It opens a pane named Run target with the output.
Newest version has better support literally for any generated Makefiles, through the compiledb
Three steps:
install compiledb
pip install compiledb
run a dry make
compiledb -n make
(do the autogen, configure if needed)
there will be a compile_commands.json file generated
open the project and you will see CLion will load info from the json file.
If you your CLion still try to find CMakeLists.txt and cannot read compile_commands.json, try to remove the entire folder, re-download the source files, and redo step 1,2,3
Orignal post: Working with Makefiles in CLion using Compilation DB
To totally avoid using CMAKE, you can simply:
Build your project as you normally with Make through the terminal.
Change your CLion configurations, go to (in top bar) :
Run -> Edit Configurations -> yourProjectFolder
Change the Executable to the one generated with Make
Change the Working directory to the folder holding your executable (if needed)
Remove the Build task in the Before launch:Activate tool window box
And you're all set! You can now use the debug button after your manual build.
Currently, only CMake is supported by CLion. Others build systems will be added in the future, but currently, you can only use CMake.
An importer tool has been implemented to help you to use CMake.
Edit:
Source : http://blog.jetbrains.com/clion/2014/09/clion-answers-frequently-asked-questions/
I am not very familiar with CMake and could not use Mondkin's solution directly.
Here is what I came up with in my CMakeLists.txt using the latest version of CLion (1.2.4) and MinGW on Windows (I guess you will just need to replace all:
g++ mytest.cpp -o bin/mytest by make if you are not using the same setup):
cmake_minimum_required(VERSION 3.3)
project(mytest)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")
add_custom_target(mytest ALL COMMAND mingw32-make WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
And the custom Makefile is like this (it is located at the root of my project and generates the executable in a bin directory):
all:
g++ mytest.cpp -o bin/mytest
I am able to build the executable and errors in the log window are clickable.
Hints in the IDE are quite limited through, which is a big limitation compared to pure CMake projects...

How to properly link Visual Studio project with OpenCV (superpack) using CMake

I am currently digitizing old VHS cassettes. For post-processing, I would like to implement a custom algorithm with C++ & OpenCV. I have already implemented a promising prototype in Matlab, but it can only process single images (reading / writing video files is not possible in my version (R2010a); also, Matlab is far too slow).
Sadly, I am - over and over again - stuck with CMake. Though I wonder ... this can't be so difficult. I have often had problems with CMake, so I will go into a lot of detail here. I hope that you can not only point out to me what I am doing wrong here, but give general advices towards my usage of CMake as well. Maybe I am doing it all wrong, I don't know.
Here is what I've done so far:
I have downloaded the OpenCV 2.3.1 superpack from sourceforge. The superpack contains OpenCV source code, includes and - most importantly - the .lib and .dll files for all major platforms. For this reason, I need not build OpenCV myself. It is already done. I need only use/link it.
I installed (i.e. extracted to) the superpack in C:\dev\vs2010sp1_win32\opencv\2.3.1.
I have renamed C:\dev\vs2010sp1_win32\opencv\2.3.1\OpenCVConfig.cmake.in to OpenCVConfig.cmake.
I have created a folder for my project C:\dev\VhsDejitterizer with the following structure:
VhsDejitterizer/
CMakeLists.txt (A)
src/
CMakeLists.txt (B)
libvhsdejitter/
CMakeLists.txt (C)
vhsdejitter/
util.h
util.cpp
main/
CMakeLists.txt (D)
main.cpp
Here are the contents of the individual CMakeLists.txt files.
/CMakeLists.txt (A)
CMAKE_MINIMUM_REQUIRED(VERSION 2.8)
PROJECT("VhsDejitterizer")
CMAKE_POLICY(SET CMP0015 OLD)
FIND_PACKAGE(OpenCV REQUIRED
NO_MODULE
PATHS "C:/dev/vs2010sp1_win32/opencv/2.3.1"
NO_DEFAULT_PATH)
ADD_SUBDIRECTORY("src")
/src/CMakeLists.txt (B)
ADD_SUBDIRECTORY("libvhsdejitter")
ADD_SUBDIRECTORY("main")
/src/libvhsdejitter/CMakeLists.txt (C)
UNSET(source_files)
FILE(GLOB_RECURSE source_files "*.h" "*.cpp")
ADD_LIBRARY(libvhsdejitter STATIC ${source_files})
TARGET_LINK_LIBRARIES(libvhsdejitter ${OpenCV_LIBS})
UNSET(source_files)
/src/main/CMakeLists.txt (D)
UNSET(source_files)
FILE(GLOB_RECURSE source_files "*.h" "*.cpp")
ADD_EXECUTABLE(main ${source_files})
TARGET_LINK_LIBRARIES(main libvhsdejitter ${OpenCV_LIBS})
UNSET(source_files)
Configuring and generating the Visual Studio .sln (...) files works well. In fact, I am not getting a single warning or error:
Configuring done
Generating done
However, my attempt to build the 'main' project in Visual Studio fails:
1>------ Build started: Project: main, Configuration: Debug Win32 ------
1>Build started 04.04.2012 14:38:47.
1>InitializeBuildStatus:
1> Touching "main.dir\Debug\main.unsuccessfulbuild".
1>CustomBuild:
1> All outputs are up-to-date.
1>ClCompile:
1> main.cpp
1>LINK : fatal error LNK1104: cannot open file '#CMAKE_LIB_DIRS_CONFIGCMAKE#/libopencv_gpu.so.#OPENCV_VERSION##OPENCV_DLLVERSION##OPENCV_DEBUG_POSTFIX#.lib'
1>
1>Build FAILED.
1>
1>Time Elapsed 00:00:00.59
========== Build: 0 succeeded, 1 failed, 2 up-to-date, 0 skipped ==========
Further details:
Operating system: Windows 7 Pro 64-bit
IDE: Visual Studio 2010 SP1
CMake version: 2.8.4
Target platform (i.e. what am I compiling/building for): Windows 32-bit
My questions:
How do I successfully build the 'main' project? I.e. how to fix that error?
What are these #VARIABLE_OR_SOMETHING#? I have tried to find out where they come from, and they seem to be set up in OpenCVConfig.cmake. But how are they supposed to work? Are they supposed to be evaluated by Visual Studio at "build-time"? If so, how are they evaluated?
You have probably noticed that I have set up quite a sophisticated folder structure. Do you have any advice on this? How do you organize your libraries and projects? Are there best-practices? Where are they documented?
Thank you and best regards, Robert
These variables are probably related to CMake's configure_file command, which allows you to specify a parameterised template document (typically with the extension ending in .in) and you can substitute CMake variables into the file to create it. They are evaluated at the time of the configure_file call, which happens when running CMake. I think what's happening is that there will be a parent CMake script to the one that you've taken which will configure that file with the contents of those variables and then use it in an add_subdirectory call. I would suggest checking for any readme that describes the top level run process (or any file which defines those variables then substitute them manually).
I have fixed it now. I think I can safely say that the whole mess was not my fault. Sorry for answering my own question.
Here is what I tried first (of course, this did not work for me, but it might work for others):
As Martin Foot pointed out in his answer, the *.in files are templates which are supposed to be filled out with proper values during a CMake configuration. However, I am using the OpenCV superpack, which includes all the binaries. For this reason I have, at no point, performed such a configuration step, because I assumed this would only be necessary if you wanted to compile something.
However, it seems that - even if you're using the superpack with prebuilt binaries - you have to configure the project in order to get your OpenCVConfig.cmake generated. Vadim Pisarevsky has stated that in the OpenCV bug tracker.
For some reason, this didn't work for me. I started up the Cmake GUI as usual, pointed it to the OpenCV directory and hit "Configure". I even hit "Generate" out of desperation. Yet, no OpenCVConfig.cmake appeared.
So I had to go on further ...
This is what actually helped:
In a recently filed bugreport related to OpenCVConfig.cmake, Sergiu Dotenco pointed out "that the currently provided OpenCVConfig.cmake is pretty fragile" etc. etc. Fortunately, Sergio has also provided a custom FindOpenCV.cmake script. By using his script I have finally been able to generate a working Visual Studio solution.
By the way, this is my current top-level CMakeLists.txt:
CMAKE_MINIMUM_REQUIRED(VERSION 2.8)
PROJECT("VhsDejitterizer")
SET(CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake-modules" ${CMAKE_MODULE_PATH})
SET(OPENCV_ROOT_DIR "C:/dev/vs2010sp1_win32/opencv/2.3.1")
#SET(OPENCV_USE_GPU_LIBS ON)
FIND_PACKAGE(OpenCV REQUIRED)
ADD_SUBDIRECTORY("src")
I have installed Sergio's FindOpenCV.cmake script in a new cmake-modules/ subfolder of the project. It is also important to note that I am using (as opposed to my original setup, where I used the "config mode") the minimal FIND_PACKAGE variant ("module mode"). Only if the minimal variant is used, CMake is looking for Find<package-name>.cmake scripts in the CMake module path. See the CMake documentation for FIND_PACKAGE.
I have also found this guide, which is about how to properly use CMake if you're a library developer: http://www.vtk.org/Wiki/CMake/Tutorials/How_to_create_a_ProjectConfig.cmake_file
I have fixed it now. I think I can safely say that the whole mess was not my fault. Sorry for answering my own question.
You fixed it?
I am having the same problem, and I followed your solution but it did not work.
When CMake executed the command
FIND_PACKAGE( OpenCV REQUIRED )
It would output:
One or more OpenCV components were not found:
calib3d
contrib
core
features2d
flann
highgui
imgproc
legacy
ml
objdetect
video
CMake Error at C:/Program Files/CMake 2.8/share/cmake-2.8/Modules/FindPackageHandleStandardArgs.cmake:91 (MESSAGE):
Could NOT find OpenCV (missing: OPENCV_CALIB3D_LIBRARY
OPENCV_CONTRIB_LIBRARY OPENCV_CORE_LIBRARY OPENCV_FEATURES2D_LIBRARY
OPENCV_FLANN_LIBRARY OPENCV_HIGHGUI_LIBRARY OPENCV_IMGPROC_LIBRARY
OPENCV_LEGACY_LIBRARY OPENCV_ML_LIBRARY OPENCV_OBJDETECT_LIBRARY
OPENCV_VIDEO_LIBRARY) (found version "2.3.1")
Finally, I used the commands
include_directories
TARGET_LINK_LIBRARIES
to include all necessary directories or files, and it works, in a awkward way!

CMAKE: Build library and link against it

I'm trying to use cmake (on Linux with GNU make and g++) to build a project with two sub-directories: MyLib and MyApp. MyLib contains source for a static library; MyApp needs to link against that library. I'm trying to build on Linux with generated makefiles using the following CMakeLists.txt:
cmake_minimum_required (VERSION 2.6)
project (MyProj)
include_directories (MyLib)
file(GLOB MyLibSrc MyLib/*.cpp)
add_library(MyLibrary STATIC ${MyLibSrc})
file(GLOB MyAppSrc MyApp/*.cpp)
add_executable(MyApplication ${MyAppSrc})
target_link_libraries(MyApplication MyLibrary)
This 'almost' works. It fails at link time because while it generates libMyLibrary.a - it is in the root. When I add:
link_directories(${MyProj_BINARY_DIR})
it makes no difference.
I've got a few (inter-linked) questions:
What's the best way to coerce cmake into building my library and executable into a 'staging directory' — say MyStage — to keep targets separate from source?
How do I convince cmake to link the application against the library?
If I wanted to build a debug and a release version, what's the best way to extend my cmake scripts to do this — making sure that the debug application links against the debug library and the release application against the release library?
I'm a relative newcomer to cmake. I've read what I can find on the web, but find myself struggling to get my library to link with my executable. This sort of a configuration, to my mind, should be quite common. An example from which to crib would be very helpful, but I've not found one.
Well, it is better to read this example and do exactly as suggested.
cmake_minimum_required (VERSION 2.6)
project (MyProj CXX)
add_subdirectory(MyLib)
add_subdirectory(MyApp)
Then for each subdirectory specified, CMakeLists.txt files are created
MyLib\CMakeLists.txt
file(GLOB SRC_FILES *.cpp)
add_library(MyLib ${SRC_FILES})
MyApp\CMakeLists.txt
file(GLOB SRC_FILES *.cpp)
add_executable(MyApp ${SRC_FILES})
target_link_libraries(MyApp MyLib)
Use "out of the source build". Make a directory used only for build and while in it, call
cmake <path to the sources, it may be relative>
Either use
link_directories(${MyProj_BINARY_DIR}/MyLib)
or make CMakeLists.txt in each subdirectory - that would be better for project larger than very small.
This is a bit tricky, check out CMAKE_BUILD_TYPE in the docs (you can set it and/or "if" by it). You can also set it from command line:
cmake -DCMAKE_BUILD_TYPE=Debug
I've discovered the 'optimal' solution to (1)... so, thought I should post it here:
SET(CMAKE_ARCHIVE_OUTPUT_DIRECTORY MyStage)
SET(CMAKE_RUNTIME_OUTPUT_DIRECTORY MyStage)
The thing that confused me previously is that static libraries are not considered a LIBRARY by Cmake - they're considered to be ARCHIVEs.
Do not add libraries and executables in the root Cmakelists.txt. Add these libraries and executables in Cmakelists.txt of subdirectories.

Resources