How to enable multiline logs instead of single-line progress-logs - makefile

Here is a sample output of my cmake:
2017/10/27 07:51:46 Platform overridden to 'RHEL5_64'
-- cmake version: 3.2.3
-- Configuring done
-- Generating done
-- Build files have been written to: /local/home/etc
[3/3] Linking CXX shared library libsample_z.so
The last line actually shows progress (as indicated by [3/3]) and thus overwritten in-place; so I cannot see all the logs (i.e the messages correspond to [1/3] and [2/3]). I want cmake to print all logs to stay on its own line, like:
Linking CXX shared library libsample_x.so
Linking CXX shared library libsample_y.so
Linking CXX shared library libsample_z.so
What can be done in cmake to log like this?

The "problem" with ninja is, that it automatically detects if you are running from a shell where it can replace the progress output in line. And there are - as of October 2017 - no command line switches or environment variables to change this behavior.
Since it checks for the console's output buffer, I found that piping the output on my Windows console somewhere else does show multi-line outputs again. So i used the following pipe command:
cmake -G "Ninja" ..
cmake --build . > CON
NOTE: That will only work if you don't have this call inside a script that needs the stdout output itself again for piping it e.g. into a log file. Meaning the output is no longer on stdout after this pipe command.

Similar to the accept answer, another option is to tick ninja into not seeing the terminal by piping to cat.
ninja | cat -
This will get you multi line output, but you will loose any console coloring as well.

From man ninja:
-v show all command lines while building

Related

How to get CMake variable of package in shellscript

I want to find the Qt5WaylandClient_PRIVATE_INCLUDE_DIRS variable which is set by the Qt5WaylandClient package. How can I get it from the shell (dash)? Something like this:
cmake -find_package(Qt5WaylandClient) -get_variable Qt5WaylandClient_PRIVATE_INCLUDE_DIRS
or
cmake -file path/to/my/CMakeLists.txt -get_variable Qt5WaylandClient_PRIVATE_INCLUDE_DIRS
CMake does have a --find-package command line option, but it is not well supported, nor well-documented. There is an answer describing its functionality here, but that's probably not what you're looking for.
Initially, it appears you could just run cmake in script mode, using -P, on a CMake file containing your find_package(Qt5WaylandCleint) command and print its variables to the console.
cmake -P MyFindQt5WaylandClient.cmake
However, running find_package() outside the confines of a CMake project does not work. It yields several errors because CMake doesn't know anything about the system or your target language. So, you must create a minimal CMake project, then run find_package(). Something like this CMakeLists.txt file should work:
cmake_minimum_required(VERSION 3.16)
project(MyProj)
find_package(Qt5WaylandClient REQUIRED)
# Print the desired variable.
message("${Qt5WaylandClient_PRIVATE_INCLUDE_DIRS}")
You can then run cmake from the command line, and this will print the Qt5WaylandClient_PRIVATE_INCLUDE_DIRS variable to the console. You can use the -S and -B command line options to specify the CMake source and binary directories, respectively.
cmake -S . -B build

Is it possible to dump the AST while building an Xcode project?

I've been doing some work on analyzing Swift projects using their ASTs, and I would like to know if it is possible to generate it somehow when building a Swift project with Xcode.
Right now, I am able to print the AST on the terminal when running the swiftc -dump-ast command for single files and simple projects. However, it becomes harder when using it for more complex projects.
For this reason, I'd like to use xcode. I already tried to pass the -dump-ast flag to the compiler in Build Settings > Swift Compiler - Custom Flags > Other Swift Flags. The flag was indeed passed to the compiler (the output does report calling swiftc with the -dump-ast flag when building). I tried to build the project both with xcode and through the xcodebuildcommand below, but neither dumped the ast.
xcodebuild -target 'CompilingTest.xcodeproj' -scheme 'CompilingTest' -
configuration "Debug" -sdk iphoneos -arch "armv7"
CONFIGURATION_BUILD_DIR="TestBuild" ONLY_ACTIVE_ARCH=NO
Now, I'm reasoning that either Xcode's build process redirects swiftc's output to some file, or it silences it somehow. Any thoughts?
Any help would be greatly appreciated.
Dumping the AST of your app is not possible to do from only changing
Xcode build settings. The main reason for this is that Xcode is making a
lot of decisions about compiler flags to pass to swiftc which
are not all compatible with dumping the AST and you cannot stop Xcode from
doing this.
But it is possible to do outside of Xcode with Xcode's help (or in
Xcode using my script mentioned below). To do this you'll need to
capture the swiftc command Xcode runs for your project, and then
change it a bit to dump the AST.
First, build your project, then go to the Report Navigator in Xcode (the
last tab in the Navigator pane on the left). From here either save your
entire build log with the save button across the top, or copy the
swiftc command directly from Xcode. You're looking for the command
called "Compile Swift Sources" for your app target (NOTE: this only
contains the compile command for the one target, if you want the AST for
multiple targets you'll need to perform these steps multiple times). If
you can't find this step, you may need to clean your project and compile
again (or look at an older build log).
After you've copied the entire swiftc command from Xcode, you'll need
to head to the command line and change the command a bit. Here's what
you'll need to do:
Remove -emit-dependencies
Remove -emit-module -emit-module-path FILEPATH
Remove -emit-objc-header -emit-objc-header-path FILEPATH
Remove -c (right before -jN and the list of files)
Remove -parseable-output
Add -dump-ast
Append > output.ast 2>&1 to your shell command
Why these?
Numbers 1, 2, 3, and 4 must be removed because when dumping the AST
you cannot also emit another type of file. You can see the logic
around these errors in the open source Swift compiler
here.
Number 4 is a case of this since -c is an alias of -emit-object.
Number 5 makes it easier for you to redirect the AST output while also
stopping swiftc from outputting other information about this
command. If you plan to parse the output of this AST command from
another program instead of redirecting the output in the shell you
might want to leave this option in (you end up getting JSON with it).
Number 6 is what causes the command to output the AST. Depending on what you want to use this output for, you can
also consider using -print-ast which prints a more
class-dump style output vs
the more verbose, classical parsable AST output of -dump-ast.
Number 7 is simple shell redirection so that you can redirect the
(probably huge) output of your AST to a single file. Change
output.ast to whatever file you want. You'll need 2>&1 because the
AST is dumped to stderr instead of stdout. The order also matters
here.
Also note that Xcode's build log escapes spaces but not other characters
that shells may not like. For example if you have & in any of your
folder / directory paths, you'll have to escape that manually.
If all of this sounds like too much work, I threw together a script to
do this processing for you and you can set it up in Xcode. You can find
it on GitHub.

Is there an easy way to COLOR-CODE the compiler outputs?

gcc (or other compilers) often generate huge text output and it's very difficult to see where the error is or miss warnings. I've done some search but havn't found a clean simple solution to color code the compiler output (so for instance warnings are yellow, errors are red, etc...)
Gcc 4.9 seems to have added this feature via the -fdiagnostics-color flag:
here's an alternative if you are looking for something very simple:
#!/bin/bash -e
make ${#} 2>&1 | perl -wln -M'Term::ANSIColor' -e '
m/Building|gcc|g++|\bCC\b|\bcc\b/ and print "\e[1;32m", "$_", "\e[0m"
or
m/Error/i and print "\e[1;91m", "$_", "\e[0m"
or
m/Warning/i and print "\e[1;93m", "$_", "\e[0m"
or
m/Linking|\.a\b/ and print "\e[1;36m", "$_", "\e[0m"
or
print; '
Just alias your make to this script and make sure it's executable...
Debian and Ubuntu gives the colorgcc package for that purpose.
And I usually run gcc (and make) thru emacs with M-x compile then the messages are colorized.
addenda
GCC 4.9 has a native colorization facility and GCC 6 - released end of April 2016 - (and probably GCC 5 too) is enabling it by default (when stdout is a terminal).
Ok, I'll just leave a notice about my own (python based) tool also :)
It is called Pluggable Output Processor and designed not only to colorize output of one particular program. Here is sample GCC output before:
After:
See colorgcc, a perl script that coulours the gcc output.
How to install and use colorgcc to colorize your gcc compiler output:
At least 3 answers here so far mention colorgcc, but NONE OF THEM EXPLAIN HOW TO INSTALL IT! (And it's not obvious). So, here's how to install the latest version in Ubuntu!
Go here and click "Clone or download" --> "Download Zip". I saved it into "~/Downloads/Install_Files"
Navigate to it in your file browser and right click it and go to "Extract Here." I now have a directory called "~/Downloads/Install_Files/colorgcc-master".
Copy the "colorgcc.pl" script to "/usr/bin/colorgcc" to "install" it (be sure to use the correct directory according to where you extracted it above): sudo cp ~/Downloads/Install_Files/colorgcc-master/colorgcc.pl /usr/bin/colorgcc
Make it executable: sudo chmod +x /usr/bin/colorgcc
Make the "~/bin" directory if it does not yet exist: mkdir ~/bin
*Make symbolic links that point to "/usr/bin/colorgcc" so that whenever you call gcc or g++ it automatically calls colorgcc instead:
ln -s /usr/bin/colorgcc ~/bin/g++
ln -s /usr/bin/colorgcc ~/bin/gcc
(if you ever want to uninstall colorgcc for some reason just delete these symbolic links "~/bin/g++" and "~/bin/gcc", and the Perl script: "/usr/bin/colorgcc" and you're done)
Done!
Here is a sample g++ output now when I call g++ -Wall -std=c++11 time_until_overflow_2.cpp -o time_until_overflow_2:
*Note: making these symbolic links in "~/bin" only works if "~/bin" is in your PATH variable in a location before the folder where the actual gcc and g++ executables are located. To ensure you have "~/bin" in your path you can view the PATH variable contents with: echo $PATH. If you don't see "/home/YOUR_USERNAME/bin" at the beginning of your path, add it with: export PATH=~/bin:$PATH.
References:
See here for more info. and for where I originally learned most of these steps: https://imranfanaswala.wordpress.com/2009/02/02/setting-up-colorgcc/. Thanks Imran Fanaswala!
~GS
you can use GilCC which is a Ruby tool that will convert GCC output to color in real-time. Right now you have two choices: Perl script (colorGCC) or GilCC and if you already work with Ruby you will like GilCC.
Unique to GilCC; GilCC has warning and errors counters and also shows compile time, very handy when you are trying to improve things. Because it is in Ruby it is cross platform. It is flexible and you can add more gems to customize it anyway you want.
The link to the download page is here.
https://github.com/gilmotta/GilCC
Although GCC 4.9 has -fdiagnostics-color option to enable colored outputs to terminals, I have created a tiny tool called 'crror' to get colorized compiler output.
It supports outputs from make as well. I can add colorize patterns for other tools if anyone requires.

How could I save the Clang diagnostics into a file?

I am writing a web server, mainly for practice, and decided to get to learn about make and makefiles. My aim is to automate builds to my vps, using Clang/LLVM to build the software, and save the diagnostics to a text file, for a script to submit to my email address. What I can not seem to achieve is to save the diagnostics to my file.
While my Clang builds successfuly, and generates the diagnostics, and my makefile seems to work, I have failed to redirect the diagnostics, both from inside the makefile, and the command line.
My makefile (working correctly, but slightly modified to accomodate the need to save results):
# Makefile to build Ironman HTTP Server
# We will use the clang frontend from the llvm compiler infrastructure
# for building
# --- targets
Ironman :
clang -o Ironman src/Ironman.c > report
# --- remove binary and executable files
clean:
rm -f Ironman
rm -f report
I suspect that ( I may be terribly wrong here) this happens because clang doesn't really return the diagnostics, it just prints them. I do not know if this is the case, and the Clang user guide doesn't suggest anything like that.
[EDIT]: I played with Clang a bit and saw that on a successful compilation it returns 0. The method that I tested it with is:
$ clang <source_file.c>
$ echo $?
0
This suggests that my theory may be correct, which complicates things :-\
Could somebody point me in the right direction?
Clang, like any other program, outputs diagnostics to stderr. You can redirect stderr to stdout like so:
Ironman :
clang -o Ironman src/Ironman.c > report 2>&1

Linker (ld) fails to produce output file?

I'm getting a strange error while trying to build Chromium on Mac. I've tracked the problem down to the following block of code, which is in a script called adjust_visibility.sh.
NEW_OBJECT="${OBJECT}.new"
ld -o "${NEW_OBJECT}" -r "${OBJECT}" \
-exported_symbols_list /dev/null -keep_private_externs
mv "${NEW_OBJECT}" "${OBJECT}"
The script fails on the last line, where mv claims that the file ${NEW_OBJECT} doesn't exist. ${NEW_OBJECT} should have been created by the linker command right above, but for some reason this isn't happening. The linker doesn't produce the new file, but it also doesn't print any errors or warnings.
I know there's no problem with the linker itself -- when I run similar commands in the terminal they work fine. The problem must have something to do with the environment that the script's executing in.
Does anyone have any idea why this linker command isn't working properly?

Resources