When using the Instruments time profiler, I often end up with results that don't make sense. They indicate that time is spent in some part of the file that is either outside of the function being inspected, or contains no executable code (comments, blank lines, etc.). This makes the results often close to useless.
I have tried a number of things to try to solve this:
Close Instruments before recompiling and re-profiling
Clean build, recompile
Restart instruments every time I profile
Updated to Xcode 4.3.2
These occasionally help, but don't always prevent the problem.
What can I do to fix this?
I was running into the same problem on code that I compiled using gcc with the -O3 optimization flag set. Googling around, I learned that Apple's time profiler does not like gcc code that is compiled with optimization flags.
Recompiling my code without the -O3 flag and rerunning the time profiler, hotspots now match the correct lines of my source code. Perhaps this relates to your problem.
Related
I'm consulting a company on how to speed up their builds and I immediately pointed them to precompiled headers and unity builds - the 10 minute full build could easily drop to 2-3 minutes. Luckily CMake 3.16 was recently released and it supports both of those, so I told them to upgrade.
The problem is the following: once they switched from CMake 2.6 to 3.16 the time it took to run CMake jumped from about 20 seconds to more than 10 minutes. Most of the time is spent in the generate phase. It does complete successfully if you gave it enough time and the code compiled successfully with unity builds, but this CMake time is unacceptable.
Here is their setup:
CMake 2.6, old style CMake with global flags/defines/includes - not modern (target-based). Nothing too fancy - no custom commands or build rules & complicated dependencies.
the compiler used is GCC 7 and they generate Makefiles - the OS is CentOS 7, Kernel: Linux 3.10.0-862.14.4.el7.x86_64
around 2000 .cpp files spread across 100 libraries and 600 executables (most of which are test executables with a single .cpp in them)
most .cpp files are gathered/globbed with aux_source_directory - we know not explicitly listing the .cpp files is an anti-pattern, but that's besides the point - I think this is irrelevant since this should happen in the configuration step and not during the generation, correct?
Here is what we observed:
we did a binary search through the different CMake versions and concluded that the huge slowdown happened between 3.15 and 3.16 - exactly when the precompiled header and unity build support was added, but I don't think those features have anything to do with the slowdown - I can't think of a reason for them to have such an impact - it must be some other refactoring or change...
we disabled all tests (that means almost all 600 executables were removed) - slimming down the number of CMake targets from 700 to a bit more than 100 - and the time it took to run CMake dropped significantly, but was still a couple of times longer than with CMake 2.6 for all the 700 targets.
we observed what was happening using strace and there were mostly lstat and access calls along with some reads and writes - but this was endless - seemed like hundreds of operations per second in a very repeatable manner. Also there was constantly an attempt to find libgflags.so which was constantly failing. Unfortunately I don't have such logs right now.
we did a callgrind profile and here is what it looks like after a couple of minutes of running: https://i.imgur.com/Z9AObso.png (the callgrind output file can be found here) - seems like most of the time is spent in ComputeLinkLibs() and getting the full name and definitions of targets and whatnot... Is having 700 targets too much? Is this an exponential problem? Why isn't it an issue with CMake 3.15?
I couldn't find any reports of anyone else having the same problem on the internet... Any ideas what to try next? Perhaps profile using Perf? Try with Ninja as a backend (report of that being faster)?
Great analysis. This will be helpful for the developers of CMake. But I doubt you will find much help here. Please open an issue.
Bonus points if you can provide a minimal example showcasing your problem. You might get this with generating tons of files.
I find that Swift is quick to mark down changes i make as compiler errors in the side panel, and then when i compile, it decides i am right after all. Is Swift just pessimistic about my code?
Can I get the compiler to wait for me to finish the line before declaring it wrong?
There is nothing pessimistic. Xcode uses the same tool - the compiler - to get the errors. However, it usually compiles only one file, using cached compiled objects for the other files. It also doesn't invoke the compiler after every change in your code, so errors that are already fixed can stay there in the side panel.
Building the project fully forces Xcode to refresh the cache and get the current list of errors from the compiler. I do agree that Xcode has many imperfections and this is one of them. When you can't find an error, just rebuild the project.
Note that IDEs for other languages often rebuild the project automatically to solve such problems. This is currently not an option in Swift because it would take too much time.
This definitely takes the cake in terms of being the largest single piece of executable code I have ever seen.
Now it was a bit easier to get this whole thing built on my Mac here (I have been trying to build LLDB on a Linux as well, and currently I'm fighting with linking to Python there), for which I am thankful, but this astoundingly large executable file has me second-guessing myself... Did I do something wrong? What is inside of this monstrous archive?
I did run this:
% otool -TV liblldb-core.a
It produces 1159 lines of output, which puts it around 350+ object files. That sounds about right, I saw the XCode project work its way through about 350 source files.
I guess my question is why does LLDB work this way, why isn't it a bit more lightweight and why doesn't it just link to LLVM and Clang code rather than doing whatever this is? OR, are the contents of this archive already ALL LLDB-specific code? I recognize building a debugger is quite the undertaking, but this is honestly just mind-boggling.
I am aware that compiling at -O3 likely bloats executable file size. I'm not about to go back and recompile this monster though (the computer nearly melted with smcFanControl reporting CPU core temps as high as 106 degrees C).
Update: I sort of chronicled some further learning I just did over here... I'm still not able to find a monstrous liblldb-core.a or anything of the sort inside of XCode.app, and I'm still a bit confused about how this whole thing works.
It's the debug info that's the real issue with liblldb-core.a. Around 720MB of the 750MB is DWARF. (You can test this yourself - ar x liblldb-core.a into a directory, then strip -S *.o and you'll have about 32MB of .o files.) DWARF without real type uniquing has some nasty bloat with C++ programs.
I can make a .tar.gz of the (stripped) LLDB framework and lldb driver program and they come in at around 10MB or so with that compression -- and this is after linking in all of the llvm and clang bits needed by lldb. Nothing particularly outrageous here, even if the intermediate steps can look crazy.
I should have noticed that I took that first screenshot inside the Debug build directory which indicates that it is huge potentially because of the build configuration I had used.
Edit: Nope, that isn't quite the answer. The Release directory actually contains a liblldb-core.a that is slightly bigger than the other one.
It does appear that this huge 700MB archive is some sort of "side effect" file. When I archived the project it produced a 369MB .xarchive file. I'm sure this is a better representation of the "content" here. I am still basically learning this stuff by stumbling over it in the dark...
Update:
Oh, okay, looking at this situation it makes a bit more sense:
I took these files out of the Release directory after building it in Xcode (and minimally tweaking it to use -O3 for Release). I can see here that the ~350MB dSYM file containing debug information is taking up the majority of the space of that .xarchive from earlier, and that actually lldb's executable code totals under 40MB in size, the majority of which resides in that framework in an executable named LLDB.
This is a lot more reasonable to me. Now that I've gotten it out of Xcode and into my Documents directory where I can be more assured no external program would delete or modify it without my knowing, I can feel good about symlinking it from here to /usr/lib/lldb.
There is a bug in RHEL5's gcc-4.3.2 with which we are stuck. As a work-around we have extracted the missing object and put it in an object file. Adding this object file to every link makes the problem go away.
While adding it directly to LDFLAGS seems like a good solution, this doesn't work since e.g. libtool cannot cope with non-la files in there.
A slightly more portable solution seems to be to directly patch the gcc spec to add this to every link. I came up with
*startfile:
+ %{shared-libgcc:%{O*:%{!O0:/PATH/TO/ostream-inst.o}}}
where ostream-inst.o is added to the list of startfiles used in the link when compiling a shared library with optimizations.
Trying to compile boost with this spec gives some errors though since its build directly sets some objects with ld's --startgroup/--endgroup.
How should I update that spec to cover that case as well, or even better, all cases?
Go through this URL Specifying subprocesses and the switches to pass to them and GCC Command Options
If this help you, thats great.
I know this is not the answer you want to hear (since you specified otherwise in your question), but you are running into trouble here and are likely to run into more since your compiler is buggy. You should find a way of replacing it, since you'll find yourself writing even more work-around code the next time some obscure build system comes along. There's not only bjam out there.
Sorry I can't help you more. You might try simply writing a .lo file by hand (it's a two-liner, after all) and insert it into your LDFLAGS.
If it is a bug of GCC 4.3, did you try to build (by compiling from sources) and use a newer GCC. GCC 4.6.2 is coming right now. Did you consider using it?
I'd like my Xcode project to go through a Build And Analyze step without generating any errors, warnings, or static analysis messages. A problem with this is that my project includes libraries that generate (possibly innocuous) warnings and static analysis messages.
I can't find a way to disable specific Clang warnings so that "my" code builds with zero issues. Is this possible?
I wasn't able to find any way to do this, and filed a bug against Clang. The team seems to want to add this functionality, but it's not there yet.
The bug is: http://llvm.org/bugs/show_bug.cgi?id=7296
Also, one can use a __clang_analyzer__ macro to ifdef out any code that one doesn't want Clang to process.
The Build and Analyze step is clang - thats the "analyze" part. It wouldn't make sense to analyze your code and not have clang tell you about issues it find. That's like going to a car wash and telling them not to touch the car.
From talking to the guy that wrote clang at WWDC, it is extremely unlikely that anything it identifies as an issue is actually not. If you think you have some examples of code that works fine but clang complains, please file a bugreport with example code so Apple can fix that.
You can disable some compiler warnings through the use of flags, but not all of them are an option.