What is the difference between Skip_build and test_without_building on Fastlane/Scanfile - xcode-ui-testing

What is the difference between Skip_build and test_without_building on Fastlane/Scanfile?
I would really execute my test cases without building everytime, I have several schemes that are executed on a shell script in a loop. I don't want to build for every iteration.
Does anybody tried those parameters before?

The skip_build parameter will omit the build command from the generated xcodebuild command that ultimately gets executed by scan. This means that if there is a built product in DerivedData, it will use it instead of recompiling your app. If there is no build product in DerivedData, it will rebuild the product.
The test_without_building parameter is the equivalent of the --test-without-building flag that you can pass to xcodebuild. This allows you to pass other flags to the command to point to the product under test.
Hope this helps! 🚀

Related

TeamCity pass codebase changes to next build

I have a TeamCity build that sometimes fails too early.
What I mean by that is that the first few steps are for "provisioning" (setting up the testing environment) and the testing of my code itself comes later.
Sometimes (for whatever reason) the build fails during one of the "provisioning" steps. This is not a problem since running the build again usually works fine.
But - the "changes" are not passed along to the next run of the build.
I am using this command as part of my build to output the "changes" that came from my codebase:
copy "%system.teamcity.build.changedFiles.file%" changelog.txt
So I need a way to tell TeamCity "hey, ignore the last run, that failure doesn't count because it didn't test my code, I want the next run to contain the same 'changes' in system.teamcity.build.changedFiles.file"
How can I do that?
Have you tried build chains with dependencies? They can be set up to only execute if the build (including tests) is successful: http://blog.jetbrains.com/teamcity/2012/04/teamcity-build-dependencies-2/

TeamCity and CTest test results

I have a number of unit tests written for my project, executed with CTest. I would like to integrate the results into my TeamCity build. I've downloaded and set up the plugin for my testing framework (Boost Test).
The problem that I have run into is that the tests run with CTest output to Testing/Temporary/LastTest.log, whereas TeamCity is trying to read the results from standard out. To get around this, my testing step is.
make test
cat Testing/Temporary/LastTest.log
which works, but feels like a hack.
Is there any way to get TeamCity to read from this file in addition to standard out? Alternatively, is there any way to tell ctest to output to standard out in addition to this LastTest.log file?
This question is similar, but I would like it to work for all output rather than just on failure: CMake: setting an environmental variable for ctest (or otherwise getting failed test output from ctest/make test automatically)
Teamcity has additional build features which allow to process CTest reports. I am not sure if it'll work or not but you could try adding an additional build feature in your build step to read CTest report.

Xcode oclint analyzer

I want to use oclint (an Objective-C static analyzer) for my projects but the official documentation does not clearly describe how to configure it.
How do I use it?
I have installed the oclint package then copied the script for capturing projects configs into a project's folder. Running this script creates an empty file named compile_commands.json and it does not fill xcodebuild.log with any data.
Thank you!
I don't think this script will add anything into xcodebuild.log. It reads from it. You may want to manually run xcodebuild and save the output into xcodebuild.log before running the script.
Check out the documentation.
I have had luck with the manual route of copying the script into my project's folder then executing it on the command line.
Though this does not integrate oclint's functionality into XCode, it does provide feedback via stdout.
From this feedback I then update my code to align with oclint's recommendation and iteratively run oclint until my code complies with oclint's rules.

How to make Xcode "Run Script" to always run

I have a "run script" step that dynamically creates resources/files that I copy into the build dirs. Every run of this script produces different content so I want it to run on every build. The script gets run correctly on a clean build however once a build is made the step is not run again since no source has been modified.
I tried setting the input of the step to /dev/random but it does not seem to trigger a changed environment and does not re run the step.
Is there a way I can set this up so this step gets run ever time build is pressed, as opposed to only when the source is modified or clean?
You should put the Run Script build phase in a separate Aggregate Target, and make your main target dependent on the Aggregate Target. The Aggregate should be built every time.

Xcode: Running a script before every build that modifies source code directly

What I did:
I have a script that
Read some configuration files to generate source code snippets
Find relevant Objective-C source files and
Replace some portions of the source code with the generated code in step 1.
and a Makefile that has a special timestamp file as a make target and the configuration files as target sources:
SRC = $(shell find ../config -iname "*.txt")
STAMP = $(PROJECT_TEMP_DIR)/$(CONFIGURATION)$(EFFECTIVE_PLATFORM_NAME).stamp
$(STAMP): $(SRC)
python inject.py
touch $(STAMP)
I added this Makefile as a "Run Script Build Phase" on top of the stack of build phases for the project target.
What happened:
The script build phase was run before compiling the source.
However, since the script modifies source code during its execution, I needed to build twice to get the most recent version of the build product. Here is what I imagine to be happening:
1st run: Xcode collects dependency information ---> no changes
1st run: Xcode runs "Run Script Build Phase" ---> source is changed behind Xcode's back
1st run: Xcode finishes build, thinking nothing needs to be updated
2nd run: Xcode collects dependency information ---> source has changed, needs rebuild!
2nd run: Xcode runs Run Script Build Phase" ---> everything is up-to-date
2nd run: Xcode proceeds to compilation
After reading Xcode documentation on Build Phases, I tried adding a source file which is known to be updated every time the script is run as the output of "Run Script Build Phases", but nothing changed. Since the number of configuration files may vary in my project, I don't want to specify every input and output file.
Question:
How do I make Xcode aware of source file changes made during "Run Script Build Phase"?
Edit:
Added that I placed the script build phase before the other build phases
Every technique mentioned so far is an overkill. Reproducing steve kim's comment for visibility:
In the build phases tab, simply drag the "Run Script" step to a higher location (e.g. before "Compile Sources").
Tested on Xcode 6
This solution is probably outdated. See the higher voted answer instead; I no longer actively use Xcode and am not qualified to vet a solution.
Use "External Target":
Select "Project" > "New Target..." from the menu
Select "Mac OS X" > "Other" > "External Target" and add it to your project
Open its settings and fill in your script setup
Open the "General" tab of the main target's settings and add the new target as it's direct dependency
Now the new "External Target" runs before the main target even starts gathering dependency information, so that any changes made during the script execution should be included in the build.
There is another, slightly simpler option that doesn't require a separate target, but it's only viable if your script tends to modify the same source files every time.
First, here's a brief explanation for anyone who's confused about why Xcode sometimes requires you to build twice (or do a clean build) to see certain changes reflected in your target app. Xcode compiles a source file if the object file it produces is missing, or if the object file's last-modified date is earlier than the source file's last-modified date was at the beginning of the first build phase. If your project runs a script that modifies a source file in a pre-compilation build phase, Xcode won't notice that the source file's last-modified date has changed, so it won't bother to recompile it. It's only when you build the project a second time that Xcode will notice the date change and recompile the file.
Here's a simple solution if your script happens to modify the same source files every time. Just add a Run Script build phase at the end of your build process like this:
touch Classes/FirstModifiedFile.m Classes/SecondModifiedFile.m
exit $?
Running touch on these source files at the end of your build process guarantees that they will always have a later last-modified date than their object files, so Xcode will recompile them every time.
As of Xcode 4, it looks like if you add the generated files to the output section of the build phase, it will respect that setting, and not generate the ... has been modified since the precompiled header was built error messages.
This is a good option if your script is only generating a handful of files each time.
I as well struggled with this for a long time. The answer is to use ento's "External Target" solution. He is WHY this problem occurs and how we use it in practice...
Xcode 4 build steps do not execute until AFTER the plist has been compiled. This is silly, of course, because it means that any pre-build steps that modify the plist won't take effect. But if you think about it, they actually DO take effect...on the NEXT build. That's why some people have talked about "caching" of plist values or "I have to do 2 builds to make it work." What happens is the plist is built, then your script runs. Next time you build, the plist builds using your modified files, hence the second build.
ento's solution is the one way I've found to actually do a true pre-build step. Unfortunately I also found that it does not cause the plist to update without a clean build and I fixed that. Here is how we have data-driven user values in the plist:
Add an External Build System project that points to a python script and passes some arguments
Add user-defined build settings to the build. These are the arguments that you pass to python (more on why we do this later)
The python script reads some input JSON files and builds a plist preprocessor header file AND touches the main app plist
The main project has "preprocess plist files" turned on and points to this preprocessor file
Using touch on the main app plist file causes the main target to generate the plist every time. The reason we pass in build settings as parameters is so our command-line build can override settings:
Add a user-defined variable "foo" to the prebuild project.
In your prebuild you can use $(foo) to pass the value into the python script.
On the command-line you can add foo=test to pass in a new value.
The python script uses base settings files and allows for user-defined settings files to override the defaults. You make a change and immediately it ends up in the plist. We only use this for settings that MUST be in the plist. For anything else it's a waste of effort....generate a json file or something similar instead and load it at run-time :)
I hope this helps...it's been a couple rough days figuring this out.
The External Target solution from #ento no longer works as of Xcode 11.5. The solution is to add all files that will be changed under Output Files in the Run Script.
Another option is to create a subproject framework with your scripts and just add it as a dependency to all targets. The phase scripts of this subproject should now be executed before all targets.

Resources