Team,
I am generating go coverage and converting them to html to view in Jenkins. but I want to ship the LCOV file .dat to sonarqube. is there a way? like set a key value in sonar-project.properties
./bazel $BAZEL_STARTUP_OPTS coverage $BAZEL_OPTS $BAZEL_EXTRA_OPTS //src/services/discovery/... \
--combined_report=lcov \
--test_output=errors --test_verbose_timeout_warnings | tee /tmp/test_results.txt
for view on Jenkins I am converting LCOV _coverage_report.dat to html. But what I want to know if I can specify this in sonarqube.properties file either html or .dat lcov? to pick it up after scanning is done?
I see this for java but not for GO
./bazel run #lcov//:bin/genhtml -- -o \coverage/discovery --ignore-errors source --no-source --function-coverage --branch-coverage --no-sort bazel-out/_coverage/_coverage_report.dat || true
I am trying to aggregate all the test.xml reports generated after a bazel test run. The idea is to then upload this full report to a CI platform with a nicer interface.
Consider the following example
$ find .
foo/BUILD
bar/BUILD
$ bazel test //...
This might generate
./bazel-testlogs/foo/tests/test.xml
./bazel-testlogs/foo/tests/... # more
./bazel-testlogs/bar/tests/test.xml
./bazel-testlogs/bar/tests/... # more
I would love to know if there is a better way to aggregate these test.xml files into a single report.xml file (or the equivalent). This way I only need to publish 1 report file.
Current solution
The following is totally viable, I just want to make sure I am not missing some obvious built in feature.
find ./bazel-testlogs | grep 'test.xml' | xargs [publish command]
In addition, I will check out the JUnit output format, and see if just concatenating the reports is sufficient. This might work much better.
In JMeter (5.1.1) I have a summary report that I'm trying to save as a timestamped file. The filename value looks like the following:
D:\Load Tests\example.com\Results\${__time(yyyy-MM-dd-HH-mm-ss,)}_summary.csv
However, rather than create the file with the result of the __time() function e.g. 2019-07-22-10-24-03_summary.csv, it's actually generating a filename called ${__time(yyyy-MM-dd-HH-mm-ss,)}_summary.csv.
I've tried creating a user-defined variable called timestamp with the value ${__time(yyyy-MM-dd-HH-mm-ss,)} and referencing it with ...\${timestamp}_summary.csv but this similarly results in $(timestamp)_summary.csv.
I saw a JMeter Archive post regarding a similar question to mine from 2006 where it's implied that listener filenames are resolved too early for functions and variables to be used, but I'm hoping that JMeter has been able to overcome this hurdle in the 13 years since then.
Is it possible to use variables for listener filenames in JMeter GUI and set them dynamically like the timestamp above?
If not, is there an alternative method of doing this using Groovy? Where would this be - in a setup thread JR223 sampler perhaps? I have tried this and seemingly managed to programatically change the filename, but no file was saved.
Update with answer:
I just needed to reverse the path delimiters from \ to /.
D:/Load Tests/example.com/Results/${__time(yyyy-MM-dd-HH-mm-ss,)}/summary.csv
I come across this issue and figure out that it works when you specify your path with the slash, instead of backlash.
For example:
D:\Load
Tests\example.com\Results\${__time(yyyy-MM-dd-HH-mm-ss,)}_summary.csv
Doesn't work. But:
./Load
Tests/example.com/Result/${__time(yyyy-MM-dd-HH-mm-ss,)}_summary.csv
Will work.
I usually don't write long answers, but you touch a bit of a sore point,
Listeners are classic example of Can't Live with You, Can't Live Without You
JMeter mindset is load testing (although can be used for functional tests)
Therefore, the moto/best practice is You shouldn't use it
Use CLI mode: jmeter -n -t test.jmx -l test.jtl
Use as few Listeners as possible; if using the -l flag as above they can all be deleted or disabled.
Don't use "View Results Tree" or "View Results in Table" listeners during the load test, use them only during scripting phase to debug your scripts.
But...in the same document it suggest it for testing/debugging
Create a simple Test Plan containing the JSR223 Sampler and Tree View Listener. Code the script in the sampler script pane, and test it by running the test.
Basically/In the end, you need to save first jtl file using -l myresults.jtl
And then convert it to CSV using JMeterPluginsCMD, example:
JMeterPluginsCMD.bat --generate-csv test.csv --input-jtl results.jtl --plugin-type ResponseTimesOverTime
Or do it the JMeter way with creating a dashboard
jmeter -g <log file> -o <Path to output folder>
You should not be using any Listeners in your tests as it violates JMeter Best Practices
Use as few Listeners as possible; if using the -l flag as above they can all be deleted or disabled.
you should be running JMeter in non-GUI mode like:
jmeter -n -t test.jmx -l summary.jtl
If you want to amend the summary.jtl filename to include timestamp - you can use date and time commands combination like:
jmeter -n -t test.jmx -l %date:~-4%-%date:~4,2%-%date:~7,2%-%time:~0,2%-%time:~3,2%-%time:~6,2%_summary.jtl
Demo:
I am creating code coverage reports for my C++ projects using gcov/lcov, and I am trying to remove all files except the ones in a certain directory from the coverage report (i.e. I do not want different dependencies in various folders to show up in the report).
However I want to do this automatically and not manually. I tried the following:
lcov -r coverage.total '!(<path>)' -o coverage.info
But then lcov comes back with Deleted 0 files. I also tried !(<path>), '[^path]*' and slight variations of these but nothing seems to work. I can manually remove the undesired folders for example the following does work:
lcov -r coverage.total '/usr/libs/*' '/usr/mylibs/*' -o coverage.info
So my question is, how can I have lcov exclude all but a specific directory?
P.S.
I am open to workarounds (for example if this can be done with a bash script)
I am using bash+CMake+gcov+lcov
P.S.
This is not a duplicate of this question. I am asking about an automated way to only include files in a specific directory in the report. (for example the current directory) I am aware of the --remove argument but that is not an automated solution.
Your help is greately appreciated!
From the llvm-cov docs:
llvm-cov show [options] -instr-profile PROFILE BIN [-object BIN,...] [[-object BIN]] [SOURCES]
The llvm-cov show command shows line by line coverage of the binaries
BIN,... using the profile data PROFILE. It can optionally be filtered
to only show the coverage for the files listed in SOURCES.
I'm using the following command:
xcrun llvm-cov show -instr-profile "${PROFDATA}" "${BINARY}" codecov_source_files > Coverage.report
Where codecov_source_files is a file with this line:
*Router.swift
So basically what I want is the report to only contain files with the suffix: Router.swift
But i'm getting a Coverage.report with all the classes in the project.
What am I missing?
It's badly worded but SOURCES is actually a list of file names, not the name of a file containing a list of filenames.
They need to be the paths to the actual source files on disk. It doesn't support wildcards or regex unfortunately.
Edit: By reading the source I have discovered that you can also list directories as SOURCES and it will recurse into them. Also there is an undocumented option -dump-collected-paths which prints the files the SOURCES match.
you can use help to look up supported commands
$ llvm-cov show --help
$ llvm-cov report --help
Maybe the following command is the function you want
--ignore-filename-regex=<string> - Skip source code files with file paths that match the given regular expression