In the set of my TeamCity configurations, I decided to make something like an aging test*. And run a single configuration for a 100 times.
Can I make in a few simple clicks?
*aging test - test that is showing, that due time/aging, results will not be changed.
As of now, this is not possible from UI. If you run one build configuration few times without any changes, they will be merged and only 1 will be executed. If you want to run 100, you have to trigger them one by one, after the previous one finished executing.
But the better solution is to trigger builds from script using REST API (for more details see the documentation here), if builds have different values in custom parameters they all will be put in the queue.
HOW: Define a dummy custom parameter, and trigger the build from script within a loop. Pass the value of iterating variable as parameter value. So, TeamCity will think those are different builds and execute all of them.
Related
I would like to create a JMeter test setup with important sampling and test variation parameters entirely controlled by csv-Files (i.e. not modifying the JMX-File). It should be run with maven.
Ideas is as follows:
sources.csv contains
sampleRate;file
40;samplesForController1.csv
30;samplesForController2.csv
5;samplesForController3.csv
...
SampleRate should determine, how often a certain number of tests (defined by parameters in the respective file) should be executed (relative to others).
How could this be achieved? Im asking for the first step here (make sure the files/testSamples are sampled/executed given the indicated sampleRate) as I think I can solve the second part (dealing with parameters in samplesForController1.csv etc.) by myself.
P.S.
I'm struggling with the options presented here: In Jmeter, I have to divide number of thread into the multiple http requests in different percentage but have to keep sequence remain same since
afaics, thread groups cannot be created on-the-fly/dynamically/progammatically
apparently, Throughput Controller needs to know its child element(s probablities) upfront (i.e. not created dynamically), otherweise, sampling is very odd (I could not get it working maintaining the desired sampleRate)
I did not try to integrate jmeter-plugins in the maven build thus far as my impression is, the available plugins/controllers also needs to know their child element upfront
It is possible to create thread groups programmatically, check out:
Five Ways To Launch a JMeter Test without Using the JMeter GUI
jmeter-from-code example project
jmeter-java-dsl project
You can use Switch Controller and use a function like __groovy() to generate the child element index, example implementation can be found in Running JMeter Samplers with Defined Percentage Probability article
It's not a problem to use JMeter Plugins with Maven, see Adding jar's to the /lib/ext directory documentation section for example setup
Environment:
I have around 100 pipelines that run on a number of triggers.
Outcome: I want to create a master pipeline that calls those 100 pipelines.
Currently, I've created a list of pipeline names and put them to an array. Then I was hoping to use forEach and execute pipeline activities to pass those names.
Issue, it seems that execute pipeline activity does not take variables or it is not obvious how to do it.
I do not want to create master pipeline manually as it can change often and I hope there must be better way to do it than manually.
You are correct that the "Invoked pipeline" setting of the Execute Pipeline activity does not support a variable value: the Pipeline name must be known at design time. This makes sense when you consider parameter handling.
One way around this is to create an Azure Function to execute the pipeline. This answer has the .Net code I leverage in my pipeline management work. It's a couple years old, so probably needs an update. If you need them to run sequentially, you'll need to build a larger framework to monitor and manage the executions, which is also discussed in that answer. There is a concurrency limit (~40 per pipeline, I believe), so you couldn't run all 100 simultaneously.
One of my automated test cases (TC) fails predictably after a prod refresh that takes place every few months.
For the TC to pass, there should be 'N/A' for values, which is a precondition. After getting the 'N/A' text, I do insert into a table to create values and then do other steps.
After the refresh, there are values (monies) instead of the 'N/A'.
What are the ways to avoid the failure? Run Keyword If and Run Keyword And Expect Failure would invalidate the original TC and and it will always pass, which I apparently don't need.
There might be other approaches too, however, one of the ways to approach this problem is
You can define init file in directory
__init__.robot
That suite setup and suite teardown in the file would run before anything in the underlying folders.
make use of set global variable with N/A and update the same when you see actual values. i.e every test case would verify whether the variable contains N/A or actual values(i.e not N/A), this can be done using Test Setup with keyword.
NOTE: You can also use set suite variable for the same
I am new to teamcity. I have created a build which should be run with different parameters. Is there a way to schedule builds to run with different parameters ?
That means, I should set x value for a parameter for a build scheduled at 9 AM and y value for a parameter for a build scheduled at 10 AM. I looked into schedule trigger,but I couldn't find a way to give custom parameters for each triggers
Convert your build configuration into template. Then, create another build based on this template. Add different parameters into those build configuration and run them on different hours with build scheduler.
If you wish to be more conservative with the number of build configurations used then create two different shell scripts that if ran by the correct trigger would produce your desired results (ex: setting env variables). Then in the Schedule Triggers add a trigger rule that excludes the shell file you do not want. In the build step if one script exists run it else run the other. But you should prefer the answer given by #Peska in most cases.
very basic example:
touch test{1,2}.sh #will create two files named ./test1.sh and ./test2.sh
#Trigger rule excludes one
./test?.sh #will run all files matching that pattern
I am trying to build a jmeter testplan, where all the test values are sent from a csv datafile.I want to add assertions(provided in the datafile) to my HTTP Request at runtime and execute the test. The reason behind doing this is to keep the plan flexible according to the number of assertions. In my case, the assertions are getting added at the runtime; however they fail to get executed. May I know what should be done to get the components added and executed in the same flow?
For example: A part of plan looks like:
XYZ
--HTTP Sampler
-- Response Assertion1
-- Response Assertion2
-- JSON Extractor
where XYZ -->keyword based transaction controller(reusable component)
Everytime I have a request of type XYZ, this chunk of components will get executed. In my case, I do not want to place anything such as Assertions, pre/post processors, extractors in the test plan already. I want to generate these components at run time and execute them (as per my test requisites).
Issue: The problem here is that I cant load the components programmatically and execute them in the same flow. The reason being, the compiler does not know beforehand what all components it needs to execute, so it bypasses the newly added components.
So, I need some alternative solution to execute this.
You can add Response Assertion (or multiple) with Pattern to test filled with a variable as ${testAssert1} and set the variable by default as empty, for example
Put in User Defined Variables name testAssert1 with empty value.
Your assertion(s) will pass until you on run time set the variable with a different value, for example using User Parameters Pre Processor.