We want to develop a new adapted tail-based sampling processor in opentelemetry-collector-contrib.
Now we change the code in opentelemetry-collector-contrib/processor/tailsamplingprocessor and then use the tools in opentelemetry-collector-releases to build a new collector image, finally deploy it in k8s to checkout its correctness.
Is there a quicker way for us to develop and debug? What should we do?
Thanks
Use Golang standard: run&create Golang unit tests for your case(s). There are already some tests (see *_test.go files) https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/tailsamplingprocessor
Related
I'm currently trying to make UI-Tests in Xamarin, where i want the app to be tested against different server versions.
Originally i wanted to do this like i did in unit-tests with parametrized tests by using [Theory] and [ClassData].
Unfortunately i haven't found anything about this in the web so far regarding this nUnit construct.
Is this even possible? or is there another way to tell my xamarin UI-Test to run all my tests once for each server version?.
Many thanks in advance.
Mav
Soo, the mistake i made was to assume that i could use xunit blocks for nunit tests.
My bad.
The solution is pretty simple and can be looked up here in the gitHub projects documentation:
https://github.com/nunit/docs/wiki/TestFixtureData
So, I have multiple selenium test projects in separate VS solutions and they all have the same Setup() methods which are executed before scenarios are run as well as all having the same TearDown() methods for after. Currently if a change is required for these methods, they have to be updated separately so I was looking in to centralising these methods to be used in all of the test projects/solutions.
I'm relatively new so apologise in advance but does anyone have experience of this with suggestions on approaches I could take? Is it even possible? My tests do not currently run in parallel so is this something I'd need to look in to?
How if you change code be
[OneTimeSetUp] and [OneTimeTearDown]?
in my opinion you should create one class for setup include setup and tear down then in yor test add setup like this public class HREmployeeList:Setup
Hopefully help or not just check this below link
http://toolsqa.com/selenium-webdriver/c-sharp/how-to-write-selenium-test-using-nunit-framework/
One thing you could do is create a new project that contains the functions for setup and teardown and then include that compiled dll in all the other projects. If you need to make a change to setup/teardown, you make the change and compile a new dll and the code change is passed to all the other projects.
Keeping long story short - you need to create a Meta framework.
By concept Meta frameworks provide a method of solving the problem with automating multiple pieces as part of a larger automation strategy. Testers define independent utility classes which can be generically used with any automation tool and can be reused between different automation projects as well. The framework provides an abstraction layer that allows the separate automation pieces to be executed and have their results reported back in a standardized way.
I have a post on the topic, so feel free to take what you need form there.
Since you've tagged VisualStudio, I'll share first the approach my team did use for sharing common functionality across Test projects. What you need is a private Nuget server. Each team compiles and supports a Nuget pack based on the service it provides. For example Selenium code, API calls etc.
Next and probably closer to your case solution would be to utilize git submodules and share the Test harness engine between your projects.
Both approaches will benefit of a Fixture Setup Patterns like Shared Fixture Construction.
Our current system is in .Net world and we are in the process of moving to FeF world (Angular) and we might keep the back end either in .net or change it to some flavor of JVM. We do not know about the server side yet but for sure, we will keep it as .Net for some time till we sort out the technology issues.
One thing I want to do during this time to do is build a CI system with language agnostic build tools. We use NAnt/MSBuild today. I would like to know as of now what are all active build tools out there, which can work with any language? I did find the following, but not sure how many are all active today. I am not trying to find which is best or not, all I want to know is the tool set and I will evaluate them based on our project requirements. I want to make sure I did not miss some build tool because I do not know the Java world.
Current list
Make
Rake
Gradle
BuildR
I did find this question but it is 5 years old.
Thanks
Interestingly if you look at the visual studio files in a text editor you will see that they are indeed build scripts. They are actually msbuild.
My goal is to be able to write core testing that I can use within a unit testing framework as well as UI testing with selenium.
For simple test like:
Scenario: Add two numbers
Given I have entered 50 into the calculator
And I have entered 70 into the calculator
When I press add
Then the result should be 120
I would create both unit tests to prove that my core API would pass as well as a Selenium test that would prove my UI is doing the correct thing as well.
I briefly tried to find anyone doing something similar through Google, but couldn't find any examples. So I guess my question is, has anyone here done anything similar?
On approach I had thought of was simple adding the feature files to a project or directory and using the add existing item as link as the solution.
Update: Adding feature files to a common directory and adding them as a link appears to be working great. The feature bindings regenerates for each project the feature file was included in so I can run unit tests in one and Selenium UI tests in the other.
First, lets start with why you might want to do this. Its laziness of the good kind.
The quality that makes you go to great effort to reduce overall energy expenditure. It makes you write labor-saving programs that other people will find useful, and document what you wrote so you don't have to answer so many questions about it. Hence, the first great virtue of a programmer. Also hence, this book. See also impatience and hubris. (p.609)
Larry Wall, Programming Perl
Except it isn't, because we aren't going to reduce our overall energy expenditure.
When you are using SpecFlow, the easy part to keep up to date is the plain text. You will find yourself refactoring the [Binding]s again and again, but the scenarios tend to be quite easy to work with, and need very little revision once they have been agreed.
In addition the [Binding]s are global. Load them in from any assembly and they are available to the SpecFlow runner. In respect of what you are trying this actually makes things harder as you need to put effort in to keep the UI bindings from being mixed up with the non-UI bindings.
Also consider the way that SpecFlow actually runs the tests from feature files. It's a two stage process.
When you save the .feature file the SpecFlow VS plugin generates a .feature.cs file.
When you run your test engine (e.g. NUnit) it ignores the plain text and uses compiled code from .feature.cs
So if you start using linked .features I have no idea if the SpecFlow plugin will generate .feature.cs for both instances of the file. (If you try this please let us know)
Second lets consider the features themselves. I think you will constantly finding yourself compromising your tests to make them fit the other place they are used. Already in the example you have given you have on the screen. If you are working with just the core API then there won't be a screen, so do we change this to fit better in a non-UI scenario?
Finally you have another thing to consider, just how useful will your tests be. If you have already got a test that tests the Core API, then what will it mean to run same test via Selenium. All you will really test is the UI layer. In my current employment we have a great number of regression tests that perform this very kind of testing, running up a client that connects to a server and manipulating the UI to get the desired scenarios enacted. These are the most fragile tests we have due to their scale. They constantly break and we basically have to check our entire codebase to find the line that broke them. Often something like 10-100 of them break just for a one line change. If these tests weren't so important to the regression cycle then the effort in maintaining them would just be too much. In my own personal projects I tend to remove these tests completely and instead with UIs, I avoid testing the View layer. With WPF MVVM, I execute Commands and test for results in ViewModels. If somebody then decides the TextBox should be a ComboBox or that it will work better in mauve, then my testing is isolated.
In short, there is a reason you can't find anything about this on Google :-)
In general (see http://martinfowler.com/bliki/TestPyramid.html), one should limit the number of automated tests that test the UI directly, and prefer tests that start at the presentation layer (just below the view layer), or below.
SpecFlow is agnostic; the tests can be implemented using e.g. Selenium at the UI layer or just MSTest or NUnit at any of the layers below.
However having said that I appreciate that you will have situations where you are doing ATDD and want to implement SpecFlow scenarios to match each of the acceptance criteria. Some of the criteria will be perfectly fine to test at a lower architectural level, but one or two of them may be specific to the GUI-- for example testing Login and ensuring that the user is redirected to the home page after successful login. If using Angular2 or React routing (see https://en.wikipedia.org/wiki/Single-page_application), that redirect is likely done in the GUI layer itself.
I don't have a perfect answer yet, but as a certified SpecFlow trainer, I have a vested interest in this! The way I am currently leaning is to use a complementary tool like CucumberJS for the front-end specific tests (such as testing React router redirects) and SpecFlow for tests at lower architectural layers. Our front-end uses Node.JS/Express and our backend is .NET Core. The idea is that the front-end tests mostly use the front-end only with mocked out AJAX calls to the backend (see sinonjs), and the back-end tests use EF Core with the in-memory option (see docs.efproject.net/en/latest/providers/in-memory/. So the tests all run fast.
Of course, you still need a few tests that actually go all the way through, but those are different-- we should call those integration tests. I do not believe that acceptance tests need to be integration tests. That way, you have a suite of acceptance tests from doing ATDD, plus a relatively small set of integration tests that test all the way front-to-back. The integration tests run more slowly and require more maintenance, so you separate them out into a different part of the CI/CD build chain.
I hope this makes sense. It is not so much solving the problem as avoiding the problem.
OK, so we all know the daily build is the heart beat of a project, but whats the single best way of automating it?
We have perl scripts wrapping our pipeline which includes ClearCase, VS2005 (C++), Intel FORTRAN, Inno setup. We use cron jobs on UNIX to schedule the build, and host a simple Apache web server to view and monitor the build. All in all its rather complex, I would like to know whats the best off the shelf solution that people use?
And yes I did say FORTRAN no escaping it sometimes, it works, no point doing a huge re-implementation project for some tried and tested FEA code that just works.
A new one to me that I've heard is quite slick is hudson - also with MSBuild support.
We're in the process of implementing CC.Net. So far it seems like it would fit your model pretty well.
Out of the box it offers automated building, results tracking and notification. I'm not sure how detailed the build-in-progress monitoring is though.
There are many tools that specifically handle this:
Cruise Control
Hudson
Continuum
The tools have out of the box support for the most common build types. They all also support some sort of "run this script" type build process.
In the end you should use the nicer build tools (MSBuild, Ant, Maven, Make, ...) where you can and fill the gaps for the odder tools with custom scripts. The automated build can just invoke these in the right order.
Here is the best resource we found to help us pick a Continuous Integration tool. We have been evaluating 5 or 6 tools on this page.
http://confluence.public.thoughtworks.org/display/CC/CI+Feature+Matrix
We use TeamCity - but then its a simple C#/Java development - maybe your pipeline can done via scripts it can drive?
I have had success using Visual Build Pro.
CC.NET is very powerful. Used it and was really happy about it. Even the status icon in the systray. It's a small detail, but it gives you a good overview of the project's "health". You immediately feel motivated to fix the tests when you see it red.
Now we use a self-baked series of scripts. Since we write Python, compilation is non-existant, so the only problem is running the tests.
If you're working with Visual Studio, be sure to check out Team Foundation Build to see if it will suit your situation.
It looks like Buck Hodges' blog post on the VS 2008 version is a good resource, too.
I know this is a really old question, but it's still coming up in searches, so someone should mention Jenkins - the open source continuation of Hudson.
From the Jenkins wiki:
Among those things, current Jenkins focuses on the following two jobs:
Building/testing software projects continuously, just like CruiseControl or DamageControl. In a nutshell, Jenkins provides an easy-to-use so-called continuous integration system, making it easier for developers to integrate changes to the project, and making it easier for users to obtain a fresh build. The automated, continuous build increases the productivity.
Monitoring executions of externally-run jobs, such as cron jobs and procmail jobs, even those that are run on a remote machine. For example, with cron, all you receive is regular e-mails that capture the output, and it is up to you to look at them diligently and notice when it broke. Jenkins keeps those outputs and makes it easy for you to notice when something is wrong.
It was originally built with Java in mind, so it integrates well with lots of other Java tools, but you can use it with any language, including all those mentioned by the OP.