I have a feature "Importing articles from external website".
In my first scenario I test importing a list of links from the external website.
Feature: Importing articles from external website
Scenario: Searching articles on example.com and return the links
Given there is an Importer
And its URL is "http://example.com"
When we search for "demo"
Then the Importer should return 25 links
And one of the links should be "http://example.com/demo.html"
In my steps I have the 25 links in a #result array.
In my second scenario I want to take one of the links and test the fact that I parse the article correctly.
Now obviously I do not want to go to the external website every time, especially now that the first scenario passes.
How do I proceed here so I can keep testing without making the HTTP requests for the first scenario? Or should I run it once and persist the #result array across the rest of the scenarios so I can keep working with the actual result set?
This is intentionally very difficult to do! Sharing state between tests is generally a Very Bad Thing, not least because it forces your tests to run in sequence (your first scenario MUST run before the subsequent ones, not something Cucumber supports explicitly).
My suggestion would be to re-think your testing strategy. Hitting external services in tests is a great way to make them run slowly and be unreliable (what happens when the external service goes down?). In this case I'd suggest using something like webmock or vcr to create a fake version of the external site, that returns the same response as you'd expect from the real site, but you can hit as many times as you like without the worry of performance or unavailability.
I found that it is technically possible to use
##global_variable in step definition to share the global state.
However, just like other people points, it may not be a good idea.
I tried to avoid repeated login steps in the similar scenarios. Again, it may not be a good practice. Use the trick when really necessary
You shouldn't share state between scenarios. A scenario describes a piece of the intended behavior for the whole system, and it should be possible to run just one scenario. E.g. if you have run the entire test suite, and you find that a single scenario fails, you should be able to run just that one scenario in order to investigate what went wrong.
Your problem arises because you try to contact external systems. That is not advisable. Not only does it make you test suite run more slowly, but it also makes the test dependent on the external system, to which you have no control. If the external system is not running, your tests are not running. If the external tests does not contain the data you expect, your tests will fail, even though there are no bugs in your own system. And you end up letting your tests be controlled by what you expect to be in the external systems, instead of controlling what is in the external system based on what you need to test.
Instead you should mock out the external system, and let your scenarios control what the mocked system will deliver:
Scenario: Query external system
# These two lines setup expected data in a mocked version of the external system
Given there the system x contains an article named "y"
And the article contains the text "Lorep ipsum"
When I query for article "y"
Then I should see the text "Lorem ipsum"
This scenario is independent of any actual data in external systems, as it explicitly specifies what needs to be there. And more importantly, it clearly describes how your own system should behave.
The scenario in that form can also be communicated to stakeholders, and they can validate the scenarios without any prior knowledge to any test data present in those external systems.
It may take some time getting a proper framework running, but in the end, it will be worth it.
Making scenario dependent or sharing the data between scenarios is not a good practice.
some solutions
1) Cucumber provide Background tag to run preconditions for each scenario.
2) Cucumber provides hooks #Before and #after which can be customized for each scenario.
I use a file. I have a case to create a new user, then I want to logout and log back in with that same user in other features.
I generate the user with:
#randomName = [*('a'..'z')].sample(8).join
Then I save the user to a file:
File.open("randomName.txt", 'w') {|f| f.write("#{#randomName}") }
Later, when I need that data in other feature, I use:
#randomName = data = File.read("randomName.txt")
I haven't seen anything that makes me want to use any of these DI little frameworks. I simply created a static initializer in a class where I store my session data, and had all step definition classes extend that class. It works and I don't have to add any more libraries to my project.
public class MyAbstractClass {
public static final Object param1;
public static final Object param2;
public static final Object param3;
static {
// initialize params here
}
public class MyStepDefinition extends MyAbstractClass {}
If you require data that can change over time, simply declare them non-static.
Related
I am trying to understand which of the following two options is the right approach and why.
Say we have GetHotelInfo(hotel_id) API that is being invoked from the Web till the Controller.
The logic of the GetHotelInfo is:
Invoke GetHotelPropertyData() (Location, facilities…)
Invoke GetHotelPrice(hotel_id, dates…)
Invoke GetHotelReviews(hotel_id)
Once all results come back, process and merge the data and return 1 object that contains all relevant data of the hotel.
Option 1:
Create 3 different repositories (HotelPropertyRepo, HotelPriceRepo,
HotelReviewsRepo)
Create GetHotelInfo usecase that will use these 3 repositories and
return the final result.
Option 2:
Create 3 different repositories (HotelPropertyRepo, HotelPriceRepo,
HotelReviewsRepo)
Create 3 different usecases (GetHotelPropertyDataUseCase,
GetHotelPriceUseCase, GetHotelReviewsUseCase)
Create GetHotelInfoUseCase that will orchestrate the previous 3
usecases. (It can also be a controller, but that’s a different topic)
Let’s say that right now only GetHotelInfo is being exposed to the Web but maybe in the future, I will expose some of the inner requests as well.
And would the answer be different if the actual logic of GetHotelInfo is not a combination of 3 endpoints but rather 10?
You can see a similar method (called Get()) in "Clean Architecture with GO" from Manato Kuroda
Manato points out that:
following Acyclic Dependencies Principle (ADP), the dependencies only point inward in the circle, not point outward and no circulation.
that Controller and Presenter are dependent on Use Case Input Port and Output Port which is defined as an interface, not as specific logic (the details). This is possible (without knowing the details in the outer layer) thanks to the Dependency Inversion Principle (DIP).
That is why, in example repository manakuro/golang-clean-architecture, Manato creates for the Use cases layer three directories:
repository,
presenter: in charge of Output Port
interactor: in charge of Input Port, with a set of methods of specific application business rules, depending on repository and presenter interface.
You can use that example, to adapt your case, with GetHotelInfo declared first in hotel_interactor.go file, and depending on specific business method declared in hotel_repository, and responses defined in hotel_presenter
Is expected Interactors (Use Case class) call other interactors. So, both approaches follow Clean Architecture principles.
But, the "maybe in the future" phrase goes against good design and architecture practices.
We can and should think the most abstract way so that we can favor reuse. But always keeping things simple and avoiding unnecessary complexity.
And would the answer be different if the actual logic of GetHotelInfo is not a combination of 3 endpoints but rather 10?
No, it would be the same. However, as you are designing APIs, in case you need the combination of dozens of endpoints, you should start considering put a GraphQL layer instead of adding complexity to the project.
Clean is not a well-defined term. Rather, you should be aiming to minimise the impact of change (adding or removing a service). And by "impact" I mean not only the cost and time factors but also the risk of introducing a regression (breaking a different part of the system that you're not meant to be touching).
To minimise the "impact of change" you would split these into separate services/bounded contexts and allow interaction only through events. The 'controller' would raise an event (on a shared bus) like 'hotel info request', and each separate service (property, price, and reviews) would respond independently and asynchronously (maybe on the same bus), leaving the controller to aggregate the results and return them to the client, which could be done after some period of time. If you code the result aggregator appropriately it would be possible to add new 'features' or remove existing ones completely independently of the others.
To improve on this you would then separate the read and write functionality of each context into its own context, each responding to appropriate events. This will allow you to optimise and scale the write function independently of the read function. We call this CQRS.
In cypress
I am trying to create a random number, in one spec, posting it to the first website
Navigate to another website with different origin, want to assert that this random number generated in the first test, is available on the second website since they are linked in the backend
This won't happen as same code to create random number would run again, in case we generate in before() block, thus creating and assigning another random number to that variable, since before() block will run again in case we visit another url
few ways like using -
describe function scope --> will run again and assign new random number
static class properties --> will run again ...
java singleton principle --> will run again ...
cypress commands in cypress/support/command.js --> will run again ...
local storage --> doesn't work, local storage gets cleared when visiting another url
None of them worked, although if we were not to visit another url, all of them would work normally
This is documented here https://github.com/cypress-io/cypress/issues/2636
that it is not possible to use cy.visit to visit different origin websites and maintain state
I have two solutions for such scenarios
use cy.writeFile to write in the first run, and cy.readFile to read in next run, since it actually writes to file system, this gets persisted and thus solves our problem, we are able to maintain state
Use JSON file for data, and thus cy.fixture().then(str => str) which will remain common across both test runs
I have a trie-based word detection algorithm for a custom dictionary. Note that regular expressions are too brittle with this dictionary as entries may contain spaces, periods, etc.
I've implemented the algorithm in a local C# app that reads in the dictionary from file and stores the trie in memory (it's compact, so no RAM size issues at all). Now I would like to use this algorithm in an MVC 3 app on a cloud host like AppHarbor, with the added twist that I want a web interface to enable adding/editing words.
It's fast enough that loading the dictionary from file and building the trie every time a user uploads their text would not be an issue (< 1s on my laptop). However, if I want to enable admins to edit the dictionary via the web interface, that would seem tricky since the dictionary would potentially be getting updated while a user is trying to upload text for analysis.
What is the best strategy for storing, loading, and updating the trie in an MVC 3 app?
I'm not sure if you are looking for specific implementation details, or more conceptual ideas about how to handle but I'll throw some ideas out there for now.
Actual Trie Classes - Here is a good C# example of classes for setting up a Trie. It sounds like you already have this part figured out.
Storing: I would persist the trie data to XML unless you are already using a database and have some need to have it in a dbms. The XML will be simple to work with in the MVC application and you don't need to worry about database connectivity issues, or the added cost of a database. I would also have two versions of the trie data on the server, a production copy and a production support copy, the second for which your admin can perform transactions against.
Loading In your admin module of the application, you may implement a feature for loading the trie data into memory, the frequency of data loading depends on your application needs. It could be scheduled or available as a manual function. Like in wordpress sites, if a user should access it while updating they would receive a message that the site is undergoing maintenance. You may choose to load into memory on demand only, and keep the trie loaded at all times except for if problems occurred.
Updating - I'd have a second database (or XML file) that is used for applying updates. The method of applying updates to production would depend partially on the frequency, quantity, and time of updates. One safe method might be to store transactions entered by the admin.
For example:
trie.put("John", 112);
trie.put("Doe", 222);
trie.Remove("John");
Then apply these transactions to your production data as needed via an admin function. If needed put your site into "maint" mode. If the updates are few and fast you may be able to code the site so that it will hold all work until transactions are processed, a user might have to wait a few milliseconds longer for a result but you wouldn't have to worry about mutating data issues.
This is pretty vague but just throwing some ideas out there... if you provide comments I'll try to give more.
1 Store trie in cache:
It is not dynamic data, and caching helps us in other tasks (like concurrency access to trie by admin and user)
2 Make access to cache clear:
:
public class TrieHelper
{
public Trie MyTrie
{
get
{
if (HttpContext.Current.Cache["myTrieKey"] == null)
HttpContext.Current.Cache["myTrieKey"] = LoadTrieFromFile(); //Returns Trie object
return (Trie)HttpContext.Current.Cache["myTrieKey"];
}
}
3 Lock trie object while adding operation in progress
public void AddWordToTrie(string word)
{
var trie = MyTrie;
lock (HttpContext.Current.Cache["myTrieKey"])
{
trie.AddWord(word);
} // notify that trie object locking when write data to file is not reuired
WriteNewWordToTrieFile(word); // should lock FileWriter object
}
}
4 If editing is performs by 1 admin at a time - store trie in xml file - it will be easy to implement logic of search element, after what word your word should be added (you can create function, that will use MyTrie object in memory), and add it, using linq to xml.
I've got a kind'a the same but 10 times bigger :)
The client design it's own calendar with questions ans possible answer in the meanwhile some is online and being used by the normal user.
What I come up was something as test and deploy. The Admin enters the calendar values and set it up correctly and after he can use a Preview button to see if it's like he needs/wants, then, to make the changes valid to all end users, he need to push Deploy.
He, as an ADMIN, will know that, until he pushes the DEPLOY button, all users accessing the Calendar will have the old values. Soon he hits deploy all is set in the Database, and pushed the files he uploaded into Amazon S3 (for faster access).
I update the Cache with the new calendar and the new Calendar object is cached until the App pool says otherwise or he hit the Deploy button again.
You could do something like this.
As you are going to perform your application in the cloud environment, I'd suggest you to take a look at CQRS and durable messaging and provide some concurrency model (possibly, optimistic concurrency and intelligent conflict detection http://skillsmatter.com/podcast/design-architecture/cqrs-not-just-for-server-systems 5:00)
Also, obviously, you need to analyze your business requirements more precisely because, as Udi Dahan mentioned, race conditions are result of the lack of business analysis.
I'm still uncertain about how best to use Mocks when doing development from the outside-in (i.e. write test first that mimics the customer's requirement)
Assume my client's requirement is "Customer can cancel her order".
I can write a test for this - Test_Customer_Can_Cancel_Her_Order.
Using TDD I then write the business class and mock the data access layer (or any layer that is 'deeper' than my business logic layer). Fantastic, my test passes and I am only testing the business layer and not any layers underneath.
Now what I am wondering is... when do I delve deeper and start writing and testing the data access layer? Directly after the top-level test? Only when I write the integration test? Maybe it doesn't matter? As long as it gets done at some point?
Just wondering...
This is a tricky question. Some people think that it's ok to test "private" code, some people think it's not. It is certainly easier, as you don't have to do complex setups to reproduce an application state that lets you test a small function that you want to add (you can test that function directly). I used to be among those people, but now I'd say that it's better to just test public APIs (create tests and setups that do and see only what the user can do and see). The reason for this is that if you only test public API, you empower yourself with unlimited re-factoring potential. If you test private code, you will have to change your tests if you choose to re-factor (making you less prone to "refactor mercilessly", thus leading to bad code).
However, I have to say, testing only public APIs do lead to more complex test frameworks.
So, to answer to your question more specifically, your next step is to try to think of a situation where there would be a problem with your current order cancellation code and write a new high-level setup to reproduce that problem. You mentioned data access tests. Create a high level setup where the database is corrupt, and test that your application behaves gracefully.
Oh, if for GUI applications, it's a little trickier, as you don't really want to get to the point where you test mouse clicks and read pixels. That's just silly. If you use (and you should) the MVC system, stop at the controller level.
Use integration tests for query and other data access testing.
I always write tests to test the DAO layer, which isn't testing business logic, but I feel it is important to test the CRUD features. This has gotten me into a lot of trouble because if my database is corrupt for some reason my tests have the high possibility of failing. What I do to prevent these DAO type tests from failing is, first do the testing in a non-production database. Then for each CRUD/DAO test I
find objects that may have been left around from a previous test and if exist I delete them.
I create objects I want to test
I update the objects I want to test
I clean up or delete the objects I created.
This sequence helps me to make sure my database is in a condition where my tests will not fail if run twice and the first time the test stopped half way in between.
Another way is to wrap your CRUD tests in a transaction and at the end of the test rollback the transaction so the database is in the state that it began.
I'm trying to follow TDD (i'm newbee) during development of Service class which builds tasks passed by Service clients. The built objects are then passed to other systems. In other words this service takes tasks but returns nothing as a result - it passes built tasks to other Services.
So I'm wondering how can I write test for it because there is nothing to assert.
I'm thinking about using mocks to track interactions inside the service but I'm a little bit afraid of using mocks because I will be tied up with internal implementarion of the service.
Thanks all of you in advance!
There's no problem using mocks for this, since you are effectively going to be mocking the external interface of the components that are used internally in the component. This is really what mocking is intended for, and sound like a perfect match for your use case.
When doing TDD it should also allow you to get those quick turnaround cycles that are considered good practice, since you can just create mocks of those external services. These mocks will easily allow you to write another failing test.
You can consider breaking it up in a couple classes. One responsible to build the list of tasks that will be executed, and the other responsible to execute the list of tasks it is handed. This way you can directly test the code that build the lists of tasks.
That said, I want to add a sample I posted on another question, regarding how I view the TDD process when external systems are involved.
Lets say you have to check whether
some given logic sends an email, logs
the info on a file, saves data on the
database, and calls a web service (not
all at once I know, but you start
adding tests for each of those). On
each test you don't want to hit the
external systems, what you really want
to test is if the logic will make the
calls to those systems that you are
expecting it to do. So when you write
a test that checks that an email is
sent when you create an user, what you
test is if the logic calls the
dependency that does that. Notice that
you can write these tests and the
related logic, without actually having
to implement the code that sends the
email (and then having to access the
external system to know what was sent
...). This will help you focus on the
task at hand and help you get a
decoupled system. It will also make it
simple to test what is being sent to
those systems.
Not sure what language you're using so in psuedo-code it could be something like this:
when_service_is_passed_tasks
before_each_test
mockClients = CreateMocks of 3 TaskClients
tasks = GetThreeTasks()
myService = new TaskRouter(mockClients)
sends_all_to_the_appropriate_clients
tasks = GetThreeTasks()
myService.RouteTaks(tasks)
Assert mockClients[0].AcceptTask(tasks[0]) was called
Assert mockClients[1].AcceptTask(tasks[1]) was called
Assert mockClients[2].AcceptTask(tasks[2]) was called
if_one_routing_fails_all_fail
tasks = GetTasksWhereOneIsFailing()
myService.RouteTaks(tasks)
Assert mockClients[0].AcceptTask(*) was not called
Assert mockClients[1].AcceptTask(*) was not called
Assert mockClients[2].AcceptTask(*) was not called