I have a coded web test being executed in visual studio load test. I deploy it so it runs in Azure. The analysis is pretty cool, but I can't seem to figure out how to correlate remote server metrics into my load test results (i.e. metrics of the server receiving my load test requests).
From what I can tell, it seems like I might have to install the "agent" software on my remote server. How would I do this with an Azure cloud service deployment? It doesn't seem possible with a cloud service deployment.
Also, it looks like a load test can aggregate App Insight data, but from what I can tell, App Insight doesn't give me the low-lever metrics I'm looking for. I was "perf"-like data: Counters for CPU/Disk/Network/Kernel/etc. Please correct me if I'm wrong.
Does anyone know of a way to automatically include remote server metrics into a Visual Studio load test?
A load test can collect performance counters from other computers. It should be able to collect any counter that Perfmon running on the computer running the load test can see. If you set the permissions and firewalls etc so the computer that runs the tests can run Perfmon and collect the counters then the load test should also be able to collect them.
A load test that is run in the cloud using VSTS is run under a Microsoft account from one of their computers. This means setting the permissions etc on the servers being load tested is difficult because the accounts are unknown.
If a load test is run on your own computers then you know the accounts running the tests and can set the server permissions to allow those accounts the appropriate accesses.
Visual Studio agents can be used on your computers to run the load tests. They are most commonly used when the load required (the number of virtual users) is too high for one computer. Agents do not run on the servers being tested. The Agent software is not used when a cloud load test is run by VSTS. (Actually it might be used by Microsoft, but it will be almost invisible to you when you run a test with VSTS.)
Related
I want to perform load testing of Single Page Application. So i have created JMeter script for that and it is working fine from my single machine for concurrent users. But i want to create actual concurrent users scenario from different machine or location. For that reason i am trying to use Azure Load Testing service and there want to use JMeter script which i have created on my local machine.
Please let me know if Azure Load testing service support below points : -
Is it possible to test performance of API call which are going to other servers (non Azure).
Is it possible to track performance report like JMeter for each API call individually.
Please let me know if any clue on above problems.
Thanks
As per documentation
Enable developers and testers to generate high-scale load and run simulations that reveal actionable insights into app performance, scalability, and capacity with a fully managed load-testing service. Create tests quickly without prior knowledge of load testing tools, or upload your existing Apache JMeter scripts. Gain specialized recommendations backed by comprehensive metrics and analytics, and support continuous improvement through automated continuous integration and continuous delivery (CI/CD) workflows—all with a testing service built for Azure.
So it should be possible to run your JMeter scripts from Azure. If your system under test is reachable from the Internet you it doesn't matter where it's deployed.
If not and your application is only accessible from your local Intranet - you will have to go for JMeter Distributed Testing
Our cloud-deployed app is composed of (simplified):
A web front-end
Some back-end services
A database
When developing the front-end, I can easily debug by running the front-end locally, and redirecting its back-end calls to the actual services, since their endpoint routes are public.
However, I also want to debug back-end service code, together with the front-end. I can run back-end services locally, but they can't access the database, since the database doesn't have any publicly-accessible endpoint.
Question: How can I conveniently develop the service code? I can think of these options:
Expose the database publicly, maybe just the dev env's database. This doesn't sound like a good practice security-wise, and I haven't found a way to do it in my cloud platform (CloudFoundry).
Test everything using local unit- and component- tests. We do this, but can't cover everything, and certainly not the integration with the front-end.
Deploy my code changes to a dev environment, and test that way. This is what we do now, but:
It's a much slower development turn-around that running locally
I can't connect a debugger to the deployed app, so I must debug using logs, which again is slow
We have a limited number of dev environments, and this creates contention for them.
Somehow deploy a replica of the database locally as well, using some kind of test data.
Tech details: For cloud we use CloudFoundry over AWS. My back-end services are written in C# + .NET core 5. Locally, we develop them using Visual Studio 2019 on Windows.
For now, I managed to expose my database locally using an SSH tunnel. Specifically, by running cf ssh [AppName] -L [local_port]:[db_hostname]:[port], with [db_hostname]:[port] taken from the app's configuration, taken by running cf env [AppName].
Came across an article https://devblogs.microsoft.com/devops/cloud-based-load-testing-service-eol/ which says that cloud-based load testing service has been deprecated.
in this article, at one point it says that,
"Running a load test in the cloud using Visual Studio (both
auto-provisioned agents and self-provisioned ‘bring-your-own’ agents):
You can continue to run the tests but self-hosted with Test
Controller/Test Agent option outlined above."
If all my load testing agents, controllers are created as VMs in Azure - and I use my VSTS subscription to run these load tests then do I need to worry? The above paragraph is a little confusing, hence asking.
I searched for about one week for the oppurtunity to monitor different windows events, for example the SQL-event(or service) in AzureRM virtual machines.
I tried it with different LogAnalytics queries, Runbooks, Powershell scripts to connect to the vm, etc. But everything I tried doesn't work.
Do you have any suggestions?
The solution should inform me, when a windows service stopped.
Best regards!
Azure Monitor, which now includes Log Analytics and Application Insights, provides sophisticated tools for collecting and analyzing telemetry that allow you to maximize the performance and availability of your cloud and on-premises resources and applications. It helps you understand how your applications are performing and proactively identifies issues affecting them and the resources they depend on.
Activity log alerts are the alerts that get activated when a new activity log event occurs that matches the conditions specified in the alert. You can follow the following document to set an activity log alert: https://learn.microsoft.com/en-us/azure/monitoring-and-diagnostics/alert-activity-log?toc=/azure/azure-monitor/toc.json
Helow
Is it possible to use Jmeter to make a performance test on a mobile app which doesn´t connect to internet, such as calendar or calculator; an app that doesn´t use http protocol?
Thank you.
No.
JMeter acts on protocol level, it doesn't actually clicks links or buttons, it sends requests over variety of supported protocols (not limited to HTTP) and records response times.
If your application doesn't have a backend, i.e. it is completely standalone, you normally don't need to load test it as it won't have more than one user. The only thing you can do is to check resources impact like CPU or RAM usage, battery consumption, most long running code parts, etc. Check out the following materials:
Android Performance Profiling Tools
iOS App Performance: Instruments & beyond
Performance Profiling on Windows Phone 8.1 with Visual Studio
For more detailed explanation of the concept of simulating multiple non-browser applications users check out How to Run Performance Tests of Desktop Applications Using JMeter guide.
You don't need internet connection in order to test like a local website, service or app. And yes, it is possible.
If your app is locally then the app will be available on a localhost port ( db, api, website). If you plan to test from the UI perspective a mobile or desktop app then you are using the wrong tool and you should use other tools like Ranorex or Test Complete.