https://docs.microsoft.com/en-us/azure/devops/test/load-test/reference-qa?view=azure-devops
Q: What are virtual users?
A: Virtual users create load by accessing your app or web site all at the same time during your test run. That way, you can test performance under more realistic or projected conditions. Virtual users are simulated by test agents.
Q: How many virtual users can I configure in my load test?
A: In the full version of Visual Studio Enterprise, the number of virtual users is unlimited. In Visual Studio Enterprise trial version, the virtual user count is limited to 250.
Q: Is there a difference between what I can analyze during a running test versus a completed test?
A: Yes, these are the differences:
-
Performance counters A smaller subset of the performance counter data is available while a test is running.
-
Views When the load test run has completed, the Summary View and Details View are available.
Performance counters A smaller subset of the performance counter data is available while a test is running.
Views When the load test run has completed, the Summary View and Details View are available.
Q: Can virtual users simulate pausing between test steps?
A: Yes, you can specify think times to simulate the time spent by a user on a web page.
Q: Why should I use Cloud-based Load Testing?
A: If you don't want to set up machines for load testing, or you don't have available resources, you can use the Cloud-based Load Testing service. It sets up virtual machines in the cloud that will run your load test. Note that your web site must be publicly available on the internet for load testing using Azure DevOps to access it.
Q: Are there any limits when running the cloud-based load tests?
A: Yes. Based on where you're running the test, each test run duration limit is:
- Visual Studio IDE: 48 hours
- Azure DevOps Load test page:
- URL-based load tests: 48 hours
- JMeter load tests: 4 hours
- Azure portal: 1 hour
Q: Can I have other test types, besides web performance tests, in a load test mix?
A: Yes, you can include unit tests and coded web tests, but not coded UI tests.
Q: Does it support any other Web Test file formats?
A: At present only Visual Studio Web Test format files are supported. We'd be pleased to hear from you if you need support for other file formats. Email us at vsoloadtest@microsoft.com.
Q: What are test agents? How do they relate to my test run?
A: Test agents are computing resources, like CPU, memory, and network, that generate load by simulating virtual users. Test agents use agent cores to create virtual users. Each core creates at least 1 virtual user.
For load test runs in Azure DevOps with the Visual Studio IDE, you can specify the number of cores to use. For example, if you get errors when you run your test, you might have to increase the number of cores.
Otherwise, your tests and the number of virtual users that you specify determine how many cores and agents are used.
Q: What is the maximum test duration and number of concurrent users?
A: The limitations for load testing in the Azure Portal depend on the web application service tier license type, as follows:
| License type | Max duration (mins) | Max user load (VUser) |
|---|---|---|
| Free | 1 | 40 |
| Shared | 30 | 1,000 |
| Basic/Standard/Premium | 60 | 20,000 |
Q: Where do I specify the number of cores for runs in Azure DevOps with the Visual Studio IDE?
A: You can do that here:
What do the values mean?
| Cores | Agents |
|---|---|
| 0 | (Default) The number of cores is based on the number of virtual users that you specify for your test. |
| 1 | Your test run will use 1 agent. |
| 2 - 10 | Each agent will always use 2 cores. |
| 11 - 40 | Each agent will always use 4 cores. |
| 41 - 200 | Each agent will always use 8 cores. |
The maximum number of cores for each test run is 200. If your test run needs more cores, you can run up to 10 load tests at the same time.
The minimum number of virtual users per agent core is 1. If your load test requires more cores, contact vsoloadtest@microsoft.com.
The number of agents also depends on your text mix (web performance test or unit test). If you have only web performance tests, we suggest using between 600 and 2,500 virtual users for every two cores. If you have unit tests, the agent count depends on what your unit tests do. This means you will have to test if you have enough agents by running a shorter duration load test run or use goal-based load testing.
Q: What are virtual user minutes (VUMs)? How many minutes will my load test use?
A: If your test run uses 25 or more virtual users per core, then VUMs = (max virtual user load for your test run) * (test run duration in minutes).
If your test run uses fewer than 25 users per core, then VUMs = (number of cores) * (25 virtual users per core) * (test run duration in minutes).
The minimum values used to calculate VUMs are 25 virtual users and 1 minute. If your test run values are smaller than the minimum values, then those values are rounded up to meet the minimums. For example, if your test run specifies 20 virtual users for 30 seconds, then your test run will actually run with 25 virtual users for 1 minute = 25 VUMs, not 15 VUMs.
Also, test run duration is in minutes, not seconds. For example, if your test run duration is 5 minutes and 15 seconds, then that duration is rounded up to 6 minutes.
A minimum of 250 virtual user minutes, including the warm-up period, is deducted from your subscription for:
- Completed runs, based on the full duration of the run
- Aborted runs, based on the elapsed run duration
No comments:
Post a Comment