Please visit the following link:
http://msdn.microsoft.com/en-us/library/aa730850(vs.80).aspx
Monday, March 30, 2009
Web and Load Test FAQs
Please visit the following link:
http://social.msdn.microsoft.com/Forums/en-US/vstswebtest/thread/bd3b1caf-ca7b-408e-b415-3b8e3465bb03
http://social.msdn.microsoft.com/Forums/en-US/vstswebtest/thread/bd3b1caf-ca7b-408e-b415-3b8e3465bb03
Web Test Authoring and Debugging Techniques for VS 2008
Please visit the following link:
http://blogs.msdn.com/edglas/archive/2007/12/02/web-test-authoring-and-debugging-techniques-for-vs-2008.aspx
http://blogs.msdn.com/edglas/archive/2007/12/02/web-test-authoring-and-debugging-techniques-for-vs-2008.aspx
Web Test Authoring and Debugging Techniques--Visual Studio 2005 Technical Articles
Please visit the following link:
http://msdn2.microsoft.com/en-us/library/ms364082(VS.80).aspx
http://msdn2.microsoft.com/en-us/library/ms364082(VS.80).aspx
Friday, March 27, 2009
How to capture multiple browser windows using WebTest?
Dear All,
We have a web application when we click on a hyperlink it opens in a new browser window.i want to check functionality of that hyperlink.but, webtest not recording the hyperlink window.
Please guide how to capture it?
Thanks,
Swaroop
-------------------------------------------------------------------------------
Are you saying that when you click in the new window the requests are not recorded in the origningal window?
--------------------------------------------------------------------------------
Blog - http://blogs.msdn.com/slumley/default.aspx
--------------------------------------------------------------------------------
Hi Kasireddy,
"Web Test Recorder" cannot record more then one browser at a time.
If you want to record both browser request you may want to use Fiddler - Please erad this post on how to Recording Web Test Using Fiddler
Good Luck
-------------------------------------------------------------------
After you record the first webpage(a.webtest), you can also also record the second page (b.webtest)
Just put all of the webrequests in b.webtest into a.webtest.
Besides fiddler, it is another way.
--------------------------------------------------------------------------------
Ray
----------------------------------------------------------------------------------
Hi Slumley,
Copying new browser URL into Record browser window solved the problem.
Thanks,
Swaroop
We have a web application when we click on a hyperlink it opens in a new browser window.i want to check functionality of that hyperlink.but, webtest not recording the hyperlink window.
Please guide how to capture it?
Thanks,
Swaroop
-------------------------------------------------------------------------------
Are you saying that when you click in the new window the requests are not recorded in the origningal window?
--------------------------------------------------------------------------------
Blog - http://blogs.msdn.com/slumley/default.aspx
--------------------------------------------------------------------------------
Hi Kasireddy,
"Web Test Recorder" cannot record more then one browser at a time.
If you want to record both browser request you may want to use Fiddler - Please erad this post on how to Recording Web Test Using Fiddler
Good Luck
-------------------------------------------------------------------
After you record the first webpage(a.webtest), you can also also record the second page (b.webtest)
Just put all of the webrequests in b.webtest into a.webtest.
Besides fiddler, it is another way.
--------------------------------------------------------------------------------
Ray
----------------------------------------------------------------------------------
Hi Slumley,
Copying new browser URL into Record browser window solved the problem.
Thanks,
Swaroop
List separator for csv files in web test
I use csv files in my web tests as data source. On my PC those tests run fine, but sometimes on other PC there is a problem with list separator. Where can I change list separator as I like (it didn't work with computer regional settings)?
Thanks!
----------------------------------------------------------------------------------
Do you mean that you are running on a machine with a different language. Check for this registry key and see what values you have for the 2 different machines:
Key: HKEY_CURRENT_USER\Control Panel\International
Name: sList
Blog - http://blogs.msdn.com/slumley/default.aspx
---------------------------------------------------------------------------------
This issue wont help. When I ran my test from VS, list separator isn't taken from regional settings (this reg key). I'm running my tests on machines with one language (in key sList separator are equal), but in VS on some PC a different list separator is needed...
---------------------------------------------------------------------------------
If you are still having trouble, I suggest using a different datasource such as xml, excel or a database.
--------------------------------------------------------------------------------
Blog - http://blogs.msdn.com/slumley/default.aspx
Thanks!
----------------------------------------------------------------------------------
Do you mean that you are running on a machine with a different language. Check for this registry key and see what values you have for the 2 different machines:
Key: HKEY_CURRENT_USER\Control Panel\International
Name: sList
Blog - http://blogs.msdn.com/slumley/default.aspx
---------------------------------------------------------------------------------
This issue wont help. When I ran my test from VS, list separator isn't taken from regional settings (this reg key). I'm running my tests on machines with one language (in key sList separator are equal), but in VS on some PC a different list separator is needed...
---------------------------------------------------------------------------------
If you are still having trouble, I suggest using a different datasource such as xml, excel or a database.
--------------------------------------------------------------------------------
Blog - http://blogs.msdn.com/slumley/default.aspx
IP Switching Question
Hi,
When setting a range for the IP switching feature, does it have to be a range of IP addresses that are not in use already? I know about not using an address already configured on the chosen nic from Bill's blog, I am talking about addresses that may be in use by other physical hosts on the network.
The reason I am asking is that common sense says they should be free otherwise there will be a conflict, but I ran a test the other day using a range that I later found out had loads of real hosts! And while none of the IP's were actually used during the test (they were not web tests), the agent logs show that all the binds succeeded with no problems. And then today I ran the same load test with no changes and the moment I started it, the agent went off the network! I checked the event log when it came back and the reason was that on the very final IP address bind of the range, it had a duplicate IP conflict with another box on the network and as a result, disabled its own nic!
So I am really confused now as I did a ping scan and found that virtually all of the addresses I used in my range do have real hosts allocated and also, why did it work first time round, then second time round only fail on the very last one it tried to bind?
Cheers
Dan
----------------------------------------------------------------------------------
can anyone confirm whether the ip addresses for ip switching need to be free ones or if it doesn't matter if they are in use?
i really need to know this as i cannot chance using this again as if it goes bad i have to call out a support guy to physically reset the network card!
----------------------------------------------------------------------------------
They need to be free.
--------------------------------------------------------------------------------
Blog - http://blogs.msdn.com/slumley/default.aspx
When setting a range for the IP switching feature, does it have to be a range of IP addresses that are not in use already? I know about not using an address already configured on the chosen nic from Bill's blog, I am talking about addresses that may be in use by other physical hosts on the network.
The reason I am asking is that common sense says they should be free otherwise there will be a conflict, but I ran a test the other day using a range that I later found out had loads of real hosts! And while none of the IP's were actually used during the test (they were not web tests), the agent logs show that all the binds succeeded with no problems. And then today I ran the same load test with no changes and the moment I started it, the agent went off the network! I checked the event log when it came back and the reason was that on the very final IP address bind of the range, it had a duplicate IP conflict with another box on the network and as a result, disabled its own nic!
So I am really confused now as I did a ping scan and found that virtually all of the addresses I used in my range do have real hosts allocated and also, why did it work first time round, then second time round only fail on the very last one it tried to bind?
Cheers
Dan
----------------------------------------------------------------------------------
can anyone confirm whether the ip addresses for ip switching need to be free ones or if it doesn't matter if they are in use?
i really need to know this as i cannot chance using this again as if it goes bad i have to call out a support guy to physically reset the network card!
----------------------------------------------------------------------------------
They need to be free.
--------------------------------------------------------------------------------
Blog - http://blogs.msdn.com/slumley/default.aspx
Wednesday, March 25, 2009
Need to get all urls from a web page
Hi ,
I m doing my first webtest using VS i m trying to get all urls from the page and them later use those urls.but i tried different things to get urls for example when i place extrace rule for html tag.start tag = a and attribute = href then it returns only first this is main issue.
2nd how i can get my my context variable value .following is the code.
Thanks
awais
WebTestRequest request1 = new WebTestRequest("http://www.worldgolf.com/courses/usa/alabama/");
ExtractAttributeValue extractionRule1 = new ExtractAttributeValue();
extractionRule1.TagName = "a";
extractionRule1.AttributeName = "href";
extractionRule1.MatchAttributeName = "";
extractionRule1.MatchAttributeValue = "";
extractionRule1.HtmlDecode = true;
extractionRule1.Required = true;
extractionRule1.Index = 0;
extractionRule1.ContextParameterName = "";
extractionRule1.ContextParameterName = "values";
request1.ExtractValues += new EventHandler(extractionRule1.Extract);
//string a = this.Context["address"].ToString();
yield return request1;
aq
-------------------------------------------------------------------------------------
Hi Awais,
Here is a complete coded webtest that shows retreiving all URL's from anchor tags that contain hrefs. The urls are all stored in a List, and will only add unique urls.
1 //------------------------------------------------------------------------------
2 //
3 // This code was generated by a tool.
4 // Runtime Version:2.0.50727.3521
5 //
6 // Changes to this file may cause incorrect behavior and will be lost if
7 // the code is regenerated.
8 //
9 //------------------------------------------------------------------------------
10
11 namespace TestExperimentMar09
12 {
13 using System;
14 using System.Collections.Generic;
15 using System.Text;
16 using Microsoft.VisualStudio.TestTools.WebTesting;
17
18
19 public class WebTest3Coded : WebTest
20 {
21 List urlList = new List();
22
23 public WebTest3Coded()
24 {
25 this.PreAuthenticate = true;
26
27 }
28
29 public override IEnumerator GetRequestEnumerator()
30 {
31
32 this.PostRequest += new EventHandler(WebTest3Coded_PostRequest);
33 WebTestRequest request1 = new WebTestRequest("http://msdn.microsoft.com/");
34 request1.ExpectedResponseUrl = "http://msdn.microsoft.com/en-us/default.aspx";
35 yield return request1;
36 request1 = null;
37
38
39 }
40
41 void WebTest3Coded_PostRequest(object sender, PostRequestEventArgs e)
42 {
43 //lets make sure that we have a valid HTML response before we start to work on the Response.HtmlDocument
44 if (e.Response.IsHtml)
45 {
46 //check a filtered list of html tags that are anchor tags
47 foreach (HtmlTag htmlTag in e.Response.HtmlDocument.GetFilteredHtmlTags(new string[] {"a"}))
48 {
49 //makes sure our htmlTag has some attributes, otherwise no need to continue
50 if (htmlTag.Attributes != null)
51 {
52 //loop through all the htmlTag.attributes to see if what we need is there
53 foreach (HtmlAttribute attributeTag in htmlTag.Attributes)
54 {
55 //lets make sure we dont have a bad htmlTag
56 if ( !string.IsNullOrEmpty(attributeTag.Name) )
57 {
58 //we are only concerned with href attributes , other checks could be done here
59 if (attributeTag.Name.Equals("href", StringComparison.InvariantCultureIgnoreCase))
60 {
61 //if the list has the url we dont need to add it again
62 if (! this.urlList.Contains(attributeTag.Value))
63 {
64 //add the url to our master list
65 this.urlList.Add(htmlTag.GetAttributeValueAsString("href"));
66 }
67
68 }
69 }
70 }
71 }
72 }
73 }
74 }
75 }
76 }
77
------------------------------------------------------------------------------
Thanks a lot :)
awais
-------
aq
I m doing my first webtest using VS i m trying to get all urls from the page and them later use those urls.but i tried different things to get urls for example when i place extrace rule for html tag.start tag = a and attribute = href then it returns only first this is main issue.
2nd how i can get my my context variable value .following is the code.
Thanks
awais
WebTestRequest request1 = new WebTestRequest("http://www.worldgolf.com/courses/usa/alabama/");
ExtractAttributeValue extractionRule1 = new ExtractAttributeValue();
extractionRule1.TagName = "a";
extractionRule1.AttributeName = "href";
extractionRule1.MatchAttributeName = "";
extractionRule1.MatchAttributeValue = "";
extractionRule1.HtmlDecode = true;
extractionRule1.Required = true;
extractionRule1.Index = 0;
extractionRule1.ContextParameterName = "";
extractionRule1.ContextParameterName = "values";
request1.ExtractValues += new EventHandler
//string a = this.Context["address"].ToString();
yield return request1;
aq
-------------------------------------------------------------------------------------
Hi Awais,
Here is a complete coded webtest that shows retreiving all URL's from anchor tags that contain hrefs. The urls are all stored in a List
1 //------------------------------------------------------------------------------
2 //
3 // This code was generated by a tool.
4 // Runtime Version:2.0.50727.3521
5 //
6 // Changes to this file may cause incorrect behavior and will be lost if
7 // the code is regenerated.
8 //
9 //------------------------------------------------------------------------------
10
11 namespace TestExperimentMar09
12 {
13 using System;
14 using System.Collections.Generic;
15 using System.Text;
16 using Microsoft.VisualStudio.TestTools.WebTesting;
17
18
19 public class WebTest3Coded : WebTest
20 {
21 List
22
23 public WebTest3Coded()
24 {
25 this.PreAuthenticate = true;
26
27 }
28
29 public override IEnumerator
30 {
31
32 this.PostRequest += new EventHandler
33 WebTestRequest request1 = new WebTestRequest("http://msdn.microsoft.com/");
34 request1.ExpectedResponseUrl = "http://msdn.microsoft.com/en-us/default.aspx";
35 yield return request1;
36 request1 = null;
37
38
39 }
40
41 void WebTest3Coded_PostRequest(object sender, PostRequestEventArgs e)
42 {
43 //lets make sure that we have a valid HTML response before we start to work on the Response.HtmlDocument
44 if (e.Response.IsHtml)
45 {
46 //check a filtered list of html tags that are anchor tags
47 foreach (HtmlTag htmlTag in e.Response.HtmlDocument.GetFilteredHtmlTags(new string[] {"a"}))
48 {
49 //makes sure our htmlTag has some attributes, otherwise no need to continue
50 if (htmlTag.Attributes != null)
51 {
52 //loop through all the htmlTag.attributes to see if what we need is there
53 foreach (HtmlAttribute attributeTag in htmlTag.Attributes)
54 {
55 //lets make sure we dont have a bad htmlTag
56 if ( !string.IsNullOrEmpty(attributeTag.Name) )
57 {
58 //we are only concerned with href attributes , other checks could be done here
59 if (attributeTag.Name.Equals("href", StringComparison.InvariantCultureIgnoreCase))
60 {
61 //if the list has the url we dont need to add it again
62 if (! this.urlList.Contains(attributeTag.Value))
63 {
64 //add the url to our master list
65 this.urlList.Add(htmlTag.GetAttributeValueAsString("href"));
66 }
67
68 }
69 }
70 }
71 }
72 }
73 }
74 }
75 }
76 }
77
------------------------------------------------------------------------------
Thanks a lot :)
awais
-------
aq
Analysis on 1000 Vusers
Affuu - Posted on Tuesday, March 24, 2009 1:50:06 AM
I want to do an analysis by loading an application with 1000 users and the SUT,controller,agent are on the same system will I be able to do it,if yes I want to know what will be the Run time settings i need to do,I want to go for constant load of 1000 users,please help me by providing more information on this,I need to check out the CPU utilization,RPS,Memory utilization etc.What counters exactly i need to select so that I can come to a conclusion that how many agents i need to add if I increase the load from 1000 to 2000 to 3000....
What will be the best configuration i.e the number of agents,controller,SUT(its only one system) and SQL express(load store)
One more thing when I run the webtest before adding it to load few of the dependant requests are failing but the mail request page is displaying fine do i need to rectify/make it pass completely before adding to load or is it fine as long as the main requests are rendered successfully
please help me as I am new to VSTS 2008 tool
------------------------------------------------------------------------------------
Hi
For hardware sizing and the number of required agents, please refer to Performance Considerations section in Load Agent and Load Controller Installation and Configuration Guide for more information. In a test rig, you just need 1 controller.
There are many performance counter category available by default. You can use counters in Processor and Memory category to check the CPU and memory usage. If you are testing an ASP.NET application, Performance Counters for ASP.NET might also be helpful.
It's correct to fix web tests before adding them in load tests. About the failing of dependent requests, please take a look at Masking a 404 error in a dependent request. If you are in a different situation, could you give us more details?
-------------------------------------------------------------------------------------
Hi Aff,
you basically need more than 1 computer.. try to get such as 1 computer for Controller and a few PCs for Agents.. by putting agent and controller at the same computer, the result will not be accurate.
To load 1000 users for a simple system, that will be fine.
But, make sure you start at small number of users, and check the CPU load if it can handle such number of users.
YOu can check the CPU utilization and etc from the perfmon in ur server.
Run time setting is based on what is ur need. If you think all this 1000 users will be using the system for 1 hour flat, then set it as 1 hour.
Benny.
-------------------------------------------------------------------------------------
Hi Affu,
Let me clairfy one part of your question. Are you saying that your Application you want to test (the System Under Test) will also be on the same box as your Controller, Agent?
This would invalidate the testing given that you want to monitor the CPU utilization, Memory consumption, etc.
The controller, agents also require memory, cpu, etc. Also, if you are executing the load test in the local configuration, not using a controller, agent, then your test process will only use a Single CPU on the system, even if you have more. Thats why its important to size your Controller, Agent(s) correctly so you will not skew your results by overloading your client.
Hope this helps.
Robert
I want to do an analysis by loading an application with 1000 users and the SUT,controller,agent are on the same system will I be able to do it,if yes I want to know what will be the Run time settings i need to do,I want to go for constant load of 1000 users,please help me by providing more information on this,I need to check out the CPU utilization,RPS,Memory utilization etc.What counters exactly i need to select so that I can come to a conclusion that how many agents i need to add if I increase the load from 1000 to 2000 to 3000....
What will be the best configuration i.e the number of agents,controller,SUT(its only one system) and SQL express(load store)
One more thing when I run the webtest before adding it to load few of the dependant requests are failing but the mail request page is displaying fine do i need to rectify/make it pass completely before adding to load or is it fine as long as the main requests are rendered successfully
please help me as I am new to VSTS 2008 tool
------------------------------------------------------------------------------------
Hi
For hardware sizing and the number of required agents, please refer to Performance Considerations section in Load Agent and Load Controller Installation and Configuration Guide for more information. In a test rig, you just need 1 controller.
There are many performance counter category available by default. You can use counters in Processor and Memory category to check the CPU and memory usage. If you are testing an ASP.NET application, Performance Counters for ASP.NET might also be helpful.
It's correct to fix web tests before adding them in load tests. About the failing of dependent requests, please take a look at Masking a 404 error in a dependent request. If you are in a different situation, could you give us more details?
-------------------------------------------------------------------------------------
Hi Aff,
you basically need more than 1 computer.. try to get such as 1 computer for Controller and a few PCs for Agents.. by putting agent and controller at the same computer, the result will not be accurate.
To load 1000 users for a simple system, that will be fine.
But, make sure you start at small number of users, and check the CPU load if it can handle such number of users.
YOu can check the CPU utilization and etc from the perfmon in ur server.
Run time setting is based on what is ur need. If you think all this 1000 users will be using the system for 1 hour flat, then set it as 1 hour.
Benny.
-------------------------------------------------------------------------------------
Hi Affu,
Let me clairfy one part of your question. Are you saying that your Application you want to test (the System Under Test) will also be on the same box as your Controller, Agent?
This would invalidate the testing given that you want to monitor the CPU utilization, Memory consumption, etc.
The controller, agents also require memory, cpu, etc. Also, if you are executing the load test in the local configuration, not using a controller, agent, then your test process will only use a Single CPU on the system, even if you have more. Thats why its important to size your Controller, Agent(s) correctly so you will not skew your results by overloading your client.
Hope this helps.
Robert
Friday, March 20, 2009
Coded web test - How to Send an email after test run and after test success or fail
http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=2724147&SiteID=1
Hi,
I want to send a successful email to a user through smtp server if test run success and on the other hand if web test fail, then i want to sent a failed email message to the user as notification.
I can send normal email from the coded scripts.
But I don't know how to retrieve web test run result status: pass or fail .........
I want to know the exact command or code as if I can able to send an email after the test run and after it's run success or fail
I very much appreciate your help. Noted that I am using VSTS 2005
Thanks,
Jaeed
-----------------------------------------------------------------------------------------
Answer Re: Coded web test - How to Send an email after test run and after test success or fail Was this post helpful ?
In VSTS 2008, in your coded web test, you can call this.Outcome.ToString(). In VSTS 2005, the property is not exposed.
Thanks.
------------------------------------------------------------------------------------------------
Re: Coded web test - How to Send an email after test run and after test success or fail
Thanks a lot Yutong. Jaeed
Hi,
I want to send a successful email to a user through smtp server if test run success and on the other hand if web test fail, then i want to sent a failed email message to the user as notification.
I can send normal email from the coded scripts.
But I don't know how to retrieve web test run result status: pass or fail .........
I want to know the exact command or code as if I can able to send an email after the test run and after it's run success or fail
I very much appreciate your help. Noted that I am using VSTS 2005
Thanks,
Jaeed
-----------------------------------------------------------------------------------------
Answer Re: Coded web test - How to Send an email after test run and after test success or fail Was this post helpful ?
In VSTS 2008, in your coded web test, you can call this.Outcome.ToString(). In VSTS 2005, the property is not exposed.
Thanks.
------------------------------------------------------------------------------------------------
Re: Coded web test - How to Send an email after test run and after test success or fail
Thanks a lot Yutong. Jaeed
How to add local text or tif or pdf files from local PC into the web test scripts
How to add local text or tif or pdf files from local PC into the web test scripts
Hello,
One of our site has a feature to upload document(text, tiff or pdf files) through its User Interface and after uploading the documents they get inserted into some database table and finally they show up to the UI.
I tried to record this by using VSTS 2008 beta/VSTS 2005 SP1 as a web test and it was not working.
I wonder - is there any alternative way or best way to make the script workable for uploading document(tiff file) from local location through the website by VSTS
Thanks,
------------------------------------------------------------------------------------------------
Answer Re: How to add local text or tif or pdf files from local PC into the web test scripts Was this post helpful ?
add the file to your test solution and add it to the deployment section of your test run configuration.
So far this is what we've had to do any time we had a test that uploads a file of any sort to the server
--Chuck
----------------------------------------------------------------------------------------------
Answer Re: How to add local text or tif or pdf files from local PC into the web test scripts Was this post helpful ?
You shouldn't have to add it to the solution, but Chuck is right, you need to add it to the deployment section of your testrunconfig
--Mike
-------------------------------------------------------------------------------------------
Re: How to add local text or tif or pdf files from local PC into the web test scripts
Thanks Chuck and Mike. It helped.
------------------------------------------------------------------------------------------
Re: How to add local text or tif or pdf files from local PC into the web test scripts Was this post helpful ?
Hi Chuck and Mike,Same scenario discussed by Jaeed but let me explain.I have a simple web page with uploading files (any txt, doc or pdf) from any location and after uploading all the files displayed to some UI. I record a web test and during the recording I have successfully upload the text file on my required application. So then according to ur solutions I then add the files to deployment area and then re-run my recorded test, The run successfully but non of the file present in deployment area is uploaded to the required applicaiton. Is there any thing which i missing.Your response in this regard will be highly appreciated.Regards,Waseem.
---------------------------------------------------------------------------------------------
Re: How to add local text or tif or pdf files from local PC into the web test scripts Was this post helpful ?
I presume you are still at the stage of getting the webtest working, and we are running in webtest mode, not loadtest mode.
So did your test generate any errors?
Open up the test results and walk through and examine the responses from the server for each step of the test.. did everything go as expected, or did you get some kind of error from the server.. (especially if you end up trying to do something like upload a file that already exists.. I'd expect some kind of different response from the server than if the file did not already exist (as in some kind of overwrite confirmation)
One thing to keep in mind with VSTS is that when page content is dynamic, if the right page URL is returned in response to the request, in the abscence of other validation checks VSTS will consider that request to be a success, NO MATTER what is actually on the page.
In that instance a very limited number of things could cause the action to fail:
unable to find a match for an extraction rule
total abscence of any hidden values if hidden values are being extracted from the page
404 error for some dependant resource
For example, if I'm doing something to update a user's credit card data on a commerce site.. after posting the data I might get back a page called updatecc.aspx.. On that page might be text to the effect of "Credit Card Updated" or "Invalid Credit Card" or "expiration date required" or some other error..
If I don't add a validation to that page to look for the success message, or to look for error messages and FAIL if they are found, then VSTS can't tell if the actual page content was correct, and the test will pass by default just because a page with the right URL came back.
--Chuck
----------------------------------------------------------------------------------------------
Re: How to add local text or tif or pdf files from local PC into the web test scripts Was this post helpful ?
Hi Chuck,I upload the files on my web application using the same recorded script by changing its some parameters. After recording the test I simply go on the step on which upload URL is present, I then change the file path of FileUploadParameter under the FromPostParameters from its properties and give the path of another file to which I want to upload and then re-run the test and I got success. But still confusing with the option which you told me for files uploading through 'Deployment' area under Test -> Edit Test Run Configurations -> Deployment .Could u plz tell me the steps by executing them I am able to upload the files through Test -> Edit Test Run Configurations -> Deployment option instead of changing the file path of FileUploadParameter .Regards,Waseem
-----------------------------------------------------------------------------------------------
Re: How to add local text or tif or pdf files from local PC into the web test scripts Was this post helpful ?
One of the MS folks might be able to explain this better (and will hopefully correct me if I have any of this wrong), but my general understanding is along these lines.
Webtests get run from more than a single machine.. They could be run by other people in your org (assuming they are checked into source control) and they can end up being run on Agent systems when run as part of a larger scale loadtests or on a dedicated workstation used for smaller scale loadtests.
So the deployment section of the test run configuration is a way to tell the system 'these are files I need to run this test' the items there might be datafiles used for databindings, or files being uploaded, or for some other purpose, but when the test runs it will need access to those files.
This is why (I believe) when a test records a file upload action, what is recorded in the test is only the filename, and not the full path. Because the system is going to expect to find that file in the deployment directory (which is created when the test runs), which is the only place it can reliably know the file will be able to be found. (after all, an agent machine, or a co-worker's system likely does not have access to your hard drive, and may not have access to the same network shares you do.)
So adding the files to the deployment list doesn't upload them anywhere, it's not an alternative to the file upload parameter.. it's a way to make sure whatever you spec in that parameter is available to the test no matter where the test is run.
One thing I'm not clear on is where the system looks for the file when it goes to deploy it. It seems the file needs to be in the same directory as your test. (which is why with my stuff being checked into source control and all, I've been adding the file to the solution.. I'm not sure if it's the 'best' way of dealing with it, but it's working for me so far.)
--Chuck
Hello,
One of our site has a feature to upload document(text, tiff or pdf files) through its User Interface and after uploading the documents they get inserted into some database table and finally they show up to the UI.
I tried to record this by using VSTS 2008 beta/VSTS 2005 SP1 as a web test and it was not working.
I wonder - is there any alternative way or best way to make the script workable for uploading document(tiff file) from local location through the website by VSTS
Thanks,
------------------------------------------------------------------------------------------------
Answer Re: How to add local text or tif or pdf files from local PC into the web test scripts Was this post helpful ?
add the file to your test solution and add it to the deployment section of your test run configuration.
So far this is what we've had to do any time we had a test that uploads a file of any sort to the server
--Chuck
----------------------------------------------------------------------------------------------
Answer Re: How to add local text or tif or pdf files from local PC into the web test scripts Was this post helpful ?
You shouldn't have to add it to the solution, but Chuck is right, you need to add it to the deployment section of your testrunconfig
--Mike
-------------------------------------------------------------------------------------------
Re: How to add local text or tif or pdf files from local PC into the web test scripts
Thanks Chuck and Mike. It helped.
------------------------------------------------------------------------------------------
Re: How to add local text or tif or pdf files from local PC into the web test scripts Was this post helpful ?
Hi Chuck and Mike,Same scenario discussed by Jaeed but let me explain.I have a simple web page with uploading files (any txt, doc or pdf) from any location and after uploading all the files displayed to some UI. I record a web test and during the recording I have successfully upload the text file on my required application. So then according to ur solutions I then add the files to deployment area and then re-run my recorded test, The run successfully but non of the file present in deployment area is uploaded to the required applicaiton. Is there any thing which i missing.Your response in this regard will be highly appreciated.Regards,Waseem.
---------------------------------------------------------------------------------------------
Re: How to add local text or tif or pdf files from local PC into the web test scripts Was this post helpful ?
I presume you are still at the stage of getting the webtest working, and we are running in webtest mode, not loadtest mode.
So did your test generate any errors?
Open up the test results and walk through and examine the responses from the server for each step of the test.. did everything go as expected, or did you get some kind of error from the server.. (especially if you end up trying to do something like upload a file that already exists.. I'd expect some kind of different response from the server than if the file did not already exist (as in some kind of overwrite confirmation)
One thing to keep in mind with VSTS is that when page content is dynamic, if the right page URL is returned in response to the request, in the abscence of other validation checks VSTS will consider that request to be a success, NO MATTER what is actually on the page.
In that instance a very limited number of things could cause the action to fail:
unable to find a match for an extraction rule
total abscence of any hidden values if hidden values are being extracted from the page
404 error for some dependant resource
For example, if I'm doing something to update a user's credit card data on a commerce site.. after posting the data I might get back a page called updatecc.aspx.. On that page might be text to the effect of "Credit Card Updated" or "Invalid Credit Card" or "expiration date required" or some other error..
If I don't add a validation to that page to look for the success message, or to look for error messages and FAIL if they are found, then VSTS can't tell if the actual page content was correct, and the test will pass by default just because a page with the right URL came back.
--Chuck
----------------------------------------------------------------------------------------------
Re: How to add local text or tif or pdf files from local PC into the web test scripts Was this post helpful ?
Hi Chuck,I upload the files on my web application using the same recorded script by changing its some parameters. After recording the test I simply go on the step on which upload URL is present, I then change the file path of FileUploadParameter under the FromPostParameters from its properties and give the path of another file to which I want to upload and then re-run the test and I got success. But still confusing with the option which you told me for files uploading through 'Deployment' area under Test -> Edit Test Run Configurations -> Deployment .Could u plz tell me the steps by executing them I am able to upload the files through Test -> Edit Test Run Configurations -> Deployment option instead of changing the file path of FileUploadParameter .Regards,Waseem
-----------------------------------------------------------------------------------------------
Re: How to add local text or tif or pdf files from local PC into the web test scripts Was this post helpful ?
One of the MS folks might be able to explain this better (and will hopefully correct me if I have any of this wrong), but my general understanding is along these lines.
Webtests get run from more than a single machine.. They could be run by other people in your org (assuming they are checked into source control) and they can end up being run on Agent systems when run as part of a larger scale loadtests or on a dedicated workstation used for smaller scale loadtests.
So the deployment section of the test run configuration is a way to tell the system 'these are files I need to run this test' the items there might be datafiles used for databindings, or files being uploaded, or for some other purpose, but when the test runs it will need access to those files.
This is why (I believe) when a test records a file upload action, what is recorded in the test is only the filename, and not the full path. Because the system is going to expect to find that file in the deployment directory (which is created when the test runs), which is the only place it can reliably know the file will be able to be found. (after all, an agent machine, or a co-worker's system likely does not have access to your hard drive, and may not have access to the same network shares you do.)
So adding the files to the deployment list doesn't upload them anywhere, it's not an alternative to the file upload parameter.. it's a way to make sure whatever you spec in that parameter is available to the test no matter where the test is run.
One thing I'm not clear on is where the system looks for the file when it goes to deploy it. It seems the file needs to be in the same directory as your test. (which is why with my stuff being checked into source control and all, I've been adding the file to the solution.. I'm not sure if it's the 'best' way of dealing with it, but it's working for me so far.)
--Chuck
Is it possible to add console application to the webtest
Is it possible to add console application to the webtest
I wanna give some user input from the console and web test will take this input as a variable and process this variable like console program.
Is it possible???
Thanks
Jaeed
-------------------------------------------------------------------------------------------------
Re: Is it possible to add console application to the webtest Was this post helpful ?
if it was possible, I think it would break things when you went to run the webtest as part of a loadtest with multiple users. And how should the test measure the test time, etc for the test when it spends time waiting on your user input?
Does the input really need to be from the console? or could it be via a datasource such as a file? that would be fairly easy to implement.. but taking console input poses all sorts of problems as far as I could predict..
--Chuck
Re: Is it possible to add console application to the webtest
Thanks
---------------------------------------------------------------------------------------------------------------
Re: Is it possible to add console application to the webtest Was this post helpful ?
I haven't attempted to use this in a loadtest (and don't expect to) but look at:
http://www.mdibb.net/net/using_the_visual_basic_input_box_in_c/
------------------------------------------------------------------------------------------------
Using the Visual Basic Input Box in C#In all versions of Visual Basic as far back as I can remember, there has been the venerable InputBox function which has been great for quickly getting an input from the user. VB.NET has it too, but C# doesn't, and it is sorely missed.
However, there is a simple solution. We can simply "borrow" the InputBox from VB.NET, by adding a reference to the required assemblies. To do this in Visual Studio, follow the steps below
In the Solution Explorer, right click on the "References" folder and select "Add Reference..."
In the ".NET" tab, scroll down and select the "Microsoft.VisualBasic" entry, then click "Ok". If you've done that ok then you should now see the reference added to the References folder. Interestingly, in .NET we can specify the location of the InputBox on screen, which I don't think you could do in the old VB days. Anyway, to use it in your code you could use code based on this example: String Prompt = "Please enter your name"; String Title = "Input Required"; String Default = "Bob"; Int32 XPos = ((SystemInformation.WorkingArea.Width/2)-200); Int32 YPos = ((SystemInformation.WorkingArea.Height/2)-100); String Result = Microsoft.VisualBasic.Interaction.InputBox(Prompt,Title,Default,XPos,YPos) if(Result != "") { MessageBox.Show("Hello " + Result); } else { MessageBox.Show("Hello whoever you are..."); } Here we are just asking for a user's name (with a default of "Bob") using a (roughly) centred InputBox, and then just saying hello with a standard message box. Unfortunately, along with inheriting the InputBox itself, .NET has also inherited its String return type - you wont find a nice object encapsulating the response like we get with the DialogResult. Because of this we need to check the string returned - if the user cancels the dialog, the string will be zero-length, otherwise it will be whatever the user typed into the input box. Note here that I've used the full name for the InputBox function. There is nothing to stop you sticking using Microsoft.VisualBasic; at the top of your class to avoid having to type out the full thing each time. Nothing special is required for compilation or distribution if you use this. I hope you find this useful - it has saved me a fair bit of time and effort when all you need is some simple text input and dont need a dedicated form.
Back 22.07.2006.
Thanks for the info, I have been looking for this for months :)
" src="http://www.mdibb.net/images/commentname.gif"> Function " src="http://www.mdibb.net/images/arrow.gif" width=2> 05.10.2006
Thank you so much!
" src="http://www.mdibb.net/images/commentname.gif"> Pie-Pacifique " src="http://www.mdibb.net/images/arrow.gif" width=2> 01.02.2007
Its a very good article, that helped me a lot
" src="http://www.mdibb.net/images/commentname.gif"> Anand " src="http://www.mdibb.net/images/arrow.gif" width=2> 27.04.2007
Thanks a lot. It was very useful.
" src="http://www.mdibb.net/images/commentname.gif"> Ponnusamy G " src="http://www.mdibb.net/images/arrow.gif" width=2> 07.06.2007
I wanna give some user input from the console and web test will take this input as a variable and process this variable like console program.
Is it possible???
Thanks
Jaeed
-------------------------------------------------------------------------------------------------
Re: Is it possible to add console application to the webtest Was this post helpful ?
if it was possible, I think it would break things when you went to run the webtest as part of a loadtest with multiple users. And how should the test measure the test time, etc for the test when it spends time waiting on your user input?
Does the input really need to be from the console? or could it be via a datasource such as a file? that would be fairly easy to implement.. but taking console input poses all sorts of problems as far as I could predict..
--Chuck
Re: Is it possible to add console application to the webtest
Thanks
---------------------------------------------------------------------------------------------------------------
Re: Is it possible to add console application to the webtest Was this post helpful ?
I haven't attempted to use this in a loadtest (and don't expect to) but look at:
http://www.mdibb.net/net/using_the_visual_basic_input_box_in_c/
------------------------------------------------------------------------------------------------
Using the Visual Basic Input Box in C#In all versions of Visual Basic as far back as I can remember, there has been the venerable InputBox function which has been great for quickly getting an input from the user. VB.NET has it too, but C# doesn't, and it is sorely missed.
However, there is a simple solution. We can simply "borrow" the InputBox from VB.NET, by adding a reference to the required assemblies. To do this in Visual Studio, follow the steps below
In the Solution Explorer, right click on the "References" folder and select "Add Reference..."
In the ".NET" tab, scroll down and select the "Microsoft.VisualBasic" entry, then click "Ok". If you've done that ok then you should now see the reference added to the References folder. Interestingly, in .NET we can specify the location of the InputBox on screen, which I don't think you could do in the old VB days. Anyway, to use it in your code you could use code based on this example: String Prompt = "Please enter your name"; String Title = "Input Required"; String Default = "Bob"; Int32 XPos = ((SystemInformation.WorkingArea.Width/2)-200); Int32 YPos = ((SystemInformation.WorkingArea.Height/2)-100); String Result = Microsoft.VisualBasic.Interaction.InputBox(Prompt,Title,Default,XPos,YPos) if(Result != "") { MessageBox.Show("Hello " + Result); } else { MessageBox.Show("Hello whoever you are..."); } Here we are just asking for a user's name (with a default of "Bob") using a (roughly) centred InputBox, and then just saying hello with a standard message box. Unfortunately, along with inheriting the InputBox itself, .NET has also inherited its String return type - you wont find a nice object encapsulating the response like we get with the DialogResult. Because of this we need to check the string returned - if the user cancels the dialog, the string will be zero-length, otherwise it will be whatever the user typed into the input box. Note here that I've used the full name for the InputBox function. There is nothing to stop you sticking using Microsoft.VisualBasic; at the top of your class to avoid having to type out the full thing each time. Nothing special is required for compilation or distribution if you use this. I hope you find this useful - it has saved me a fair bit of time and effort when all you need is some simple text input and dont need a dedicated form.
Back 22.07.2006.
Thanks for the info, I have been looking for this for months :)
" src="http://www.mdibb.net/images/commentname.gif"> Function " src="http://www.mdibb.net/images/arrow.gif" width=2> 05.10.2006
Thank you so much!
" src="http://www.mdibb.net/images/commentname.gif"> Pie-Pacifique " src="http://www.mdibb.net/images/arrow.gif" width=2> 01.02.2007
Its a very good article, that helped me a lot
" src="http://www.mdibb.net/images/commentname.gif"> Anand " src="http://www.mdibb.net/images/arrow.gif" width=2> 27.04.2007
Thanks a lot. It was very useful.
" src="http://www.mdibb.net/images/commentname.gif"> Ponnusamy G " src="http://www.mdibb.net/images/arrow.gif" width=2> 07.06.2007
Re: Question about the number of total tests in load testing
Question about the number of total tests in load testing Was this post helpful ?
Hi All,
I'm currently working on a load testing of reading one piece of property of a certain piece of equipment. What I'm confused about right now is, no matter how big user load I apply with, the number of passed tests increases to 198 and then stops increasing. The %processor time and %user time goes down to less than 1% immediately when the number of passed tests reaches 198. It's approximately happening around 3mins after the test starts to run, varing a bit depending on the number of users.
After the "198" issue occurs, everything else goes fine until the test finishes. (I set the run duration as 10 mins) So there is no fail tests, there is no other strange signs, but the passed tests number just equals to the total tests number which is 198.
It seems like 198 is a limit or something like that, but i didn't set any threshold to 198 though. Is there anyone who has an idea what it's going on here? Any help is greatly appreciated. Thanks!
Jane
26 Sep 2007, 12:15 AM UTC
Dennis Stone - MSFT
Moderator Posts 361
Re: Question about the number of total tests in load testing Was this post helpful ?
What kind of tests does your loadtest contain (web, unit, both)? Are you using any data sources in those tests, if so what kind of access method do you have specified.
http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1177487&SiteID=1
26 Sep 2007, 1:39 AM UTC
Jaeed Posts 11
Re: Question about the number of total tests in load testing Was this post helpful ?
Hi Jane,
You probably use unique access method for data source binding and that is why you cannot cross 198. Use Other data access method.
In my test, I observed the same situation, but I have overcome that and I got passed thousands of tests.
I think it would help you.
Thanks,
Abdur
26 Sep 2007, 4:12 PM UTC
JaneQQ Posts 13
Re: Question about the number of total tests in load testing Was this post helpful ?
Hello Abdur,
Thank you for your reply. I just don't quite understand what the "unique access method for data source binding" means...I do use a unique database to store the properties and equipment which are going to be read from. Do you mean the access to database or to a piece of property is unique??
Hope i can get more details. Thanks a lot!!
Jane
26 Sep 2007, 4:20 PM UTC
JaneQQ Posts 13
Re: Question about the number of total tests in load testing Was this post helpful ?
Hi,
My loadtest contains unit tests. In my tests, I created 5 pieces of equipment in the database and then created one property for each of these 5 equipment in a different table but the same database. The test method called "ReadPropertyTest" is going to read properties of equipment one for each read. And the user can only read 1 time each test. This is the basic idea that my test does.
So I don't know how to answer your question...I'm not sure what data sources i'm using and what kinda access method I've specified. Where should i check it out from??
Thanks !!
Jane
26 Sep 2007, 4:29 PM UTC
Michael Taute - MSFT
Moderator Posts 166
Re: Question about the number of total tests in load testing Was this post helpful ?
There are three different methods of accessing the data in a bound datasource. Sequential, random, and unique.
What she's saying is if you have your access method set to unique, one iteration of the test will run for each entry in your database and then it will stop. Ideally in this case though it would bubble up an out of data error and abort the test though.
It's easy enough to check... just highlight the datasource table you're binding to and view its properties.
26 Sep 2007, 5:23 PM UTC
Jaeed Posts 11
Re: Question about the number of total tests in load testing Was this post helpful ?
Jane,
Sorry I was in outside. And you got already answer from Michael.
Thanks Micheal for helping jane while I was not here.
Jaeed
28 Sep 2007, 7:23 PM UTC
JaneQQ Posts 13
Re: Question about the number of total tests in load testing Was this post helpful ?
Hi Michael,
Sorry for the delay in getting back to you. I was moved over to another issue and now i'm back. Regarding to what you've indicated here, I observed my test. In the unit test's property based on which I built up my load test, I noticed that the Data Access Method item is always sequential. And the other choice is Random, no option for unique. So I believe I've been using Sequential data access method all the time, if it is the same data access method as you were talking about.
I doubt i understood correctly because I thought I should check the data table in SQL server database, in which i stored my test data into. I checked actually, but i didn't find a data table's property called Data Access Method. You know what I mean? More specifically, my test is used to add more pieces of equipment or properties to an hierarchy of equipment model, I store all these equipment/property data in a database of SQL server on local host. Do you think I should check the data access method of this table or unit test of visual studio?
No matter which one is the thing you referred to, I didn't see anything set to Unique. Unless SQL server database use Unique data access method by default??
What am i going to do next?? Frustrated....
28 Sep 2007, 7:24 PM UTC
JaneQQ Posts 13
Re: Question about the number of total tests in load testing Was this post helpful ?
Hi Jaeed,
Thank you for your concern. That's totally fine. The issue is I still stucked on this thing...I guess i need more help.
Thanks;
Jane
28 Sep 2007, 7:52 PM UTC
Bill Barnett - MSFT
Moderator Posts 372
Answer Re: Question about the number of total tests in load testing Was this post helpful ?
I'll try to help. First of all, some of the earlier responses were asking about your data binding settings, but it's not clear to me from your responses wether or not your unit tests even use the VSTS data binding feature. In other words, do your unit test methods use data source attributes like the example shown in the docs here: http://msdn2.microsoft.com/en-us/library/ms182527(VS.90).aspx?
If not, then your problem has nothing to do with the VSTS data binding feature.
Do your unit tests do any other database access, or do something like invoke a Web service that access a database?
One possibility is that for some reason the 199th unit test gets stuck and never completes. Do you think that is a possibility? Can you describe what the unit test does? Does it send a request to a server and wait for a response? You could add some code to your unit tests to log to a file upon entering and leaving your unit test method which would help determine if the unit test itself is getting hung.
Bill
Hi All,
I'm currently working on a load testing of reading one piece of property of a certain piece of equipment. What I'm confused about right now is, no matter how big user load I apply with, the number of passed tests increases to 198 and then stops increasing. The %processor time and %user time goes down to less than 1% immediately when the number of passed tests reaches 198. It's approximately happening around 3mins after the test starts to run, varing a bit depending on the number of users.
After the "198" issue occurs, everything else goes fine until the test finishes. (I set the run duration as 10 mins) So there is no fail tests, there is no other strange signs, but the passed tests number just equals to the total tests number which is 198.
It seems like 198 is a limit or something like that, but i didn't set any threshold to 198 though. Is there anyone who has an idea what it's going on here? Any help is greatly appreciated. Thanks!
Jane
26 Sep 2007, 12:15 AM UTC
Dennis Stone - MSFT
Moderator Posts 361
Re: Question about the number of total tests in load testing Was this post helpful ?
What kind of tests does your loadtest contain (web, unit, both)? Are you using any data sources in those tests, if so what kind of access method do you have specified.
http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1177487&SiteID=1
26 Sep 2007, 1:39 AM UTC
Jaeed Posts 11
Re: Question about the number of total tests in load testing Was this post helpful ?
Hi Jane,
You probably use unique access method for data source binding and that is why you cannot cross 198. Use Other data access method.
In my test, I observed the same situation, but I have overcome that and I got passed thousands of tests.
I think it would help you.
Thanks,
Abdur
26 Sep 2007, 4:12 PM UTC
JaneQQ Posts 13
Re: Question about the number of total tests in load testing Was this post helpful ?
Hello Abdur,
Thank you for your reply. I just don't quite understand what the "unique access method for data source binding" means...I do use a unique database to store the properties and equipment which are going to be read from. Do you mean the access to database or to a piece of property is unique??
Hope i can get more details. Thanks a lot!!
Jane
26 Sep 2007, 4:20 PM UTC
JaneQQ Posts 13
Re: Question about the number of total tests in load testing Was this post helpful ?
Hi,
My loadtest contains unit tests. In my tests, I created 5 pieces of equipment in the database and then created one property for each of these 5 equipment in a different table but the same database. The test method called "ReadPropertyTest" is going to read properties of equipment one for each read. And the user can only read 1 time each test. This is the basic idea that my test does.
So I don't know how to answer your question...I'm not sure what data sources i'm using and what kinda access method I've specified. Where should i check it out from??
Thanks !!
Jane
26 Sep 2007, 4:29 PM UTC
Michael Taute - MSFT
Moderator Posts 166
Re: Question about the number of total tests in load testing Was this post helpful ?
There are three different methods of accessing the data in a bound datasource. Sequential, random, and unique.
What she's saying is if you have your access method set to unique, one iteration of the test will run for each entry in your database and then it will stop. Ideally in this case though it would bubble up an out of data error and abort the test though.
It's easy enough to check... just highlight the datasource table you're binding to and view its properties.
26 Sep 2007, 5:23 PM UTC
Jaeed Posts 11
Re: Question about the number of total tests in load testing Was this post helpful ?
Jane,
Sorry I was in outside. And you got already answer from Michael.
Thanks Micheal for helping jane while I was not here.
Jaeed
28 Sep 2007, 7:23 PM UTC
JaneQQ Posts 13
Re: Question about the number of total tests in load testing Was this post helpful ?
Hi Michael,
Sorry for the delay in getting back to you. I was moved over to another issue and now i'm back. Regarding to what you've indicated here, I observed my test. In the unit test's property based on which I built up my load test, I noticed that the Data Access Method item is always sequential. And the other choice is Random, no option for unique. So I believe I've been using Sequential data access method all the time, if it is the same data access method as you were talking about.
I doubt i understood correctly because I thought I should check the data table in SQL server database, in which i stored my test data into. I checked actually, but i didn't find a data table's property called Data Access Method. You know what I mean? More specifically, my test is used to add more pieces of equipment or properties to an hierarchy of equipment model, I store all these equipment/property data in a database of SQL server on local host. Do you think I should check the data access method of this table or unit test of visual studio?
No matter which one is the thing you referred to, I didn't see anything set to Unique. Unless SQL server database use Unique data access method by default??
What am i going to do next?? Frustrated....
28 Sep 2007, 7:24 PM UTC
JaneQQ Posts 13
Re: Question about the number of total tests in load testing Was this post helpful ?
Hi Jaeed,
Thank you for your concern. That's totally fine. The issue is I still stucked on this thing...I guess i need more help.
Thanks;
Jane
28 Sep 2007, 7:52 PM UTC
Bill Barnett - MSFT
Moderator Posts 372
Answer Re: Question about the number of total tests in load testing Was this post helpful ?
I'll try to help. First of all, some of the earlier responses were asking about your data binding settings, but it's not clear to me from your responses wether or not your unit tests even use the VSTS data binding feature. In other words, do your unit test methods use data source attributes like the example shown in the docs here: http://msdn2.microsoft.com/en-us/library/ms182527(VS.90).aspx?
If not, then your problem has nothing to do with the VSTS data binding feature.
Do your unit tests do any other database access, or do something like invoke a Web service that access a database?
One possibility is that for some reason the 199th unit test gets stuck and never completes. Do you think that is a possibility? Can you describe what the unit test does? Does it send a request to a server and wait for a response? You could add some code to your unit tests to log to a file upon entering and leaving your unit test method which would help determine if the unit test itself is getting hung.
Bill
VSTS 2008 Beta2 close and restart automatically
VSTS 2008 Beta2 close and restart automatically Was this post helpful ?
Hi,
I ran a lot of load test by using VSTS 2008 Beta 2 version and I found that this is awsome and is going to beat the Mercury Loadrunner/QTP very soon.
But I notice that VS for tester 2008 beta 2 gets close and restart automatically when I tried to debug and ran some test though this behaviors are not always. It does this once in a while specially when it finds someting unusual I think.
I think it should through error message instead of closing down the VS and restart automatically.
You can share your thought about this and I appreciate that a lot.
Thanks.
-- Jaeed
-------------------------------------------------------------------------------------------
Re: VSTS 2008 Beta2 close and restart automatically Was this post helpful ?
Hi Jaeed, glad to hear you like the product, but sorry to hear you're having some trouble with it shutting down unexpectedly.
If you are able to get a set of steps that can reproduce the crash consistently it would greatly help us to track down and fix the issue you're seeing.
http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1177487&SiteID=1
------------------------------------------------------------------------------------------------
Re: VSTS 2008 Beta2 close and restart automatically Was this post helpful ?
I ran a web test and from web test viewer I closed a pop up window that was came from test scripts(recording script was captured and it had some javascripts window). I wish I can give u the screen shot, but this editor doesn't have attachment capability.
but FYI,
message was : Microsoft Visual Studio has encountered a problem and needs to close... window came up and if i clik on Don't Send error report button, the VS got shut down.
I think this will help you.
Thanks.
-------------------------------------------------------------------------------------------------
Re: VSTS 2008 Beta2 close and restart automatically Was this post helpful ?
Please send us the screen shot: slumley at domain microsoft.com
Blog - http://blogs.msdn.com/slumley/default.aspx
----------------------------------------------------------------------------------------------
Re: VSTS 2008 Beta2 close and restart automatically Was this post helpful ?
Please check your email and find out the screen shots and I have reported to by clicking on send report button.
Thanks
--------------------------------------------------------------------------------------------------
Answer Re: VSTS 2008 Beta2 close and restart automatically Was this post helpful ?
I hope you resolved this offline with Sean. Please upgrade to RTM, if the problem persists contact support.
Thanks,
Ed.
Ed Glas [MSFT]
Hi,
I ran a lot of load test by using VSTS 2008 Beta 2 version and I found that this is awsome and is going to beat the Mercury Loadrunner/QTP very soon.
But I notice that VS for tester 2008 beta 2 gets close and restart automatically when I tried to debug and ran some test though this behaviors are not always. It does this once in a while specially when it finds someting unusual I think.
I think it should through error message instead of closing down the VS and restart automatically.
You can share your thought about this and I appreciate that a lot.
Thanks.
-- Jaeed
-------------------------------------------------------------------------------------------
Re: VSTS 2008 Beta2 close and restart automatically Was this post helpful ?
Hi Jaeed, glad to hear you like the product, but sorry to hear you're having some trouble with it shutting down unexpectedly.
If you are able to get a set of steps that can reproduce the crash consistently it would greatly help us to track down and fix the issue you're seeing.
http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1177487&SiteID=1
------------------------------------------------------------------------------------------------
Re: VSTS 2008 Beta2 close and restart automatically Was this post helpful ?
I ran a web test and from web test viewer I closed a pop up window that was came from test scripts(recording script was captured and it had some javascripts window). I wish I can give u the screen shot, but this editor doesn't have attachment capability.
but FYI,
message was : Microsoft Visual Studio has encountered a problem and needs to close... window came up and if i clik on Don't Send error report button, the VS got shut down.
I think this will help you.
Thanks.
-------------------------------------------------------------------------------------------------
Re: VSTS 2008 Beta2 close and restart automatically Was this post helpful ?
Please send us the screen shot: slumley at domain microsoft.com
Blog - http://blogs.msdn.com/slumley/default.aspx
----------------------------------------------------------------------------------------------
Re: VSTS 2008 Beta2 close and restart automatically Was this post helpful ?
Please check your email and find out the screen shots and I have reported to by clicking on send report button.
Thanks
--------------------------------------------------------------------------------------------------
Answer Re: VSTS 2008 Beta2 close and restart automatically Was this post helpful ?
I hope you resolved this offline with Sean. Please upgrade to RTM, if the problem persists contact support.
Thanks,
Ed.
Ed Glas [MSFT]
What is the relationship between vusers and threads?
What is the relationship between vusers and threads? Was this post helpful ?
What is the relationship between vusers and threads? If set Constant User Count = 1000, is that means the constant 1000 vuser running during the test?
----------------------------------------------------------------------------------------------
Re: What is the relationship between vusers and threads? Was this post helpful ?
That is correct.
If you set a constant user count of 1000 you will see 1000 concurrent scenarios running at any given point during your load test.
----------------------------------------------------------------------------------------------
Re: What is the relationship between vusers and threads? Was this post helpful ?
Thanks for the quick reply!
So can we simply understand as 1 vuser = 1 thread ?
---------------------------------------------------------------------------------------------
Answer Re: What is the relationship between vusers and threads? Was this post helpful ?
The only relationship I know of is that vusers take threads to run on agents, and will likely generate threads on the system under test.. depending on what the tests are doing I suspect that a single vuser might generate more than one thread on the agent system that is running the user. How many threads get spawned to deal with the load generated by the vuser will again depend on what kind of requests the vuser is making, and the software running on the SUT.
Vuser is a Virtual user. the number of vusers you utilize is basically controlling how many webtests or unit tests will run in parallel in a loadtest scenario. How that equates to real users depends on factors like what actions the tests are doing, time between tests, and think times.
the number of vusers you use also controls the maximum simultanious (or nearly so due to travelling over a network) requests that can be generated at the same time against your SUT.
low think times and little to no time betweeen test iterations allows a single vuser to simulate the requests/time load of many real users. This is a good technique for something like a perfomance test, or measuring performance deltas due to code/hardware changes, where you want a consistent repeatable test.
a larger number of vusers, using realistic randomized think times, and longer times between test iterations could generate the same load in terms of transactions/hour as the no-think test, however the load would less constant, with a higher number of potential simultanious requests. This type of config is good for a more realistic classic 'load test' where you want ebb and flow in the level of requests at any one instant and it's not as important the test be repeatable
--Chuck
------------------------------------------------------------------------------------------------
Answer Re: What is the relationship between vusers and threads? Was this post helpful ?
"Threads" are a confusing term which is why we try not to use it.
A better way to think of it is:
1 Concurrent user = 1 user at a machine running through the scenario(s) you have defined.
So 1000 concurrent users = 1000 of those scenarios running at the same time.
What is the relationship between vusers and threads? If set Constant User Count = 1000, is that means the constant 1000 vuser running during the test?
----------------------------------------------------------------------------------------------
Re: What is the relationship between vusers and threads? Was this post helpful ?
That is correct.
If you set a constant user count of 1000 you will see 1000 concurrent scenarios running at any given point during your load test.
----------------------------------------------------------------------------------------------
Re: What is the relationship between vusers and threads? Was this post helpful ?
Thanks for the quick reply!
So can we simply understand as 1 vuser = 1 thread ?
---------------------------------------------------------------------------------------------
Answer Re: What is the relationship between vusers and threads? Was this post helpful ?
The only relationship I know of is that vusers take threads to run on agents, and will likely generate threads on the system under test.. depending on what the tests are doing I suspect that a single vuser might generate more than one thread on the agent system that is running the user. How many threads get spawned to deal with the load generated by the vuser will again depend on what kind of requests the vuser is making, and the software running on the SUT.
Vuser is a Virtual user. the number of vusers you utilize is basically controlling how many webtests or unit tests will run in parallel in a loadtest scenario. How that equates to real users depends on factors like what actions the tests are doing, time between tests, and think times.
the number of vusers you use also controls the maximum simultanious (or nearly so due to travelling over a network) requests that can be generated at the same time against your SUT.
low think times and little to no time betweeen test iterations allows a single vuser to simulate the requests/time load of many real users. This is a good technique for something like a perfomance test, or measuring performance deltas due to code/hardware changes, where you want a consistent repeatable test.
a larger number of vusers, using realistic randomized think times, and longer times between test iterations could generate the same load in terms of transactions/hour as the no-think test, however the load would less constant, with a higher number of potential simultanious requests. This type of config is good for a more realistic classic 'load test' where you want ebb and flow in the level of requests at any one instant and it's not as important the test be repeatable
--Chuck
------------------------------------------------------------------------------------------------
Answer Re: What is the relationship between vusers and threads? Was this post helpful ?
"Threads" are a confusing term which is why we try not to use it.
A better way to think of it is:
1 Concurrent user = 1 user at a machine running through the scenario(s) you have defined.
So 1000 concurrent users = 1000 of those scenarios running at the same time.
VSTS-response time measurement accuracy?
response time measurement accuracy? Was this post helpful ?
results are quite different when benchmark between vsts2005 loadtest and loadrunner, wonder what exactlyvsts and how it measured? and if there are benchmark whitpaper from Microsoft or somewhere? Thanks!
performance and capacity
------------------------------------------------------------------------------------------------
Answer Re: response time measurement accuracy? Was this post helpful ?
In the VSTS results, are you looking at "Avg. Response Time" for requests, or "Avg. Page Time"? Are the values greater or less than the response times reported by LoadRunner?
An important thing to understand in the VSTS load test results is the difference between “Avg. Response Time” for requests and “Avg. Page Time” for pages.
Consider a web page (home.aspx) that contains 10 images. When a Web test that contains a request to home.aspx runs, there will be 11 HTTP requests sent (1 for home.aspx and 1 for each of the 10 images) just as there would be with a browswer (ignoring caching for simplicity). The round trip times for receive each of these 11 individual requests is recorded. The averages for each URL are shown in the "Requests" table in the load test results (and can also be added to the graphs). The average request response time across all URLs is reported in the performance counter "Avg. Response Time" with the instance name "_Total" which is included in the default graph created by VSTS.
However, you may be more interested in the "Avg. Page Time". The page time is the time measured from the time a page request is sent until all bytes for the page and all of the dependent requests (for images and such) are received.
So, going back to the example, if the HTTP response to the request to get home.aspx takes 3.4 seconds and the time to get the 10 embedded images is 0.1 seconds each, the load test results would show:
In the Requests table: Request Response Time home.aspx 3.4 image1.gif 0.1 image2.gif 0.1 etc.
In the Pages table: Page Page Time home.aspx 4.4
On the default graph: Counter Instance Category Avg Avg. Response Time _Total LoadTest:Request 0.4
If you add Page Time to the graph: Avg. Page Time _Total LoadTestage 4.4
The Avg. Response Time for requests is much lower than the Avg. Page Time because there are many fast requests to images.
Other recommended reading:
"Monitoring and Analyzing a Load Test Result": http://msdn2.microsoft.com/en-us/library/aa730850(vs.80).aspx
"Advanced Load Testing Features of Visual Studio Team System": http://msdn2.microsoft.com/en-us/teamsystem/aa718897.aspx
-------------------------------------------------------------------------------------------------
Re: response time measurement accuracy? Was this post helpful ?
Thank you very much Bill!I think if I put BeginTransaction() and EndTransaction(), it will be no confusing.
performance and capacity
results are quite different when benchmark between vsts2005 loadtest and loadrunner, wonder what exactlyvsts and how it measured? and if there are benchmark whitpaper from Microsoft or somewhere? Thanks!
performance and capacity
------------------------------------------------------------------------------------------------
Answer Re: response time measurement accuracy? Was this post helpful ?
In the VSTS results, are you looking at "Avg. Response Time" for requests, or "Avg. Page Time"? Are the values greater or less than the response times reported by LoadRunner?
An important thing to understand in the VSTS load test results is the difference between “Avg. Response Time” for requests and “Avg. Page Time” for pages.
Consider a web page (home.aspx) that contains 10 images. When a Web test that contains a request to home.aspx runs, there will be 11 HTTP requests sent (1 for home.aspx and 1 for each of the 10 images) just as there would be with a browswer (ignoring caching for simplicity). The round trip times for receive each of these 11 individual requests is recorded. The averages for each URL are shown in the "Requests" table in the load test results (and can also be added to the graphs). The average request response time across all URLs is reported in the performance counter "Avg. Response Time" with the instance name "_Total" which is included in the default graph created by VSTS.
However, you may be more interested in the "Avg. Page Time". The page time is the time measured from the time a page request is sent until all bytes for the page and all of the dependent requests (for images and such) are received.
So, going back to the example, if the HTTP response to the request to get home.aspx takes 3.4 seconds and the time to get the 10 embedded images is 0.1 seconds each, the load test results would show:
In the Requests table: Request Response Time home.aspx 3.4 image1.gif 0.1 image2.gif 0.1 etc.
In the Pages table: Page Page Time home.aspx 4.4
On the default graph: Counter Instance Category Avg Avg. Response Time _Total LoadTest:Request 0.4
If you add Page Time to the graph: Avg. Page Time _Total LoadTestage 4.4
The Avg. Response Time for requests is much lower than the Avg. Page Time because there are many fast requests to images.
Other recommended reading:
"Monitoring and Analyzing a Load Test Result": http://msdn2.microsoft.com/en-us/library/aa730850(vs.80).aspx
"Advanced Load Testing Features of Visual Studio Team System": http://msdn2.microsoft.com/en-us/teamsystem/aa718897.aspx
-------------------------------------------------------------------------------------------------
Re: response time measurement accuracy? Was this post helpful ?
Thank you very much Bill!I think if I put BeginTransaction() and EndTransaction(), it will be no confusing.
performance and capacity
Validation rule - how to verify a list of values is listed in a Combo box.
Hello,
I am new to the validation rules using VSTS 2008 Test Edition. Any suggestions would be much appreciated.
One of things in my script is to verify a list of values (i.e. Monday – Sunday) is listed in a combo box. How to do it?
Thanks in advance!
HaYin
-------------------------------------------------------------------------------------------------
Yin
The "Find Text" validation rule can solve your problem. It can be used to check whether html code contains a specified string. For example, you can try to create a Find Text rule and set the text to find as some thing like "".
Bill Wang
-----------------------------------------------------------------------------------------------
To check a range of values like this will require a custom validation rule. The following article explains how to create a custom validation rule.
http://msdn.microsoft.com/en-us/library/ms182556(VS.80).aspx
You will be implementing a method called "Validate" in the in the rule. THe rule will set a pass/fail indicator via the ValidationEventArgs parameter. For instance…
public override void Validate(object sender, ValidationEventArgs e)
{
e.Message = "It didn't work";
e.IsValid = false;
}
The validate method will be supplied with the response body as a string and an HTMLDocument via the event args (e.Response.BodyAsString and e.Response.HTMLDocument). The HTMLDocument that is supplied by the response is very limited in what it can provide you. It parses HTML tags out of the response but does not provide any information about HTML structure. It is intentionally lightweight to avoid unnecessary overhead in the web test (especially when the web test may be executions thousands of time during a load test). It is sufficient for many validation scenarios, but not all.
For instance, you could do something like...
public override void Validate(object sender, ValidationEventArgs e)
{
WebTestResponse response = e.Response;
HtmlDocument doc = response.HtmlDocument;
IEnumerable tags = doc.GetFilteredHtmlTags("option");
foreach (HtmlTag tag in tags)
{
foreach (HtmlAttribute attribute in tag.Attributes)
{
if (string.Equals(attribute.Name, "value", StringComparison.OrdinalIgnoreCase))
{
string value = attribute.Value; // note this is the attribute "value" not the inner text of the tag.
}
}
}
}
Again, the HTMLDocument class only finds tags in the response but does not provide structure information. So, in the code snippet above you will not know which combo box each of the options belongs to. If this is sufficient for your needs then you can use the HTMLDocument class.
A colleague of mine is about to post a Validation Rule sample to CodePlex that will demonstrate how to do more complex searches through the HTML using e.Response.HtmlDocument. It would be a sample that you could refer to if needed. There should be an announcement on the forums when this sample is posted.
You could also consider using a full HTML Dom to parse the response and then navigate the document.
Let me know if you need additional information.
Thanks,
Rick
------------------------------------------------------------------------------------------------
This should work however you may need to tweak it depending on your source.
1) Add a Find Text validation rule
2) Change Use Regular Expression to True
3) Plop the following into Find Text
(?=[\w\W]*Sunday)(?=[\w\W]*Monday)(?=[\w\W]*Tuesday)(?=[\w\W]*Wednesday)(?=[\w\W]*Thursday)(?=[\w\W]*Friday)(?=[\w\W]*Saturday)
I tested this in a RegEx tester using the following source:
------------------------------------------------------------------------------------------
Thank you all for such quick response!
Looks like writing custom validation rules will be a common way to verify rules because the out-of-box validation rules are so limited (which is not a good thing for testers, like me, who are not good at writing code).
For this case, I tried to use Lewis’ suggestion and it worked! Thanks Lewis. One question – actually the range of values I want to check is the 12 months, so I tried to copy & paste the following to the Find Text parameter in the Add Validation Rule dialog box:
(?=[\w\W]*January)(?=[\w\W]*February)(?=[\w\W]*March)(?=[\w\W]*April)(?=[\w\W]*May)(?=[\w\W]*June)(?=[\w\W]*July)(?=[\w\W]*August)(?=[\w\W]*September)(?=[\w\W]*October)(?=[\w\W]*November)(?=[\w\W]*December)
However, it doesn’t work due to the text is too long. So what I did was to have 2 validation rules – one is “(?=[\w\W]*January)(?=[\w\W]*February)(?=[\w\W]*March)(?=[\w\W]*April)(?=[\w\W]*May)(?=[\w\W]*June)”;
another is “(?=[\w\W]*July)(?=[\w\W]*August)(?=[\w\W]*September)(?=[\w\W]*October)(?=[\w\W]*November)(?=[\w\W]*December)”. Is it right way to do it OR there are other better ways to do it without writing a custom validation rule? What does ‘[w\W]’ mean? Thanks for your time Lewis.
Rick – I am looking forward to seeing the Validation Rule sample posting. It will for sure give users great ideas on what and how to do with the custom validation rules.
Thanks all.
HaYin
-----------------------------------------------------------------------------------------------
If you can get away with it, you can shorten the months to read:
(?=[\w\W]*Jan)(?=[\w\W]*Feb)(?=[\w\W]*Mar)(?=[\w\W]*Apr)(?=[\w\W]*May)(?=[\w\W]*Jun)”“(?=[\w\W]*Jul)(?=[\w\W]*Aug)(?=[\w\W]*Sep)(?=[\w\W]*Oct)(?=[\w\W]*Nov)(?=[\w\W]*Dec)
According to my handy Regular Expressions book [\w\W]* means
\w = any alphanumeric in upper or lower case and undersore
\W = any non-alphanumeric or non underscore
putting both in brackets tells it to match one or the other
and the asterisk tells it to match zero or more of the set
----------------------------------------------------------------------------------------------------
I am new to the validation rules using VSTS 2008 Test Edition. Any suggestions would be much appreciated.
One of things in my script is to verify a list of values (i.e. Monday – Sunday) is listed in a combo box. How to do it?
Thanks in advance!
HaYin
-------------------------------------------------------------------------------------------------
Yin
The "Find Text" validation rule can solve your problem. It can be used to check whether html code contains a specified string. For example, you can try to create a Find Text rule and set the text to find as some thing like "".
Bill Wang
-----------------------------------------------------------------------------------------------
To check a range of values like this will require a custom validation rule. The following article explains how to create a custom validation rule.
http://msdn.microsoft.com/en-us/library/ms182556(VS.80).aspx
You will be implementing a method called "Validate" in the in the rule. THe rule will set a pass/fail indicator via the ValidationEventArgs parameter. For instance…
public override void Validate(object sender, ValidationEventArgs e)
{
e.Message = "It didn't work";
e.IsValid = false;
}
The validate method will be supplied with the response body as a string and an HTMLDocument via the event args (e.Response.BodyAsString and e.Response.HTMLDocument). The HTMLDocument that is supplied by the response is very limited in what it can provide you. It parses HTML tags out of the response but does not provide any information about HTML structure. It is intentionally lightweight to avoid unnecessary overhead in the web test (especially when the web test may be executions thousands of time during a load test). It is sufficient for many validation scenarios, but not all.
For instance, you could do something like...
public override void Validate(object sender, ValidationEventArgs e)
{
WebTestResponse response = e.Response;
HtmlDocument doc = response.HtmlDocument;
IEnumerable
foreach (HtmlTag tag in tags)
{
foreach (HtmlAttribute attribute in tag.Attributes)
{
if (string.Equals(attribute.Name, "value", StringComparison.OrdinalIgnoreCase))
{
string value = attribute.Value; // note this is the attribute "value" not the inner text of the tag.
}
}
}
}
Again, the HTMLDocument class only finds tags in the response but does not provide structure information. So, in the code snippet above you will not know which combo box each of the options belongs to. If this is sufficient for your needs then you can use the HTMLDocument class.
A colleague of mine is about to post a Validation Rule sample to CodePlex that will demonstrate how to do more complex searches through the HTML using e.Response.HtmlDocument. It would be a sample that you could refer to if needed. There should be an announcement on the forums when this sample is posted.
You could also consider using a full HTML Dom to parse the response and then navigate the document.
Let me know if you need additional information.
Thanks,
Rick
------------------------------------------------------------------------------------------------
This should work however you may need to tweak it depending on your source.
1) Add a Find Text validation rule
2) Change Use Regular Expression to True
3) Plop the following into Find Text
(?=[\w\W]*Sunday)(?=[\w\W]*Monday)(?=[\w\W]*Tuesday)(?=[\w\W]*Wednesday)(?=[\w\W]*Thursday)(?=[\w\W]*Friday)(?=[\w\W]*Saturday)
I tested this in a RegEx tester using the following source:
------------------------------------------------------------------------------------------
Thank you all for such quick response!
Looks like writing custom validation rules will be a common way to verify rules because the out-of-box validation rules are so limited (which is not a good thing for testers, like me, who are not good at writing code).
For this case, I tried to use Lewis’ suggestion and it worked! Thanks Lewis. One question – actually the range of values I want to check is the 12 months, so I tried to copy & paste the following to the Find Text parameter in the Add Validation Rule dialog box:
(?=[\w\W]*January)(?=[\w\W]*February)(?=[\w\W]*March)(?=[\w\W]*April)(?=[\w\W]*May)(?=[\w\W]*June)(?=[\w\W]*July)(?=[\w\W]*August)(?=[\w\W]*September)(?=[\w\W]*October)(?=[\w\W]*November)(?=[\w\W]*December)
However, it doesn’t work due to the text is too long. So what I did was to have 2 validation rules – one is “(?=[\w\W]*January)(?=[\w\W]*February)(?=[\w\W]*March)(?=[\w\W]*April)(?=[\w\W]*May)(?=[\w\W]*June)”;
another is “(?=[\w\W]*July)(?=[\w\W]*August)(?=[\w\W]*September)(?=[\w\W]*October)(?=[\w\W]*November)(?=[\w\W]*December)”. Is it right way to do it OR there are other better ways to do it without writing a custom validation rule? What does ‘[w\W]’ mean? Thanks for your time Lewis.
Rick – I am looking forward to seeing the Validation Rule sample posting. It will for sure give users great ideas on what and how to do with the custom validation rules.
Thanks all.
HaYin
-----------------------------------------------------------------------------------------------
If you can get away with it, you can shorten the months to read:
(?=[\w\W]*Jan)(?=[\w\W]*Feb)(?=[\w\W]*Mar)(?=[\w\W]*Apr)(?=[\w\W]*May)(?=[\w\W]*Jun)”“(?=[\w\W]*Jul)(?=[\w\W]*Aug)(?=[\w\W]*Sep)(?=[\w\W]*Oct)(?=[\w\W]*Nov)(?=[\w\W]*Dec)
According to my handy Regular Expressions book [\w\W]* means
\w = any alphanumeric in upper or lower case and undersore
\W = any non-alphanumeric or non underscore
putting both in brackets tells it to match one or the other
and the asterisk tells it to match zero or more of the set
----------------------------------------------------------------------------------------------------
Recording tool causing scripts to fail
During playback of the automation script, when the following HTTP request is sent, the request fails because the dependent request fail. Our Developers think this is a problem with the recording tool
https://xxxxxxxxxxx/xxxxxxxxx/VR.NET/public/xxxxxxxxxx.aspx?xxxxxxxxxxx=xxxxxxxx&page=&sortid=0&DueDate=Unknown&RequestIcon=..%2f..%2fimages%2fblue.gif&lsoeid=xxxxxx&Anthem_CallBack=trueThe following dependent request fails.https://xxxxxxxxxxx/%22/xxxxxxxx/VR.NET/images/tab_left.gif/%22
-------------------------------------------------------------------------------------------------
Can you either post the full response of the top level request that thius dependent is in, or email me the full response.
https://xxxxxxxxxxx/xxxxxxxxx/VR.NET/public/xxxxxxxxxx.aspx?xxxxxxxxxxx=xxxxxxxx&page=&sortid=0&DueDate=Unknown&RequestIcon=..%2f..%2fimages%2fblue.gif&lsoeid=xxxxxx&Anthem_CallBack=trueThe following dependent request fails.https://xxxxxxxxxxx/%22/xxxxxxxx/VR.NET/images/tab_left.gif/%22
-------------------------------------------------------------------------------------------------
Can you either post the full response of the top level request that thius dependent is in, or email me the full response.
FAILED:An existing connection was forcibly closed by the remote host
hi everybody,
i am doing load test for 10 users,20users,30users working fine upto to this point if increase to 40 users iam getting error as below.
and also getting someimes for 30users as request time out . iam new to load test can any body help me out.
FAILED:An existing connection was forcibly closed by the remote host,
can any body help me on this error.
thanks.
ven
-------------------------------------------------------------------------------------------------
ven
It seems to be an SQL Server message. Please provide the exact error message and version information about platform and SQL Server. Since the system works well for low pressure, it's also possible that high pressure cause some request time out. So, please also let us know your hard ware sizing and what kind of operations you performed for each user.
Bill Wang
i am doing load test for 10 users,20users,30users working fine upto to this point if increase to 40 users iam getting error as below.
and also getting someimes for 30users as request time out . iam new to load test can any body help me out.
FAILED:An existing connection was forcibly closed by the remote host,
can any body help me on this error.
thanks.
ven
-------------------------------------------------------------------------------------------------
ven
It seems to be an SQL Server message. Please provide the exact error message and version information about platform and SQL Server. Since the system works well for low pressure, it's also possible that high pressure cause some request time out. So, please also let us know your hard ware sizing and what kind of operations you performed for each user.
Bill Wang
How to control webtest?
I'm new to VSTS webtest. I tried record and play. There are two questions:
1. Can I control the webtest sequence? If test1.webtest failed, go to test2.webtest, otherwise, go to test3.webtest?
2. Can I change post parameters? Instead of using textBoxUsername=abc, can I change it as User Name = abc? textBoxUsername is id, and User Name is label.
Thanks,
Cecilia
--------------------------------------------------------------------------------------------
1. There is no easy way to sequence as you suggest, based on outcome. You can serailize tests using an "orderedtest". What you could do is to generate code, then merge the generated code into one large coded web test, then put the logic in there. You will have to learn a bit about coded web tests to do this, but it is not that difficult if you are familiar with C# or VB. See http://msdn2.microsoft.com/en-us/library/ms364082.aspx#wtauthd_topic6
2. Yes, most strings can be replaced with parameters, those parameters can be changed, possibly driven from extraction rules, or from a data source. For more on this, take a look at http://msdn2.microsoft.com/en-us/library/ms364082.aspx#wtauthd_topic5 and look for the section called Handling View State and Other Dynamic Parameters.
In general if you take a look at the main page of this forum you will find a lot of useful information.
-------------------------------------------------------------------------------------------
Thanks for your replying.
I'm still struggling with my 2nd question.
I can use data source or context parameter to input different value for username and password, but how can I parameterize textboxUsername to User Name, and btnLogin to Login? If I delete them from Form Post Parameters, and add User Name = {{uservalue}}, Login=Login, the test failed.
Any suggestions?
----------------------------------------------------------------------------------------------
When you are editing
the value filed, the drop down should show "add data source", that can be bound to a source of data such as a CSV file, XML file or DB file. Please take a look at http://msdn2.microsoft.com/en-us/library/ms243182(VS.80).aspx
It may not be 100% obvious, but the {{???}} strings that the data binding inserts can be further edited to add constant text around them (if you need it)
---------------------------------------------------------------------------------------------
Thanks for replying.
My question is not how to use datasource. My question is: Can VSTS use control label or name, instead of control id?
Thanks,
Cecilia
--------------------------------------------------------------------------------------
I'm not quite sure what you are asking for.
What is being specified is a post parameter, that us, an arbitrary name value pair. That should normally correspond with the name of a control, but that depends on the web page itself. There is ano hard and fast rule.
Am I still missing the point of your question?
-----------------------------------------------------------------------------------------
In short, no there is no way to do this. The Name is what gets posted back to the web server, and it is what the web server uses to determine which parameter has been set, so you can't change this value. Nor is there a way to provide an alias or friendly name for the parameter.
Ed.
Ed Glas [MSFT]
---------------------------------------------------------------------------------------------
about sequence in webtest ,, you can do this by using ordertest ,, you can arrange ur tests of defferent types in order test then from Test >> Window >> test view .. then select the order test >> right Click on it then run selection ... you will get what u want
-------------------------------------------------------------------------------------------
Very Interesting.
Srikanth Manohar
----------------------------------------------------------------------------------------
1. Can I control the webtest sequence? If test1.webtest failed, go to test2.webtest, otherwise, go to test3.webtest?
2. Can I change post parameters? Instead of using textBoxUsername=abc, can I change it as User Name = abc? textBoxUsername is id, and User Name is label.
Thanks,
Cecilia
--------------------------------------------------------------------------------------------
1. There is no easy way to sequence as you suggest, based on outcome. You can serailize tests using an "orderedtest". What you could do is to generate code, then merge the generated code into one large coded web test, then put the logic in there. You will have to learn a bit about coded web tests to do this, but it is not that difficult if you are familiar with C# or VB. See http://msdn2.microsoft.com/en-us/library/ms364082.aspx#wtauthd_topic6
2. Yes, most strings can be replaced with parameters, those parameters can be changed, possibly driven from extraction rules, or from a data source. For more on this, take a look at http://msdn2.microsoft.com/en-us/library/ms364082.aspx#wtauthd_topic5 and look for the section called Handling View State and Other Dynamic Parameters.
In general if you take a look at the main page of this forum you will find a lot of useful information.
-------------------------------------------------------------------------------------------
Thanks for your replying.
I'm still struggling with my 2nd question.
I can use data source or context parameter to input different value for username and password, but how can I parameterize textboxUsername to User Name, and btnLogin to Login? If I delete them from Form Post Parameters, and add User Name = {{uservalue}}, Login=Login, the test failed.
Any suggestions?
----------------------------------------------------------------------------------------------
When you are editing
the value filed, the drop down should show "add data source", that can be bound to a source of data such as a CSV file, XML file or DB file. Please take a look at http://msdn2.microsoft.com/en-us/library/ms243182(VS.80).aspx
It may not be 100% obvious, but the {{???}} strings that the data binding inserts can be further edited to add constant text around them (if you need it)
---------------------------------------------------------------------------------------------
Thanks for replying.
My question is not how to use datasource. My question is: Can VSTS use control label or name, instead of control id?
Thanks,
Cecilia
--------------------------------------------------------------------------------------
I'm not quite sure what you are asking for.
What is being specified is a post parameter, that us, an arbitrary name value pair. That should normally correspond with the name of a control, but that depends on the web page itself. There is ano hard and fast rule.
Am I still missing the point of your question?
-----------------------------------------------------------------------------------------
In short, no there is no way to do this. The Name is what gets posted back to the web server, and it is what the web server uses to determine which parameter has been set, so you can't change this value. Nor is there a way to provide an alias or friendly name for the parameter.
Ed.
Ed Glas [MSFT]
---------------------------------------------------------------------------------------------
about sequence in webtest ,, you can do this by using ordertest ,, you can arrange ur tests of defferent types in order test then from Test >> Window >> test view .. then select the order test >> right Click on it then run selection ... you will get what u want
-------------------------------------------------------------------------------------------
Very Interesting.
Srikanth Manohar
----------------------------------------------------------------------------------------
WebTest: How to automate flash based web site?
WebTest: How to automate flash based web site? Was this post helpful ?
Dear All,I am working on automation of a web site which is full of flash component/controls. In VSTS webtest the flash controls/objects are not getting recorded/tracked. How should I go for automation of such site? Let me know if I can go with coding and what should be the approach?Any comment/help in this regard would be appreciable.
Thanks,Salil
--------------------------------------------------------------------------------------------
Answer Re: WebTest: How to automate flash based web site? Was this post helpful ?
Our Webtests work at the http layer, the recorder only captures the http traffic from the browser. So if the flash controls on your site are making http requests (for example, how ajax does) then that will get captured. The actual mouse movements and UI manipulations are not captured though.
------------------------------------------------------------------------------------------------
Re: WebTest: How to automate flash based web site? Was this post helpful ?
Also, are you using VSTS 2008? The VSTS 2005 Web test recorder did not record Ajax requests or Java script requests, and would likely miss these.
Dear All,I am working on automation of a web site which is full of flash component/controls. In VSTS webtest the flash controls/objects are not getting recorded/tracked. How should I go for automation of such site? Let me know if I can go with coding and what should be the approach?Any comment/help in this regard would be appreciable.
Thanks,Salil
--------------------------------------------------------------------------------------------
Answer Re: WebTest: How to automate flash based web site? Was this post helpful ?
Our Webtests work at the http layer, the recorder only captures the http traffic from the browser. So if the flash controls on your site are making http requests (for example, how ajax does) then that will get captured. The actual mouse movements and UI manipulations are not captured though.
------------------------------------------------------------------------------------------------
Re: WebTest: How to automate flash based web site? Was this post helpful ?
Also, are you using VSTS 2008? The VSTS 2005 Web test recorder did not record Ajax requests or Java script requests, and would likely miss these.
Are Web Tests "Functional" tests?
Ed Glas's blog on VSTS load testing
Are Web Tests "Functional" tests?
When most people talk about automated functional testing, they are talking about UI Automated functional testing, where the test drives the UI (clicks buttons, types in text, etc.).
In fact there are many other strategies for functional testing, and most “unit tests” written today are not true Unit tests (with a capital U) that isolate and only test a single class, but test from the API down. For example, a unit test that tests a web service, or a client API that hits a server, are really functional tests.
Web tests work at the http layer, as Sean explains in this blog post. Really a better name for these is "Web Server Tests", or "Web Application Tests" since they actually test the server. The first thing users think when they see "web test" is a test that drives the browser. The browser preview window in web test playback only adds to this confusion.
Web tests do not actually instantiate IE when running, but only send and receive http requests. The main reason we chose to drive web tests at the HTTP layer is we wanted to be able to drive load efficiently in a load test. Instantiating IE, even just the guts of IE (the web browser control), consumes a lot more memory and constrains the amount of load that can be generated from a single machine. Web tests are super-efficient when running.
So what does and does not get tested by a web test? Obviously you can test your server code this way, which is super-valuable as that’s where most of your business logic is. Given a particular request, you can validate that the server sends back the correct response. However, since the test is not driven through IE, it will not test any client side code or behaviors, such as any java script code (including Ajax code). However, you can simulate the HTTP requests that Ajax will make and validate that the server is sending back the right response. Web tests do validation using Validation Rules, which enable you to validate the response sent back by the server.
There are advantages and disadvantages to driving at the protocol layer vs. the UI. UI tests are easier to understand and debug, but more fragile. To really be effective at scripting web tests you have to understand what is happening at the http layer, which most developers and testers are not usually concerned with. Most are . Also doing things like adding validation is easier in a UI automation tool, harder in web tests (although this is more a tooling problem than protocol vs. UI, we have some features lined up to make this easier).
So the bottom line answer is "yes", web tests are functional tests, but they work at the http layer rather than the UI layer.
Published Wednesday, February 27, 2008 5:31 PM by edglas
Are Web Tests "Functional" tests?
When most people talk about automated functional testing, they are talking about UI Automated functional testing, where the test drives the UI (clicks buttons, types in text, etc.).
In fact there are many other strategies for functional testing, and most “unit tests” written today are not true Unit tests (with a capital U) that isolate and only test a single class, but test from the API down. For example, a unit test that tests a web service, or a client API that hits a server, are really functional tests.
Web tests work at the http layer, as Sean explains in this blog post. Really a better name for these is "Web Server Tests", or "Web Application Tests" since they actually test the server. The first thing users think when they see "web test" is a test that drives the browser. The browser preview window in web test playback only adds to this confusion.
Web tests do not actually instantiate IE when running, but only send and receive http requests. The main reason we chose to drive web tests at the HTTP layer is we wanted to be able to drive load efficiently in a load test. Instantiating IE, even just the guts of IE (the web browser control), consumes a lot more memory and constrains the amount of load that can be generated from a single machine. Web tests are super-efficient when running.
So what does and does not get tested by a web test? Obviously you can test your server code this way, which is super-valuable as that’s where most of your business logic is. Given a particular request, you can validate that the server sends back the correct response. However, since the test is not driven through IE, it will not test any client side code or behaviors, such as any java script code (including Ajax code). However, you can simulate the HTTP requests that Ajax will make and validate that the server is sending back the right response. Web tests do validation using Validation Rules, which enable you to validate the response sent back by the server.
There are advantages and disadvantages to driving at the protocol layer vs. the UI. UI tests are easier to understand and debug, but more fragile. To really be effective at scripting web tests you have to understand what is happening at the http layer, which most developers and testers are not usually concerned with. Most are . Also doing things like adding validation is easier in a UI automation tool, harder in web tests (although this is more a tooling problem than protocol vs. UI, we have some features lined up to make this easier).
So the bottom line answer is "yes", web tests are functional tests, but they work at the http layer rather than the UI layer.
Published Wednesday, February 27, 2008 5:31 PM by edglas
How do I filter sessionid from a header?
e.g header: ARPT=YMXXXKS192.168.100.218CKQIY; path=/,JSESSIONID=3F523783B77C8E8EDEE5F5FE5B8FE99A; Path=/ I just want to grab JSESSIONID and value.
-----------------------------------------------------------------------------------------------
Code Snippet
using System.Net;
public class ExtractJSESSIONID : WebTestRequestPlugin
{
public override void PostRequest(object sender, PostRequestEventArgs e)
{
foreach (Cookie cook in e.Response.Cookies)
{
if (cook.ToString().StartsWith("JSESSIONID="))
{
e.WebTest.Context.Add("JSESSIONID", cook.Value.ToString());
}
}
}
}
-----------------------------------------------------------------------------------------------
Code Snippet
using System.Net;
public class ExtractJSESSIONID : WebTestRequestPlugin
{
public override void PostRequest(object sender, PostRequestEventArgs e)
{
foreach (Cookie cook in e.Response.Cookies)
{
if (cook.ToString().StartsWith("JSESSIONID="))
{
e.WebTest.Context.Add("JSESSIONID", cook.Value.ToString());
}
}
}
}
Urgent problem with An existing connection was forcibly closed by the remote host
hi everybody,
i am doing load test for 10 users,20users,30users working fine upto to this point if increase to 40 users iam getting error as below.
and also getting someimes for 30users as request time out . iam new to load test can any body help me out.
FAILED:An existing connection was forcibly closed by the remote host,
can any body help me on this error.
thanks.
-----------------------------------------------
It seems to be an SQL Server message. Please provide the exact error message and version information about platform and SQL Server. Since the system works well for low pressure, it's also possible that high pressure cause some request time out. So, please also let us know your hard ware sizing and what kind of operations you performed for each user.
-----------------------------------------------------------
Congratulations, you have found your first stress bug in your product using load testing
I have also seen this error in applications outside of SQL server. I believe it is a generic TCP/IP error. Your web server could be crashing, or calling response.Close().
At any rate, I suggest you take this problem to the developer of your product. They will probably have web server logs that correspond to this failure and will know what the problem is.
Thanks,
-JP
------------------------------------------------------
venkatanna wrote:
yes my web server is crashing ,how can i solve this problem.?.
thanks
file a bug with your developers. or the IT folks that will administer the site.
Seriously, fixing the site is not the tester's job.
there are a LOT of things that can affect webserver performance, everything from webserver settings, the physical hardware, to the code that runs the site.
this type of thing generally has to be debugged by the site developer and IT folks.. they will perhaps want to look at your test results and review the performance data that was logged to see if they identify a bottleneck (like cpu time, memory, disk access). Hopefully they also have good error logging inside the code and on the site itself, and can review those logs as well.
Website performance tuning is a HUGE subject area, and is well outside the scope of this forum..
-------------------------------------
Hi,
I have an urgent problem need help. We have an Asp.net application runing on window 2003 server. One part of the application is socket programming. When customer hits our site a little bit heavier, after serveral thoudsands hits, I statrt to get a bunch of
An existing connection was forcibly closed by the remote host
exceptions, then if the customer continue hits the site, the other function throw out of memory exception, The result is our site not responding any more. The attached is the socket call programming, any help will be appreciated! Thank you.
IPAddress[] ips = Dns.GetHostAddresses(m_sAVSHost); IPEndPoint oIPEndPoint = new IPEndPoint(ips[0], iAvsPort); Socket oSocket = new Socket(oIPEndPoint.Address.AddressFamily, SocketType.Stream, ProtocolType.Tcp); try { oSocket.Connect(oIPEndPoint); byte[] bSendBytes = Encoding.UTF8.GetBytes(sAvsParam); oSocket.Send(bSendBytes, bSendBytes.Length, SocketFlags.None);
byte[] bRecvBytes = new byte[4096]; oSocket.Receive(bRecvBytes, SocketFlags.None); rtnVal = Encoding.UTF8.GetString(bRecvBytes).Trim(); } catch (Exception ex) { VVXmlListenerException.LogException(new VVXmlListenerException(ex)); } finally { oSocket.Shutdown(SocketShutdown.Both); oSocket.Close(); }
-----------------------------------------
I would run a packet trace to find out what's actually going over the network. (Use a tool like Ethereal or Netmon.)
On all the occasions I've seen this exception, it has meant exactly what it says: the connection was forcibly closed. So this suggests that whatever your code here is connecting to is closing the connection. (Or you're going through a firewall, proxy, or other intermediary which is closing it.) The first step would be to confirm that - look at the actual packets going across the network to see what's really happening - that will most likely confirm that the error you're getting is consistent with the incoming network packets.
And if that's what you see, then the problem is external to the code you're showing us here. You need to look into why the external system - the thing that this code connects to - is kicking you off.
What does this code connect to? (I.e. what is m_sAVSHost?)
----------------------------------------------------------
For instructions on how to get a netmon trace see here:
http://blogs.msdn.com/dgorti/archive/2005/10/29/486887.aspx
For instructions on how to get a System.Net tracing log go here:
http://blogs.msdn.com/dgorti/archive/2005/09/18/471003.aspx
Mariya
---------------------------------------------------------------
I guess this is because the number of clients exceed the backlog set in the listen request of the server.
Please take a look at (5) & (6) th points in
https://blogs.msdn.com/malarch/archive/2006/06/26/647993.aspx.
Try to avoid starting many clients at the same time.. One simple solution is to Add a random sleep before starting the clients.
Malar
-----------------------------------------------------------------
Customer now resends the data when the other side closes the connection
Mariya
-----------------------------------------------------------------
I am living diffuculty at the same problem if you have solved the problem can you help me please? Thank you in advance
----------------------------------------------------------------
The problem is caused by the remote port not being available. You get a ICMP packet with "Destination unreachable (Port unreachable)" and that causes an exception when you try to "listenerPort.Receive" The error is generated by sending data to an unreachable port and then windows receiving an ICMP packet with the error. The packet is in the receive buffer and when you try to read it you end up with an exception. All you have to do is add a TRY and then igonore the error and go back to listening to the port.
You do not get an exception when you send but that is where the problem lies.
Raymond Ortiz
---------------------------------------------
i am doing load test for 10 users,20users,30users working fine upto to this point if increase to 40 users iam getting error as below.
and also getting someimes for 30users as request time out . iam new to load test can any body help me out.
FAILED:An existing connection was forcibly closed by the remote host,
can any body help me on this error.
thanks.
-----------------------------------------------
It seems to be an SQL Server message. Please provide the exact error message and version information about platform and SQL Server. Since the system works well for low pressure, it's also possible that high pressure cause some request time out. So, please also let us know your hard ware sizing and what kind of operations you performed for each user.
-----------------------------------------------------------
Congratulations, you have found your first stress bug in your product using load testing
I have also seen this error in applications outside of SQL server. I believe it is a generic TCP/IP error. Your web server could be crashing, or calling response.Close().
At any rate, I suggest you take this problem to the developer of your product. They will probably have web server logs that correspond to this failure and will know what the problem is.
Thanks,
-JP
------------------------------------------------------
venkatanna wrote:
yes my web server is crashing ,how can i solve this problem.?.
thanks
file a bug with your developers. or the IT folks that will administer the site.
Seriously, fixing the site is not the tester's job.
there are a LOT of things that can affect webserver performance, everything from webserver settings, the physical hardware, to the code that runs the site.
this type of thing generally has to be debugged by the site developer and IT folks.. they will perhaps want to look at your test results and review the performance data that was logged to see if they identify a bottleneck (like cpu time, memory, disk access). Hopefully they also have good error logging inside the code and on the site itself, and can review those logs as well.
Website performance tuning is a HUGE subject area, and is well outside the scope of this forum..
-------------------------------------
Hi,
I have an urgent problem need help. We have an Asp.net application runing on window 2003 server. One part of the application is socket programming. When customer hits our site a little bit heavier, after serveral thoudsands hits, I statrt to get a bunch of
An existing connection was forcibly closed by the remote host
exceptions, then if the customer continue hits the site, the other function throw out of memory exception, The result is our site not responding any more. The attached is the socket call programming, any help will be appreciated! Thank you.
IPAddress[] ips = Dns.GetHostAddresses(m_sAVSHost); IPEndPoint oIPEndPoint = new IPEndPoint(ips[0], iAvsPort); Socket oSocket = new Socket(oIPEndPoint.Address.AddressFamily, SocketType.Stream, ProtocolType.Tcp); try { oSocket.Connect(oIPEndPoint); byte[] bSendBytes = Encoding.UTF8.GetBytes(sAvsParam); oSocket.Send(bSendBytes, bSendBytes.Length, SocketFlags.None);
byte[] bRecvBytes = new byte[4096]; oSocket.Receive(bRecvBytes, SocketFlags.None); rtnVal = Encoding.UTF8.GetString(bRecvBytes).Trim(); } catch (Exception ex) { VVXmlListenerException.LogException(new VVXmlListenerException(ex)); } finally { oSocket.Shutdown(SocketShutdown.Both); oSocket.Close(); }
-----------------------------------------
I would run a packet trace to find out what's actually going over the network. (Use a tool like Ethereal or Netmon.)
On all the occasions I've seen this exception, it has meant exactly what it says: the connection was forcibly closed. So this suggests that whatever your code here is connecting to is closing the connection. (Or you're going through a firewall, proxy, or other intermediary which is closing it.) The first step would be to confirm that - look at the actual packets going across the network to see what's really happening - that will most likely confirm that the error you're getting is consistent with the incoming network packets.
And if that's what you see, then the problem is external to the code you're showing us here. You need to look into why the external system - the thing that this code connects to - is kicking you off.
What does this code connect to? (I.e. what is m_sAVSHost?)
----------------------------------------------------------
For instructions on how to get a netmon trace see here:
http://blogs.msdn.com/dgorti/archive/2005/10/29/486887.aspx
For instructions on how to get a System.Net tracing log go here:
http://blogs.msdn.com/dgorti/archive/2005/09/18/471003.aspx
Mariya
---------------------------------------------------------------
I guess this is because the number of clients exceed the backlog set in the listen request of the server.
Please take a look at (5) & (6) th points in
https://blogs.msdn.com/malarch/archive/2006/06/26/647993.aspx.
Try to avoid starting many clients at the same time.. One simple solution is to Add a random sleep before starting the clients.
Malar
-----------------------------------------------------------------
Customer now resends the data when the other side closes the connection
Mariya
-----------------------------------------------------------------
I am living diffuculty at the same problem if you have solved the problem can you help me please? Thank you in advance
----------------------------------------------------------------
The problem is caused by the remote port not being available. You get a ICMP packet with "Destination unreachable (Port unreachable)" and that causes an exception when you try to "listenerPort.Receive" The error is generated by sending data to an unreachable port and then windows receiving an ICMP packet with the error. The packet is in the receive buffer and when you try to read it you end up with an exception. All you have to do is add a TRY and then igonore the error and go back to listening to the port.
You do not get an exception when you send but that is where the problem lies.
Raymond Ortiz
---------------------------------------------
Question on Test mix based on user pace - How many tests??
Hi, All
I met a problem when I using VSTS2008 SP1 to implement the loadtest.
First of all, I create two common unittest in the test project.
Then I add the loadtest to the test project, using a constant load of 2 Vusers. Selecting those unittests I had witten. In the "Test Mix" option. I choose the "Test mix based on user pace" model. I apply "60 Tests Per User Per Hour"
OK. run the loadtest for 10 min
Based on my scenario. The 2 vusers should run 120 tests in total per hour (60*2). If I run 10min, The total runs of the loadtest should be 20, but the actual test run is "40", double of the expected result.
Does my calculation wrong? or there are some other option I was missed.
Thank you.
-------------------------------------------------------------------------------------------------
Hi
This is a bit tricky. Note the definition of "tests per user per hour". It is used to configure the frequency of each test in the test mix. Since you added 2 unit tests, 2 users, 60 tests per user per hour and run 10 minutes, the calculation should be:
60 * 2 tests * 2 users = 240 tests per hour.
Since you run the load test for 10 minutes, so the total tests have run should be 240 / 6=40 tests.
I met a problem when I using VSTS2008 SP1 to implement the loadtest.
First of all, I create two common unittest in the test project.
Then I add the loadtest to the test project, using a constant load of 2 Vusers. Selecting those unittests I had witten. In the "Test Mix" option. I choose the "Test mix based on user pace" model. I apply "60 Tests Per User Per Hour"
OK. run the loadtest for 10 min
Based on my scenario. The 2 vusers should run 120 tests in total per hour (60*2). If I run 10min, The total runs of the loadtest should be 20, but the actual test run is "40", double of the expected result.
Does my calculation wrong? or there are some other option I was missed.
Thank you.
-------------------------------------------------------------------------------------------------
Hi
This is a bit tricky. Note the definition of "tests per user per hour". It is used to configure the frequency of each test in the test mix. Since you added 2 unit tests, 2 users, 60 tests per user per hour and run 10 minutes, the calculation should be:
60 * 2 tests * 2 users = 240 tests per hour.
Since you run the load test for 10 minutes, so the total tests have run should be 240 / 6=40 tests.
Password Encryption in VSTS
Of course, I work at a company that, for some reason, does not like the idea of unencrypted passwords being available to anyone that has access to the Webtests that we are creating.
Have I missed any built-in option that provides this functionality? If not, can I add this to the wish list?
-------------------------------------------------------------------------------------------
that would be a nice feature.. LoadRunner supports something like that so you can encrypt passwords, either in the test itself, or your datafiles..
Of course it's really just a speed bump to anyone who has access to the tool, since they could run it and look at the request details in the test results to see the acutal values being submitted.. (presuming they know or learn enough about how the tool works to do that)
Even if you can encrypt them, there is a very Strong argument here for a complete seperate set of userID''s and passwords on your test environment that are not used anywhere else, especially not in a production environment.
-----------------------------------------------------------------------------------------------
That's right, thanks Chuck. To be clear, encrypting the passwords does not make them secure, it just obfuscates them.
To really secure the password, we would have to tie the encryption algorithm to a machine, specific user, or use a shared secret (a cert file) that unless you had the cert file you couldn't decrypt the password. Or provide a central secure store for the user names and passwords.
We would also have to ensure the password does not show up in any log file or post request display.
But I agree Lewis, obfuscation is definitely the first step. We will likely go up to the shared secret option in our next release.
Ed.
---------------------------------------------------------------------------------------------
This is one of those instances where I think you REALLY need to take the view that any and every account in your test environment is presumed to be insecure. Those accounts should have no rights outside the test domain, they should be used ONLY for the test domain. Because ultimately the more you work to make it so those things are 'secured' the more difficult you make it to work with the tool and diagnose things if something goes wrong.. detecting a typo in a password becomes nearly impossible for example.
Ultimately the test system has to be able to submit the password for authentication, which pretty much means that anyone who can run the tests (perhaps in a debug mode) can get to the passwords. In that respect it's a lot like trying to secure a CD or DVD etc (and anyone who's been following that scene knows how effective those efforts have been).. You've also got run logs, recorded values, fiddler logs, and a host of other places a password could get stored in the clear simply because the system didn't know (or know at that TIME) that it was a password and should be obscured.
So were it me I would pretty much plan on a presumption that all test accounts are not secure, and thus limit them to the test sandbox. I think any encryption you have available is mostly to make your corporate IT folks happy, and to comply with any company directive that no passwords be stored in cleartext EVER.. (which is a not unreasonable policy)
Speaking of which, for sites not using integrated security (eg. most public websites) it's not a bad idea to peek at the db and see how user passwords are stored for your system.. If it's not at least something like a salted hash, then somebody needs a lesson in basic security
------------------------------------------------------------------------------------------------
Someone messing with your test environment and invalidating your results? why when would that ever happen?
Yeah I so Totally get where you are coming from in that respect.. and preventing some 'well meaning' person from 'helping' you with your testing is a great reason to be able to obfuscate passwords in scripts and script data..
(Note that 'helping' here is used in much the same way that a cat will 'help' you read a book by either laying ON the book, or otherwise interposing their body between your eyes and said book..)
------------------------------------------------------------------------------------------
Have I missed any built-in option that provides this functionality? If not, can I add this to the wish list?
-------------------------------------------------------------------------------------------
that would be a nice feature.. LoadRunner supports something like that so you can encrypt passwords, either in the test itself, or your datafiles..
Of course it's really just a speed bump to anyone who has access to the tool, since they could run it and look at the request details in the test results to see the acutal values being submitted.. (presuming they know or learn enough about how the tool works to do that)
Even if you can encrypt them, there is a very Strong argument here for a complete seperate set of userID''s and passwords on your test environment that are not used anywhere else, especially not in a production environment.
-----------------------------------------------------------------------------------------------
That's right, thanks Chuck. To be clear, encrypting the passwords does not make them secure, it just obfuscates them.
To really secure the password, we would have to tie the encryption algorithm to a machine, specific user, or use a shared secret (a cert file) that unless you had the cert file you couldn't decrypt the password. Or provide a central secure store for the user names and passwords.
We would also have to ensure the password does not show up in any log file or post request display.
But I agree Lewis, obfuscation is definitely the first step. We will likely go up to the shared secret option in our next release.
Ed.
---------------------------------------------------------------------------------------------
This is one of those instances where I think you REALLY need to take the view that any and every account in your test environment is presumed to be insecure. Those accounts should have no rights outside the test domain, they should be used ONLY for the test domain. Because ultimately the more you work to make it so those things are 'secured' the more difficult you make it to work with the tool and diagnose things if something goes wrong.. detecting a typo in a password becomes nearly impossible for example.
Ultimately the test system has to be able to submit the password for authentication, which pretty much means that anyone who can run the tests (perhaps in a debug mode) can get to the passwords. In that respect it's a lot like trying to secure a CD or DVD etc (and anyone who's been following that scene knows how effective those efforts have been).. You've also got run logs, recorded values, fiddler logs, and a host of other places a password could get stored in the clear simply because the system didn't know (or know at that TIME) that it was a password and should be obscured.
So were it me I would pretty much plan on a presumption that all test accounts are not secure, and thus limit them to the test sandbox. I think any encryption you have available is mostly to make your corporate IT folks happy, and to comply with any company directive that no passwords be stored in cleartext EVER.. (which is a not unreasonable policy)
Speaking of which, for sites not using integrated security (eg. most public websites) it's not a bad idea to peek at the db and see how user passwords are stored for your system.. If it's not at least something like a salted hash, then somebody needs a lesson in basic security
------------------------------------------------------------------------------------------------
Someone messing with your test environment and invalidating your results? why when would that ever happen?
Yeah I so Totally get where you are coming from in that respect.. and preventing some 'well meaning' person from 'helping' you with your testing is a great reason to be able to obfuscate passwords in scripts and script data..
(Note that 'helping' here is used in much the same way that a cat will 'help' you read a book by either laying ON the book, or otherwise interposing their body between your eyes and said book..)
------------------------------------------------------------------------------------------
Subscribe to:
Comments (Atom)
Unix/Linux Commands
Flow control Syntax break Exit a loop. continue Exit the current iteration of the loop and proceed with the next iteration. Ctrl+C Key com...
-
%systemroot%\system32\inetsrv\config\applicationHost.config https://docs.microsoft.com/en-us/iis/configuration/system.applicationhos...
-
hi everybody, i am doing load test for 10 users,20users,30users working fine upto to this point if increase to 40 users iam getting error a...
