Re: Question about the number of total tests in load testing

Question about the number of total tests in load testing Was this post helpful ?

Hi All,

I'm currently working on a load testing of reading one piece of property of a certain piece of equipment. What I'm confused about right now is, no matter how big user load I apply with, the number of passed tests increases to 198 and then stops increasing. The %processor time and %user time goes down to less than 1% immediately when the number of passed tests reaches 198. It's approximately happening around 3mins after the test starts to run, varing a bit depending on the number of users.

After the "198" issue occurs, everything else goes fine until the test finishes. (I set the run duration as 10 mins) So there is no fail tests, there is no other strange signs, but the passed tests number just equals to the total tests number which is 198.

It seems like 198 is a limit or something like that, but i didn't set any threshold to 198 though. Is there anyone who has an idea what it's going on here? Any help is greatly appreciated. Thanks!

Jane






26 Sep 2007, 12:15 AM UTC
Dennis Stone - MSFT
Moderator Posts 361
Re: Question about the number of total tests in load testing Was this post helpful ?

What kind of tests does your loadtest contain (web, unit, both)? Are you using any data sources in those tests, if so what kind of access method do you have specified.


http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1177487&SiteID=1


26 Sep 2007, 1:39 AM UTC
Jaeed Posts 11
Re: Question about the number of total tests in load testing Was this post helpful ?

Hi Jane,

You probably use unique access method for data source binding and that is why you cannot cross 198. Use Other data access method.

In my test, I observed the same situation, but I have overcome that and I got passed thousands of tests.

I think it would help you.

Thanks,
Abdur


26 Sep 2007, 4:12 PM UTC
JaneQQ Posts 13
Re: Question about the number of total tests in load testing Was this post helpful ?


Hello Abdur,

Thank you for your reply. I just don't quite understand what the "unique access method for data source binding" means...I do use a unique database to store the properties and equipment which are going to be read from. Do you mean the access to database or to a piece of property is unique??

Hope i can get more details. Thanks a lot!!

Jane

26 Sep 2007, 4:20 PM UTC
JaneQQ Posts 13
Re: Question about the number of total tests in load testing Was this post helpful ?

Hi,

My loadtest contains unit tests. In my tests, I created 5 pieces of equipment in the database and then created one property for each of these 5 equipment in a different table but the same database. The test method called "ReadPropertyTest" is going to read properties of equipment one for each read. And the user can only read 1 time each test. This is the basic idea that my test does.

So I don't know how to answer your question...I'm not sure what data sources i'm using and what kinda access method I've specified. Where should i check it out from??

Thanks !!
Jane



26 Sep 2007, 4:29 PM UTC
Michael Taute - MSFT
Moderator Posts 166
Re: Question about the number of total tests in load testing Was this post helpful ?

There are three different methods of accessing the data in a bound datasource. Sequential, random, and unique.

What she's saying is if you have your access method set to unique, one iteration of the test will run for each entry in your database and then it will stop. Ideally in this case though it would bubble up an out of data error and abort the test though.

It's easy enough to check... just highlight the datasource table you're binding to and view its properties.


26 Sep 2007, 5:23 PM UTC
Jaeed Posts 11
Re: Question about the number of total tests in load testing Was this post helpful ?


Jane,

Sorry I was in outside. And you got already answer from Michael.

Thanks Micheal for helping jane while I was not here.


Jaeed


28 Sep 2007, 7:23 PM UTC
JaneQQ Posts 13
Re: Question about the number of total tests in load testing Was this post helpful ?


Hi Michael,

Sorry for the delay in getting back to you. I was moved over to another issue and now i'm back. Regarding to what you've indicated here, I observed my test. In the unit test's property based on which I built up my load test, I noticed that the Data Access Method item is always sequential. And the other choice is Random, no option for unique. So I believe I've been using Sequential data access method all the time, if it is the same data access method as you were talking about.

I doubt i understood correctly because I thought I should check the data table in SQL server database, in which i stored my test data into. I checked actually, but i didn't find a data table's property called Data Access Method. You know what I mean? More specifically, my test is used to add more pieces of equipment or properties to an hierarchy of equipment model, I store all these equipment/property data in a database of SQL server on local host. Do you think I should check the data access method of this table or unit test of visual studio?

No matter which one is the thing you referred to, I didn't see anything set to Unique. Unless SQL server database use Unique data access method by default??

What am i going to do next?? Frustrated....


28 Sep 2007, 7:24 PM UTC
JaneQQ Posts 13
Re: Question about the number of total tests in load testing Was this post helpful ?

Hi Jaeed,

Thank you for your concern. That's totally fine. The issue is I still stucked on this thing...I guess i need more help.

Thanks;
Jane


28 Sep 2007, 7:52 PM UTC
Bill Barnett - MSFT
Moderator Posts 372
Answer Re: Question about the number of total tests in load testing Was this post helpful ?

I'll try to help. First of all, some of the earlier responses were asking about your data binding settings, but it's not clear to me from your responses wether or not your unit tests even use the VSTS data binding feature. In other words, do your unit test methods use data source attributes like the example shown in the docs here: http://msdn2.microsoft.com/en-us/library/ms182527(VS.90).aspx?

If not, then your problem has nothing to do with the VSTS data binding feature.

Do your unit tests do any other database access, or do something like invoke a Web service that access a database?

One possibility is that for some reason the 199th unit test gets stuck and never completes. Do you think that is a possibility? Can you describe what the unit test does? Does it send a request to a server and wait for a response? You could add some code to your unit tests to log to a file upon entering and leaving your unit test method which would help determine if the unit test itself is getting hung.

Bill