Appendix: Some Notes from the Badboy Forum
http://www.badboy.com.au/forum/viewtopic.php?t=1534&highlight=data+result
Posted: Sun Jun 17, 2007 9:50 am
Hi maximilien,
maximilien wrote: |
I am running the load testing using badboy. I am wondering can the badboy record down the response time for each of the iteration. (the raw data to ploting the graph)? |
Code: |
BrowserThread : Sending result [1 1 0 0 0 18.000000 ] for item 5 |
Quote: |
Also, I would like to check can badboy recording the number of pages being executed within a seconds? It would be very useful for us to generate the Page/seconds information. |
http://www.badboy.com.au/forum/viewtopic.php?t=658&highlight=data+result
Hi Albert,
Albert wrote: |
As loading variables from an Excel file is possible under Badboy, I would now require to execute the other way round action, i.e save new data created whilst playing a script (specific content retrieved on the website) on a new Excel file. |
Code: |
<col1>, <col2>, <col3>, ... |
Quote: |
If not existing, admit this would be a nice feature. Isnt' it? |
- - - -- -
Hi John,
jayers wrote: |
We’re using the export feature to get a CSV then we’re formatting the data with pivot tables… is it possible to export the results directly to a database? This would make it really easy for our team to run queries that would populate charts automatically. |
Code: |
var con = new ActiveXObject("ADODB.Connection"); |
Code: |
var con = badboy.plugin.createObject("ADODB.Connection"); |
___ - - - - - - - - -- - - - - - -- - - -- -
This is only “not under load’
http://www.badboy.com.au/forum/viewtopic.php?t=913&highlight=response
Hi George,
There are a
George Pickles wrote: |
Now my question is... how do I save the response times to a CSV or XLS file for easy analysis at a later date. Using the XML save item option give me LOTS of Data I have no nee for! |
A problem with t
Code: |
badboy.saveResponseTimes("c:\\responseTimes.csv") |
This can then be sav
Code: |
badboy.setVariable('responseTime', |
-- - - - - - -- -- -
Badboy and SOAP
http://www.badboy.com.au/forum/viewtopic.php?t=1528&highlight=data+result
4. You can schedule it off on the command line using task manager in windows, or just fire it off.
Things to think about: what is your most likely slow point? Is it going to be the network, the machine running the tests (should be dedicated - no one using it), the machine serving up the website (same thing - no one else should be on while testing), the database (same thing), IE (are you using standard [expected end-user] settings for memory, et. al.)? If you go to the JMeter sites/forums, there's a lot of information that is useful.
-----------------------------------
Viewstate Note
Short answer - ViewState is a generally volatile value (think "temp" variable) -- you should not be explicitly handling it; your script probably retains a specific value for it, and that's breaking subsequent playbacks.
I recommend examining your script, looking for any place where the viewstate value may be getting set, and deleting anything that directly sets or holds a viewstate value. Let Badboy's browser handle this itself.
For web automation tools that use a rich useragent (UA), it's normally not necessary to explicitly handle volatile or transient variables like this; in fact, I've usually found that while most recorders will capture them anyway, having done so the scripts often won't play back. Badboy, since it operates with an instance of MSIE, is one such tool where in general you should not attempt to explicitly handle such session-related tokens as ViewState -- let the MSIE (Badboy) browser handle it on its own.
As a separate issue, I've seen folks new to automated web testing have problems with this before, so in the interests of a bit of web testing education, over on the general web testing discussion forum I've posted "Handling State Tokens in Web Automation". Understanding the material in it can save you lots of trouble and man-hours later.
**Web developers and veterans at automated web testing will know all of it already, so it'll be of no value to you.**
Good luck with your testing.
Thanks fjw for the very comprehensive & clear post - this is certainly one of the issues that most often holds up folks who are new to automated testing (not just in Badboy - nearly all automated test solutions run into this).
I'll add that Badboy has an automatic mechanism for handling session tokens - it's currently only implemented for two cases:
Adding Variables
Automatic Variables are configured in the Badboy Preferences dialog. To add an Automatic Variable, go to the Preferences menu and select "Preferences" and then click on the "Variables" tab. The figure below shows how the Automatic Variables tab looks:
This must be fixed : viewstate
Changes toc Variables
When you play your script you can make changes to Variables just like you can to ordinary variables. In fact, Variables work just like ordinary variables in nearly all respects. One difference, however, is important: values of Variables are not saved with your script. Instead they are saved as part of the Badboy Preferences. This means that although your script can modify the values of c variables, other scripts will not see the modifications. If you want to make a permanent change to the value of an ic variable then you have to do it by changing the variable in the Preferences.
If you are familiar with the concept of environment variables, you can think ofc Variables as being a bit like environment variables for Badboy. Just like environment variables, scripts get the values of the Automatic Variables when they are opened. After that they can be modified but the changes are only visible to the Badboy process that is running the script.
Predefined Variables
Badboy comes with some Variables built in when you install it. These are common variables that are helpful to applications built on web frameworks such as Java Servlets and ASP.NET. They help by capturing session id's and form states to ensure scripts play back properly, even when the server uses different unique values for each web session. If you don't need these variables you can delete them, however they shouldn't cause you any trouble and can help make testing applications built on these common frameworks much easier.
It looks like Badboy is searching for a specific sytnax of the viewstate which does not match the sytax our application is using.
This is the syntax from our xhtml source:
<input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE" value="/xxxxxxxxxxxxx==" />
And the regex which Badboy is using:
name="__VIEWSTATE" value="([^"]*)"
A solution may be to change the regex:
name="__VIEWSTATE" (?:id="__VIEWSTATE" )?value="([^"]*)"
I am not sure if this is the cause of the problems we've been having, but after making this change to the automatic variable I assume we'll known shortly
No problem! We also had to add another automatic variable:
name: EVENTVALIDATION
expression: name="__EVENTVALIDATION" (?:id="__EVENTVALIDATION" )?value="([^"]*)"
In addition to the change to the viewstate expression, this has corrected all of our event state problem
BB starts up in recording mode. You simply type in a URL.
Also once you convert the script into the correct mode you cannot see it play back on the right hand part of the screen.
Appendices
In a nutshell:
1. Record your test script using Bad Boy, and parameterize everything necessary (e.g. logins, add, edit, search forms, whatever).
2. Set up your datasources so they will be pulled into your parameters.
3. Set up the the script to thread.
4. You can schedule it off on the command line using task manager in windows, or just fire it off.
Things to think about: what is your most likely slow point? Is it going to be the network, the machine running the tests (should be dedicated - no one using it), the machine serving up the website (same thing - no one else should be on while testing), the database (same thing), IE (are you using standard [expected end-user] settings for memory, et. al.)? If you go to the JMeter sites/forums, there's a lot of information that is useful.
I don't know if anybody else has done similar stuff, atleast didn't find anything in the forum. Instead of using Mouse Click (cascading), my solution for a modal dialog that pops in my application is to introduce "Threaded Item".
So what I did was start a thread just before the step that results in a modal message dialog pop up, and in the thread, I pause for some seconds, and then subsequently send keystroke {VK_RETURN} or {VK_ESCAPE} depending on the message box that has "OK" and "Cancel".
So when the main program pops up dialog, the thread spawned before sends the keystroke and dismisses the dialog.
Using Different Data Across Threads
You may encounter situations where your script needs to ensure different data is used by each thread. For example, each thread might need its own login account if your system only allows a user to login to a single session at a time.
To achieve this easily, Badboy provides a special "threadNum" variable that you can use to access different data in different threads. An example of how to do this is illustrated in the following steps:
Tip on handling pop ups such as the stm log out
I don't know if anybody else has done similar stuff, atleast didn't find anything in the forum. Instead of using Mouse Click (cascading), my solution for a modal dialog that pops in my application is to introduce "Threaded Item".
So what I did was start a thread just before the step that results in a modal message dialog pop up, and in the thread, I pause for some seconds, and then subsequently send keystroke {VK_RETURN} or {VK_ESCAPE} depending on the message box that has "OK" and "Cancel".
So when the main program pops up dialog, the thread spawned before sends the keystroke and dismisses the dialog.