Appendix: Some Notes from the Badboy Forum
http://www.badboy.com.au/forum/viewtopic.php?t=1534&highlight=data+result

Posted: Sun Jun 17, 2007 9:50 am 
Hi maximilien,

maximilien wrote:

I am running the load testing using badboy. I am wondering can the badboy record down the response time for each of the iteration. (the raw data to ploting the graph)?
It would be very helpful to generate the load test matrix (for eg. 90%-tile of the response time, Max-Time, Min Time, Average Time)


Unfortunately it's not really possible to get the individual response times in a simple way. I agree that that would be a useful and important function when you want to compute more detailed statistics (such as percentiles). If you are using MSHTML Threads then in fact you could add JScript items to your script that will log the response time for the previous response.

Alternatively, if you turn the logging up to DEBUG level (see "Logging" tab in Preferences) then you will see the individual times logged like so:

Code:

BrowserThread : Sending result [1 1 0 0 0 18.000000 ] for item 5


So - with enough effort you could use this to reconstruct the times. I've logged a feature request to add support for logging or getting at the "real" data much easier than this!

Quote:

Also, I would like to check can badboy recording the number of pages being executed within a seconds? It would be very useful for us to generate the Page/seconds information.


Badboy counts the responses received as the "Played" number in the summary statistics. So if you want the "Pages/Seconds" number then you could divide that by the length of the overall test. (Again, unfortunately, Badboy doesn't track the actual time taken for the whole test, it only counts the response times themselves).

Hope this helps!

Simon

http://www.badboy.com.au/forum/viewtopic.php?t=658&highlight=data+result

Hi Albert,

Albert wrote:

As loading variables from an Excel file is possible under Badboy, I would now require to execute the other way round action, i.e save new data created whilst playing a script (specific content retrieved on the website) on a new Excel file.


The easiest way to make a file Excel can load is to save it in CSV (Comma Separated Values) format. If what you want is the response times, Badboy has that feature built in. However if you want something else, you can often make a CSV file quite easily by saving the content using a Save Item, and making an expression that has the columns you want to appear in the Excel file in the form:

Code:

<col1>, <col2>, <col3>, ...
<blank line>


The main problem with this is if the values might contain commas - then it can get more difficult.

Quote:

If not existing, admit this would be a nice feature. Isnt' it?
Please let me know on how to proceed for such an operation.


I agree! However it depends a bit what you are trying to save. Usually CSV format is pretty good, and if you want something more flexible, you can write to a database, eg: see this thread

http://www.badboy.com.au/forum/viewtopic.php?t=1142

But if you have a case where neither of these works well, I'd love to hear more details and write up a specific feature request.

Hope this helps!

Simon.

 - - - -- -

Hi John,

jayers wrote:

We’re using the export feature to get a CSV then we’re formatting the data with pivot tables… is it possible to export the results directly to a database? This would make it really easy for our team to run queries that would populate charts automatically.

It might be worth looking into using a technique as mentioned here:

http://www.badboy.com.au/forum/viewtopic.php?t=1142

Basically, you could put a JScript item into your script and write to the database directly with that. Note that with Badboy 2.0 you can avoid the security warnings with this technique by marking the JScript item as having "plugin" privileges (just a checkbox to tick in the properties) and then use instead of:

Code:

var con = new ActiveXObject("ADODB.Connection");

something like this:

Code:

var con = badboy.plugin.createObject("ADODB.Connection");

Hope this helps!

___ - - - - - - - - -- - - - - - -- - - -- -

This is only “not under load’

http://www.badboy.com.au/forum/viewtopic.php?t=913&highlight=response

Hi George,


There are a

George Pickles wrote:

Now my question is... how do I save the response times to a CSV or XLS file for easy analysis at a later date. Using the XML save item option give me LOTS of Data I have no nee for!

few ways you could do this. One is very simple, if it works for you. Badboy has a built in function to save the response times as CSV data. You can call this function from your script itself using JScript - just drop a JScript item from the Toolbox as the last item in your script and enter:


A problem with t

Code:

badboy.saveResponseTimes("c:\\responseTimes.csv")

his is that it will overwrite the file each time - it doesn't, unfortunately, have an "Append" option.

If you happen to know or be willing to learn a little XSLT then there is another option: you can use the "Save Item" to save as Badboy XML and give it a style sheet to transform the data to CSV form. Then you can set the Save Item to "Append Mode" which will add the results to the end of the file over time.

Finally, you could access the
response times via JScript and set those in a variable, using some JScript like so:


This can then be sav

Code:

badboy.setVariable('responseTime',
  badboy.summary(1,"averageResponseTime"));

ed to a file with a Save Item as an expression - this way you can save your own CSV format to load in Excel.

This seems like something that should be simpler to do - I will add a feature request to make this a built in function in the future as I think it's an important and useful thing to be able to do.

Hope this helps!

Simon.

-- - - - - - -- -- -

Badboy and SOAP

http://www.badboy.com.au/forum/viewtopic.php?t=1528&highlight=data+result


4. You can schedule it off on the command line using task manager in windows, or just fire it off.

Things to think about: what is your most likely slow point? Is it going to be the network, the machine running the tests (should be dedicated - no one using it), the machine serving up the website (same thing - no one else should be on while testing), the database (same thing), IE (are you using standard [expected end-user] settings for memory, et. al.)? If you go to the JMeter sites/forums, there's a lot of information that is useful.

-----------------------------------

Viewstate Note

Short answer - ViewState is a generally volatile value (think "temp" variable) -- you should not be explicitly handling it; your script probably retains a specific value for it, and that's breaking subsequent playbacks.

I recommend examining your script, looking for any place where the viewstate value may be getting set, and deleting anything that directly sets or holds a viewstate value. Let Badboy's browser handle this itself.

For web automation tools that use a rich useragent (UA), it's normally not necessary to explicitly handle volatile or transient variables like this; in fact, I've usually found that while most recorders will capture them anyway, having done so the scripts often won't play back. Badboy, since it operates with an instance of MSIE, is one such tool where in general you should not attempt to explicitly handle such session-related tokens as ViewState -- let the MSIE (Badboy) browser handle it on its own.


As a separate issue, I've seen folks new to automated web testing have problems with this before, so in the interests of a bit of web testing education, over on the general web testing discussion forum I've posted "Handling State Tokens in Web Automation". Understanding the material in it can save you lots of trouble and man-hours later.

**Web developers and veterans at automated web testing will know all of it already, so it'll be of no value to you.**

Good luck with your testing.

Thanks fjw for the very comprehensive & clear post - this is certainly one of the issues that most often holds up folks who are new to automated testing (not just in Badboy - nearly all automated test solutions run into this).

I'll add that Badboy has an automatic mechanism for
handling session tokens - it's currently only implemented for two cases:

  • .NET ViewState
  • Java Servlet URL Rewriting (jsessionid).

  • It might not be covered in the help at time of writing, but with the 1.6 release there should be a comprehensive topic in the Help under "Automatic Variables" to explain how this works. But the gist of it is that Automatic Variables do exactly the process described above of extracting the relevant session ids from the page response and ensuring that it is sent in future transactions.

    Of course there are many many more cases that could be handled, and possibly cases where the ones that are handled by default could be refined. Anyone who gets the
    handling correct for any particular web framework they are using would be most welcome to send them to us (using the Feedback link is probably the best way) and we can build those into Badboy to make it handle those in the future.

    Cheers,

    Simon.
  •  

    Adding Variables

    Automatic Variables are configured in the Badboy Preferences dialog. To add an Automatic Variable, go to the Preferences menu and select "Preferences" and then click on the "Variables" tab. The figure below shows how the Automatic Variables tab looks:

     

     

    This must be fixed : viewstate

    Changes toc Variables

    When you play your script you can make changes to Variables just like you can to ordinary variables. In fact, Variables work just like ordinary variables in nearly all respects. One difference, however, is important: values of Variables are not saved with your script. Instead they are saved as part of the Badboy Preferences. This means that although your script can modify the values of c variables, other scripts will not see the modifications. If you want to make a permanent change to the value of an ic variable then you have to do it by changing the variable in the Preferences.

    If you are familiar with the concept of environment variables, you can think ofc Variables as being a bit like environment variables for Badboy. Just like environment variables, scripts get the values of the Automatic Variables when they are opened. After that they can be modified but the changes are only visible to the Badboy process that is running the script.

    Predefined Variables

    Badboy comes with some Variables built in when you install it. These are common variables that are helpful to applications built on web frameworks such as Java Servlets and ASP.NET. They help by capturing session id's and form states to ensure scripts play back properly, even when the server uses different unique values for each web session. If you don't need these variables you can delete them, however they shouldn't cause you any trouble and can help make testing applications built on these common frameworks much easier.

    It looks like Badboy is searching for a specific sytnax of the viewstate which does not match the sytax our application is using.

    This is the syntax from our xhtml source:
    <input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE" value="/xxxxxxxxxxxxx==" />

    And the regex which Badboy is using:
    name="__VIEWSTATE" value="([^"]*)"

    A solution may be to change the regex:
    name="__VIEWSTATE" (?:id="__VIEWSTATE" )?value="([^"]*)"

    I am not sure if this is the cause of the problems we've been having, but after making this change to the automatic variable I assume we'll known shortly

    No problem! We also had to add another automatic variable:

    name: EVENTVALIDATION
    expression: name="__EVENTVALIDATION" (?:id="__EVENTVALIDATION" )?value="([^"]*)"

    In addition to the change to the
    viewstate expression, this has corrected all of our event state problem

     

    • when running in the correct mode, you don’t get responses to your requests.
  • Can’t run any load with the script in “default” mode. You must convert the script. After conversion, you can no longer edit the script.
  • Before recording any script you must SHUT DOWN, bad boy inorder to get it to record properly.
  • BB starts up in recording mode. You simply type in a URL.

    Also once you convert the script into the correct mode you cannot see it play back on the right hand part of the screen.

    Appendices

    In a nutshell:
    1. Record your test script using Bad Boy, and parameterize everything necessary (e.g. logins, add, edit, search forms, whatever).
    2. Set up your datasources so they will be pulled into your parameters.
    3. Set up the the script to
    thread.
    4. You can schedule it off on the command line using task manager in windows, or just fire it off.

    Things to think about: what is your most likely slow point? Is it going to be the network, the machine running the tests (should be dedicated - no one using it), the machine serving up the website (same thing - no one else should be on while testing), the database (same thing), IE (are you using standard [expected end-user] settings for memory, et. al.)? If you go to the JMeter sites/forums, there's a lot of information that is useful.

    I don't know if anybody else has done similar stuff, atleast didn't find anything in the forum. Instead of using Mouse Click (cascading), my solution for a modal dialog that pops in my application is to introduce "Threaded Item".

    So what I did was start a
    thread just before the step that results in a modal message dialog pop up, and in the thread, I pause for some seconds, and then subsequently send keystroke {VK_RETURN} or {VK_ESCAPE} depending on the message box that has "OK" and "Cancel".

    So when the main program pops up dialog, the thread spawned before sends the keystroke and dismisses the dialog.

    Using Different Data Across Threads

    You may encounter situations where your script needs to ensure different data is used by each thread. For example, each thread might need its own login account if your system only allows a user to login to a single session at a time.

    To achieve this easily, Badboy provides a special "threadNum" variable that you can use to access different data in different threads. An example of how to do this is illustrated in the following steps:

    • Create a variable (for example, "foo") and add a list of values to it. In this example, one value will be used by each thread that you run.
    • When you reference your variable in your script, do so in the form:
    •     ${foo[${threadNum}]}_ - - - -- - - - - -

    Tip on handling pop ups such as the stm log out

    I don't know if anybody else has done similar stuff, atleast didn't find anything in the forum. Instead of using Mouse Click (cascading), my solution for a modal dialog that pops in my application is to introduce "Threaded Item".

    So what I did was start a
    thread just before the step that results in a modal message dialog pop up, and in the thread, I pause for some seconds, and then subsequently send keystroke {VK_RETURN} or {VK_ESCAPE} depending on the message box that has "OK" and "Cancel".

    So when the main program pops up dialog, the thread spawned before sends the keystroke and dismisses the dialog.

    BB Appndx