FunkLoad CHANGES

Author: Benoit Delbosc
Address:
bdelbosc _at_ nuxeo.com
Revision: CHANGES.txt 53638 2009-06-03 12:07:52Z bdelbosc

Abstract

This document describes changes between FunkLoad versions.

Table of Contents

FunkLoad SVN trunk

Svn:http://svn.nuxeo.org/pub/funkload/trunk
Target:1.10.1

Bug fixes

  • Fixing gnuplot charts when concurrent users are not in ascending order or have duplicates. (like -c 1:10:50:10:1 instead of -c 1:10:20:30)
  • Prevent fl-record to generate invalid python test case name.
  • Fix regex to check valid resources urls, it was too restrictive.
  • Fix cookie support, there was an extra ";" if there was only one cookie, thanks to Daniel Swärd.

FunkLoad 1.10.0

Package:http://pypi.python.org/packages/source/f/funkload/funkload-1.10.0.tar.gz
Svn:http://svn.nuxeo.org/pub/funkload/tags/1.10.0
Released date:2009-03-23

New features

  • Easier installation on Debian Lenny/Ubuntu 8.10, See INSTALL.
  • Support setuptools console_scripts entry point for funkload scripts. Thanks to Ross Patterson.
  • Added a two sample buildout configurations: a minimal one that just contains what is necessary to run tests and a default one that also enables recording. Thanks to Tom Lazar.
  • Added a --report-directory option to fl-build-report to set report directory path.
  • Added a --label option to fl-run-bench which the report generator then appends to the output directory of the report. This is to make it easier to keep different bench runs apart from each other. Thanks to Tom Lazar.
  • Added a --pause option to fl-run-test, the test runner will wait for the ENTER key to be pressed between requests.
  • Added a --loop option to fl-record, this way it is easy to type a comment then hit Ctrl-C to dump the script and continue with the next action.

Bug fixes

  • Only attempt to retrieve syntactically valid URLs links.
  • Fix report generation for bench with only one cycle.
  • Fix differential report average percent profit chart.
  • Fix possible error on differential report ReStructuredText.

FunkLoad 1.9.0

Package:http://pypi.python.org/packages/source/f/funkload/funkload-1.9.0.tar.gz
Svn:http://svn.nuxeo.org/pub/funkload/tags/1.9.0
Released date:2008-11-07

New features

  • Switch to gnuplot 4.2 for charts generation. Gnuplot script and data are kept with the report enabling charts customization. No more gdchart charting woes.

  • Enhanced charts displaying more information, view a report example.

  • New diff report option, comparing 2 reports is a long and error prone task, fl-build-report has been enhanced to produce a differencial report. This new feature requires gnuplot.

    To produce a diff report:

    fl-build-report --diff path/to/report-reference path/to/report-challanger
    

    View a diff report example.

  • Post method handles custom data and content type. For example to post a xmlrpc by hand

    from funkload.utils import Data
    ...
        data = Data('text/xml', """<?xml version='1.0'?>
    <methodCall>
    <methodName>getStatus</methodName>
    <params>
    </params>
    </methodCall>
    """)
        self.post(server_url, data, description="Post user data")
    
  • The recorder translates properly application/xml or any content type using the new Data class (see above exemple).

  • New test script provided with fl-install-demo to bench the JBoss Seam Booking application.

FunkLoad 1.8.0

Package:http://pypi.python.org/packages/source/f/funkload/funkload-1.8.0.tar.gz
Svn:http://svn.nuxeo.org/pub/funkload/tags/1.8.0/
Released date:2008-07-28

New features

  • Upgrate to python-gdchart2 using libgd2 (gdchart1 is deprecated).

Bug fixes

  • Handle redirect code 303 and 307 like a 302.

FunkLoad 1.7.0

Package:http://funkload.nuxeo.org/funkload-1.7.0.tar.gz
Svn:http://svn.nuxeo.org/pub/funkload/tags/1.7.0/
Released date:2008-07-23

New features

  • Recorder and test take care of JSF MyFaces tokens which make FunkLoad ready to record/play Nuxeo EP or any JBoss Seam application.
  • Change API of listHref to be able to search on link content as well as link url. pattern parameter is renamed into url_pattern and a new content_pattern can be supply.

Bug fixes

  • fix: # 1838: Upload doesn't work for CherryPy.
  • fix: # 1834: multiple redirects not handled properly (jehiah patch).
  • fix: # 1837: post() is sent as GET when no params defined.
  • Apply patch from Dan Rahmel to fix fl-record on non posix os.

FunkLoad 1.6.2

Package:http://funkload.nuxeo.org/funkload-1.6.2.tar.gz
Svn:http://svn.nuxeo.org/pub/funkload/tags/1.6.2/
Released date:2007-04-06

Bug fixes

  • fix: 'Page stats' percentiles are wrong.

FunkLoad 1.6.1

Package:http://funkload.nuxeo.org/funkload-1.6.1.tar.gz
Svn:http://svn.nuxeo.org/pub/funkload/tags/1.6.1/
Released date:2007-03-09

Bug fixes

  • Support of python 2.5.
  • Fix: 1.6.0 regression invalid encoding of parameters when posting several times the same key.

FunkLoad 1.6.0

Package:http://funkload.nuxeo.org/funkload-1.6.0.tar.gz
Svn:http://svn.nuxeo.org/pub/funkload/tags/1.6.0/
Released date:2007-02-27

New features

  • Do not send cookie with deleted value, this fix trouble with Zope's CookieCrumbler and enable benching of Plone apps, Thanks to Lin.
  • Works with Ruby CGI, fixing webunit mimeEncode and adding Content-type header on file upload, Thanks to Bryan Helmkamp.
  • # 1283: Patching webunit to support http proxy by checking $http_proxy env. Thanks to Greg, (note that https proxy is not yet supported).
  • Adding Mirko Friedenhagen --with-percentiles option to fl-build-report to include percentiles in statistic tables and use 10%, 50% and 90% percentil instead of min, avg and max in charts. This is now the default option, use --no-percentiles for the old behaviour.
  • Upgrade to setuptools 0.6c3
  • FunkLoadTestCase.conf_getList accept a separator parameter

Bug fixes

  • fix: # 1279: Browser form submit encoding, default encoding for post is now application/x-www-form-urlencoded, multipart mime encoding is used only when uploading files.
  • Patching webunit mimeEncode method to send CRLF as describe in RFC 1945 3.6.2, patch provided by Ivan Kurmanov.
  • fix: response string representation url contains double /
  • fix: xmlrpc url contains basic auth credential in the report
  • fix: # 1300: easy_install failed to install docutils from sourceforge, upgrading ez_install 0.6a10

FunkLoad 1.5.0

Package:http://funkload.nuxeo.org/funkload-1.5.0.tar.gz
Svn:http://svn.nuxeo.org/pub/funkload/tags/1.5.0/
Released date:2006-01-31

New features

  • # 1284: TestCase: support of doctest

    There is a new FunkLoadDocTest class that ease usage of FunkLoad from doctest:

    >>> from funkload.FunkLoadDocTest import FunkLoadDocTest
    >>> fl = FunkLoadDocTest()
    >>> response = fl.get('http://localhost/')
    >>> 'HTML' in response.body
    True
    >>> response
    <response url="http://127.0.0.1:80//" code="200" message="OK" />
    

    If you use python2.4, the test runner fl-run-test is able launch doctest from a plain text file or embedded in python docstring:

    $ fl-run-test -v doctest_dummy.txt
    Doctest: doctest_dummy.txt          Ok
    -----------------------------------------------------
    Ran 1 test in 0.077s
    OK
    

    And the --debug option makes doctests verbose:

    $ fl-run-test -d doctest_dummy.txt
    ...
    Trying:
        fl = FunkLoadDocTest()
    Expecting nothing
    ok
    Trying:
        fl.get('http://localhost/')
    Expecting:
        <response url="http://127.0.0.1:80//" code="200" message="OK" />
    ok
    Ok
    ----------------------------------------------------------------------
    Ran 1 test in 0.051s
    OK
    
  • Test runner can use a negative regex to select tests. For example if you want to launch all tests that does not ends with 'foo'

    fl-run-test myFile.py -e '!foo$'
    
  • # 1282: TestRunner: more verbosity

    The new fl-run-test option --debug-level=2 will produce debug output on each link (images or css) fetched.

  • Improve firefox view in real time by using approriate file extention for the content type.

  • CPSTestCase is up to date for 3.4.0, use CPS338TestCase for a CPS 3.3.8.

Bug fixes

  • fix # 1278: BenchRunner: UserAgent from config file is not set

FunkLoad 1.4.1

Package:http://funkload.nuxeo.org/funkload-1.4.1.tar.gz
Svn:http://svn.nuxeo.org/pub/funkload/tags/1.4.1/
Released date:2005-12-16

Bug fixes

  • fix # 1201: Erroneous page stats
  • fix # 934: REPORT: Charts should display origin

FunkLoad 1.4.0

Package:http://funkload.nuxeo.org/funkload-1.4.0.tar.gz
Svn:http://svn.nuxeo.org/pub/funkload/tags/1.4.0/
Released date:2005-12-08

New features

  • New --loop-on-pages option for fl-run-test.

    This option enable to check response time of some specific pages inside a test without changing the script, which make easy to tune a page in a complex context. Use the debug option to find the page numbers. For example:

    fl-run-test myfile.py MyTestCase.testSomething -l 3 -n 100
    

    Run MyTestCase.testSomething, reload one hundred time the page 3 without concurrency and as fast as possible. Output response time stats. You can loop on many pages using slice -l 2:4.

  • New --accept-invalid-links option for fl-run-test and fl-run-bench

    Don't fail if css/image links are not reachable.

  • New --list option for fl-run-test to list the test names without running them.

  • # 936: TestRunner: use regexp to load test

    New --regex or -e option for fl-run-test to filter test names that match a regular expression.

  • # 939: Browser: Provide an option to disable image and links load

    New --simple-fetch option for fl-run-test and fl-run-bench.

  • # 937: TestRunner: Add an immediate fail option

    New --stop-on-fail option for fl-run-test that stops tests on first failure or error.

  • # 933: Report: Add global info

    Adding total number of tests, pages and requests during the bench.

  • CPSTestCase.listDocumentHref is renamed into cpsListDocumentHref

  • FunkLoadTestCase.xmlrpc_call is renamed into xmlrpc (xmlrpc_call is still working)

  • Some code cleaning, cheesecake index 460/560 ~82%.

  • New epydoc API documentation.

  • fl-run-test is now able to run standard unittest.TestCase.

Bug fixes

  • # 1183: updating ez_setup to fix broken sourceforge docutils download

FunkLoad 1.3.1

Package:http://funkload.nuxeo.org/funkload-1.3.1.tar.gz
Svn:http://svn.nuxeo.org/pub/funkload/tags/1.3.1/
Released date:2005-11-10

Bug fixes

  • fix # 1115: Recorder: impossible to generate test

FunkLoad 1.3.0

Package:http://funkload.nuxeo.org/funkload-1.3.0.tar.gz
Svn:http://svn.nuxeo.org/pub/funkload/tags/1.3.0/
Released date:2005-11-08

New features

  • # 944: Recorder: replace TestMaker recorder.

    Providing a fl-record command that drive a TCPWatch proxy. See INSTALL to setup TCPWatch.

  • # 1041: Browser: implement an addHeader method.

    FunkLoadTestCase provides new methods setUserAgent, addHeader and clearHeaders.

  • # 1088: TestRunner / BenchRunner: use compatible command line option

    • All fl-* executables have a --version option to display the FunkLoad version.
    • All fl-run-* are now in color mode by default. Use --no-color options for monochrome output. You need to remove the -c option for fl-run-test and -C for fl-run-bench in your scripts.
    • Changing fl-run-bench short option -d into -D for duration, keeping -d for debug mode.
    • Removing fl-run-test short option -D to not conflict with new -D option of fl-run-bench, you now have to use the long format --dump-directory.

Bug fixes

  • fix # 935: Browser: doesn't handle Referer header.

FunkLoad 1.2.0

Package:http://funkload.nuxeo.org/funkload-1.2.0.tar.gz
Svn:http://svn.nuxeo.org/pub/funkload/tags/1.2.0/
Released date:2005-10-18

New features

  • Credential and Monitor services have been refactored they are now true unix daemon service, controllers are now in pure python (no more bash scripts).
  • Switching from distutils to setuptools using EasyInstall, installing FunkLoad is now just a question of sudo easy_install funkload.
  • Moving demo examples into the egg, just type fl-install-demo to extract the demo folder

Bug fixes

  • fix # 1027 Report: min and max page response time are wrong.
  • fix # 1017 Report: request charts is alway the same.
  • fix # 1022 Monitor: no cpu usage monitoring on linux 2.4.x
  • fix # 1000 easy_install failed to install funkload.
  • fix # 1009 Report: remove error scale in graph if there is no errors.
  • fix # 1008 Report: missing legend.

FunkLoad 1.1.0

Package:http://funkload.nuxeo.org/funkload-1.1.0.tar.gz
Svn:http://svn.nuxeo.org/pub/funkload/tags/1.1.0/
Released date:2005-10-07

New features

  • FunkLoadTestCase: adding exists method.
  • FunkLoadTestCase: support XML RPC test/bench using xmlrpc_call.
  • FunkLoadTestCase: adding a regex pattern to listHref.
  • FunkLoadTestCase: new setUpCycle and tearDownCycle methods to configure bench between cycle.
  • FunkLoadTestCase: Patching webunit to send a User-Agent header.
  • # 950 Report: display failure and error (first part).
  • # 948 Report: provide the 5 slowest requests.
  • # 941 Demo: provide usefull examples.
  • CPSTestCase: add cpsVerifyUser, cpsVerifyGroup, cpsSetLocalRole, cpsCreateSite, cpsCreateSection.
  • ZopeTestCase: adding zopeRestartZope, zopeFlushCache, zopePackZodb, zopeAddExternalMethod.
  • Lipsum: handle iso 8859-15 vocabulary.
  • Lipsum: adding random phone number and address generator.
  • credentiald: add methods listCredentials and listGroups.
  • Moving TODO and bugs to trac: http://svn.nuxeo.org/trac/pub/report/12
  • Improve documentation.

Bug fixes

  • # 971 Report: the network load monitor should record network speed instead of cumulative downlaod
  • XML result file is resetted at beginning of a test or bench.
  • Fix threadframe module requirement.
  • No more python 2.3 dependency for scripts fl-*-ctl

FunkLoad 1.0.0

Location:http://funkload.nuxeo.org/funkload-1.0.0.tar.gz
Released date:2005-09-01

First public release.


See the INSTALL file for requirement and installation.

See README for more information about FunkLoad.