Unit testing sucks. Tests are boring to write, nobody wants to run them, and thus they are subject to bitrot. For years, the only (actively) maintained tests were the parserTests to help handle regressions in MediaWiki’s wikitext parser. There were some PHPUnit-based ones in /tests and some custom TAP-based ones in /t. Neither worked perfectly, and neither is complete by any stretch of the word.

The past couple of days, things have at least improved. We’ve nuked /t entirely, meaning unit tests are going to be PHPUnit-based from here on out. All useful tests in /t were ported over first. Also, Mark has written a wrapper for the parser tests allowing them to be invoked as a normal part of the unit testing routine (this needs a rewrite to put the code in native PHPUnit format, rather than the hacky method we’re using now). While working on this, a couple of things have become apparent:

  1. Globals suck. Period. Trying to set up an instance of MediaWiki–which relies wayyyy too heavily on globals–in a non-global context (which PHPUnit does not run in) is a pain. I plan to be much more agressive on globals post-branching.
  2. We need more unit tests. Code coverage is far from complete, but at least by unifying the tests we can try to encourage people to write more tests.
  3. Need to work with Brion to redo the magic upload-test-results-to-CR bridge. PHPUnit can output test results in a variety of formats (TAP, XML…), and CodeReview should use this, rather than reinventing the wheel ourselves.

Finally, all the tests have been moved from /tests to /maintenance/tests, removing yet another top-level directory in the install. Oh yeah, code review is pretty much caught up too, which is a Good Thing.