(Release 1.2. The RegTest script is in the public domain.)
RegTest is a very simple script for writing IF unit tests. You write down a list of commands and the output you want from each one. RegTest will run the list through your game, and check for that output.
RegTest can work with any interpreter which uses stdin/stdout, such as DumbFrotz or Glulxe/CheapGlk. (But see "Restrictions", below.)
All of what RegTest does is defined by a test file. The easiest way to explain it is to paste one in. (With a soupçon of syntax coloring, for documentation's sake.)
# advent-regtest: test script for regtest.py # For a full description, see <http://eblong.com/zarf/plotex/regtest.html> ** game: /Users/zarf/Documents/IF/Advent.ulx ** interpreter: /Users/zarf/bin/glulxec -q * south-from-start # A simple, one-command test. > south You are in a valley in the forest beside a stream tumbling along a rocky bed. * in-well-house # Test the opening text, followed by two commands. Lines starting # with "!" are negated; lines starting with "/" are regular expressions. Welcome to Adventure! Crowther Woods > go east There is tasty food here. some keys !grue > get all /b[aeiou]ttle.*water !/^Taken * test-that-fails # All three of the tests in this run will fail. > go east There is a bucket of cheese here. /[xqz] ! Inside Building
The first two lines are comments. Lines beginning with "#", and blank lines, are ignored.
The next two lines (beginning with "**") define test parameters -- the location of the game file and interpreter. The game will be the Glulx version of Adventure (compiled with Inform 6). The interpreter will be Glulxe/CheapGlk. I've defined pathnames in my computer's filesystem; you'd want to change those, of course. (You can also supply these values from the command line.)
A line beginning with "** precommand:" is an extra command that will be stuck onto the beginning of every test defined in the file.
The rest of the test file is a set of tests. Each test is a separate run through the game. A test contains a sequence of commands. A command can contain various checks, validating the output of that command.
(All the "**" lines should appear before the tests begin.) (Okay, you could customize the game file or interpreter for a specific test if you really wanted. But why?) (This is a rhetorical question.)
The line "* south-from-start" defines the beginning of the first test. south-from-start is the test name. (You can name tests anything you want; it's just a convenient label.)
This test contains just one command -- south. The next line is a check: RegTest will search the command's output for this line. It's the room description for the room to the south, obviously.
The second test is called in-well-house. Here we start by performing some checks on the banner text of the game. (Note that this test is a fresh start; the previous "south" command was in a different run.) RegTest verifies that "Welcome to Adventure!" occurs somewhere in the game's initial output. Then it looks for "Crowther" and "Woods", which also occur. (These aren't complete lines, but that's fine -- the check line just has to occur somewhere in one of the paragraphs that begin the game. The two name tests happen to occur in the same line; that's fine too.)
After the initial text, we go east. We're applying three different checks to the output of "go east". RegTest verifies that "There is tasty food here." and "some keys" both occur. (Remember, we're looking only at the output of the latest command, not the entire transcript.)
A check line starting with "!" is negated: RegTest verifies that none of the output contains the word grue. Which is good, because there are no grues in Colossal Cave.
These are independent checks; order doesn't matter. (The line about the keys actually occurs before the one about the food.)
The idea is that you don't usually want to verify every single character of your game output. During development, you're going to be changing descriptions, adding objects, and so on. But you still might want to write a test sequence for particular actions. By checking only for the important bits of each response, you don't have to fix the test every time a room description or timer event changes.
The next command demonstrates regular expressions. A check line that begins with "/" is matched as a regular expression. (See the Python documentation for the syntax of regular expressions.) Here we have a (contrived) regex which matches the output line "stream: The bottle is now full of water."
A line starting with "!/" is, unsurprisingly, a negated regex check. The line "!/^Taken" verifies that no line of the output begins with the word Taken. (The word occurs within several lines, but not at the beginning of any.)
To run all tests, paste the test script into a file, and then type:
python regtest.py TESTFILE
When you do this, you will see the output:
* south-from-start * in-well-house * test-that-fails <LiteralCheck "There is a bucket of cheese here...">: not found <RegExpCheck "[xqz]">: not found <LiteralCheck !"Inside Building">: inverse test should fail FAILED: 3 errors
The first two tests show no problems. The third fails three checks. When you see failures, you'll probably want to re-run a single test:
python regtest.py TESTFILE -v test-that-fails
This runs only the specified test. The -v (or --verbose) argument displays the complete transcript of the test run, with the failures marked, so you can see exactly what went wrong.
You can run several tests by naming them all, or by using a glob-style wildcard. (You'll probably have to quote the wildcard to keep your shell from mangling it.)
python regtest.py TESTFILE south-from-start in-well-house python regtest.py TESTFILE 'test-*'
These options are available:
**game: line in the test script.)
**interpreter: line in the test script.)
**precommand: lines in the test script.)
RegTest handles IF output in a very simplistic way. It will only work with a game that abides by these rules:
The prompt must always be ">" at the beginning of a line.
In particular, Inform's "if the player consents" (yes/no) questions will confuse RegTest -- it won't recognize them as input requests. The same goes for menu-based input.
">" at the beginning of a line must always be a prompt.
If your game prints ">" at the beginning of a line, even if text follows, RegTest will think it is an input request and fire the next command.
(Were I ambitious, I would enhance RegTest to work with RemGlk-enabled interpreters. Then it could recognize all sorts of input requests correctly, and also test the contents of the status window. I have not gotten off my ass to do this work.)
Last updated March 10, 2013.
Other IF scripts: PlotEx