Robust Grammar Development with Parse::RecDescent and Test::More

By:

A client of my employer asked for us to validate a data feed from them, a feed used to create customer statements and invoices. They provided a detailed spec (about 45 pages). After reading it for about 20 minutes, I realised it was almost like a BNF description, and that P::RD could handle this fairly easily.

However, I had tried P::RD on two previous occasions, with poor results. This was my fault, not P::RD’s. At those times, I knew a lot less and didn’t possess the required skills. This time, however, I must have been feeling pretty cocky, so I decided to have another bash. But this time I also wanted to make sure, early on, that I was on the right track.

So I pondered how I could test that the grammar I was developing was going to correctly parse the feed, and only let through valid documents. Moreover, I wanted to be able to do something as simple as ‘make test’ to prove it’s correctness.

Reading the P::RD POD, I quickly found how to pass grammars to code in test files, and automate the testing of valid and invalid documents. This resulted in a complete implementation of a validation tool, for those 45 pages of spec, in about a day and a half of coding effort.

This talk shows that technique, plus the complete grammar, to show how one might create a solution like this.

Unfortunately, the actual data feed didn’t match the spec, but we found that out really quickly !!! In the end the client opted to make every field optional, (including things like address and total due) because ‘we validate too much’. Sigh.

Stream: Perl

Paper: Robust Grammar Development with Parse::RecDescent and Test::More