This is something that's been on my mind a lot over the last few years, and now that I've got a blog, it's the first real piece of content so I can get it off my chest.
I've done a lot of C/C++ code as a professional software engineer, and quite a bit more as a hobby programmer too, and have often come across libraries that are written to parse (that is, load from disk and interpret) various forms of text files. I've even used some and built some of my own when I've thought that a proper parser (e.g. ANTLR) might be overkill.
But I'm not convinced there really is such a thing as parser overkill, as a recent incident demonstrates.
I was attempting to use the TinyXML library for a personal project, based on SDL (Simple Direct-media Layer). I'd created my internal structure and an effective internal DTD (since TinyXML doesn't use them) to ensure that the read-in XML structure was correct.
I went to run my code on a sample XML file I'd created as a testbench, when "bam"... my program died (fortunately the computer didn't, since it was about 2 weeks old at that point!). Some of the attributes had to have integer values, but I got a strange thing with one attribute which didn't give the value I'd wanted.
Double-checking my file shoed the problem. With my new computer setup I've got a wireless USB keyboard, but it's really sensitive and possibly buggy - it seems every now and again to either not send characters I know I've typed or send additional ones which I know I haven't. And so, sitting in the middle of the integer, was an unobtrusive letter "n". I'm guessing "n" for nowhere since my fingers were nowhere near the "n" key as my keyboard continued its private vendetta.
Removing the "n" resulted in the correct behaviour of my software and I was happy initially. But after about 2 seconds' worth of happiness I felt quite infuriated about the whole process. No parsing library should ever report a successful conversion of an integer value when the integer is only partly converted. It should report a straight-out error if the text string contains characters other than 0-9 and possibly + and - depending on whether you want signed or unsigned.
The key to the problem lies in very basic C/C++ conversion. atoi (which is, IIRC, what TinyXML uses) has the problem that it converts only up to the point where it gets a non-integral character and returns that (see cplusplus.com's description). Therefore, "89n745" will convert successfully, but will return 89. In my case, the entire token has to convert to an integer, since the entire token is intended to be the one integer.
I've encountered this problem frequently before, and one way to solve it is to convert the integer back to a string and do a string comparison between the two... but that only works if you know the numerical format (e.g. how many leading zeroes?). In one case, a few years ago, we weren't sure about using BOOST and we did know the format exactly since we didn't allow leading zeroes, so we used this trick as a back-validation step. Sure, it added extra processing (string compares are expensive operations) - but correct parsing was non-negotiable, so we suffered the performance hit.
My solution this time was to forget TinyXML's conversion functions. Instead, I grab the attribute value as a string always and then use the BOOST lexical_cast system to do the conversion, since it throws a bad_lexical_cast exception if it can't convert the entire argument.
BTW, this is the only real complaint I have about TinyXML.
No comments:
Post a Comment