Perl, Compiled Regex's and FastCGI

Recently I have been having intermittent problems with various Foswiki plugins when running under FastCGI. Sometimes a plugin would only partially render, or would not render at all. Other times it would be fine. On a test install of Foswiki, it always seemed to work fine. As always with intermittent problems, it can be very frustrating to track down the problem…

It turns out the problem was caused by compiled regular expressions. In Perl, you can use the /o modifier to compile a regex which includes variables to increase the performance when it is next used. Some of the Foswiki plugins were using the modifier to compile regexes which rendered their macros.

This works fine under plain CGI, as on each request the Perl is re-compiled, so we know that the parameters of a macro will not change and the regex can be compiled to increase performance. However, under FastCGI, the Perl code stays compiled over a number of requests. If the parameters for the macro changes, the regex will not be re-compiled and the plugin will not render correctly.

So the fix is to remove the /o modifier. But what about performance? Well, according to the Perl FAQ, since 5.6 Perl knows when a variable for the regex has been changed and will recompile if it needs to. If the variable does not change, it won’t recompile.

So as long as you are not working with a Perl older than 5.6 (which you really shouldn’t be in 2010), you should not need to use the /o modifier. Ever. Perl just does the Right Thing.

I have started removing the /o modifier from some plugins where I have had intermittent problems in the past. I would really like to remove them from Perl code everywhere, if only I had the time…

Note: Although I have talked specifically about Foswiki and FastCGI here, the same would probably apply to any Perl code running running persistently (mod_perl, POE, etc).