February 27, 2003
Engrossed in Work
I have been so engrossed with an issue at work over the past few days that I have barely taken time to check my email or read any weblogs, which is unusual. It's been such a consuming process that once it was resolved I went home and didn't turn a computer on all evening. That hasn't happened in a long time.
Over the past few days I spent almost every waking moment researching the issue and trying to find concrete pointers to the problem, thinking about ways to resolve the problem, implementing a fix, or running load tests against our site to see if the problem had gone away.
The issue was that very sporadically one of our Apache/mod_perl processes would die during XSLT transformation of XML (using libxml2 and libxslt). The frequency of the issue was directly proportional to the length of time the process had been in memory.
One might think all I ever do here is fix problems . . .
So I found the problem:
I know that looks fine and dandy, but what can't be seen here is that the Twig was declared globally in one of our libraries that parses and gets the XML ready for transformation. Declaring the twig globally meant that for as long as an Apache process was in memory, all XML would be sent through the same Twig object. This was difficult to pinpoint because only a portion of our request use the XML tranformation, and additionally, the Twig only became "corrupt" after a certain combination of XML requests had been passed through it.
The good news is that the fix solved issues we were having in more than one place, which makes the effort even more worth it.
Now . . . back to doing some actual development.
Posted by mike at February 27, 2003 11:26 AM