Over the past two weeks, I’ve been pair programming with Eric. Together, we’ve been building out a suite of acceptance tests for Ivanhoe to provide a basic check on critical, user-facing functionality as we refactor parts of the codebase. While learning to write basic tests has been relatively straightfoward for me, due to the natural language character of acceptance tests with RSpec and Capybara, we have run into a couple of hang-ups in getting the test suite functional. I want to briefly describe one of these problems and how we resolved it.
Database Transactions and WordPress Problems
After getting a number of tests working, Eric and I decided to straighten out the database transactions occuring before running the suite and between each test. Before the suite runs, we set up a testing database from fixtures, and between each test we want to reset the database. This reset ensures that any given test is unaffected by database events from previous tests. Initially, this was to be handled with two Ruby gems: database_cleaner and sequel. The database reset wasn’t working when we attempted to run it however. Eric realized that, due to the testing running in a different process than the application, we would not be able to use the default transaction strategy. Given that WordPress uses a MySQL database, the deletion strategy was not available, leaving us only with truncation. Truncation, however, without some fine tooling, would have cut off more of the database than necessary, requiring a more extensive rebuild from the fixtures. We decided that it would make more sense to write more restrained database transactions ourselves in the
While this provided an intial solution, it led to lengthy test times, and lengthy test times lead to a distinct lack of excitement for running tests regularly, substantially hindering the effectiveness of testing at all. We set out to refactor the database transactions to reduce testing time, but in doing so created a perhaps more significant problem. Running our tests led to two significant problems in WordPress:
- TinyMCE, the WYSIWYG editor used by default in the WordPress editor, was no longer working, with relevant failing tests indicating that TinyMCE was not defined on pages with the WordPress editor
- an admin user, with full privileges, was able to login, but not able to access the back-end Dashboard at all, being given instead a message that the user did not have the necessary permissions to access the page.
As part of this work, though, I tried to create a post through the WordPress Dashboard to check whether TinyMCE was being loaded along with the editor when called through WordPress core (vs. when called through the front-end forms in Ivanhoe). Doing so revealed the second issue, that even as an admin user I was not able to access the Dashboard due to permissions. Checking the database tables against those of a working installation of WordPress and Ivanhoe, I discovered that the
During our weekly pair programming time, we set out to figure out how and why the testing script was causing this mis-copy of the database table. Through a repeated cycle of starting with a clean database, running the script with different system actions commented out, and checking the database, we eventually discovered that the database fixtures themselves, loaded during the test script, were creating the erroneous key-value pairs in the
wp_usermeta table. After generating a clean set of fixtures, our tests worked, passing as expected. Both errors in WordPress were resolved. To ensure that we don’t run into this problem again, we used a rake task to create backup copies of the clean database tables and then used those in
spec_helper.rb to ensure clean data between tests.
We still don’t know when the error in the fixtures was introduced, and I’ll be spending at least a bit of time trying to figure that out, but, for now the testing suite for Ivanhoe is working. We’ll continue building that out for better coverage and begin refactoring the Ivanhoe codebase. When I mentioned to Wayne and Ammon that we had figured out the error, and it turned out not to be the script at all, neither was surprised. They were, I suspect, not suprised by two things: that it took several days of work to debug a problem in WordPress and that it wasn’t the script at all, just bad fixtures.