October 21 to 23, 2011

UC Berkeley, California

A culmination of the brightest Drupal minds in the technology hub of the world


Distributing Drupal Content Through Custom Feeds

At Common Sense Media we partner with several well-known companies (Best Buy, NBC Universal, Comcast among others) who are licensed to re-package our reviews of various media content available on our site. In fact if you watch TV and have children you are likely to have made use of our reviews as they have been added to channel guides for parents to keep track of what their children watch.
We were tasked with the re-engineering of the daily feeds that provide our partners with this data as the old system was unscalable and hard to extend for the new features that our clients have been requesting. We've designed, built, and deployed a new system that uses a custom Drupal module that is integrated with the Services contrib module in Drupal 6. This custom module was designed to pull data kept mostly in CCK for distribution in XML and/or JSON, remain performant for even very large data sets, and keep current within short time frames while being extensible for future requirements. The module acts as a facade to an engine that uses PHP5 OOP and employs Unit testing for each of the major classes that comprise the system. For the current version we opted to pull data using SQL since using Views for simple data transfer seemed to incur too much overhead to be lightweight but the system maintains modular access to data so that future versions of Views and Solr can be swapped out without effect on the related classes if/when we decide to go in that direction. Real-time data is maintained using the module's nodeapi for concurrence with editorial entries.
We'll explain how we came about this strategy, how we built it, and how we hope to extend it going forward.