SANDcamp, San Diego, CA - February 27, 2015
The Great ConsolidationEntertainment Weekly migration case study
Who are we?
• Four Kitchens Web Chefs
• Jon Peck - drupal.org/u/fluxsauce
• Matt Grill - drupal.org/u/drpal
• Patrick Coffey - drupal.org/u/pcoffey
Project overviewWhat did we do & who did it?
Entertainment Weekly
• Magazine that covers film, TV, music, Broadway theater, books, and pop culture
• 40th largest in the US
• ew.com
• 13.7 million consumers / week
Migration scope
• WordPress
• 10 blogs
• 51,616 terms
• 101,581 posts
• 90,435 images
• Vignette V6
• 117,326 articles
• 201,986 images
• 97,611 terms
• 4,924 galleries
Development Teams
• Four Kitchens - 3 developers to 6
• Time Inc. - 1 developer to 4
• Zoom, HipChat for regular communication
• Video really helps
• As does GitHub selfies :-)
Timeline
• First commit: April 29th, 2014
• Launch: January 29th, 2015
• 17 sprints
Delegation of responsibilities
• Four Kitchens
• Project Management
• Data structures
• Migration
• Implement design, advertising
• Time Inc.
• Product Owner
• Design
• Workflow
• Infrastructure, build process
ProcessStandards and documentation
Definition of Readiness and Completion• Stakeholders
• Context necessary for development
• Developers
• Code reviews, PO acceptance, demo, docs
• Evolving framework
Code and style standards
• Algorithmically enforced
• PHP_CodeSniffer (rules from coder)
• jshint - syntax
• jscs - code style
• EditorConfig settings
Peer code reviews
• Functional - does it…
• Fulfill the intent of the story?
• Use best practices?
• Avoid technical debt?
Development Environments
• Documented project-specific process
• Superset of existing instructions
• Greatly reduced on-boarding time
• Provided plentiful feedback to DCMS
• Beta tested new VMs
Branching standard
• Reduces clutter
• Improves communication, navigation
• Best practices
• sprint-xx (kept for one sprint)
• SEGEWCMSM-YY (deleted after merge)
• Tags added when deleting sprint branch
Front-endTheming, performance and advertising
THEMING
https://www.drupal.org/project/aurora
Aurora, mobile first theming
• Used Aurora as base theme
• HTML5, Sass & Compass
• Mobile theme is sub-theme of Aurora
• Desktop is a sub-theme of Mobile
Gulp task management
• Compiles Sass
• Runs JavaScript checkers
• jshint
• jscs
JAVASCRIPT
JavaScript global scoping
• Created global JS object for all EW functions
• Best practice
• Centralized point of entry
• Easy access
Atomic JavaScript functionality• Break related functions into individual files
• Only load what’s needed
• No monolithic structure
• Improves front-end performance
PREPROCESSING
Balancing performance with business logic• Node template data built in pre-processing
• Does not use views, panels
• Clean & reusable
• Intuitive structure
• Helper and abstractions
BEACONS
Global metadata object
• Useful for integrations requiring metadata
• Avoids repetition
• Facilitates data collection by crawlers
Beacon performance
• Delay loading of beacons (non-blocking)
• Use defer for injecting JS tags
ADVERTISING
Collaborating with DCMS
• TGX improvements
• Single-request mode
• More native GPT functionality
• Ad Manager
• Patches, recommendations
Custom ad rendering to maximise performance• No logic in templates
• Data attributes contain values
• Avoids inline JS
• Improves rendering performance
• Ad renders in footer (after page load)
Editorial UI & Workflow
Packages and Channels
• Automatic dynamic content
• Editors define rules for selection
• Optional manual curation
Custom content types
• Used existing DCMS features to begin
• Rounds of PO, editor feedback
• Usability key to adoption
Collaboration with DCMS
• Provided feedback on tools
• Search interfaces
• Transient messages
• Crop tool
• Auto save
Dynamic Entity References
• Lots of aggregate views in EW
• Editorial control of views, but with overrides
• Custom module that “fills in” empty parts of a view based on context
• View lists 15, editor defines 5, DER populates remaining
Dynamic ER challenges
• Modular, extensible, reusable
• Accurate dynamically curated nodes
• High performance
• Editoral expectations for caching
• State Machine integration
Content migrationsWordPress, Vignette to Drupal
WORDPRESS
Challenges
• 10 different blogs
• Different structures
• Multiple taxonomies
• Invalid markup
• Custom shortcodes, filters
• WordPress Migrate module not enough
Solutions
• Interviewed other teams, shared code
• Pre-processing
• Strip comments
• Combine into one monolithic file
• Separates authors, tags, images
Shortcodes and Filters
• Custom markup
• Created custom short code system
• Most will not be migrated
• Pre-processing solution
• Render as HTML
• Used WordPress, existing code
WordPress Migrate
• Can’t use directly…
• Can extend!
• Helps normalize WXR structure
• Focus on custom logic
Mapping terms on import
• Separates monolithic taxonomy
• Specific vocabularies
• New content types (people, creative works)
• CSV Spreadsheet used for defining rules
• Curated by Editors, PO
VIGNETTE
Exporting from Vignette
• Coordinated with TI team
• Defined XML structure
• Gave feedback on export UI
• Resolve edge cases
Importing
• XML content is sanitized and validated
• Transforms overloaded taxonomies to content
• People, Creative Works
• Articles, Packages, Galleries
MIGRATION PERFORMANCE
Improving migration performance• Validate all XML before ingestion
• Eliminate all PHP errors, warnings
• Avoid redundant migrations
• Separate in pre-import
• Reduce overall number of migrations
• Implement cache_counts
Reducing migration overhead
• Disabled solr, pathauto, metatag, others during migration
• Project requirement was 2.5; you should use 2.7
• Ran migrations in parallel
• Ensure HW resources are sufficient
Migration redirects
• Minimize Drupal performing redirects
• Redirect farm (rules, 1:1 redirects)
• Exported if source, target known
• DNS changes
• Redirects from migrated servers
Performance and caching
Frontend performance
• CSS
• Sass and Compass optimized
• Targeted - only load what’s needed
• JavaScript
• Linted, strict standards
• In the footer (non-blocking)
• Global objects (less is more)
Frontend caching strategy
• Editorial 1 minute publish to live
• Ensure cacheable headers are set
• Shorter TTL on Akamai than Varnish
• Ex: 5 minutes on Akamai, 10 on Varnish
• Send purges on publishing
Backend performance
• Eliminate PHP errors
• Cache, EXPLAIN custom queries
• Minimize unnecessary overhead
• Module count
• Benchmarking
Performance testing
• Load test production with migrated data
• blitz.io
• New Relic for introspection
• Exercised different content types, exceeded TTLs
• Front-end testing
• WebPageTest.org
Questions?
Thank you!
Top Related