Drupal Global Training Day by Drupal Mumbai 6th Sep - Drupal Terminologies
Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011
-
Upload
jbarclay -
Category
Technology
-
view
2.430 -
download
0
description
Transcript of Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011
![Page 1: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/1.jpg)
Feeds Module for Drupal 7
Use Cases, Overview, and Walktroughs
![Page 2: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/2.jpg)
Who Am I?
• johnbarclay on drupal.org• University of Illinois College of
Education web developer• Pixotech.com (Pixo) Consultant• Maintain LDAP module• Like drupal community/dev model• Need to be interrupted
![Page 3: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/3.jpg)
Who are You?
• Who has used feeds in drupal 6• Who has used feed in drupal 7• Who knows they have a use case for
feeds?• Who may have a use case for feeds
(importing or synching data into drupal)
• Be thinking of these for middle part of talk.
![Page 4: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/4.jpg)
Goals
• In love with feeds. Using like crazy the last couple of months.
Want to:• sell the module it• give leg up and warnings for new
users• give away a book
![Page 5: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/5.jpg)
Breakdown
• Terms, Module Features, General Use Cases. 10 minutes. Keep me in check.
• A simple node import walk through. 10 minutes.
• Examples of use cases we’ve implemented or need to implement. 20 minutes. Start thinking.
• Advanced Feeds. Gotchas and Tools.
![Page 6: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/6.jpg)
What is Feeds Module?
Feeds is a pluggable system for importing or aggregating content into Drupal. •Investing time into learning it can make user import, node import, etc. obsolete.•Very extensible. Class Based with stack of interchangeable plugins + hooks.•Supports exportables/features.
![Page 7: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/7.jpg)
![Page 8: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/8.jpg)
Use Cases for Site Builders
• Data Sources: RSS Feeds, XML Feeds, CSV Files, SOAP, WDSL, LDAP, Databases...
• Data Targets: Nodes, Users, Taxonomy, ...
![Page 9: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/9.jpg)
Use Cases For Developers
• For data ingestion, only need to write plugins for unique data sources or targets. Can leverage other plugins and core feeds functionality (batch, unique identifiers, user interface).
• For migrations and many use cases don’t need any code at all
• Supports exportables/features
![Page 10: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/10.jpg)
Architecture
FetcherFetcher ParserParser ProcessorProcessor
Importermappermapper
![Page 11: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/11.jpg)
Know the Lingo
• Feeds Glossary very helpful http://drupal.org/node/622710
• Key to understanding documentation
![Page 12: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/12.jpg)
Fetcher
• Fetcher gets data.• Specific to transport/storage type• crawling, directory, file upload, http,
ldap query, sql,
![Page 13: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/13.jpg)
Parser
• Parser Normalizes Data. • Specific to data structure• Known Data Structures: CSV, RSS,
LDAP Entry, OPML, SQL Recordset Customizable Data Structures: Querypath, Xpath, Xpath HTML
• http://drupal.org/node/856644
![Page 14: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/14.jpg)
Processor
• Processor “does stuff”. Generally stores feed items in a data structure. Could send out emails or do other obscure use cases.
• Node, Taxonomy Term, User built in.• Data, Commerce Products, Self Node
Processor, etc.
![Page 15: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/15.jpg)
(field) Mappers
• http://drupal.org/node/856780• required to know where data from
feed goes into a field.• basic core field types in Feeds
module• contrib mappers in contrib modules
![Page 16: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/16.jpg)
Other Terms
• plugin – fetcher, processor, parser• feed – body of data (csv file)• feed item – individual item (csv line)• feed node – confusing. instance of
importer stored in node.• mapping – source to target field
mapping• guid and unique target
![Page 17: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/17.jpg)
Import CSV Example
![Page 18: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/18.jpg)
Examples for Learning
• In the simpletest folder in feeds are example feeds sources and test configurations
![Page 19: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/19.jpg)
Our Use Cases
• What are you using feeds for?• What do you need to either import on
a one time basis or synch continually.
![Page 20: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/20.jpg)
My Common Use Cases
• MSSQL Data Synch to Nodes & Taxonomy
• XML Feed to Nodes• LDAP Query to User Records• CSV to Nodes to Taxonomy
![Page 21: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/21.jpg)
Where are Plugins
• Processors and Mappers: for core structures such as node, user, and taxonomy in feeds module. Outside core, should be in related contrib module or own module.
• http://drupal.org/node/856644
![Page 22: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/22.jpg)
7.x version not ready
• Lots of bugs and RTBC patches• If it does the job great. Read issues.• When importing large amounts of
data, everything has to work or need an alternative approach.
• my notes: http://drupal.org/node/1301604
• if you use a patch, RTBC!
![Page 23: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/23.jpg)
Feeds Tamper
• Handy for edge cases• trim, regex, explode, etc.
![Page 24: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/24.jpg)
Write Plugins
• search the sandbox• use other plugins as templates• extend other plugins rather than
rewriting• use file logging for debugging
because of batch nature of execution. dpm() and tracing problematic.
![Page 25: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/25.jpg)
Add to documentation
• http://drupal.org/node/1307732• feeds are confusing
![Page 26: Drupal 7 Feeds Intro Drupal Camp Indianapolis 2011](https://reader030.fdocuments.us/reader030/viewer/2022020720/54b6f0284a795902588b45bf/html5/thumbnails/26.jpg)
7.x-2.x gotchas
• http://drupal.org/node/1301604