Post on 10-Dec-2014
description
Talk CalDAV To Me: Integrating Bedework into OAE
Chris Tweney <chris@media.berkeley.edu>Senior Software DeveloperEducational Technology ServicesUC BerkeleyJune 14, 2011
myBerkeley
Instance of Sakai OAE with customizations
Esp. those that allow advisors to communicate to students
myBerkeley widgets
myBerkeley widgets
myBerkeley Notifications
Notification Dependencies
Calendaring FundamentalsiCalendar specificationiCal4jCalDAV protocolWebDAV protocol
iCalendarSimple text-based format for
calendar data exchange
iCalendar's fundamental objects are:◦Events (VEVENT)◦To-dos (VTODO)◦Journal entries (VJOURNAL)◦Free-busy info (VFREEBUSY)
iCalendar
BEGIN:VCALENDAR
PRODID:-//Ben Fortuna//iCal4j 1.0//EN
VERSION:2.0
CALSCALE:GREGORIAN
BEGIN:VTODO
DTSTAMP:20110603T222847Z
DTSTART:20110505T151506
DUE:20110505T151506
SUMMARY:Pay your bill
UID:f84347ab-575b-4274-9436-a5ac906381f9
DESCRIPTION:Pay your bill by the deadline of May 5.
END:VTODO
END:VCALENDAR
Usually you encounter iCalendar data in the form of files with a “.ics” extension. Here’s a typical (abbreviated) example:
iCalendar is defined in RFC 5545http://tools.ietf.org/html/rfc5545
iCalendar Parts
ical4jOpen-source Java library to
wrangle messy iCalendar data
iCalendar records with time zone data get very complicated; ical4j makes it pretty simple to work with them
ical4j home page:http://wiki.modularity.net.au/ical4j/index.php?
title=Main_Page
ical4j Java Exampleprotected Calendar buildVTodo(String summary) {
CalendarBuilder builder = new CalendarBuilder();
Calendar calendar = new Calendar();
calendar.getProperties().add(new ProdId("-//Ben Fortuna//iCal4j 1.0//EN"));
calendar.getProperties().add(Version.VERSION_2_0);
calendar.getProperties().add(CalScale.GREGORIAN);
TimeZoneRegistry registry = builder.getRegistry();
VTimeZone tz = registry.getTimeZone("America/Los_Angeles").getVTimeZone();
calendar.getComponents().add(tz);
DateTime due = new DateTime(DateUtils.addDays(new Date(), new Random().nextInt(28)));
VToDo vtodo = new VToDo(due, due, summary);
vtodo.getProperties().add(new Uid(UUID.randomUUID().toString()));
vtodo.getProperties().add(CalDavConnector.MYBERKELEY_REQUIRED);
vtodo.getProperties().add(new Description("this is the description, it is long enough to
wrap at the ical specified standard 75th column"));
vtodo.getProperties().add(Status.VTODO_NEEDS_ACTION);
calendar.getComponents().add(vtodo);
return calendar;
}
ical4j pitfall: Line foldingPer the iCalendar RFC, long lines
of text get newlines inserted at the 75th column
Sadly, ical4j does not handle this by default
Need to specify “ical4j.unfolding.relaxed=true” in an ical4j.properties file
CalDAVDialect of WebDAV that provides
calendar functionality
Much syntax and structure is inherited from its underlying specifications: WebDAV and iCalendar
CalDAV is defined in RFC 4791:http://tools.ietf.org/html/rfc4791
Sakai 2 CalDAV
GA Tech had a CalDAV project in Sakai 2 that was never released due to gaps in Zimbra's API
Zach Thomas left excellent docs that hugely helped our efforts
Sakai 2 CalDAV Doc Page:https://confluence.sakaiproject.org/display/CALDAV/Developer's+Guide
CalDAV, WebDAV, HTTP
Learn by snooping
Speaking CalDAV with curl
curl –u user:pass -X PROPFIND http://localhost:8080/ucaldav/user/300846/calendar/
Doing a PROPFIND is a shortcut way to get a user's whole calendar:
To curl, "-X PROPFIND" means do an HTTP PROPFIND method.
HTTP has several funky methods you've never heard of unless you've worked with WebDAV.
Speaking CalDAV with curl
curl –u user:pass -X PROPFIND http://localhost:8080/ucaldav/user/300846/calendar/
HTTP/1.1 207 Multi-Status<?xml version="1.0" encoding="UTF-8" ?>
<DAV:multistatus xmlns:DAV="DAV:" xmlns="urn:ietf:params:xml:ns:caldav" xmlns:ical="http://www.w3.org/2002/12/cal/ical#"> <DAV:response> <DAV:href>/ucaldav/user/300846/calendar/00137ac8-1638-4193-ac3e-c172dfe3fd35.ics</DAV:href> <DAV:propstat> <DAV:prop> <DAV:getcontenttype>text/calendar; charset=UTF-8</DAV:getcontenttype> <DAV:getcontentlength>339</DAV:getcontentlength> <DAV:getlastmodified>Tue, 31 May 2011 21:23:32 +0000</DAV:getlastmodified> <DAV:displayname>00137ac8-1638-4193-ac3e-c172dfe3fd35.ics</DAV:displayname> <DAV:getetag>"20110531T212332Z-0"</DAV:getetag> <DAV:current-user-principal> <DAV:href>/ucaldav/principals/users/admin/</DAV:href> </DAV:current-user-principal> <DAV:resourcetype/> <DAV:getcontentlanguage>en</DAV:getcontentlanguage> <DAV:creationdate>20110531T212332Z</DAV:creationdate> </DAV:prop> <DAV:status>HTTP/1.1 200 ok</DAV:status> </DAV:propstat> </DAV:response> … more entries snipped …</DAV:multistatus>
CalDAV and WebDAV challenges
Not your everyday HTTP request
Weird methods (PROPFIND, OPTIONS, PUT, etc)
Multi-status responses (HTTP 207)
Fortunately…
Jackrabbit, on which OAE is based, ships with a reasonable WebDAV Java client
We can use this to make our basic CalDAV connections
CalDavConnector.getCalendarUris()
getCalendarUris() is our simplest method
Just return URI wrapper objects for a user's whole calendar
Complexity's a problem
CalDAV and WebDAV are◦Complicated◦Not noob-friendly◦RFCs are almost the only docs
available
Impatience is a virtue
Need lots of trial and errorRedeploying a server takes 60-
90sRunning a test takes 2s
◦2s is too little time to get distracted
Integration-test-driven development
JUnit to the rescue!
"Unit" tests that actually talk to a running Bedework server are the solution
Write the test first…Before doing anything else I write
a test:
public class CalDavConnectorTest() {
@Test() public void getCalendars() throws CalDavException { List<CalendarURI> uris = this.calDavConnector.getCalendarUris(); }
}
…Then make it passNow we'll make it pass in the
hackiest way imaginable
public class CalDavConnector() {
public void getCalendars() throws CalDavException { return new ArrayList<CalendarURI>(); }
}
…Iterate until done
PropFindMethod propFind = executeMethod(new PropFindMethod(this.userHome.toString())); MultiStatusResponse[] responses = propFind.getResponseBodyAsMultiStatus().getResponses(); for (MultiStatusResponse response : responses) { if (response.getHref().endsWith(".ics")) { Status[] status = response.getStatus(); if (status.length == 1 && status[0].getStatusCode() == HttpServletResponse.SC_OK) { DavPropertySet propSet = response.getProperties(HttpServletResponse.SC_OK); DavProperty etag = propSet.get(DavPropertyName.GETETAG); try { CalendarURI calUri = new CalendarURI( new URI(this.serverRoot, response.getHref(), false), etag.getValue().toString()); uris.add(calUri); } catch (ParseException pe) { throw new CalDavException("Invalid etag date", pe); } } } }
Keep adding implementation code until it does what's needed:
Unit testing heresy
These tests talk to and expect a running Bedework server
This makes them heretical according to true unit test dogma
Tests are not supposed to require an external server
Gracefully failing test
This test will succeed even if the Bedework server does not respond (because we catch the IOException):
public class CalDavConnectorTest() {
@Test() public void getCalendars() throws CalDavException { try { List<CalendarURI> uris = this.calDavConnector.getCalendarUris(); } catch (IOException ioe) { LOGGER.error("Trouble contacting server", ioe); } }
}
PROPFIND only gives you URIs…curl –u user:pass -X PROPFIND http://localhost:8080/ucaldav/user/300846/calendar/
HTTP/1.1 207 Multi-Status<?xml version="1.0" encoding="UTF-8" ?>
<DAV:multistatus xmlns:DAV="DAV:" xmlns="urn:ietf:params:xml:ns:caldav" xmlns:ical="http://www.w3.org/2002/12/cal/ical#"> <DAV:response> <DAV:href>/ucaldav/user/300846/calendar/00137ac8-1638-4193-ac3e-c172dfe3fd35.ics</DAV:href> <DAV:propstat> <DAV:prop> <DAV:getcontenttype>text/calendar; charset=UTF-8</DAV:getcontenttype> <DAV:getcontentlength>339</DAV:getcontentlength> <DAV:getlastmodified>Tue, 31 May 2011 21:23:32 +0000</DAV:getlastmodified> <DAV:displayname>00137ac8-1638-4193-ac3e-c172dfe3fd35.ics</DAV:displayname> <DAV:getetag>"20110531T212332Z-0"</DAV:getetag> <DAV:current-user-principal> <DAV:href>/ucaldav/principals/users/admin/</DAV:href> </DAV:current-user-principal> <DAV:resourcetype/> <DAV:getcontentlanguage>en</DAV:getcontentlanguage> <DAV:creationdate>20110531T212332Z</DAV:creationdate> </DAV:prop> <DAV:status>HTTP/1.1 200 ok</DAV:status> </DAV:propstat> </DAV:response> … more entries snipped …</DAV:multistatus>
…which you then must get with a REPORT method
curl –u user:pass -X REPORT http://localhost:8080/ucaldav/user/mtwain/calendar/ -d "<?xml version="1.0" encoding="UTF-8"?>
<C:calendar-multiget xmlns:C="urn:ietf:params:xml:ns:caldav" xmlns:D="DAV:">
<D:prop>
<D:getetag/>
<C:calendar-data/>
</D:prop>
<D:href>http://localhost:8080/ucaldav/user/mtwain/calendar/d12ad881-5769-4d17-85ac-fa0a0196ec04.ics
</D:href>
</C:calendar-multiget>"
(Note: Double quotes not escaped for clarity)
REPORT method's response<DAV:response>
<DAV:href>/ucaldav/user/mtwain/calendar/02bfa2cd-6c04-40f4-93da-896d54fc2987.ics</DAV:href>
<DAV:propstat>
<DAV:prop>
<DAV:getetag>"20110602T211046Z-0"</DAV:getetag>
<calendar-data><![CDATA[BEGIN:VCALENDAR
PRODID://Bedework.org//BedeWork V3.7//EN
VERSION:2.0
BEGIN:VTODO
CATEGORIES:MyBerkeley-Required
CATEGORIES:MyBerkeley-Archived
CREATED:20110601T171120Z
DESCRIPTION:test
… [ snip ]
]]></calendar-data>
</DAV:prop>
<DAV:status>HTTP/1.1 200 ok</DAV:status>
</DAV:propstat>
</DAV:response>
Hairy REPORT SyntaxCalDAV's REPORT syntax is hairy
XML
Have to extend Jackrabbit's ReportInfo classes
This blog entry by Ricardo Martin Camarero has very useful starter code:◦ http://blogs.nologin.es/rickyepoderi/index.php?/archives/15-
Introducing-CalDAV-Part-II.html
REPORT is foundation for search
REPORT methods with parameters let you search by date
Bedework difficulties
Bedework's search implementation spotty
No support for search on X-props
No support for search on CATEGORIES
Filtering on myBerkeley server
Due to Bedework bugs we're forced to do some filtering on the myBerkeley server side
E.g. Required/Not Required fields
Caching on myBerkeley server
CalendarURI wrapper class◦URI: Locates calendar◦Etag: Uniquely identifies its contents
(sort of like a SHA hash)
Caching not implemented yet, but easy enough since all calendars are keyed by CalendarURI
Implementing CalDavProxyServlet
Also done with test-driven development
Write tests against JSON files that contain servlet's expected inputs
When servlet's finished, UI devs use those JSON files as an API reference
Improvements to Nakamura's CalendarService
Future work: Store and search calendars using Nakamura's CalendarService
Add support for VTODO to CalendarService
Beyond CalendarService
Future: Create a full-blown CalDAV Calendar Provider component for Nakamura
Store calendars externally, transparently refer to them within Nakamura code