Stonemind WEB A Bare Bones Approach to Displaying Remote iCalendar Data in Zope

A Bare Bones Approach to Displaying Remote iCalendar Data in Zope

December 16, 2011December 16, 2011| Adam SmithAdam Smith| 0 Comment | 4:58 pm
Categories:

This is an approach I took to download iCalendar data from a remote site, then process and display it within Zope with minimum dependencies. I have tried to document not only the specifics of the approach, but what motivated me to do it and what limitations exist.

This was my first exposure to the iCalendar format. Many thanks to those on the icalendar-dev mailing list who patiently helped me with my newbie questions, Max M and Martijn Faassen in particular. Also, the Radicalendar webmaster (whose name I never got) was very helpful in this process, at one point making a very quick change to the way line endings were generated in his scripts. The help I received from these gurus made me look good to my client, and this documentation is a token of my appreciation.

Why I took this approach and why maybe you shouldn’t.

I was creating a custom Web application in Zope for a client who was already using the open source and publicly hosted calendar called Radicalendar (www.radicalendar.org). They wanted to continue using Radicalendar, but integrate the display of the calendar data seemlessly into their Web application. I started looking for what I thought would be the easiest, bare bones way of doing this.

First, DTML Calendar came to mind. Yes, its in DTML which I don’t know well, but it was available, and it looked like it could be configured easily despite not knowing DTML well. Basically, DTML Calendar allows you to display just about any date oriented information in a flexible calendar format with minimal work.

Radicalendar offers the ability to download their data in iCalendar format, and sure enough, I found a nice iCalendar library for Python. All that was needed was to glue these two pieces together. My approach ends up looking more complicated than it really is; there are a number of little pieces that all work together.

If you find yourself in a similar situation, the following may help you. If you are using one of the many good frameworks built on top of Zope such as CMF, Plone, Silva, etc. you may first want to search their documenation or Google. There may be iCalendar support either built into your framework or available from a third party that will allow you to more easily accomplish this task in your architecture. Such approaches probably use the same iCalendar libraries anyway.

The primary limitation of this approach is that it does not really handle recurring events at all. This seems very tricky to accomplish to me, but perhaps as the Python iCalendar library continues to develop an easy way of handling this will emerge.

My approach in brief:

The following files are discussed beginning with those closest to the backend data and ending with those in the interface:

  • calendar_data, a File object in the ZODB that will store the downloaded calendar file.
  • getWebDoc.py, an External Method to download remote files.
  • download_calendar, a Python Script that calls getWebDoc.py to populate calendar_data.
  • getIcalEvents.py, an External Method to parse the iCalendar data.
  • getAllEvents, a Python Script that passes the calendar_data data to getIcalEvents.py to get events as an list.
  • display_calendar, a DTML Method that calls getAllEvents and uses the DTML Calendar tag to display the events it receives.

My approach in detail:

I have initially implemented this approach using the following software/versions:

  • Python 2.3.4
  • Zope 2.7.4
  • iCalendar Python library 0.10
  • DTML-Calendar 1.0.17

This approach uses two existing components: a Python iCalendar library and a DTML based Zope Calendar product. You should install these first. You can get the iCalendar libraries and good documentation at:

http://codespeak.net/icalendar/

I uncompressed and moved the resulting package into my Python installation’s library directory (lib/python2.4/site-packages/icalendar), the same place you install things like PIL and MySQLdb.

Next, download DTML-Calendar from:

http://www.zope.org/DevHome/Members/teyc/CalendarTag

This site also has good documentation on its use. This ia a Zope product and so should be uncompressed and moved into your Zope instance’s “Products” directory. You will need to restart Zope to load it.

Now we can start to glue these two components together. Note that for simplicity, I assume that these files are added to the root of your ZODB or somewhere they can be directly “acquired”. If you put these into a subfolder, you may have to change how objects are referenced.

Begin by logging into the Zope Management Interface (ZMI) and adding a “File” object, called “calendar_data”. The purpose of this file is to hold our iCalendar data that is downloaded from the remote site. We do this so that each time we need this data to display our calendar, we do not have to rely on a fast network connection to the remote site, but can use this local version, and periodically update it from the remote data in a manner separate from displaying the data.

Now add an External Method that can be used to download the remote iCalendar data:

  1. Copy the following function to a file named “getWebDoc.py” in your Zope instance’s “Extensions” directory.
  2. In the ZMI, add an “External Method” in the root folder or somewhere it can be acquired.
  3. Populate the external method object in the following manner:
    • id: getWebDoc
    • title: getWebDoc
    • module name: getWebDoc
    • function name: getWebDoc

def getWebDoc(self, url=”): “”” get the indicated document “”” if url: from urllib import urlopen return urlopen(url).read() return ”

I purposely kept this very generic; it needs to be in an External Method because of its use of the urllib library which for security reasons is not accessible to normal Zope Python Scripts. To use it, add a Python Script like the following, which I saved in the ZODB as “download_calendar”:

url = “http://www.radicalendar.org/mods/radicalendar/icalendar/TCActivist.ics”doc = context.getWebDoc(url=url)# just to be safe, line endings are a common issueif doc.count(‘\r\n’) == 0: doc.replace(‘\n’, ‘\r\n’)cal = context[‘calendar_data’]cal.manage_edit(cal.title, cal.content_type, filedata=doc)if hasattr(context.REQUEST, ‘HTTP_REFERER’): return context.REQUEST.RESPONSE.redirect(context.REQUEST[‘HTTP_REFERER’])return

With this Python script, I can allow Web site managers to manually update the calendar data by providing a link to this script on the Web site. For example, by adding this snippet to a Zope Page Template:

<a href=”” tal:attributes=”href here/download_calendar/absolute_url” tal:condition=”python: user.has_role(‘Manager’)”> manually update calendar data from Radicalendar </a>

And I can use a cron job and wget to automatically update the calendar data, say, every day at 4:00 AM:

0 4 * * * /usr/local/bin/wget -q –http-user <username> –http-passwd <password> http://<base_path>/download_calendar

(Type: crontab -e on the command line, and substitute your own information in the angle () brackets.)

Now that we have a way to retrieve and store the iCalendar data, we need to process it and display it. Begin by adding an External Method called “getIcalEvents” that uses the iCalendar library to parse the data we have downloaded into an list of dictionaries containing the attributes we are interested in:

  1. Copy the following function to a file named “getIcalEvents.py” in your Zope instance’s “Extensions” directory.
  2. In the ZMI, add an “External Method” in the root folder or somewhere it can be acquired.
  3. Populate the external method object in the following manner:
    • id: getIcalEvents
    • title: getIcalEvents
    • module name: getIcalEvents
    • function name: getIcalEvents

def getIcalEvents(self, ical_string=”): “”” given an icalendar formatted string, get all events as a list of dictionaries “”” import time allEvents = [] try: if ical_string: from icalendar import Calendar thiscalendar = Calendar.from_string(ical_string) if thiscalendar: for event in thiscalendar.walk(‘vevent’): tempEvent = {} org = event.get(‘organizer’, “) if org: tempEvent[’email’] = org tempEvent[‘name’] = org.params.get(‘cn’, “).replace(‘\,’, ‘,’) tempEvent[‘dtstart’] = event.get(‘dtstart’, “) if tempEvent.get(‘dtstart’, “): start = time.strptime(tempEvent.get(‘dtstart’, “), ‘%Y%m%dT%H%M%S’) tempEvent[‘dtstart_date’] = time.strftime(‘%Y%m%d’, start) tempEvent[‘dtstart_time’] = time.strftime(‘%I:%M %p’, start) tempEvent[‘sort_on’] = time.strftime(‘%H%M%S’, start) tempEvent[‘dtend’] = event.get(‘dtend’, “) if tempEvent.get(‘dtend’, “): end = time.strptime(tempEvent.get(‘dtend’, “), ‘%Y%m%dT%H%M%S’) tempEvent[‘dtend_date’] = time.strftime(‘%Y%m%d’, end) tempEvent[‘dtend_time’] = time.strftime(‘%I:%M %p’, end) tempEvent[‘summary’] = event.get(‘summary’, “).replace(‘\,’, ‘,’) tempEvent[‘description’] = event.get(‘description’, “).replace(‘\,’, ‘,’) tempEvent[‘location’] = event.get(‘location’, “).replace(‘\,’, ‘,’) tempEvent[‘categories’] = event.get(‘categories’, “).replace(‘\,’, ‘,’) tempEvent[‘url’] = event.get(‘url’, “) allEvents.append(tempEvent) else: return None except Exception, e: return None return allEvents

Each item in the list represents an event and is a dictionary of the event attributes we are interested in. We can now use this data more generically, iterating over it in a manner similar to database query results. This function has to be in an External Method because of its use of the iCalendar library which for security reasons is not directy accessible to normal Zope Python Scripts. To use this function, we create a Python Script called “getAllEvents” to feed it our calendar data:

ical_string = context.calendar_data.data# just to be safe, line endings are a common issueif ical_string.count(‘\r\n’) == 0: ical_string = ical_string.replace(‘\n’, ‘\r\n’)allEvents = context.getIcalEvents(ical_string=ical_string)if allEvents: allEvents.sort(lambda x, y: cmp(x.get(‘sort_on’, ”), x.get(‘sort_on’, ”)))return allEvents

The final step is to create a DTML Method named “display_calendar” that calls this script to retrieve our calendar data and apply it to the DTML Calendar tag:

<div id=”calendar”> <dtml-let allEvents=”getAllEvents()”> <dtml-calendar controls=”yes” modes=”week” lang=”en” theme=”yb”> <dtml-call “setCalendar(‘cellattrs’, ((‘class’, ‘day’), (‘width’, ‘14%’)))”> <dtml-call “setCalendar(‘align’, ‘left’)”> <dtml-call “setCalendar(‘valign’, ‘top’)”> <dtml-call “setCalendar(‘leftbgcolor’, ”)”> <dtml-call “setCalendar(‘daynames’, ”)”> <dtml-let thisDay=”date.strftime(‘%Y%m%d’)”> <dtml-let current_day=”DateTime().strftime(‘%Y%m%d’)”> <dtml-if expr=”thisDay == current_day”> <div class=”current_dayNumber”> <dtml-var date fmt=”%A, %B %d”> </div> </dtml-if> <dtml-if expr=”thisDay != current_day”> <div class=”dayNumber”> <dtml-var date fmt=”%A, %B %d”> </div> </dtml-if> </dtml-let> <dtml-in prefix=”loop” expr=”allEvents”> <dtml-let start=”loop_item.get(‘dtstart_date’, ”)”> <dtml-let end=”loop_item.get(‘dtend_date’, ”)”> <dtml-if expr=”start==thisDay or end==thisDay or (thisDay>start and thisDay<end)”> <dtml-let event=”loop_item”> <div class=”anEvent”> <a href=”<dtml-var expr=”event.get(‘url’, ”)”>” class=”event_title” title=”<dtml-var expr=”event.get(‘description’, ”).replace(‘\\n’,’\n’)”>” alt=”<dtml-var expr=”event.get(‘description’, ”).replace(‘\\n’,’\n’)”>”> <dtml-var expr=”event.get(‘summary’, ”).replace(‘\\n’,'<br>’)”> </a> <dtml-if expr=”event.get(‘location’, ”)”> at <dtml-var expr=”event.get(‘location’, ”).replace(‘\\n’,'<br>’)”> </dtml-if> <br> <dtml-var expr=”event.get(‘dtstart_time’, ”)”> <dtml-if expr=”event.get(‘dtend_time’, ”)>event.get(‘dtstart_time’, ”)”> – <dtml-var expr=”event.get(‘dtend_time’, ”)”> </dtml-if> <dtml-if expr=”thisDay>start”> <br> <span class=”event_cont”> (Continued from <dtml-var expr=”event.get(‘dtstart’, ”).strftime(‘%m/%d/%y’)”>) </span> </dtml-if> </div> </dtml-let> </dtml-if> </dtml-let> </dtml-let> </dtml-in> </dtml-let> </dtml-calendar> </dtml-let></div>

The final step is to include this DTML method in your Zope Page Template by adding a snippet like this:

<tal:block replace=”structure here/display_calendar” />

Related Post