Post on 11-May-2015
description
APEREO OAEBootcamp, Miami 2014
WHAT WOULD YOU LIKE TO LEARN?
OAE @ MIAMI• Sunday: Development Bootcamp
• Monday @ 10am: State of the Project
• Monday @ 11am: Project talks
• Monday @ 1pm: Architectural Overview
• Tuesday @ 1pm: The FOLD
• Tuesday @ 2pm: Stories from the Frontlines of innovation
• Wednesday @ 11.45am: Apereo & ESUP: Brothers in arms
TOPICS 1. OAE Project
2. Hilary System Architecture
3. Node.js Exercise
4. Back-end Exercise
5. UI Architecture
6. Front-end Exercise
Support for academic collaborationand networking
PROJECT GOALS• User-led
• Multi-tenant platform
• Cloud-ready
• SaaS
• Cross-institutional
• Used at large scale
INTEGRATION
• Integrate OAE with other systems
• Integrate other systems with OAE
MULTI-TENANCY• Support multiple institutions on same installation
• Easily created, maintained and configured
• Each institution has its own URL, branding, skinning and users
• Share infrastructure
• Institutions can share and collaborate with other institutions
TOPICS 1. Project Goals
2. Hilary System Architecture
3. Node.js Exercise
4. Back-end Exercise
5. UI Architecture
6. Front-end Exercise
OAE ARCHITECTUREThe Apereo OAE project is made up of 2 distinct source code platforms:
• “Hilary”
• Server-side RESTful web platform that exposes the OAE services
• Written entirely using server-side JavaScript in Node.js
• “3akai-ux”
• A client-side / browser platform that provides the HTML, JavaScript and CSS that make up the UI of the application
OAE ARCHITECTURE
APPLICATION SERVERS• Written in server-side JavaScript, run in Node.js
• Light-Weight
• Single-Threaded
• Event-Driven
• Non-Blocking
• Uses callbacks/promises and an event queue to stash work to be done after I/O or other heavy processes complete
• App servers can be configured into functional specialization:
• User Request Processor
• Activity Processor
• Search Indexer
• Preview Processor
APACHE CASSANDRA• Authoritative data source
• Provides high-availability and fault-tolerance without trading away performance
• Regarding CAP theorem, Cassandra favours Availability and Partition Tolerance over Consistency, however consistency is tunable on the connection-level (we always use “quorum”)
• Uses a ring topology to shard data across nodes, with configurable replication levels
• Used by: Netflix, eBay, Twitter
ELASTICSEARCH• Lucene-backed search platform
• Built for incremental scaling and high-availability
• Exposes HTTP RESTful APIs for indexing and querying documents
• RESTful query interface uses JSON-based Query DSL
• Powers all of Hilary search functions (Library Search, Members / Memberships Search)
• Used by: GitHub, FourSquare, StackOverflow, WordPress
RABBITMQ• Light-weight message queue platform written in
Erlang
• Used for distributing tasks to specialized application server instances
• Supports active-active queue mirroring for high availability
• Used by: Joyent
REDIS• REmote DIctionary Server
• Fills a variety of functionality in OAE:
• Broadcast messaging (could move to RabbitMQ)
• Locking
• Caching of basic user profiles
• Holds volatile activity aggregation data
• Comes with no managed clustering solution (yet), but has slave replication for active fail-over
• Some clients manage master-slave switching, and distributed reads for you
• Used by: Twitter, Instagram, StackOverflow, Flickr
ETHERPAD• Open Source collaborative editing application written
in Node.js
• Originally developed by Google and Mozilla
• Licensed under Apache License v2
• Powers collaborative document editing in OAE
• Collaborative documents sharded by ID to dedicated Etherpad instances
NGINX
• HTTP and reverse-proxy server
• Used to distribute load to application servers, etherpad servers and stream file downloads
• Useful rate-limiting features based on source IP
• Proxy web socket connections to Hilary and Etherpad
• Used by: Netflix, WordPress.com
TOPICS 1. Project Goals
2. Hilary System Architecture
3. Node.js Exercise
4. Back-end Exercise
5. UI Architecture
6. Front-end Exercise
IDE• OAE Team uses mostly Sublime Text 2 and Aptana Studio 3
• http://www.sublimetext.com/2 • http://www.aptana.com/products/studio3/download
• Useful things in an IDE:
• Syntax highlighting • Automatic “linting” • Variable assistance (don’t expect much from dynamic
scripting language) • File-system navigation
GITHUB• OAE Team uses it for :
• Version control • Issue Tracking • Code Integration Workflow (Pull Requests)
• TravisCI Integration for :
• Running unit tests on all commits and pull requests • Packaging up and uploading all commits to Amazon S3 so they can be easily pulled
down and deployed • Packaging up and uploading all tagged releases to Amazon S3 for production releases
• Coveralls Integration for :
• Reporting on trends in unit tests code coverage (e.g., indicates if test coverage drops from 90.2% to 90.15%)
TOPICS 1. Project Goals
2. Hilary System Architecture
3. Node.js Exercise
4. Back-end Exercise
5. UI Architecture
6. Front-end Exercise
GOAL
NPM• Used to download packages and manage versions
• Packages can be installed from:
• Central NPM repo: npm install oae-tasklist • Git: npm install git+https://github.com/oaeproject/oae-tasklist • File-system directory: npm install ../oae-tasklist
• package.json file describes metadata about a module and its dependencies
• When a package is installed, they are placed either locally in a ./node_modules directory or globally (npm install -g) in a shared location such as /usr/local/share/npm/lib/node_modules
EXTENDING OAE WITH NPM• NPM module represents a logical set of functionality (e.g., a
back-end REST API, or a set of related widgets)
• NPM modules in 3akai-ux are searched for custom widgets
• NPM modules in Hilary (that start with oae-) are searched for init.js and rest.js to integrate to the application container
• New dependencies can be added to package.json file and then fetched / installed running “npm install”
PACKAGE.JSON
• Describes metadata about a package:
• Version • Author • Unique package id
• Required for installing a package as a module
TOPICS 1. Project Goals
2. Hilary System Architecture
3. Node.js Exercise
4. Back-end Exercise
5. UI Architecture
6. Front-end Exercise
UI ARCHITECTURE
UI ARCHITECTURE
UI ARCHITECTURE
• In essence
• HTML • CSS • JavaScript
• Talks to Hilary via REST APIs
WIDGETS• Modular UI components
• HTML Fragment • JavaScript • CSS • I18n bundles • Manifest file
• Loaded into DOM
• Re-usable
WIDGETS
TOOLBOX
JS frameworks
CSS frameworks
3rd party plugins
OAE UI API
OAE CSS components
TOOLBOX
JS frameworks
CSS frameworks
3rd party plugins
OAE UI API
OAE CSS components
JS FRAMEWORKS
• jQuery
• RequireJS
• underscore.js
• DOM manipulation
• Cross-browser abstraction
• Events
• Pretty much everything
• Utility toolbelt
• Manipulate objects, arrays, etc.
• File and module loader
• Load files on the fly
• Necessity to keep things modular
• Optimisation built-in
TOOLBOX
JS frameworks
CSS frameworks
3rd party plugins
OAE UI API
OAE CSS components
CSS FRAMEWORKS
• Twitter Bootstrap
• Font Awesome
TWITTER BOOTSTRAP
TWITTER BOOTSTRAP
• Most popular CSS framework
• Responsive (!)
• Documentation
• Basic components, styles, etc.
• Override where necessary
BOOTSTRAP GRID• Predefined classes make up rows and columns
• 12 available columns to span
• e.g., 3 x `.col-xs-4` columns
• 4 media-queries
Device Class WidthPhones .col-xs- < 768pxtablets .col-sm- ≥ 768px
Desktops .col-md- ≥ 992pxDesktops .col-lg- ≥ 1200px
BOOTSTRAP GRID
FONT AWESOME
FONT AWESOME• Icon font
• No more images
• Style with CSS
• Skinning
• Easy
TOOLBOX
JS frameworks
CSS frameworks
3rd party plugins
OAE UI API
OAE CSS components
3RD PARTY PLUG-INS
• jQuery plug-ins
• Bootstrap plug-ins
3RD PARTY PLUG-INS• Autosuggest
• History.js
• Fileupload
• Validation
• Markdown
• etc.
INCLUDE YOUR OWN
TOOLBOX
JS frameworks
CSS frameworks
3rd party plugins
OAE UI API
OAE CSS components
OAE UI API• Wrapper for REST requests
• Authentication • Content • Comments • Discussions • Following • Groups • Users • Admin
OAE UI API• Utilities
• i18n • l10n • Template rendering • Widget loading • Notifications • XSS escaping • etc.
OAE UI API
TOOLBOX
JS frameworks
CSS frameworks
3rd party plugins
OAE UI API
OAE CSS components
OAE CSS COMPONENTS
• Re-usable HTML fragments
• Consistency
• Design guide
OAE CSS COMPONENTS• Clips
• Visibility icon
• Large options
• Thumbnails
• Clip
• Left hand navigation
• Tiles
• List items
• List header
• Skinnable Panel
WIDGET BEST PRACTICES
• Namespacing
• i18n
• UI Templating
NAMESPACING• Widgets share same container
• Avoid clashes
• Namespace:
• HTML IDs • CSS classes • jQuery selectors
I18N
• UI available in multiple languages
• Standard .properties files
• 2 types of bundles
• Core bundles • Widget bundles
I18N
Translation priority
1. Widget user language file
2. Widget default language file
3. Container user language file
4. Container user language file
I18N
__MSG__TRANSLATION_KEY__
I18N
Complete
Partial
I18N
L10N
• API methods for localizing:
• Number formatting • Date formatting
UI TEMPLATING
• TrimPath
• Avoids lots of DOM manipulation
• Pass in JSON data
• Supports if statements, for loops, etc.
UI TEMPLATING
• Templates are defined in between <!-- -->
• oae.api.util.template().render(...)
UI TEMPLATING• Template
<div id="example_template"><!--
<h4>Welcome {firstName}.</h4> You are ${profile.age} years old--></div>
• Input oae.api.util.template().render($("#example_template"), {
“firstName”: “John”,“profile”: {
“placeofbirth”: “Los Angeles”,“age”: 45
}});
• Result<h4>Welcome John.</h4> You are 45 years old.
UI TEMPLATING• Template
<div id="example_template"><!--
{if score >= 5}<h1>Congratulations, you have succeeded</h1>
{elseif score >= 0}<h1>Sorry, you have failed}
{else}<h1>You have cheated</h1>
{/if}--></div>
• Input oae.api.util.template().render($("#example_template"), {
“score”: 6});
• Result <h1>Congratulations, you have succeeded!</h1>
UI TEMPLATING• Template
<div id="example_template"><!--
{for conference in conferences}<div>${conference.name} (${conference.year})</div>
{forelse}<div>No conferences have been organized</div>
{/for}--></div>
• Input oae.api.util.template().renderTemplate($("#example_template"), {
“conferences”: [ {“name”: “Apereo Miami”, “year”: 2014}, {“name”: “Apereo San Diego”, “year”: 2013}, {“name”: “Sakai Atlanta”, “year”: 2012}]
});
• Result <div>Apereo Miami (2014)</div> <div>Apereo San Diego (2013)</div> <div>Sakai Atlanta (2012)</div>
UI RELEASE PROCESSES• Grunt
• Task-based build system implemented in JavaScript
• Similar in theory of operation to Make, Rake
• Rich ecosystem of plug-ins to do most tasks
• Easy to implement new task when a plugin doesn’t exist yet
• Used for running test suites, production builds, linting tools
UI RELEASE PROCESSES• Production Build
• Optimizes the static assets to reduce throughput, request frequency, and optimize caching across versions
• Require.js Optimization:
• Concatenate JavaScript dependencies (reduces number of web requests significantly) • Minify / Uglify JavaScript files (reduces payload sizes significantly, even when gzip enabled
on web server)
• Hash optimization:
• Hash the contents of static assets and append result to the filename, then cache them indefinitely on the browsers
• When the files change, the hash in the filename changes to force reloading of the updated asset
• If files never change across version, client never reloads file until their cache is cleared
TOPICS
1. Project Goals
2. Hilary System Architecture
3. Node.js Exercise
4. Back-end Exercise
5. UI Architecture
6. Front-end Exercise