Directory Directory ServicesServices

The University of Queensland LDAP Experience

Intro - What Is A Directory ?A directory is something which through organisation and categorisation assists people in finding things.TV Guides, Telephone Books and Library Catalogues are all directories.Online Directories are dynamic and flexible directories.Online Directories can also allow personalised access including configuration and security.Active Directory is and example of a Network OS tailored online directory.Lotus Notes and Microsoft Exchange are examples of Application Specific Directories.Internet DNS is an example of a purpose specific directory.LDAP and X500 are examples of general purpose directory systems, this presentation will be looking at LDAP Implementation at The University of Queensland.An online directory can organise itself hierarchically, starting at the country then the organisation – e.g. ‘o=The University of Queensland, c=AU’. Alternately a directory can organise itself hierarchically using a domain naming convention much like the DNS structure.

Intro - What Is LDAP ?It all started with X.500 which developed during the 1980s and was formalised in an 1990 and included 8 recommendations (X501, X509, XF511, X518, X519, X520, X521 and X525).

LDAP was initially created as a front end for X500 directory services (at The University of Michigan) as the protocol stack for X500 was too demanding for the client computing of the day. It was released in 1992 as copyrighted but free software and called U-M LDAP.

The client protocol (LDAPv2) become a de-facto standard and was published in RFC 1823.

At then end of 1995 The University of Michigan released an updated U-M LDAP (V3.2) which included the first LDAP server (SLAPD).

LDAPv3 in the late 1990s was extended to include Internationalisation, Referrals, Security (SASL and TLS), Extensibility and Feature and Schema Discovery.

Intro - How Is UQ Using LDAP ?

Configurable and secure mass e-mailer for allowing certain staff access to various groups of staff and students (for example allowing certain staff to e-mail all members of a particular superannuation fund, or lecturers to e-mail all students in one of their subjects).University wide access to central authentication and through that enable development of single sign on.University wide access to central authorisation allowing access to facilities (web sites, samba shares and other logins) based on age (over 18), courses (subject material) and many other criteria.Enabling a university wide mail directory for e-mail clients (including Eudora, Netscape Mail and Outlook).Replacement of the printed phone, e-mail and location directory with an online directory – increasing privacy and improving access.Directory enabling the university portal (my.UQ) for configuration information and online directory facilities.

Intro – Presentation

University Database SystemsUniversity Database Systems

Central Collation DatabaseCentral Collation Database

Data Consolidation and ImportData Consolidation and Import

Export ProcedureExport Procedure

Directory ServersDirectory Servers

Directory ApplicationsDirectory Applications

UQ - University Databases

University Database SystemsUniversity Database Systems

Databases - RequirementsA single data source which defines the existence of each object type (people, units etc)A single unique identifier representing instances of each object type (people, units etc)Data sources responsible for information about instances of each object typeA mechanism for defining single unique login ids for each person, and an optional external authentication systemA way of linking people to membership of each of the organisational structuresCo-operation with the owners of these data sourcesA security policy to protect the information contained in the directory from each of these data sources.

Databases – PRISMPrism has an INGRES backend.Prism is an account management system used by Information Technology Services at UQ for management and billing of central computing and dial-in facilities.Prism was chosen as the database responsible for creating directory entries for people because it currently creates and maintains login ids for all categories of persons (including Staff and Students).Prism also creates the entries for the authentication system Kerberos, as the UQ LDAP services pass authentication through to Kerberos – this was an additional reason to choose PRISM for entry creation for persons.Prism also provides a unique identifier for each person, e-mail aliases and also unique identifiers for the person in the staff and student systems.

Databases – AURIONAURION has an Oracle backend.

AURION is the primary system used by UQ to maintain personal and corporate information about all staff.

AURION has a separate feed into PRISM for automatic creation of Staff accounts and Kerberos entries.

The information the directory obtains from AURION includes employment information, superannuation information, full name details and a link to the organisational unit employing them.

Databases – STUDENT 2000Student 2000 has an Oracle backend.

Student 2000 is a comprehensive database system (Peoplesoft) which stores information about all students, their personal, enrolment details and course information.

As well as all the information about the student, Student 2000 provides the directory with lists of classes and timetables for subjects.

Student 2000 has a complete directory of all subjects taught within the university and this information is imported into the LDAP directory in a structured (hierarchical) layout.

Databases – UQORG/UQROLESUQOrg currently has an MS-SQL 6.5 backend

The UQOrg database and associated database UQRoles provide important structural information for the directory.

The UQOrg database allows creation of a hierarchy of business units and also a hierarchy of geographic facilities (campus, building, floor and room)

The UQRoles database supplies information connecting different roles performed by different persons with the units (from UQOrg) locations and facilities (from UQOrg) and subjects (from Student 2000). Additional ‘free floating’ roles are also defined that are not tied to any specific area.

Databases - OtherPABX database gives information about the telephone and fax numbers for staff as well as room numbers. PABX is currently a text file manual feed.

Convocation (FUTURE) database will provide information about alumni

Research Services (FUTURE) database will provide information about research areas, pointers to research material allowing searching and contact with groups of similar interest.

Union (FUTURE) databases allowing access to both staff and students by their respective unions, allowing contact for resources and information to those groups.

UQ - Database Import

University Database SystemsUniversity Database Systems

Data Consolidation and ImportData Consolidation and Import

Import – RequirementsA structured system that accepts input data from any data source.

Regularly reads the data sources for additions, deletions and modifications.

Input injection procedures checking for error conditions, duplicates and changes.

Assumes a primary key from each data source.

Allows multiple value fields for a given primary key.

Input Injection Procedures (Oracle PL/SQL – for database speed)

Input Processing Scripts (Perl Code – for flexibility)

Input Data Sources for People (Configuration Files)

Import – Structure

PRISMPRISM

PRISMPRISM Student2000Student2000AURIONAURION PABXPABX

PeoplePeople

AddAdd DeleteDelete

UnitsUnits

Units

Subjects

Locations

SubjectsSubjects LocationsLocations

AURIONAURION S2000S2000 PABXPABX etc…etc…

UQ - Collation Database

University Database SystemsUniversity Database Systems

Central Collation DatabaseCentral Collation Database

Data Consolidation and ImportData Consolidation and Import

Collation – RequirementsThe collation database is designed to store basic information about each object type (People, Units etc), in such a way that it can represent directory information.

The collation database is therefore not suitable as a normal data warehouse.

It must be capable of storing large quantities of data and optimised for updating, not for reporting or browsing.

Each object type (People, Units etc) needs to be flagged as ‘altered’ to allow the export procedure to only export modified, deleted or added records.

The database should preferably reside on the same server as the master directory server.

Attribute

Collation – Data Structure

LocationLocation

SubjectSubject

UnitUnit

People

Staff, Student

People

Staff, Student IDID

IDID

IDID

IDID

People ID Attribute Value

Unit ID Attribute Value

Subject ID Attribute Value

Location ID Attribute Value

Category -1 Default Value

Category 0 Force Value

* -1 Default Value

* 0 Force ValueNote: * is a wildcard on category, 0 is a wildcard for a

mandatory value on ID and -1 is a wildcard for a default value

Note: The People table has sub categories such as Student, Staff etc)

UQ - Export Procedure

University Database SystemsUniversity Database Systems

Central Collation DatabaseCentral Collation Database

Data Consolidation and ImportData Consolidation and Import

Export ProcedureExport Procedure

Export – RequirementsProcedures capable of regularly (each hour) looking at the collation database and creating, deleting and modifying entries within the directory server.

For organisational structures be capable of modifying hierarchical structures, given the limitation of the LDAP directory that moving non-leaf entries is not inherently possible.

Logging error conditions and logging entries that failed on updates.

Export Scripts (Perl)

Collation Database

Export – Structure

Master DirectoryMaster Directory

Export PeopleExport People Export UnitExport Unit Export SubjectsExport Subjects Export LocationExport Location

Full Entries

for People

Hierarchical

Information Including

Member People

PeoplePeople UnitUnit SubjectsSubjects LocationLocation

Export – Hierarchy

StudentStudent Vice-ChancellorVice-ChancellorSubjectSubject LocationLocation

o=The University of Queensland, c=AUo=The University of Queensland, c=AU

StaffStaff

GattonGattonIpswichIpswich

St LuciaSt LuciaAssociateAssociateCasualCasualPermanentPermanent

People

People People People

ARTSARTS SCIENCESCIENCE

UnitUnit Secretary and Registrar

Secretary and Registrar

UnitUnit

Information Technology Services

Information Technology Services

Staff Full Read Mail

Pointers to

real entries from

groups below each

unit, subjects and locations

UQ - Directory Servers

University Database SystemsUniversity Database Systems

Central Collation DatabaseCentral Collation Database

Data Consolidation and ImportData Consolidation and Import

Export ProcedureExport Procedure

Directory ServersDirectory Servers

Directory – RequirementsAs the directory was being used for authentication it had to be primarily reliable with little or no downtime.

As the directory contained sensitive information it had to be secure from prying eyes yet open to those who were permitted access.

Due to the volume of requests to the directory it needed to be responsive, but at the same time allow large scale queries to those who needed them.

Due to the nature of some e-mail clients the directory had to be standard both in the protocols used (LDAP) and in the attributes that the clients expected to exist.

Kerberos AuthenticationKerberos Authentication

ldap.uq.edu.auldap.uq.edu.au

Master DirectoryMaster Directory

Secondary LDAP RouterSecondary LDAP RouterPrimary LDAP RouterPrimary LDAP Router

Primary Slave DirectoryPrimary Slave Directory Secondary Slave DirectorySecondary Slave Directory

E10K A

E10K B E10K C

ReplicationReplication ReplicationReplication

80% load80% load20% load20% load80% load80% load 20% load20% load

DNS RotaryDNS Rotary

Directory – Structure

Directory – Security/PrivacySecurity is implemented using the Netscape Directory Server ACI access control system.Privacy is enabled through additional attributes (prefixed with pub_) which are publicly viewable versions of certain personal information – e.g. pub_displayname corresponds to the standard attribute cn (common name).A person can disable public viewing and searching on these attributes through a web interface built into the university web portal.The ACIs give read and search controls for anonymous users to ‘pub_*’, and deny all other anonymous access.ACIs can control access individually to different parts of the directory based on login details, client IP address, target attributes and most importantly membership of groups.Through membership of control groups within each organisational unit, the directory knows which persons are allowed access to staff within that unit (or students within that subject etc).

Directory – Reliability/ResponseRedundancy is firstly achieved via the dual slave directory servers, by attempting connection on one then the other.

Secondly the LDAP routers both load-share to both directories, if one directory fails then the second takes up the full load.

Thirdly the DNS ldap.uq.edu.au is a rotary, so that load is shared between both LDAP router in case of hardware failure.

By ensuring that only short searches are done on the slave servers and the longer searches on the master, the response time for most users is normally 1 or 2 seconds with half a million operations per day.

In future we want to implement a smart DNS that only serves the addresses of the operational LDAP routers – this would allow for automatic recovery on hardware failure.

Directory – CompatibilityThe LDAP routers also allow attribute translation – that is a search for one attribute can be mapped onto another search at the two slaves.Returned attributes get mapped the other way.This allows us to map the public attributes (e.g. pub_displayname) to and from the standard attributes (e.g. cn).Pub_displayname, pub_email, pub_phone and pub_ouname are currently mapped to ‘cn’, ‘mail’, ‘telephonenumber’ and ‘ouname’.This is currently working with Eudora, Netscape Mail and Outlook Express – giving them an expanded address book that is University Wide.

UQ - Applications

University Database SystemsUniversity Database Systems

Central Collation DatabaseCentral Collation Database

Data Consolidation and ImportData Consolidation and Import

Export ProcedureExport Procedure

Directory ServersDirectory Servers

Directory ApplicationsDirectory Applications

Applications – AdminAdmin applications talk via LDAP directly to the master directory server.

Mass e-Mailer application authenticates and authorises application users, then searches and returns e-mail addresses.

Directory and security management applications written in Borland Kylix (Linux and Win32) talk to any of the servers.

Applications – CentralCentral applications talk to slave 1 and fail over to slave 2.

SI-Net Student Information Network, allows students to enrol, select subject etc online – authenticated via LDAP

my.UQ web portal including Netscape Mail and Netscape Calendar – authenticated an authorised via LDAP

Applications – ClientClient applications talk to ldap.uq.edu.au

Eudora E-Mail Mac and Windows

Netscape (not version 6) E-Mail

Microsoft Outlook Express

Microsoft Outlook

UQ - Summary

University Database SystemsUniversity Database Systems

Central Collation DatabaseCentral Collation Database

Data Consolidation and ImportData Consolidation and Import

Export ProcedureExport Procedure

Directory ServersDirectory Servers

Directory ApplicationsDirectory Applications

Summary - Hardware/SoftwareHardware: Three domains on Enterprise 10000 hardware shared with other applications (1 x 8 processor, 2 x 16 processor)

Operating System : Solaris 8

Collation Database : Oracle 8.1.6

Directory Server: Netscape Directory Server 4.12

LDAP Router: iPlanet Directory Access Router 2.1

Kerberos Plugin for Directory Server: Written in C

Custom Server Software: Perl 5 and Oracle PL/SQL

Custom Client Software: PHP/Web and Kylix (Win32 and Linux).

Summary - IssuesAn additional requirement for inserting changes from the directory back into the collation database was implemented via an audit log which Netscape Directory Server implements.Searching, especially un-index attributes is very time consuming so good design of the schema and indexing is needed.Replication using Netscape Replication services creates a redundant system – however re-initialising replication can cause significant problems (if the master is corrupted), this was a several hour process.The directory server requires around 2GBytes of physical memory, fast disks for updates and a large number of available file handles for socket communications. It performs well on a single processor – but at times was using up to 3 of the Enterprise 10000 processors.Scripts were written to transfer all the access logs for the master and slaves to a central location for processing. The access logs are very complex and at this stage only simple analysis has been done.Netscape and iPlanet recommend that Netscape Directory servers that are required for high traffic should be on stand alone servers – rather than on shared servers. We have had some issues with that and are still resolving these problems.

Summary - Read about LDAPUnderstanding and

Deploying Ldap Directory Services

(MacMillan Network Architecture and Development Series)

by

Tim Howes, Mark C. Smith, Gordon S. Good, Timothy A.

Howes

1st edition

(January 1999) New Riders Publishing

ISBN: 1578700701