System Administration Part 2Lecture Notes, Theory Questions and Lab Exercises

Erik Hjelmas

September 20, 2011

Contents

1 Infrastructure Basics: DHCP and DNS 1

1.1 DHCP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Namespaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2.1 Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2.2 RFC 1178 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.3 DNS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.3.1 History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.3.2 Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.3.3 Naming Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.3.4 DNS Query . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.3.5 Dynamic DNS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.3.6 DJBDNS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.3.7 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.3.8 DNS admin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

1.4 Infrastructure Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

1.5 Our Infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

1.6 Theory questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

1.7 Lab exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2 Configuration Management: Many Hosts 17

2.1 Infrastructure Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.1.1 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.1.2 Uniformity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

i

CONTENTS

2.2 Infrastructure Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.2.1 Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.2.2 Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.3 Configuration Management Systems . . . . . . . . . . . . . . . . . . . . . 19

2.3.1 Implementations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.3.2 Host identities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.3.3 Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.3.4 Push vs. Pull . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.4 Cfengine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.5 Theory questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

2.6 Lab exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.6.1 Cfengine install . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.6.2 Getting started with pull-based configuration management . . . 27

3 User Accounts: Identites and Authentication 29

3.1 Local User Accounts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.1.1 Unix/Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.1.2 Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3.1.3 Local vs domain accounts . . . . . . . . . . . . . . . . . . . . . . . 30

3.2 User Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3.3 LDAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.3.1 Directory Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.3.2 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.3.3 DN/RDN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.3.4 Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.3.5 Schema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.3.6 LDIF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.3.7 Case: FEIDE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.4 Kerberos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.5 Active Directory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.6 User Account Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.6.1 Choosing usernames . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.7 Theory questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

3.8 Lab exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

ii

Chapter 1

Infrastructure Basics: DHCP andDNS

1.1 DHCP

see slides and lab from “Automatic Installations”...

1.2 Namespaces

• Flat namespaces: UIDs, SIDs, usernames, hostnames, ...

• Hierarchical namespaces: DNS names, email addresses, file and directory paths,...

• Namespaces need policies, procedures and centralized management

1.2.1 Policy

• Defining names

– Formulaic (pc-*)

– Thematic (Star Trek names)

– Functional (www1)

– Descriptive (www1-oslo)

• Access control

• Scope: diameter and thickness

1

CHAPTER 1. INFRASTRUCTURE BASICS: DHCP AND DNS

• Consistancy

• Reuse

This is from Limoncelli et al., section 8.1.1:

Maybe the names need to be protected, e.g. to protect email addresses from spamming.Scope means how many parts of the company (diameter, think of it as geography fora big multinational company), and how many services (email, userID, VPN username,etc) use these names. If a username is used in several user databases, will they be kept insync? e.g. same password? this is called level of consistency, we have high consistencyif the same username means the same attributes in all user databases (e.g. identity man-agement solutions). Reuse is important since we very much like the hostname pompeland want to use that for an important server, but pompel has actually been used as ahostname for a different host a few years ago, might this cause problems? maybe, itdepends on how long ago it was used and what it was used for.

• Procedures: additions, changes and deletions

• Centralized management of namespaces provides consistancy

1.2.2 RFC 1178

What not to do when choosing hostnames:

• Don’t overload other terms already in common use.

• Don’t choose a name after a project unique to that machine.

• Don’t use your own name.

• Don’t use long names.

• Avoid alternate spellings.

• Avoid domain names.

• Avoid domain-like names.

• Don’t use antagonistic or otherwise embarrassing names.

• Don’t use digits at the beginning of the name.

• Don’t use non-alphanumeric characters in a name.

• Don’t expect case to be preserved.

2

1.3. DNS

What to do:

• Use words/names that are rarely used.

• Use theme names.

• Use real words.

• Don’t worry about reusing someone else’s hostname.

• There is always room for an exception.

1.3 DNS

• DNS is a distributed database with (key,value) pairs

• DNS provides the following primary services

– IPaddr-to-name mapping

– name-to-IPaddr mapping

– aliases

– mail routing

• Might also be used for

– Other lookups (Service records, certificates, etc)

– Load distribution

– RBL/SPF (spam prevention)

– Finding authorative information about your network!

1.3.1 History

1971 RFC226, HOSTS.TXT (Peggy Karp)

1981 RFC799, DNS concepts (David Mills)

1982 RFC819, DNS structure (Zaw-Sing Su & Jon Postel)

1983 RFC882/883, Hostname lookup, authority and delegation (Paul Mockapetris)

1984 RFC920, Outline of work to be done andTLDs/TopLevelDomains (Jon Postel)

1985 Start of DNS, first name registered (symbolics.com or think.com)

3

CHAPTER 1. INFRASTRUCTURE BASICS: DHCP AND DNS

• A bunch of RFCs has followed(http://www.dns.net/dnsrd/rfc)

• A huge “war” of companies and academia also followed, the problem solvedtoday

• Names can be registered at one of the “ICANN Accredited Registrars”

• In Norway: NORID responsible for .no (and accredits registrars) (http://www.norid.no)

1.3.2 Tools

• A DNS server is usually divided into a resolver/cache and an authorative server. Thefollowing servers are common

– BIND (Unix) ∼70%

– MSWindows DNS server

– DJBDNShttp://mydns.bboy.net/survey/

• To query DNS servers we usually use the dig program (man dig) from the Debiandnsutils package or nslookup on Windows

1.3.3 Naming Scheme

Fully Qualified Host Name A FQHN is either the FQDN of a host (ie, a completelyspecified domain name ending in a TLD), or the numeric IP address of a host.

Fully Qualified Domain Name (most common!) A FQDN is a domain name that in-cludes all higher level domains relevant to the entity named.

• e.g hostname=www, domainname=hig.no,FQDN/FQHN=www.hig.no

• Sometimes in DNS context we have to specify the top node of the DNS hierarchyas well (the dot at the end): www.hig.no.

• The top level node has a serie of root servers which has information about all theTLDs

• The current TLDs can be found athttp://www.icann.org

• Information about the root servers can be found athttp://www.root-servers.org

4

1.3. DNS

• The thirteen root servers are usually listed in a configuration file in the DNS server(e.g @ in djbdns and named.root in Bind) which can be retrieved from ftp.rs.internic.net/domain/named.root

Domain Name Space

= resource recordsassociated with name

see also: RFC 1034 4.2:How the database is divided into zones.

= zone of authority,managed by a name server

"delegated subzone"

"zon

e de

lega

tion

"

NS RR ("resource record")names the nameserverauthoritative fordelegated subzone

When a system administratorwants to let another administratormanage a part of a zone, the firstadministrator's nameserver delegatespart of the zone to anothernameserver.

A domain is everything under .no, but the .no zone excludes all its subdomains whichit has delegated. E.g. hig.no is in the .no domain but not in the .no zone.

1.3.4 DNS Query

• DNS works by passing around Resource Records (RRs) through port 53 (TCP iflarger than 512 bytes; else UDP)

• The most important resource records are

SOA Start Of Authority

NS Name Server

MX Mail eXchanger

A Address, define the canonical name of an IP address

CNAME Canonical Name, define an alias

PTR PoinTer Record, define the reverse mapping (the IP address of a fqdn)

• A query for a RR can be either recursive or iterative (the RD bit is either set or not)

Always remember that you can use wireshark to inspect network traffic, in this caseDNS records (and also to check if it UDP or TCP is being used).

5

CHAPTER 1. INFRASTRUCTURE BASICS: DHCP AND DNS

A host can have multiple names, e.g. in a small organization you might have a serverrunning smtp and imap services and have the namessmtp.example.com, imap.example.com, mail.example.com andmikke.example.com. mikke is probably the original name of the server (sometimescalled the canonical name), while smtp, imap and mail, are names identifying the ser-vice the host offers. They should be implemented either as additional A records or asCNAMEs. Implementing them as A records is probably the best option since use of CNAMEsleads to twice as many lookups (a CNAME maps to an A record, so you have to loop up theA record to find the IP address). Separating names into canonical hostname and servicenames is a good idea since you might have to move the service to a different host in thefuture.

DEMO: dig utelunch.hig.no

DNS Recurser

rootnameserver

org.nameserver

wikipedia.org.nameserver

198.41.0.4

204.74.112.1

207.142.131.234

"Where's www.wikipedia.org?"

1

2

3

"Try 204.74.112.1"

"Try 207.142.131.234"

"It's at xxx.xx.xx.xxx"

Typical sequence

• Look in /etc/hosts (assuming /etc/nsswitch.conf has a linehosts: files dns)

• Look in /etc/resolv.conf for the IP address of a name resolver

• Ask the name resolver a recursive query for www.wikipedia.org

• (1) The name resolver checks to see if it has the A record forwww.wikipedia.org, or wikipedia.org or .org, it does not so it has to contact(send an iterative query to) one of the root servers

• The root server replies with the NS records of the .org zone and their correspond-ing A records (glue records!)

• (2) It sends one of the .org servers an iterative query, e.g. one of the .org servers

• The .org authorative server replies with the NS records ofwikipedia.org along with their A records (glue records)

• (3) It then sends an iterative query to one of the wikipedia.org authoritativeservers which answers with the A record of www.wikipedia.org

6

1.3. DNS

Web Browser

Mail Client

Your Computer

Client Programs Operating System

DNS Resolver

localcache

Your ISP

DNS Resolver

localcache

minicache

cache timeout:1-30 min

recursive

DNSsearch

Note that replies are cached and the TTL (Time To Live) field is important, its commonlyset to 24 hours, meaning that the reply you get might not be correct if the site havechanged its authorative DNS within the last 24 hours.

1.3.5 Dynamic DNS

• What to do with DHCP clients needing a name?

– Get a new name with every new IP address?

– Always have the same name? Dynamic DNS!

• Practical and widely used (Active Directory, dyndns.com, ...)

• A security nightmare?

– clients changing server configuration...

Dynamic DNS means that a client can notify an authorative DNS server that it has anew IP address, and the server will update the clients DNS entry to map the samename to the new IP address. This can also apply to other kinds of DNS records (e.g.service records (SRV) in active directory). The fundamental security problem here isthe concept of a client being able to force a configuration change on the server. Thismight not be a problem, but this should ring a bell in our security conscious minds.

1.3.6 DJBDNS

NOTE: WE WILL NOT USE DJBDNS THIS YEAR, BUT WE MENTION IT SHORTLYHERE DUE TO ITS SECURITY DESIGN AND ITS SIMPLIFIED CONFIGURATIONFILES.

• http://cr.yp.to/djbdns.htmlhttp://www.djbdns.orghttp://www.lifewithdjbdns.org

• DJBDNS is a dns server package with a security guarantee (cash reward)

7

CHAPTER 1. INFRASTRUCTURE BASICS: DHCP AND DNS

• It is simple, fast and “secure”

– many small programs running with their own UIDs

– the programs run chrooted

– all “unnecessary” features are removed, feel free to write our own patches

– all data is entered into textfiles in a simple format, then compiled into fastaccessible binary files (.cdb)

http://cr.yp.to/djbdns/guarantee.html

http://article.gmane.org/gmane.network.djbdns/13864

See also DJBDNS security last year:http://en.wikipedia.org/wiki/Dan_Kaminsky

We remember Dan Kaminsky from the OS course when we mentioned the Sony rootkit.Dan discovered how to guess transacation IDs in DNS in order to do cache poisoning,since the number of transaction IDs is a 16-bit number, which is to small. To createadditional randomness a good design choice is to also randomize the use of UDP sourceport which is also a 16-but number. DJBDNS is the only (by my knowledge) dns serverwhich does this by design.

• Key components (programs)

dnscache the recursive resolver

tinydns the authorative nameserver (only UDP)

axfrdns for zone transfers (and TCP replies)

Note the design principle: separate functionality into small components instead of puttingall code into one program. If djbdns was just one big program then a bug in e.g. tinydnswould also affect dnscache.

1.3.7 Example

.borg.trek::queen.borg.trek # SOA,NS@borg.trek::queen.borg.trek # MX=queen.borg.trek:192.168.25.10 # A,PTR+mail.borg.trek:192.168.25.10 # A=scout.borg.trek:192.168.25.1 # A,PTR=probe.borg.trek:192.168.25.11 # A,PTR=cube.borg.trek:192.168.25.20 # A,PTR.25.168.192.in-addr.arpa::queen.borg.trek# SOA,NS

8

1.4. INFRASTRUCTURE DESIGN

This is an example configuration file (/etc/tinydns/root/data) fortinydns see http://cr.yp.to/djbdns/tinydns-data.html for explanation of the syn-tax. This is the simplest possible format with no specification of TTLs or other parame-ters, defaults are used.

1.3.8 DNS admin

• Setting TTLs

• Primary and secondary DNS (priorities)

• Secure Zone transfers

• Integrating with Windows, tsig’s

• Services which do reverse look-up

• Load-balancing

• Split horizon

• DNSSEC

1.4 Infrastructure Design

9

CHAPTER 1. INFRASTRUCTURE BASICS: DHCP AND DNS

Version Control CVS/SubVersion for tracking OS configuration files, OS and appli-cation binaries and source code, and tools and administrative scripts.

Gold Server ”Never log into a machine to change anything on it. Always make thechange on the gold server and let the change propagate out.”

Host Install Tools We manage all of our desktop machines identically, and we man-age our server machines the same way we manage our desktop machines.

Ad Hoc Change Tools Push-based ad hoc change tools such as scp/ssh, r-commands(rsync, rdist, rcp, rsh) and expect scripts are detrimental to use on a regular basis, butmight be necessary in a crisis.

Directory Servers Hostname and/or services resolution, UID and GID mappings.

Authentication Servers Single point of authentication for our users. It’s useful to notethat there are really only four elements to a user’s account in UNIX – the encryptedpassword, the other info contained in /etc/passwd (such as UID), the info contained in/etc/group, and the contents of the home directory. To make a user you have to createall of these. Likewise, to delete a user you have to delete all of these.

Time Synchronization Correct timestamps needed for backups, log entries, state en-gines (make), several network protocols (authentication), cron. Shy away from any toolwhich periodically pops machines into the correct time, NTP is your only good choice.

Network File Servers NFSv2/3/4, Samba, CIFS.

File Replication Servers /etc/ files can be replicated with cfengine.

Client File Access Uniform filesystem namespace.

Client O/S Update OS specific tools through cfengine.

Client Configuration Management Cfengine.

10

1.5. OUR INFRASTRUCTURE

Client Application Management OS specific apps through cfengine, or network mountedfilesystems with shared apps (Licence manager needed?)

Mail SMTP servers properly linked to the DNS system

Printing cross-platform nightmares? Samba... Printer accounting, security (where doesthe classified document end up? insert employee smartcard for retrieving the documentfrom printer?).

Monitoring Syslog to a central server, cfengine here as well.

”Can I grab a random machine and throw it out the tenth-floor window without ad-versely impacting users for more than 10 minutes?”

1.5 Our Infrastructure

This is the base scenario which will will start on, and build on for six iterations.

DEMO: hand out instructions for access.

11

CHAPTER 1. INFRASTRUCTURE BASICS: DHCP AND DNS

1.6 Theory questions

1) How do you configure a typical DHCP server?

2) How do you configure a typical DHCP client?

3) What is a hierarchical namespace? Give two examples related to system administra-tion.

4) Describe some of the key recommendations of RFC1178 (choosing hostnames).

5) Why does access control matter for namespaces?

6) What are the advantages and disadvantages of dynamic DNS?

7) Why should services be linked to aliases (CNAMEs or additional A records) in DNS?

8) What do you have to consider carefully if you want to move the authorative DNSservice to a new host (with a different IP address).

9) How does a reverse DNS look-up work?

10) Briefly describe the contents of the resource records MX, CNAME and PTR.

11) Give some examples of ”Ad Hoc Change Tools” in Traugott and Huddlestons virtualmachine infrastructure model.

12) Write a cfengine script which will install the software package binclock on Debianand Ubuntu hosts. Use promise-type packages and the body:

body package_method apt {package_changes => "bulk";package_list_command => "/usr/bin/dpkg -l";package_list_name_regex => "ii\s+([ˆ\s]+).*";package_list_version_regex => "ii\s+[ˆ\s]+\s+([ˆ\s]+).*";package_installed_regex => ".*"; # all reported are installedpackage_name_convention => "$(name)";package_add_command => "/usr/bin/aptitude --assume-yes install";package_delete_command => "/usr/bin/aptitude --assume-yes remove";package_update_command => "/usr/bin/aptitude --assume-yes install";package_list_update_command => "/usr/bin/aptitude update";package_list_update_ifelapsed => "240";

}

12

1.7. LAB EXERCISES

1.7 Lab exercises

13

CHAPTER 1. INFRASTRUCTURE BASICS: DHCP AND DNS

Note: Synchronized time is critical in all sysadm settings, if you experience problemswith time sync in your virtual machines when using vmware (e.g. if you do this labsetup at home), restart vmware-tools, e.g. in debian service vmware-tools restartsince vmware-tools takes care of syncing virtual machines’s time with the host com-puter. In unvirtualized scenarios time syncronization is commonly taken care of by thentp daemon. We will talk briefly about this in the lecture.

IMPORTANT: When installing servers/services, take your time and read the information pro-vided to you during the installation process.

1) DHCP

Install DHCP server on queen so lin and win gets network access automatically.Configure the dhcp server to provide IP addresses in the range .101 to .110. Use128.39.243.2 as default nameserver initially.

2) DNS

In this lab exercise you will install authorative DNS for the borg.trek domain on cubeand at the same time a caching dns/recursive resolver also on cube which everyonecan use instead of 128.39.243.2. The caching dns/recursive resolver on cube will havethe addtional power that it can answer queries for the borg.trek domain (128.39.243.2will not be able to do that and you should think about why it is so).

On cube, check what ports are open and listening in the base installation by defaultwith netstat -apnob tcp and netstat -apnob tcp.

Install Microsoft DNS Server as a “Server Role” using the Server Manager on cube.

Configure the newly installed DNS server so it only answers queries on its IPv4interface 192.168.25.20

Create a new forward lookup zone borg.trek with corresponding reverse lookupzone, and add authorative DNS entries (A and PTR records) for scout.borg.trek,queen.borg.trek, probe.borg.trek and cube.borg.trek.

Check that you can resolve your own borg.trek domain

3) Update DHCP server settings Change dhcp server setup to reflect the changes inDNS you have just completed. Do this by simply running the following cfengine3script (note that the script only restarts dhcp3-server if the config file is edited, incfengine3 terminology “if the promise had to be repaired”):

body common control {bundlesequence => { "dhcpedit", "restartdhcp" };

}

bundle agent dhcpedit {files:

14

1.7. LAB EXERCISES

"/etc/dhcp/dhcpd.conf"edit_line => dhcpline;

}

bundle edit_line dhcpline {replace_patterns:"domain-name-servers 128.39.243.2"

replace_with =>with("domain-name-servers 192.168.25.20");

}

body replace_with with(x) {replace_value => "$(x)";occurrences => "all";promise_repaired => { "need_dhcp_restart" };

}

bundle agent restartdhcp {commands:need_dhcp_restart::"service isc-dhcp-server restart";

}

You can download this script (wget it) fromhttp://www.hig.no/˜erikh/sysadm/dhcpedit.cf

Check that you can ping queen.borg.trek from win after an ipconfig /renew

4) Active Directory

Install Active Directory Domain Services as a “Server Role” using the Server Man-ager on cube.

Turn cube into a Domain Controller.

(a) (Do not choose advanced mode installation)(b) “Create a new domain in a new forest”(c) FQDN of the forest root domain: “borg.trek”(d) Forest functional level: “2008” (not R2), see

http://technet.microsoft.com/en-us/library/understanding-active-directory-functional-levels(WS.10).aspx

(all other options are fine by default)When the process is completed you will be asked to reboot. After rebooting, fromnow on you can no longer log in as Administrator, you have to use usernameBORG\Administrator (in other words DOMAIN\Username).

15

CHAPTER 1. INFRASTRUCTURE BASICS: DHCP AND DNS

Check that the dns server for cube now is set to use itself.

Check what ports are open now after the active directory and dns install is complete,are ports 53 and 389 open?

You now need to setup DNS such that it is properly linked to AD. AD needs to dy-namically update DNS with SRV-records, such that other clients can locate servicesusing SRV records.

Create an OU (Organizational Units) in AD for server computers (an OU called“Servers”) and client computers (an OU called “Clients”) using the cmdletNew-ADOrganizationalUnit

Stop non-administrative users from being able to join computers to the domainborg.trek by changing the domain’s MachineAccountQuota setting to 0.

Create computer objects (“prestaging computer accounts”) in AD for the serversprobe and queen, and for the clients lin and win, using the cmdlet New-ADComputer.

Join all hosts to Active Directory using the cmdlet Add-Computer on Windows andthe likewise-open software on Linux. For help on Linux see http://www.youtube.com/watch?v=sVT-0t4d48I.

DNS IS NOT UPDATED CORRECTLY WITH SOA AND SRV RECORDS WHENDNS IS INSTALLED BEFORE AD !!!DNS SAYS “the dns server is waiting for active directory...”THE INFO IN c:\Windows\System32\config\netlogon.dns should have been inDNS server ...

Activate LINK TO DNS; AND INITIATE SECURE DYNAMIC DNS

List the A records in cube’s DNS, can you find win and lin?

5) Dynamic DNS

Force IP address change on lin and win (e.g. by reducing the DHCP IP range on theDHCP server, and reloading network setup on lin and win), what happens to lin andwin’s records in the authorative DNS server?

16

Chapter 2

Configuration Management: ManyHosts

2.1 Infrastructure Design

Go back and refresh the model of Traugott and Huddleston

In the next slides, all principles, corollaries and suggestions are from Mark Burgess’book “Principles of Network and System Administration”

2.1.1 Stability

Principle32 (scalability) Any model of system infrastructure must be able to scale effi-ciently to large numbers of hosts.

Principle33 (reliability) Any models of system infrastructure must have reliability asone of its chief goals. Down-time can often be measured in real money.

Corollary Reliability is safeguarded by redundancy, or backup services running in par-allel, ready to take over at a moments notice.

2.1.2 Uniformity

Principle34 (homogeneity/uniformity) A model in which all hosts are similar is i) eas-ier to understand conceptually both for users and administrators, ii) cheaper toimplement and maintain, and iii) easier to repair and adapt in the event of failure.

Corollary27 Avoid improvising system modifications on the fly (ad hoc changes), whichare not reproducible. It is easy to forget what was done, and this will make thefunctioning of the system difficult to understand and predict, for you and forothers.

17

CHAPTER 2. CONFIGURATION MANAGEMENT: MANY HOSTS

Principle35 Expressing tasks in an operating-system independent language reducestime spent debugging, promotes uniformity/homogeneity and avoids unneces-sary repetition.

Suggestion8 Use languages and tools which are independent of the operating systempeculiarities, e.g. cfengine, perl, python. More importantly, use the right tool forthe right job.

Always remember that humans make error, computers dont (unless we have programmedthem to do). If you script your tasks instead of performing them manually, you will cre-ate uniformity in your infrastructure.

2.2 Infrastructure Maintenance

2.2.1 Models

• Models of maintenance

– Reboot (the MSWindows way...)– Manual– Central control– Immunology/Autonomy

∗ Convergence and stable state

Home-users with little computing skills follow the model of rebooting if somethingdoes not work (a reboot will “clean the host”). Instead of rebooting, we can always tryto manipulate the host manually to fix errors and perform sysadm tasks. If we are abit more advanced we introduce a monitoring system so we can fix things only whenneeded. But what we really would like are hosts that fix themselves as much as theycan.

Remember the concept of idempotent operations from operating systems. An operationis idempotent if it can be applied multiple times to the end results without changing it.Convergent operations are basically the same thing but also means that there is somedefined state we want to reach and the operation has to lead towards that state). Whenthe state is reached the operation can be applied again and again without altering thatstate. In other words:

Convergence means bringing a host closer to some ideal state (stable state), which isdefined by a system policy, an operation f is convergent if

f(incorrect) = correct

andf(correct) = correct

18

2.3. CONFIGURATION MANAGEMENT SYSTEMS

2.2.2 Policy

Principle37 (disorder) Systems tend to a state of disorder unless a disciplined policy ismaintained, because they are exposed to random noise through contact with users.

Principle39 (policy) A clear expression of goals and responses prepares a site for futuretrouble and documents intent and procedure. Policy should be a protocol for systempredictability.

Users are our friends and enemies. If we didnt have users, we would not have jobsso of course our job is to serve our users the best we can. But on the other hand, ourhosts will drift away from their stable state due to users: users will fill up storage spacewith big files, submit extremely large print jobs to our printing server, start renderingan a 3D animation stealing all processor power, install additional software, unplug thenetwork cables, etc

To be able to maintain a stable state of our hosts, we need to know what that state is sup-posed to be. That is the role of policy. Policy is a set of rules of what the system shouldbe like. There are high-level (“all laptops should have updated security software”) andlow-level policies (“per user disk quota should be 1GB”). High-level policies needs tobe translated into low-level policies which we can implement in our configuration man-agement system.

2.3 Configuration Management Systems

• Components

– (version controlled) Storage of configuration

– Master server/Control center

– Agents doing the job

• High security requirements

2.3.1 Implementations

• Cfengine

• Puppet, Chef, Bcfg2

• Dell OpenManage, Dell KACE

• IBM Tivoli Framework

• HP Server Automation software

19

CHAPTER 2. CONFIGURATION MANAGEMENT: MANY HOSTS

• MS Group Policy

• MS System Center Configuration Manager

• self-made scripts :)

• etc ...

see alsohttp://en.wikipedia.org/wiki/Comparison_of_open_source_configuration_management_software

There are a plethora of systems, and you will probably all end up working with differ-ent systems. Thats we spending time learning a “platform independent” open sourcetool like cfengine is a good idea. It has been around since 1993 and manages millions ofhosts, and has many good theoretical concepts which you will make good use of whenyou learn other systems later. It is also likely that you will end up working directly withcfengine since it is a Norwegian company and several of the recent companies who hirestudents from Gjøvik, use cfengine in their infrastructures.

2.3.2 Host identities

• Our servers have static IP addresses, but what to do with our roaming clients?Use VPN!

Head-office

Regional Office

Regional Office

Remote / roaming users

Internet VPN

Internet

Sometimes we need to identify the hosts, and this can be difficult in todays mobileworld. Hosts can be identified based on IP address, DNS name if we have dynamicDNS, MAC address, maybe a separate ID scheme, public keys, there are several options.But one easy to use, and widely used solution, is using a virtual private network wherewe assign all hosts a fixed IP address.

In the lab, we will use OpenVPN, and use certificates for authenticating clients (note:using certificated means that OpenVPN uses the public key of a client for identifications

20

2.3. CONFIGURATION MANAGEMENT SYSTEMS

(the clients identity is its public key), and the client uses its corresponding private keyto authenticate itself).

2.3.3 Authentication

• In the old days (19??) IP addresses where trusted (rdist/rsh/rcp)

• In recent years we commonly use public/private keys in a challenge-response pro-tocolhttp://en.wikipedia.org/wiki/Challenge-response_authentication

We really need to pay close attention to where these key files are stored. A hosts privatekey is a password stored in “cleartext”, make sure these files are not world readable.

2.3.4 Push vs. Pull

• Push changes onto our hosts: the process initiated from the masterserver and thehosts are ’forced’ to obey

• The hosts can instead Pull changes from the masterserver and be in charge them-selves.

• Pull makes it scale better (the load is on the hosts)

• Keeping hosts autonomous is better when there’s unstable network access

• Better security: no listening server on all hosts with “write access”, hosts onlyneeds “read access” on masterserver

The most common model today is a Pull. It scales better. But many configuration man-agement systems offers both models, or like cfengine does, there’s no direct push butthe server can ask the client to run its own agent (which usually pulls, thereby we pushto force a pull :).

To be able to push, we have to have an identity for all hosts. If we have a pure pull-basedmodel, we dont necessarily need a host identity unless we are serving “confidential”configuration data.

21

CHAPTER 2. CONFIGURATION MANAGEMENT: MANY HOSTS

2.4 Cfengine

(Page 6 of M. Burgess “Cfengine 3 Concept Guide”)

And of course the cfengine scripts which are distributed should be fetched from a ver-sion controlled repository (typically subversion).

22

2.4. CFENGINE

TAVLE:

queen: all (also queen):

/masterfiles/* --> /var/cfengine/inputs/

(cf-serverd) (cf-execd as a wrapper for cf-agent)

23

CHAPTER 2. CONFIGURATION MANAGEMENT: MANY HOSTS

2.5 Theory questions

1) Name and describe at least two core principles of stable infrastructures.

2) How can cfengine be used to enforce policy?

3) Briefly describe four different models of system maintenance.

4) Describe what is meant by “a convergent operation”.

5) Describe advantages and disadvantages of at least three different ways of identify-ing hosts.

6) Why can a pull-based approach to host maintenance provide better security?

7) Draw and explain how the components cf-execd, cf-monitord, cf-agent and cf-serverdcfengine interact between a host and a policyserver.

8) What is the purpose of the cfengine component cf-monitord?

24

2.6. LAB EXERCISES

2.6 Lab exercises

25

CHAPTER 2. CONFIGURATION MANAGEMENT: MANY HOSTS

Cfengine is based on the simple model that one single text file controls everything, thefile $(sys.workdir)/inputs/promises.cf.promises.cf holds the entire configuration for us (usually by including other files in it-self). In addtition to doing the actual configuration management, cfengine also need toupdate its own configuration by distributing promises.cf (including related files) fromone master location to all hosts. This task should be completely separated from the ac-tual configuration management work, because it cannot fail!. If we make an error in thepromises.cf file, this would quickly propagate in our infrastructure and break all com-munication between cfengine components. This cannot happen, and this is why thereis a separate file failsafe.cf which has to be present in $(sys.workdir)/inputs/(in addtion to promises.cf). If cf-agent fails when executing promises.cf, it will al-ways switch to executing failsafe.cf. But it is also common to make the updatingof cfengine configuration a part of the promises.cf since cf-agent usually runs finewithout any errors (thus not using failsafe.cf). So there is also a file commonly calledupdate.cf which contains the actualbundle agent update {...} for doing the update, and promises.cf andfailsafe.cf includes this and adds it to their bundlesequence (thus failsafe.cf isactually just very small file with its only contents beingbody common control { bundlesequence => { "update" }; inputs => { "update.cf" }; }).The update.cf should never change once it is working, and must be self-contained(meaning it should not depend on other files like lib.cf). If you wonder why thisupdate bundle is not directly in failsafe.cf it is because we cannot have multiplebundlesequences, and since it is commonly a part of promises.cf, it has to be a sepa-rate file typically named update.cf

The common way to run cfengine is by executing cf-execd as a wrapper around cf-agent,and the default (BUT NOT DOCUMENTED IN THE REFERENCE MANUAL?) com-mand (exec_command) cf-execd will execute is$(sys.workdir)/bin/cf-agent -f failsafe.cf && $(sys.workdir)/bin/cf-agent

cf-agent also updates itself (since update.cf is commonly included into promises.cf),but by executing failsafe.cf first, we make sure the configuration is updated beforecf-agent parses its promises.cf file (this save us one iteration of waiting for updatedconfiguration).

Steps for creating a working cfengine infrastructure in borg.trek:

1. Install the cfengine software package on all hosts (the keys localhost.pub andlocalhost.priv have been created in $(sys.workdir)/ppkeys/, if they dont ex-ist run cf-key as root/administrator).

2. On queen (our cfengine master host), create the directory /masterfiles/inputsand place the files promises.cf, failsafe.cf and update.cf in there.

3. Edit promises.cf to reflect your settings (you probably just have to change themailto email address).

26

2.6. LAB EXERCISES

4. Copy the same three files to $(sys.workdir)/inputs/ on all hosts (includingqueen itself).

5. Execute cf-execd first on queen, then on all the other hosts (it will start runningas a daemon/service). On Windows, start the services GUI with services.mscand right click on the CfengineNovaExec service and start it.

2.6.1 Cfengine install

Important: use the flags -v (verbose) or -d (debug) to get more output from cfengine componentsin case things dont work. E.g. to find out why the cf-serverd is not accepting connections startit verbosely in the foreground with cf-serverd -Fv or if you want to find out why cf-execd isnot sending you email do a cf-execd -Fd.

Install Cfengine on all hosts with queen as the master (gold) server.

You have now a very nice scalable pull-based management infrastructure where youcan maintain all your thousands of hosts from a single masterserver using a singleconfiguration file!

2.6.2 Getting started with pull-based configuration management

Place the file putty.msi in /masterfiles and have cfengine copy it toC:\Users\Administrator\ on the Windows hosts.

Implement some of your cfengine scripts from the previous theory exercises.

27

Chapter 3

User Accounts: Identites andAuthentication

3.1 Local User Accounts

• Creating user accounts is very system specific

• User accounts contain both public and private information

• Creation:

– insert new user into user database

– insert new user into appropriate group(s)

– create a home directory

– set up the users environment

In general, educate users to always use at least two passwords, one for secure stuff andone for insecure stuff...

3.1.1 Unix/Linux

worf:x:1000:1000:Lt.Worf:/home/worf:/bin/bash

• Login name

• Optional encrypted password

– x meaning use of shadow file

• Numerical user ID

29

CHAPTER 3. USER ACCOUNTS: IDENTITES AND AUTHENTICATION

• Numerical group ID

• User name or comment field (GECOS)

• User home directory

• User command interpreter

3.1.2 Windows

• %SystemRoot%\System32\Config\SAMloaded into registry asHKEY_LOCAL_MACHINE\SAM

DEMO: see what we can find in registry

3.1.3 Local vs domain accounts

• Unix/Linux: Controlled by sequence in /etc/nsswitch.conf and /etc/pam.d/

• Windows: Controlled by choosing domain

When logging on to a Unix/Linux host, the host will attempt to lookup the usernamein a sequence of files/databases decided in the Name Service Switch configuration file.Similarly for authentication decided by the files in the Pluggable Authentication Direc-tory.

DEMO: view nsswitch.conf and pam.d

3.2 User Profiles

• The user-specific configuration data

• A separate folder on Windows

– can be roaming

• Dot-files and dot-directories on Unix/Linux

30

3.3. LDAP

3.3 LDAP

3.3.1 Directory Service

• A simple database optimized for reads and searches (lookups)

• Centralized storage of users, computers, services, printers, mailinglists, ...

• Think about it as the yellow pages phonebook

• Examples are DNS, NIS, LDAP

The ratio between reads:writes in a directory service is typically 1000:1, 10000:1 or even100000:1.

Yellow pages have entries (records) for each company or person, and possibly havemany levels, e.g. Sporting goods stores and within sporting goods stores you havemight have entries for general sporting goods, but you might also have subcategorieslike tennis stores, golf stores, etc. But in general there is very frequent searches andreads, and very rarely there are new writes (new registrations or updates).

3.3.2 Overview

• LDAP is a client-server protocol for communication with a directory service

• Defined in RFC4510 - RFC4533 ! (most important are RFC4511&RFC4512)

• LDAP directories follow the X.500 model:

– a tree of directory entries (records)

– an entry consist of a set of attributes (fields)

– an attribute has a name and one or more values

• LDAP uses TCP on port 389

Note: the actual data in the directory service can be stored in any kind of backend,e.g. flat files or an relational database, the requirement however is that the data haveto be accessed (read, searched, updated, added, modified, etc) according to the LDAPprotocol.

TAVLE:

(rdn: uid=erikh dn: uid=erikh,ou=informatikk,ou=imt,dc=hig,dc=no)

31

CHAPTER 3. USER ACCOUNTS: IDENTITES AND AUTHENTICATION

dc=no|+--dc=hig (these could be one entry dc=no,dc=hig)

|+--ou=imt

|+-ou=informatikk

|+-uid=erikh| roomNumber=A-132| mobile=93034446| .| .| .|+-uid=frodeh+-uid=tomr

.

.

.

DEMO: Luma with ldap.hig.no

DEMO: ldap contact lists and autocompletion in evolution

3.3.3 DN/RDN

• DN = RDN + Parent’s DN

• DN is the unique identifier (primary key)

• RDN is unique identifier only at its own level

3.3.4 Protocol

• StartTLS

• Bind

• Search

• Compare

• Add (atomic)

32

3.3. LDAP

• Delete (atomic)

• Modify (atomic)

• Modify DN (move entry) (atomic)

• Abandon

• Extended operation

• Unbind

StartTLS and Bind for security, StartTLS is part of LDAP while LDAP over SSL meansthat we first establish a SSL session (ldaps using port 636).

Note: prefix notation in search (as opposed to infix and postfix)

DEMO:

ldapsearch -x -b dc=hig,dc=no -h ldap.hig.no \"(mail=erik.hjelmaas@hig.no)" sn givenName mobile

echo SGplbG3DpXM= | base64 -d

ldapsearch -x -b dc=hig,dc=no -h ldap.hig.no \"(&(givenName=Erik)(ou=Institutt for \informatikk og medieteknikk))" sn givenName

ldapsearch -xt -b dc=hig,dc=no -h ldap.hig.no "(mail=frode.haug@hig.no)" sngivenName mobile jpegPhoto

3.3.5 Schema

• Entries are instance of an object

• ObjectClass is the link to Schema which defines the object

• From RFC4512 4. Directory Schema: “The schema enables the Directory systemto, for example:”

– prevent the creation of subordinate entries of the wrong object-class (e.g., acountry as a subordinate of a person)

– prevent the addition of attribute-types to an entry inappropriate to the object-class (e.g., a serial number to a person’s entry)

– prevent the addition of an attribute value of a syntax not matching that de-fined for the attribute-type (e.g., a printable string to a bit string).

33

CHAPTER 3. USER ACCOUNTS: IDENTITES AND AUTHENTICATION

3.3.6 LDIF

• Plain text format for representing LDAP directory content and update requests

• Records separated from one another by blank lines

• See examples:http://en.wikipedia.org/wiki/LDAP_Data_Interchange_Format

3.3.7 Case: FEIDE

• Feide is an identity management system on a national level for the educationalsector

• Basically a directory schema called norEduhttp://feide.no/content.ap?thisId=1346

Gjøvik: feide.hig.no (Fedora directory) as parent of ldap.hig.no (OpenLDAP) and hig.no,fadm.hig.no, ansatt.hig.no (Active Directory)

3.4 Kerberos

AuthenticationServer (AS)

Ticket-granting

Server (TGS)

request ticket-

granting ticket

once peruser logonsession

1. User logs on toworkstation andrequests service on host.

3. Workstation promptsuser for password anduses password to decryptincoming message, thensends ticket andauthenticator thatcontains user's name,network address, andtime to TGS.

ticket + session key

request service-

granting ticket

ticket + session key

once pertype of service 4. TGS decrypts ticket and

authenticator, verifies request,then creates ticket for requestedserver.

Kerberos

5. Workstation sendsticket and authenticatorto server.

6. Server verifies thatticket and authenticatormatch, then grants accessto service. If mutualauthentication isrequired, server returnsan authenticator.

request service

provide server

authenticatoronce perservice session

Figure 4.1 Overview of Kerberos

2. AS verifies user's access right indatabase, creates ticket-granting ticketand session key. Results are encryptedusing key derived from user's password.

(Stallings, W. Network Security Essentials, 2nd Ed.. Prentice-Hall, 2003.)

In Greek mythology, Kerberos is a many headed dog, the guardian of the entrance ofHades

34

3.5. ACTIVE DIRECTORY

• Developed at MIT in the 80’s• Provides a centralized authentication server to authenticate users to servers and

servers to users• Relies on conventional encryption, making no use of public-key encryption• Provides Single-Sign On• Kerberized applications (apps need to support kerberos)• Two versions: version 4 and 5• Version 4 makes use of DES (not good!)• Recommended on W2K/W3K (but with MS’s embrace and extend...)

Kerberos can have the following issues:

• Lifetime associated with the ticket-granting ticket

– If too short: repeatedly asked for password– If too long: greater opportunity to replay

• The threat is that an opponent will steal the ticket and use it before it expires

DEMO:

.

3.5 Active Directory

• LDAP, Kerberos and (Dynamic DNS)

• Forests, Trees and Domains

– Scales to really large organization!

• OUs are the common level for GPOs

• Sites are physical (not logical) groupings defined by IP subnets

• OUs should be structured primarely to facilitate administrative delegation, sec-ondary to facilite GPOs

• Multi-master replication (pull-based replication)

The Active Directory LDAP schema allows for the entries necessary for Unix/Linuxuser accounts also.

35

CHAPTER 3. USER ACCOUNTS: IDENTITES AND AUTHENTICATION

3.6 User Account Policy

• What usernames can be chosen?

• Can user accounts be shared?

• When and how to disable and archive the account?

• How long can it remain inactive?

• Password construction and aging rules

• Location (and redirection) of home directory

These are some of the possible policies that have to be addressed for user accounts.

3.6.1 Choosing usernames

• Remember the topic of Namespaces

• firstname.lastname does not scale!

• Numeric usernames OK today, but trouble on legacy systems

• Let the User choose his/her Username!

A good choice today is to let users choose their own usernames, but we should assistthem. This is how most internet services work (e.g. gmail). This will let users keep thesame name across different services.

36

3.7. THEORY QUESTIONS

3.7 Theory questions

1) Where is the local user database commonly located on Unix/Linux? on Windows?.

2) In LDAP, what is a distinguished name (DN) and a relative distringuished name(RDN)? Give examples.

3) How are client requests matched to server responses in LDAP?

4) What is the difference between using “StartTLS in LDAP” and “LDAP over SSL”(LDAPS)?

5) What is a schema in LDAP and what does it define? Give an example.

6) What is LDIF and what does an LDIF record contain? What is LDIF used for?

7) What is a site in Active Directory?

8) What should be the nr one and nr two criterias when choosing how to create OUs inActive Directory?

9) Which issues should be addressed in a user account policy?

37

CHAPTER 3. USER ACCOUNTS: IDENTITES AND AUTHENTICATION

3.8 Lab exercises

38

3.8. LAB EXERCISES

1) Prepare environment

On lin, install openldap-utils.

2) Creating an organizational unit

Create the organizational unit (OU) Drone on cube.

3) Register LDAP entries in AD

Create a LDIF file users.ldf:

dn: cn=James Tiberius Kirk, ou=drone, dc=borg, dc=trekchangetype: addcn: James Tiberius KirkobjectClass: userSAMAccountName: kirkgivenName: James Tiberiussn: KirkuserAccountControl: 66048unicodePwd::IgBwAG8ATAArAFYAbwB0ADIAIgA=

dn: cn=Jean-Luc Picard, ou=drone, dc=borg, dc=trekchangetype: addcn: Jean-Luc PicardobjectClass: userSAMAccountName: picardgivenName: Jean-Lucsn: PicarduserAccountControl: 66048unicodePwd::IgBwAG8ATAArAFYAbwB0ADIAIgA=

The password has been created usingwine stringconverter.exe \"poL+Vot2\" /encode /unicodehttp://oreilly.com/pub/a/windows/2004/03/30/serverhacks_passwords.html

Import this ldif file to create user kirk.

Check that kirk and picard has been added to domain users in Server manager -Roles - AD Domain Services - AD Users and Computers - borg.trek - drone - Users -(right click on domain users, choose properties - members)

4) Browsing AD LDAP from a client

On Lin, install Luma and attempt to browse the ldap service on cube.borg.trek. Youwill see that you are only able to retrieve some top-level “metadata”.

5) Logging in on Windows Clients

39

CHAPTER 3. USER ACCOUNTS: IDENTITES AND AUTHENTICATION

Before you can login as domain users with remote desktop you have to add Domain Usersto the list of users on Win that are allowed to log in, do this with (in powershell orcmd)SystemPropertiesRemote (Select Users - Advanced)

Check that you can login as kirk (username BORG\kirk), what is your home direc-tory?

6) Logging in on Linux Clients

Check that you can login as kirk (username BORG\kirk), what is your home direc-tory? is kirk present in /etc/passwd? Check which uids and guids kirk belong towith id.

40