course Linux system administration · Linux system administration, 2nd slide set © Jani Jaakkola...

Linux system administration course

Spring 2019: 3rd slideset

1

Linux system administration, 2nd slide set © Jani Jaakkola 2019

Dbus - Desktop Bus● The dbus-daemon implements a message passing and remote procedure call

(RPC) system and API used by Linux system daemons and desktop software.○ System bus for communication between system daemons.○ Session bus for communication between session GUI software.

● Provides a C and python bindings for type safe message passing.○ Also introspection for API enumeration.

● Can authenticate caller and callee identity.○ Can be safely used to let possibly malicious user processes to communicate with the system.○ Provides a mechanism to request administrators permission to perform the requested operation.

● Monitor a bus (system or session): dbus-monitor● Inspect and call dbus APIs: d-feet

2

Network configuration and daemons

Interfaces, configuration, WiFi

3


Network configuration● Kernel doesn’t do any network configuration. All interface configuration

happens from user space.○ Kernel has no defaults.

● Depending on distributions and settings Linux has multiple different ways to configure networks.

● Servers and laptop workstations have completely different needs.○ On laptops networks change often and need management by the user.○ On servers the network needs to be always on, so it needs to work without any user or

administrator action.

4


Network interface controller (NIC)● Physical network interface controllers (NIC) have drivers loaded by udev.

○ MAC (media access control address) address is provided by the NIC,○ MAC address of a device is a unique identifier assigned to a NIC by manufacturer for

communications at the data link layer of a network segment.○ Ethernet segment can only communicate between MAC-addresses

■ Unicast, multicast or broadcast.○ Udev generates a persistent names for a NIC based on the interface MAC address.○ No more eth0 and eth1○ List of NICs: ip link

○ NIC IP addresses: ip addr

○ Routes: ip route

○ Physical wired link status: ethtool <interface>

○ Older versions: /sbin/ifconfig and /sbin/route

5


Dynamic Host Configuration Protocol (DHCP)● A working network configuration still needs:

○ IP addresses and network routes.○ List of usable DNS servers.

■ Google and others provide free of cost DNS-services, if you are willing share your data.○ Priorities between different interfaces. Wired network is still likely to be faster than wireless.

● DHCP protocol allows client device to access this information on a network.○ DHCP is a broadcast protocol: client broadcasts its MAC-address and requests network

configuration lease. The lease is granted (or denied) for the MAC-address.○ However MAC-address spoofing is easy.

● Linux DHCP client is implemented by /sbin/dhclient○ DHCP client is needed early in the boot process, so it is installed already on initrd.

6


Wi-Fi networks● Wi-Fi is a technology for radio wireless local area networking of devices based

on the IEEE 802.11 standards (wikipedia).● Clients connect to Wi-Fi networks through access points (AP).

○ APs broadcast their status with user readable ESSID name. ○ When connected to an AP works very much like wired ethernet.○ All Wi-Fi traffic can be intercepted by other devices within wireless distance.

■ Standards provide limits on transmission power. Nothing prevents an attacker from circumventing those limits. Sometimes with a pringles can. Encryption is needed.

○ The WPA2 protocol provides encryption, with shared keys (passwords) and/or certificates.■ Everyone should always use certificates. Nobody does.

○ List Wi-Fi interfaces: iwconfig

○ Scan for APs: iwlist scan

○ WPA2 encryption is implemented by wpa_supplicant

7


Network manager● A daemon which provides an API for non admin users to manage network

device configuration.○ Network manager applets provide a GUI for network manager.○ By default simply starts dhclient on wired and wireless networks.○ Can handle ethernet, WiFi, VPN and bluetooth connections.

● From command line:○ List devices: nmcli dev ○ Show connections: nmcli connection show

○ Show connection details: nmcli connection show connection

○ Connection up/down: nmcli connection [up|down] <connection>

● Connections are saved in /etc/NetworkManager/system-connections/○ Including system wide shared passwords!

8


Static configuration: /etc/network/interfaces● On Debian and Ubuntu based systems provides a configuration file for static

NIC configuration.○ Package ifupdown○ Old system, implemented with shell scripts.○ Is still widely used.○ When configured, NetworkManager won’t touch interfaces with static configuration.

● Command line:○ Bring interface up: ifup <interface>

○ Bring interface down: ifdown <interface>

○ Query interface: ifquery <interface>

9


Simple example configurationallow-hotplug eth0

iface eth0 inet static

address 192.168.11.100

netmask 255.255.255.0

gateway 192.168.11.1

dns-domain example.com

dns-nameservers 192.168.11.1

10


Red Hat: /etc/sysconfig/● /etc/sysconfig/network

○ This file specifies routing and host information for all network interfaces. It is used to contain directives which are to have global effect and not to be interface specific.

● /etc/sysconfig/network-scripts/ifcfg-<interface-name>○ For each network interface, there is a corresponding interface configuration script. Each of these

files provide information specific to a particular network interface. ● Managed with ifup and ifdown

11


Systemd: systemd-networkd● Part of systemd that detects and configures network devices as they appear.● Can manage virtual devices, like virtual bridges and bonding devices.● Needs systemd-networkd.service running.● Configuration stored in /etc/systemd/network● Exacmple: /etc/systemd/network/20-wired.network

[Match]Name=enp1s0

[Network]#DHCP=ipv4Address=10.1.10.9/24Gateway=10.1.10.1DNS=10.1.10.1#DNS=8.8.8.8

12

Power management

13


Power Management: ACPI● Advanced Configuration and Power Interface.● Offers a firmware API for PC power management.

○ API is implemented in bytecode (AML) that needs to run by the OS at kernel level!○ It certainly looks like it was invented to make alternative OS development harder.○ The ACPI bytecode almost always contain bugs, which might get fixed with firmware updates.○ Linux kernel 2.6 blacklisted any ACPI BIOS from before January 1, 2001.

● ACPI provides:○ Temperature and fan status.○ Battery power and power usage information.○ Information about power events: suspend button, power button, laptop lid.○ Display power management.○ Wake up events when power is still on, but machine is turned off.

14


ACPI: Some selected opinionsIn November 2003, Linus Torvalds—author of the Linux kernel—described ACPI as "a complete design disaster in every way". In 2001, other senior Linux software developers like Alan Cox expressed concerns about the requirements that bytecode from an external source must be run by the kernel with full privileges, as well as the overall complexity of the ACPI specification. In 2014, Mark Shuttleworth, founder of the Ubuntu Linux distribution, compared ACPI with Trojan horses. (wikipedia)

15


ACPI: Sleep states● ACPI defines six sleep states.

○ S0: Computer is running and CPU executing instructions.○ S1: Standby. CPU stops running instructions.○ S2: CPU is powered off and data flushed to RAM.○ S3: Suspend to RAM. Only RAM remains powered. This is the usually used suspend state.○ S4: Hibernation, where RAM is flushed to disc and machine is powered off.○ S5: Soft off state: the computer is off, but some devices can still receive power to restart

computer, like keyboard, clock, modem, LAN, or USB device.○ S6: Mechanical off.

● Only states S0, S3 and S5 are actually useful. ● Hibernation is implemented by the OS in Windows and Linux.

16


ACPI: CPU power states● The CPU has power saving states, which are used when the CPU is idle.● On deeper power saving states less power is used, but it takes for the CPU to

resume computation. ● The CPU power states:

○ C0: CPU is operating normally.○ C1: Halt state. CPU is stopped, but can resume immediately.○ C2: Software visible state is maintained, but wake up takes time.○ C3: CPU is cache is flushed. On wake up cache is empty.○ C4 - C10: Deeper CPU states implemented by CPU vendors.

17


Linux CPU power management● Kernel automatically manages CPU power usage states.● CPU C sleep states when CPU is idle:

○ Can completely turn off parts of CPU.○ The deeper sleep states need longer latencies for resuming computation.

■ Might need to be disabled in cluster environments.○ Command cpupower

● CPU Performance Scaling:○ Dynamically adjust CPU frequency, including temporarily boosting it.○ Scaling governors implement algorithms to estimate the required CPU capacity○ Configured through: cpufreq and /sys/devices/system/cpu/cpufreq ○ Intel P-state with intel_pstate module leaves scaling to CPU

● Command to monitor power usage: powertop

18


Suspend and hibernate● Suspend:

○ Suspend state turns the machine off except for RAM memory and devices needed for waking up○ Power button, sleep button, keyboard○ Usually works well○ From command line: systemctl suspend

○ Or with upowerd dbus API.● Hibernate:

○ Writes RAM to a swap partition and turns machine off completely.○ Needs a swap partition (obviously)!○ Resume is done by initrd by reading from swap and switching back to saved kernel.

■ Enables keeping swap on encrypted partition.○ Does not work with secure boot!○ From command line: systemctl hibernate

○ You need to have kernel argument resume=/dev/your/swapdevice

19

Bluetooth

20


Bluetooth protocol● Bluetooth is a wireless technology standard for exchanging data over short

distances using short-wavelength UHF radio waves in the ISM band from 2.400 to 2.485 GHz (same as WiFi) from fixed and mobile devices (wikipedia)

○ Useful for low power devices.● Implements encryption on the bluetooth link with shared keys.

○ The shared keys are generated during the pairing process.● With Linux useful to wirelessly connect nearby (max 10m) devices:

○ Keyboards, mice, game controllers.○ Audio headsets and speakers.○ Mobile phones, for network tethering and (slow) file sharing.

● The Linux device needs a bluetooth dongle. It is builtin to all laptops.

21


Bluetooth pairing process● The first bluetooth device need is set to pairing mode by the user.● The other device uses bluetooth discovery and request pairing.● During pairing the encryption is vulnerable to man-in-the-middle attack.

○ A PIN-code is generated and validated between the devices for protection from attack● After the key exchange a shared secret is generated and stored on the device.● The secret key is used for later connections automatically.

22


Linux bluetooth protocol stack● Kernel drivers and userspace deamons implemented by Bluez.

○ Bluetooth HID driver for keyboards, mice and game controllers.○ D-bus API

● Bt applet and configuration GUI: blueman

● Bt service discovery: sdptool

● Command line configuration: bluetoothctl

● Dump bluetooth packets: hcidump

● Bluez configuration: /etc/bluetooth

● Bluez device status: /var/lib/bluetooth

23

Linux and audio

24


Linux and audio● During the history of Linux audio interfaces of Linux have had multiple

development problems and deficiencies.○ This has been mostly solved by inventing completely new audio APIs and platforms.

● Linux 1.0: the excellent OSS sound system (of the time) and API. ○ However, its developer Hannu Savolainen was hired to make the project commercial.○ OSS API is still in use in FreeBSD and Solaris.

● With kernel 2.6 newer ALSA API was introduced and is still in use.○ Advanced Linux Sound Architecture. ○ Alsa didn’t provide GUI tools and didn’t allow source sinks to be selected dynamically.

● Fedora 8 introduced Pulseaudio for audio mixing and redirection.○ User space daemon, which can handle software audio sources and sinks.

25


Current Linux support for sound devices● Alsa and pulseaudio can handle all modern audio devices

○ Integrated audio devices○ HDMI audio○ USB-audio devices○ Bluetooth audio with A2DP codec (Apt-X proprietary bluetooth codec does not work yet)○ Network streamed audio○ 16-bit and 24-bit audio, 48 kHz, ○ Surround setups with multiple channels: 2.1, 5.1, and 7.1.○ Passthrough digital audio: PCM, AC-5.1, Dolby Digital○ Midi devices.

● Input and output devices are dynamically selected and different software can use different sinks and sources at the same time.

26


ALSA: Advanced Linux Sound Architecture● Kernel internal and external API for sound drivers.● Provides a library for user space applications.

○ However, only used by Linux● A plugin architecture for user space audio drivers.

○ Not used by anything anymore.○ There used to be a library based dmix software audio mixer.

● Backwards compatible with OSS, although support is now disabled by default.● Command line:

○ Hardware mixer configuration: alsamixer○ List physical audio devices: aplay -l○ List all virtual audio devices: aplay -L

● Access to alsa sound devices in /dev/snd/* is managed by the OS.

27


Pulseaudio● The GUI manageable part of Linux audio stack.● One pulseaudio daemon shared between all processes in an user session.● Has an ALSA plugin that directs ALSA audio to pulseaudio daemon.● Mixes and redirects audio sources to audio sinks.● Adds latency to audio.● Configuration: /etc/pulse

● From command line: pactl

● From GUI: pavucontrol

28


More Linux audio systems● OpenAL as a library to produce 3D surround audio effects.

○ For games.● ESD Enlightened Sound Daemon was used before pulseaudio to mix audio and

transfer audio over network. Pulseaudio is compatible with it.● Jackd: audio daemon for professional studio use.

○ Aims at low latency.● Phonon and Arts: yet more audio daemons.

29

X-server and the GUI

31


X server● The X Window System (aka X11) is a client/server network protocol that's

been used for decades on a variety of different hardware platforms. It has been implemented by a number of different vendors for a wide variety of hardware platforms. (ubuntu wiki)

● X server is a root owned server process that has access to the GPU driver interfaces and input devices and which provides graphical rendering and input events to GUI client processes.

● Graphical software connect the X server to render graphics on the GPU.● Server info: xdpyinfo

● Server binary: /usr/bin/Xorg

● Server logs: /var/log/Xorg.0.log

32


X protocol● X originated at MIT in 1984● The protocol maps graphical operations to X requests and replies.

○ Works over a TCP or UNIX socket. Can be forwarded over ssh connection.○ X server used to be a separate device from the Unix server where the GUI software was run.○ Provides no GUI widgets, no desktops, no window management, not even proper font rendering.

● Provides input events from keyboards and from mice.○ Including keyboard and mice grabs and generated synthetic events.○ … which means that access to X server should only be allowed for trusted software.

● C library libX11 implements client side X protocol.● Server location specified by DISPLAY environment variable.● Authentication data is provided by XAUTHORITY environment variable.

33


X protocol extensions● The X server protocol is designed to be extensible

○ Which allows it to exists still today.● XRender: allow images with transparency to be rendered

○ Needed for proper font rendering.● Composite: allow windows to be drawn in bitmaps in memory● XINERAMA: support moving windows on multiple displays● RANDR: allow changing of display resolution● DRM2 and DRM3: Allow Direct Rendering from user space process to GPU● GLX: Provide OpenGL support● XVideo: Video scaling by GPU

34


X toolkits● Since X provides only graphical primitives the implementation of GUI toolkits

and widgets has been left to separate libraries.● Old commercial Unixes used the Motif toolkit.

○ Worked well, but was commercial and expensive. ● GTK toolkit is the most used LGPL toolkit from Gnome project.

○ Gimp toolkit.○ Written in C, but bindings exists for almost all programming languages.○ libgtk3.so

● Qt toolkit is a mature and old C++ toolkit used by the KDE project.○ Dual licensed with GPL and commercial licenses.○ Was used and bought by Nokia work Maemo/Meego project.○ libQt5*

35


X Window manager● X does not specify how windows should look and feel.● Instead there is a special X client software who manages windows and their

decorations.● All major desktop environments have their own windows managers with

window behaviour and decorations integrated:○ Gnome: Mutter○ Cinnamon: Cinnamon○ KDE: KWin○ Mate: Marco○ XFCE: xfvwm4

36


Display manager● Display manager is the software that provides graphical login services (greeter)

○ Usually just asks for user account and password, but can be configured with PAM modules.○ The login service is run with root access rights.

■ The greeter which provides the login GUI can be run by other user account.○ Can be accessed from command line with dbus-api: dm-tool○ Allow user to select the desired desktop environment.○ Allows changing language and keyboard layout.○ Managing network connections and shutting down system without login.○ Can provide guest login and manage multiple logins. ○ GDM, KDM, Lightdm

● After user has been authenticated display manager switches to the logged in user account, runs the common session startup scripts and starts login session.

37


X session startup● Common X session startup scripts are located at /etc/Xsession.d/*● All scripts are sourced in order before the session is started.

○ Sets environment variables for all processes in X session.● The scripts start some daemons needed for a session.● Starts gpg and ssh key management agents.● In the future, might be replaced with systemd session scripts.● Saves startup scripts logs and errors in .xsession-errors

38


Fonts: Freetype and Fontconfig● The X core protocol had support for fonts, but the support mechanism is useless

for modern interfaces.● Modern toolkits render fonts with Freetype font library using the XRender

extension. Fonts are rendered in use space applications. ● Freetype supports old bitmap fonts, Type 1 Adobe Postscripts fonts and

TrueType.○ Freetype also supports font bytecode hinting, automatic hinting and subpixel antialiasing.

● Fontconfig is a library for listing and finding the available fonts. ● Linux distributions install a large list of quality fonts by default.● List available Fontconfig fonts: fc-list● If you need access to MS core fonts, install ttf-mscorefonts-installer

39


Desktop environments● The Linux Desktop environments consists of all software daemons and GUI

toolkits and configuration settings needed to run a working GUI environment with a common look and feel.

○ toolkit, a toolkit configuration (with multiple themes), a window manager, panels, ...● KDE: K Desktop Environment

○ A system based on the Qt toolkit. ● Gnome: Gnome project Desktop environment

○ Based on GTK toolkit● Mate: Fork of Gnome version 2, without trying to reinvent the metaphora● Cinnamon: Fork of Gnome 3, without trying to reinvent the metaphora● Unity: old Canonical project based on GTK. Now dead.

40


Direct rendering manager (DRM)● The traditional X server architecture where graphics were drawn by a separate

server became too slow and cumbersome when 3D accelerated graphics cards became available.

● Direct rendering with DRM X extensions allow Linux GPU drivers to issue commands directly to GPU hardware through kernel DRM interface.

○ The GPU driver is implemented as a OpenGL library, which translates OpenGL primitives to GPU commands and sends them directly to the GPU.

○ This does not work with X session forwarding over sockets or ssh.○ Provides a very large API with direct access to DMA capable hardware.

■ Drivers try to protect the OS from security exploits, but it is likely that security holes are always available.

41


GPU Drivers● Intel, Nvidia, and AMD all support Linux and write Linux GPU drivers.● Intel OS driver: works well, saves battery, is slow

○ Available by default on distributions. ○ OpenGL for graphics and VAAPI for video encoding and decoding.

● Nvidia proprietary driver: works well, uses lots of power, is the fastest○ Needs separate installation. Needs DKMS recompile for all new kernels.○ OpenGL for graphics, nvdec and nvenc APIs for video encoding and decoding○ Provides CUDA drivers for GPU computation○ Works well, is the fastest driver available

● AMDGPU: works well, good for OS gaming without separate driver○ OS driver available by default on distributions. Separate “pro” proprietary downloadable.○ OpenGL for graphics, VAAPI for video encoding and decoding, OpenCL for computation

43


GPU Drivers● Intel, Nvidia, and AMD all support Linux and write Linux GPU drivers.● Intel OS driver: works well, saves battery, is slow

○ Available by default on distributions. ○ OpenGL for graphics and VAAPI for video encoding and decoding.

● Nvidia proprietary driver: works well, uses lots of power, is the fastest○ Needs separate installation. Needs DKMS recompile for all new kernels.○ OpenGL for graphics, nvdec and nvenc APIs for video encoding and decoding○ Provides CUDA drivers for GPU computation○ Works well, is the fastest driver available

● AMDGPU: works well, good for OS gaming without separate driver○ OS driver available by default on distributions. Separate “pro” proprietary downloadable.○ OpenGL for graphics, VAAPI for video encoding and decoding, OpenCL for computation

44


Video decoding● As of 2019 all major Linux GPU drivers support hardware video decoding.

○ Also, new machines are fast enough that video decoding isn’t strictly necessary.● Xvideo: the extension for scaling live video to screen.

○ /usr/bin/xvinfo

● VA API: Video Acceleration API for Intel and AMD GPU’s○ royalty-free API as well as its implementation as free and open-source library (libVA)

distributed under the MIT License.○ /usr/bin/vainfo

● NVDECODE/NVENCODE: proprietary APIs for hardware video acceleration○ NVIDIA Fermi, Kepler, Maxwell and Pascal generation GPUs.

● VDPAU: Old Nvidia API for video decoding. No updates since 2015.○ /usr/bin/vdpauinfo

45


Media software and libraries● VLC: easy to use GUI software for video viewing.

○ Support passthrough audio and hardware video decoding.● Mplayer: the command line video software which supports everything.

○ Including video encoding with mencoder● Handbrake: video encoder with a nice GUI.● Ffmpeg: general purpose library and programs for digital media decoding and

conversion.● X264: A library for h264 decoding and encoding using CPU. The best tool for

h264 available, however does not support GPU acceleration.

46

course Linux system administration · Linux system administration, 2nd slide set © Jani Jaakkola...

Documents

Transcript of course Linux system administration · Linux system administration, 2nd slide set © Jani Jaakkola...