Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 April 2019
H17702
Deployment Guide
Abstract
This deployment guide describes how to create a Red Hat OpenShift Container Platform environment on Dell EMC infrastructure for a highly available production deployment. The guide includes sample configurations and modifiable Ansible scripts.
Dell EMC Solutions
Copyright
2 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11
Deployment Guide
The information in this publication is provided as is. Dell Inc. makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.
Use, copying, and distribution of any software described in this publication requires an applicable software license.
Copyright © 2019 Dell Inc. or its subsidiaries. All Rights Reserved. Dell Technologies, Dell, EMC, Dell EMC and other trademarks are trademarks of Dell Inc. or its subsidiaries. Intel, the Intel logo, the Intel Inside logo and Xeon are trademarks of Intel Corporation in the U.S. and/or other countries. Other trademarks may be trademarks of their respective owners. Published in the USA 04/19 Deployment Guide H17702.
Dell Inc. believes the information in this document is accurate as of its publication date. The information is subject to change without notice.
Contents
3 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Contents
Chapter 1 Introduction 5
Solution overview .................................................................................................. 6
Document purpose ................................................................................................ 6
Audience ............................................................................................................... 7
We value your feedback ........................................................................................ 7
Chapter 2 Before You Start 8
Introduction ........................................................................................................... 9
Requirements and assumptions ............................................................................ 9
Cabling ................................................................................................................ 10
Preinstallation steps ............................................................................................ 11
Chapter 3 Configuring the Switches 14
Introduction ......................................................................................................... 15
Customizing sample files..................................................................................... 15
Customizing the Dell EMC Networking switches ................................................. 17
Chapter 4 Installing and Preparing the Bastion Node 20
Introduction ......................................................................................................... 21
Preparing the installation ..................................................................................... 21
Installing Red Hat Enterprise Linux 7.6 on the bastion node .............................. 22
Configuring the bastion node .............................................................................. 25
Chapter 5 Preparing Cluster Servers for OpenShift Installation 26
Introduction ......................................................................................................... 27
Preparing the cluster nodes ................................................................................ 27
Chapter 6 Deploying the OpenShift Cluster 29
Introduction ......................................................................................................... 30
Preparing for deployment .................................................................................... 30
Deploying the cluster ........................................................................................... 32
Chapter 7 Creating OpenShift User Accounts 33
Introduction ......................................................................................................... 34
Logging in to OpenShift ....................................................................................... 34
Creating user accounts and setting cluster-admin roles ...................................... 34
Chapter 8 Creating the Default Storage Class 37
Introduction ......................................................................................................... 38
Contents
4 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Creating the default storage class ....................................................................... 38
Chapter 9 Validating the OpenShift Container Platform Deployment 39
Introduction ......................................................................................................... 40
Validating the deployment ................................................................................... 40
Chapter 10 Validating OpenShift Operation 47
Introduction ......................................................................................................... 48
Deploying a sample application: ruby-ex ............................................................. 48
Deploying a sample application: Lab-Guide-Workshop ....................................... 51
Appendix A Sample Switch Configuration Files 57
Introduction ......................................................................................................... 58
Downloading the sample files .............................................................................. 58
Modifying the deployment control file .................................................................. 61
Switch configuration file examples ...................................................................... 65
Appendix B All-In-One Installation 100
Introduction ....................................................................................................... 101
Deploying OpenShift All-in-One using a single server ....................................... 101
Installing Red Hat Enterprise Linux 7.6 ............................................................. 102
Creating Docker storage ................................................................................... 104
Installing OpenShift 3.11 AIO on a PowerEdge R640 server ............................. 105
Deploying a sample Kubernetes application ...................................................... 110
Chapter 1: Introduction
5 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Chapter 1 Introduction
This chapter presents the following topics:
Solution overview ............................................................................................... 6
Document purpose ............................................................................................. 6
Audience .............................................................................................................. 7
We value your feedback ..................................................................................... 7
Chapter 1: Introduction
6 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Solution overview
Red Hat OpenShift Container Platform is an open-source application deployment platform
that is based on Kubernetes container orchestration technology. Containers are
standalone processes that run within their own environment, independent of the operating
system and underlying infrastructure. Red Hat OpenShift Container Platform helps you
develop, deploy, and manage container-based applications.
As part of Red Hat OpenShift Container Platform, Kubernetes manages containerized
applications across a set of containers or hosts and provides mechanisms for the
deployment, maintenance, and scaling of applications. The container run-time engine
packages, instantiates, and runs containerized applications.
A Kubernetes cluster consists of one or more masters and a set of nodes. Kubernetes
allocates to each pod an IP address from an internal network so that all containers within
the pod behave as if they were on the same host. Giving each pod its own IP address
means that pods can be treated like physical hosts or virtual machines for port allocation,
networking, naming, service discovery, load balancing, application configuration, and
migration. Dell EMC recommends creating a service that enables your pods to interact
rather than letting the pods talk to one another directly using their IP addresses.
A fully functioning Domain Name System (DNS) is crucial in the deployment and operation
of your container ecosystem. Red Hat OpenShift Container Platform has a built-in DNS so
that the services can be found through DNS switched virtual circuit (SVC) entries or
through the service IP/port registrations.
The Dell EMC Ready Architecture for Red Hat OpenShift Container Platform is a proven
design to help companies accelerate their container deployments and cloud-native
adoption. Dell EMC delivers tested, validated, and documented design guidance to help
customers rapidly deploy Red Hat OpenShift on Dell EMC infrastructure by minimizing
time and effort. For details, see the architecture guide on the OpenShift Container
Platform Info Hub for Ready Solutions.
Document purpose
This deployment guide describes the required infrastructure for deploying and operating
Red Hat OpenShift Container Platform and the validated deployment process for a
production-ready OpenShift Container Platform cluster. It provides the information that
you need to facilitate readiness for day 2 operations. Topics include:
• Containers and images
• Pods and services
• Projects and users
• Builds and image streams
• Deployments
• Routes
• Templates
Chapter 1: Introduction
7 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
For more information about OpenShift Container Platform, see the Overview on the
OpenShift website.
This guide provides a reference design for deploying version 3.11 of Red Hat OpenShift
Container Platform on Dell EMC PowerEdge servers and Dell EMC Networking switches.
Dell EMC strongly recommends that you complete the validation steps that are described
in this guide and ensure that you are satisfied that your application will operate smoothly
before proceeding with development or production use.
Audience
This deployment guide is for system administrators and system architects. Read the
solution reference architecture guide to familiarize yourself with the solution architecture
and design before planning your deployment. Some experience with Docker and Red Hat
OpenShift Container Platform technologies is recommended.
Appendix B All-in-One Installation provides basic instructions for installing a single-
node deployment of Red Hat OpenShift Container Platform. Red Hat refers to this
single-node deployment as “All-in-One” (AIO). These instructions are provided to
enable you to test-drive the deployment before starting the process of installing a full
high availability (HA) cluster.
We value your feedback
Dell EMC and the authors of this document welcome your feedback on the solution and
the solution documentation. Contact the Dell EMC Solutions team by email or provide
your comments by completing our documentation survey.
Authors: Michael Tondee, Scott Powers, John Terpstra, Aighne Kearney
Note: OpenShift Container Platform Info Hub for Ready Solutions on the Dell EMC Communities
website provides links to additional documentation for this solution.
Chapter 2: Before You Start
8 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Chapter 2 Before You Start
This chapter presents the following topics:
Introduction ......................................................................................................... 9
Requirements and assumptions ........................................................................ 9
Cabling ............................................................................................................... 10
Preinstallation steps ......................................................................................... 11
Chapter 2: Before You Start
9 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Introduction
This chapter provides information about the typographical conventions that are used in
this guide and describes the planning and preparation that you must complete before you
deploy the Red Hat OpenShift Container Platform cluster.
This guide uses certain typographical conventions to mark commands and command
output:
• Information that is specific to your environment is placed inside <> symbols and
presented in italics. For example: OS10(config)# hostname <hostname>
• Command output is presented with a grey background.
Before you begin, ensure that you have reviewed the following questions:
• Do you have the correct Dell EMC servers and switches and are they equipped with
the resources necessary to support your production workload?
Note the following requirements:
▪ Use only Dell EMC PowerEdge R640 or PowerEdge R740xd server nodes. Images
are tagged during Red Hat OpenShift Container Platform deployment, and the
deployment scripts cannot differentiate between mixed systems for a given node
type.
▪ Manually install and update the image streams. Image streams and templates
are not installed by default or updated when you upgrade the system.
▪ Install on-premises PowerEdge servers. Do not attempt to install OpenShift
Container Platform on nodes from a cloud provider.
• How many pods do you need to deploy and run concurrently within the cluster?
See Planning your installation on the OpenShift website.
• How many server nodes will you require in the cluster?
See Planning your installation on the OpenShift website.
• How do you plan to monitor your cluster?
See Prometheus Cluster Monitoring on the OpenShift website.
Before starting the deployment, read the following documents:
• Dell EMC Red Hat Intel OpenShift Architecture and Deployment Guide (includes
hardware design and ordering information)
• Relevant sections of the OpenShift Container Platform 3.11 Documentation
Requirements and assumptions
The customer environment must have:
▪ Red Hat Enterprise Linux Server subscriptions with OpenShift-3.11 Enterprise
▪ RHEL-7.6-20181010.0-Server-x86_64-dvd1.iso
Typographical
conventions
Installation and
configuration
checklist
Essential
reading
Chapter 2: Before You Start
10 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
▪ Eleven PowerEdge R640 servers with integrated Dell Remote Access Controller
(iDRAC) Enterprise licenses
▪ Four PowerEdge R740xd servers with iDRAC Enterprise licenses
▪ Static IP addresses set for iDRAC NICs and cabled to the Dell EMC Networking
S3048-ON switch that is uplinked to the data switches
▪ Between two and four 10 GbE or 25 GbE NICs
Note: This Ready Architecture uses four 25 GbE NICs in bond mode=4.
▪ Switches supporting LACP port channels with untagged and tagged VLANs on the
same interface
The Dell EMC Ready Architecture for Red Hat OpenShift Container Platform uses Dell
EMC Networking S5232F-ON switches. These switches are referred to as S5232F-1
and S5232F-2 in this guide.
▪ DNS wildcard entries for the *.DomainX that resolve to the openshift_cluster_ip
address in /etc/ansible/hosts
▪ Internet access to download packages and updates
The deployment procedures in this guide assume that:
• Servers and networking switches are racked, and power and network cabling has
been completed in accordance with the instructions in the architecture guide.
• Static IP addresses are set on the iDRAC NICs, and the NICs are cabled to the
S3048-ON switch.
Cabling
Cable the servers to switches, spreading a port from each dual-port adapter across the
network switches. The following table lists the recommended cabling between the NICs on
the server network daughter card and the switches.
Table 1. Server/switch wiring recommendations for integrated NICs
Server S5232F-1 S5232F-2 Switch
Ethernet port
Switch port
channel
Switch untagged
VLAN
Switch tagged VLAN
bastion em1 em2 1/1/1:1 1 20 10, 421
master1 em1 em2 1/1/1:2 2 20
master2 em1 em2 1/1/1:3 3 20
master3 em1 em2 1/1/1:4 4 20
infra1 em1 em2 1/1/2:1 5 20 421
infra2 em1 em2 1/1/2:2 6 20 421
infra3 em1 em2 1/1/2:3 7 20 421
app1 em1 em2 1/1/2:4 8 20
Chapter 2: Before You Start
11 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Server S5232F-1 S5232F-2 Switch
Ethernet port
Switch port
channel
Switch untagged
VLAN
Switch tagged VLAN
app2 em1 em2 1/1/3:1 9 20
app3 em1 em2 1/1/3:2 10 20
app4 em1 em2 1/1/3:3 11 20
stor1 em1 em2 1/1/3:4 12 20
stor2 em1 em2 1/1/4:1 13 20
stor3 em1 em2 1/1/4:2 14 20
stor4 em1 em2 1/1/4:3 15 20
The following table shows the recommended cabling for the PCIe NICs.
Table 2. Server/switch wiring recommendations for add-in NICs
Server S5232F-1 S5232F-2
Switch
Ethernet port
Switch port
channel
Switch untagged
VLAN
Switch tagged
VLAN
bastion p2p1 p2p2 1/1/9:1 1 20 10, 421
master1 p2p1 p2p2 1/1/9:2 2 20
master2 p2p1 p2p2 1/1/9:3 3 20
master3 p2p1 p2p2 1/1/9:4 4 20
infra1 p2p1 p2p2 1/1/10:1 5 20 421
infra2 p2p1 p2p2 1/1/10:2 6 20 421
infra3 p2p1 p2p2 1/1/10:3 7 20 421
app1 p2p1 p2p2 1/1/10:4 8 20
app2 p2p1 p2p2 1/1/11:1 9 20
app3 p2p1 p2p2 1/1/11:2 10 20
app4 p2p1 p2p2 1/1/11:3 11 20
stor1 p7p1 p7p2 1/1/11:4 12 20
stor2 p7p1 p7p2 1/1/12:1 13 20
stor3 p7p1 p7p2 1/1/12:2 14 20
stor4 p7p1 p7p2 1/1/12:3 15 20
Preinstallation steps
Before starting the installation, complete these steps:
1. From this github repository, download the sample scripts that Dell EMC has
provided for the installation.
Chapter 2: Before You Start
12 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
2. Modify the S5232F-1, S5232F-2, and S3048 files to match your networks and
infrastructure, and save the modified files.
3. Configure each of the switches using the modified files that you saved in the
preceding step.
4. Install the bastion node by using iDRAC remote console with Virtual Media
attached to RHEL-7.6-20181010.0-Server-x86_64-dvd1.iso.
5. On the bastion node:
Register Red Hat Enterprise Linux 7.6 and attach the Red Hat pool ID with
OpenShift subscriptions.
Install the packages and Red Hat Enterprise Linux 7.6 updates, and then
reboot and prepare RHEL 7.6 iso.
Run the following command:
# cd /root
# git clone https://github.com/dell-esg/openshift-bare-
metal/
6. Modify examples/hosts.example.com to match your infrastructure, and then
copy the file to the bastion node file /etc/ansible/hosts.
Note: Your Dell Service Tags are required in the updated hosts file.
7. From the bastion node, run the Ansible scripts that install Red Hat Enterprise
Linux 7.6 and OpenShift 3.11 onto the other 14 PowerEdge servers:
# ansible-playbook src/prerequisites/nodes_setup.yaml -k
# ansible-playbook src/keepalived-
multimaster/keepalived.yaml
# ansible-playbook /usr/share/ansible/openshift-
ansible/playbooks/prerequisites.yml
# ansible-playbook /usr/share/ansible/openshift-
ansible/playbooks/deploy_cluster.yml
# ansible-playbook src/ocs-pool/storage-class.yml
8. Log in as user system:admin, and then:
Create new OpenShift users.
Set htpasswd passwords for the users.
Add the users to the cluster-admin role.
9. Deploy a sample application and validate that the application routes are
accessible on the Red Hat OpenShift Container Platform public network.
Chapter 2: Before You Start
13 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
To uninstall OpenShift dependencies, run the following command:
ansible-playbook /usr/share/ansible/openshift-
ansible/playbooks/adhoc/uninstall.yml
To clean glusterfs disks using values from /etc/ansible/hosts, run the following
command:
ansible-playbook -i /etc/ansible/hosts -e
"openshift_storage_glusterfs_wipe=True" -e
"openshift_storage_glusterfs_registry_wipe=true"
/usr/share/ansible/openshift-ansible/playbooks/openshift-
glusterfs/uninstall.yml
Troubleshooting
Chapter 3: Configuring the Switches
14 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Chapter 3 Configuring the Switches
This chapter presents the following topics:
Introduction ....................................................................................................... 15
Customizing sample files ................................................................................. 15
Customizing the Dell EMC Networking switches ............................................ 17
Chapter 3: Configuring the Switches
15 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Introduction
This chapter describes how to customize the Dell EMC-provided sample configuration
files for your environment.
The sample configuration files and Ansible scripts that we used to validate the deployment
are available in a github repository. These files enable you to easily configure multiple
complex OpenShift 3.11 components. Before a production installation, you can modify
these files and scripts, in consultation with Dell EMC and Red Hat, to meet your
requirements.
CAUTION: If you use different hardware or need different configurations, you must modify the
scripts to ensure proper solution installation and functioning.
Customizing sample files
To customize sample files:
1. Browse to the github repository and select Current Branch: 3.11.
2. Look for the following directory:
/root/openshift-bare-metal/examples/
The /root/openshift-bare-metal/examples/example.xlsx file
contains the server and switch details that are used to document changes in your
installation.
3. Download the sample files and modify them as necessary to match your
environment.
Alternatively, you can clone the github.com repository from an existing Linux
system by running the following command:
cd /root
git clone https://github.com/dell-esg/openshift-bare-metal
4. Save the modified files.
Note:
▪ Kickstart examples are modified and used to install Red Hat Enterprise Linux
7.6 on the bastion and AIO nodes.
▪ Switch examples are modified, copied to running-configuration, and saved to
each switch, enabling you to quickly configure the three Dell EMC Networking
switches for the example.com deployment.
The following table on the Inventory tab in the example.xlsx file shows the
names, IP addresses, and switch details for your deployment.
Chapter 3: Configuring the Switches
16 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Table 3. Ready Architecture deployment servers
Name Hardware iDRAC VLAN-10 192.168.10.0/24 gw:192.168.10.1
Internal VLAN-20 192.168.20.0/24 gw:192.168.20.1
Public VLAN-421 100.82.42.0/24 gw:100.82.42.1
DNS: 100.82.42.8
W2008R2 VM 192.168.10.8 192.168.20.8 100.82.42.8
Existing Linux system
Any system 100.82.42.9
RHEL01 Bastion
R640#1 192.168.10.11 192.168.20.11 100.82.42.11
RHEL02 master1
R640#2 192.168.10.12 192.168.20.12
RHEL03 master2
R640#3 192.168.10.13 192.168.20.13
RHEL04 master3
R640#4 192.168.10.14 192.168.20.14
RHEL05 infra1
R640#5 192.168.10.15 192.168.20.15 100.82.42.15
RHEL06 infra2
R640#6 192.168.10.16 192.168.20.16 100.82.42.16
RHEL07 infra3
R640#7 192.168.10.17 192.168.20.17 100.82.42.17
RHEL08 app1
R640#8 192.168.10.18 192.168.20.18
RHEL09 app2
R640#9 192.168.10.19 192.168.20.19
RHEL10 app3
R640#10 192.168.10.20 192.168.20.20
RHEL11 app4
R640#11 192.168.10.21 192.168.20.21
RHEL12 stor1
R740xd#1 192.168.10.22 192.168.20.22
RHEL13 stor2
R740xd#2 192.168.10.23 192.168.20.23
RHEL14 stor3
R740xd#3 192.168.10.24 192.168.20.24
RHEL15 stor4
R740xd#4 192.168.10.25 192.168.20.25
S5232F-1 S5232F-ON vrrp: 192.168.10.1 vrrp: 192.168.20.1
S5232F-2 S5232F-ON vrrp: 192.168.10.1 vrrp: 192.168.20.1
5-S3048 S3048-ON
Chapter 3: Configuring the Switches
17 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
The following table shows the recommended processor model, memory capacity, and
firmware versions that we used in the installation.
Table 4. Recommended firmware versions
CPU model (all nodes)
Memory at 2,666 MHz
Dell EMC BIOS version
Dell EMC iDRAC and LifeCycle versions
Mellanox NIC firmware version
Dell EMC BOSS firmware version
2 x Intel Xeon Gold 6138
384 GB 1.6.12 3.21.26.22 14.23.10.20 2.5.13.3016
Customizing the Dell EMC Networking switches
To modify the sample files:
1. Update the S5232F-1 and S5232F-2 values to match your VLANs.
2. Place the modified S5232F-1 and S5232F-2 files on an existing Linux server that
can be reached on the switch out-of-band (OOB) management network.
3. Change the public VLAN 421 and network CIDR to a routable network in your
infrastructure that has Internet access.
4. Modify the switch OOB management network VLAN and CIDR to match the
switch OOB management network at your site.
The following table shows the networks and settings that we used in the ocp.example.com
installation.
Table 5. Sample installation networks and settings
VLAN number
Network Purpose Gateway location
10 192.168.10.0/24 iDRAC S5232F via VRRP= 192.168.10.1
20 192.168.20.0/24 Internal S5232F via VRRP= 192.168.20.1
421 100.82.42.0/24 Public External= 100.82.42.1
33 100.82.33.0/24 Switch management
network
External= 100.82.33.1
(management route 0.0.0.0/0 for each Dell EMC Switch)
The initial switch configuration settings are:
• Dell Networking serial port baud-rate speed: 115200
• Dell Networking OS10 Enterprise default passwords switch login:
▪ Username: admin
▪ Password: admin
Modifying the
sample files
Configuring the
switches
Chapter 3: Configuring the Switches
18 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
To perform the configuration:
1. Copy the updated switch configuration files to each switch and save the
configuration.
2. Power on the switches, connect to the serial debug port, and set a static IP
address.
3. Log in to each Dell EMC Networking switch, set the static IP address for the
switch OOB management port, copy the switch examples to running-
configuration, and save the switch configuration.
The following code sample shows switch S5232F-1:
S5232F-1# configure terminal
S5232F-1(config)# interface mgmt 1/1/1
S5232F-1(conf-if-ma-1/1/1)# no shutdown
S5232F-1(conf-if-ma-1/1/1)# no ip address dhcp
S5232F-1(conf-if-ma-1/1/1)# ip address 100.82.33.81/24
S5232F-1(conf-if-ma-1/1/1)# exit
S5232F-1(config)# management route 0.0.0.0/0 100.82.33.1
S5232F-1# copy scp://root@hostip/root/openshift-bare-
metal/examples/S5232F-1 running-configuration
S5232F-1# write memory
S5232F-1# reload
S5232F-2 switch
The following code sample shows how to configure the S5232F-2 switch:
S5232F-2# configure terminal
S5232F-2(config)# interface mgmt 1/1/1
S5232F-2(conf-if-ma-1/1/1)# no shutdown
S5232F-2(conf-if-ma-1/1/1)# no ip address dhcp
S5232F-2(conf-if-ma-1/1/1)# ip address 100.82.33.82/24
S5232F-2(conf-if-ma-1/1/1)# exit
S5232F-2(config)# management route 0.0.0.0/0 100.82.33.1
S5232F-2# copy scp://root@hostip/root/openshift-bare-
metal/examples/S5232F-2 running-configuration
S5232F-2# write memory
Chapter 3: Configuring the Switches
19 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
S5232F-2# reload
S3048 switch
The following code sample shows how to configure the S3048 switch:
S3048# configure terminal
S3048(config)# interface mgmt 1/1
S3048(conf-if-ma-1/1/1)# no shutdown
S3048(conf-if-ma-1/1/1)# ip address 100.82.33.83/24
S3048(conf-if-ma-1/1/1)# exit
S3048(config)# management route 0.0.0.0/0 100.82.33.1
S3048# copy scp://root@hostip//root/openshift-bare-
metal/examples/S3048 running-configuration
S3048# write memory
S3048# reload
Chapter 4: Installing and Preparing the Bastion Node
20 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Chapter 4 Installing and Preparing the Bastion Node
This chapter presents the following topics:
Introduction ....................................................................................................... 21
Preparing the installation ................................................................................. 21
Installing Red Hat Enterprise Linux 7.6 on the bastion node ........................ 22
Configuring the bastion node .......................................................................... 25
Chapter 4: Installing and Preparing the Bastion Node
21 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Introduction
This chapter describes how to install Red Hat Enterprise Linux 7.6 on the bastion node
using automated kickstart files and the iDRAC remote console and how to configure the
bastion node for deployment.
Preparing the installation
Before you begin installing the Red Hat OpenShift Container Platform on your cluster, you
must install and configure the bastion node, which launches the next steps in the
deployment process. Choose either of the following options to use kickstart with the
iDRAC remote console.
To prepare and use an image file for use with the iDRAC remote console:
1. From an existing Red Hat Enterprise Linux 7.6 system, create a USB image by
running the following command:
mkfs.vfat -C ks_usb.img 1024
2. Mount the image by running the following command:
mount -o loop ks_usb.img /mnt
3. Place the customized bastion.ks file into the image by running the following
command:
cp bastion.ks /mnt
4. Sync and unmount the image by running the following command:
sync; umount /mnt
5. Copy ks_usb.img to the machine that will log in to the server’s iDRAC.
During the installation, you will attach this file to removable media in the iDRAC
remote console’s Virtual Media.
To prepare a USB key for use with the iDRAC remote console:
1. From an existing Red Hat Enterprise Linux 7.6 system, format a USB key by
running the following command:
mkfs.ext3 /dev/sdb
2. Mount the USB key:
mount /dev/sdb /mnt
3. Place the customized sample bastion.ks file onto the USB key:
cp bastion.ks /mnt
4. Sync and unmount the image by running the following command:
sync; umount /mnt
Option 1: Using
an image file
Option 2: Using
a USB key
Chapter 4: Installing and Preparing the Bastion Node
22 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
5. Plug the USB key into the bastion server before you power it on, and then log in
to the server’s iDRAC.
During the operating system installation, you will select the USB location to the
bastion.ks on the USB key in the iDRAC remote console.
Installing Red Hat Enterprise Linux 7.6 on the bastion node
To install Red Hat Enterprise Linux 7.6 on the bastion node:
1. Log in to the iDRAC web interface of the R640 PowerEdge server and launch a
remote console.
2. On the Virtual Media tab, connect virtual media and select Map CD/DVD.
3. Browse to the location of the Red Hat Enterprise Linux Server 7.6 installation
media and select Map Device.
4. Use either the ks_usb.img file (Option 1) or the USB key you prepared earlier
(Option 2):
▪ Option 1: In the iDRAC remote console, on the Virtual Media tab, select Map
Removable Disk. Browse to the location of the ks_usb.img file and select
Map Device.
▪ Option 2: Insert the USB key into a USB port on the R640 server.
sda = BOSS RAID1 as OS drive
sdb = ks_usb_img mounted to iDRAC
5. Select Next Boot, set Virtual CD/DVD/ISO as the next boot option.
6. Power on the system and click OK on the remote console.
7. From the Red Hat Enterprise Linux 7.6 installation menu, select Install.
8. Do not press the [Enter] key. Press the [e] key.
9. Move the cursor to the end of the line that begins with vmlinuz, and append the
following code to the end of the line:
ks=hd:sdb:/bastion.ks
Note: If only one physical RAID, SAS, or SATA disk is presented to the server, use the
device name sdb for the kickstart location in the Red Hat Enterprise Linux 7 installer.
Note: NVMe disks are enumerated as nvme0n1, nvme1n1, nvme2n1, and nvme3n1.
The following figure shows the installation command.
Chapter 4: Installing and Preparing the Bastion Node
23 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Figure 1. Installation command
10. Press [CTRL+X] to start the installation.
11. Reboot the server and accept the license.
12. Log in and verify that the bonds and networks are functioning.
13. Start an SSH session to the bastion server.
14. To register with Red Hat, attach the pool ID with OpenShift subscriptions, and
install the supporting packages, run the following commands:
subscription-manager register --username Your_RH_account --
password RH_password --force
subscription-manager attach --
pool=xxxxxxxxxxxxxxxxxxxxxxxxxxxx
subscription-manager repos --enable="rhel-7-server-rpms" --
enable="rhel-7-server-extras-rpms" --enable="rhel-7-server-
ose-3.11-rpms" --enable="rhel-7-server-ansible-2.6-rpms"
yum -y install wget git net-tools bind-utils iptables-
services bridge-utils bash-completion kexec-tools sos psacct
yum -y install openshift-ansible
yum -y install docker-1.13.1
Chapter 4: Installing and Preparing the Bastion Node
24 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
If partitions with previous data exist on the disks, clean the disks by running the following
command:
dd if=/dev/zero of=/dev/nvmeXn1 bs=1M count=1
Replace X with the target drive number such as 0, 1, 2, 3.
Example: dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=1
Start an SSH session to the bastion node, and then follow these steps:
1. Run the following commands as root:
cd /etc/sysconfig/
vi docker-storage-setup
2. Edit this file as follows:
cat /etc/sysconfig/docker-storage-setup
VG=docker-vg
WIPE_SIGNATURES=true
3. Save your changes.
4. Configure Docker storage by running the following command:
docker-storage-setup
Rounding up size to full physical extent 1.49 GiB
Thin pool volume with chunk size 512.00 KiB can address at
most 126.50 TiB of data. Logical volume "docker-pool"
created. Logical volume docker-vg/docker-pool changed.
Enable and start Docker by running the following commands:
systemctl enable docker
systemctl start docker
systemctl is-active docker
5. Log in to the Docker registry and install atomic by running the following
commands:
docker login -u Your_RH_account -p Your_RH_password
docker login https://registry.redhat.io
yum -y install atomic
yum -y install atomic-openshift-clients
mkdir /etc/dnsmasq.d
6. Install the Red Hat Enterprise Linux updates and reboot the bastion server:
yum clean all
yum makecache
yum -y update
reboot
Preparing the
node for
OpenShift
installation
Creating Docker
storage for the
bastion node
Chapter 4: Installing and Preparing the Bastion Node
25 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Configuring the bastion node
The Dell EMC Ready Architecture for the Red Hat OpenShift Container Platform uses Red
Hat Enterprise Linux (Red Hat Enterprise Linux) as the bare-metal operating system for all
nodes. Setting up the bastion node involves obtaining the Red Hat Enterprise Linux 7.6
ISO image from the Red Hat customer portal and transferring it to the bastion node. For
automated provisioning, copy the image content to the appropriate directories.
Follow these steps:
1. Copy the Red Hat Enterprise Linux 7.6 ISO image to the bastion /root
directory, mount ISO, and then copy the contents to /tftp/rhel/media/ by
running the following commands:
cd /root/
mkdir -p /mnt/media
mount -o loop rhel-server-7.6-x86_64-dvd.iso /mnt/media
mkdir -p /tftp/rhel/media
cd /mnt/media
cp -r * /tftp/rhel/media/
2. On the bastion node, log in and git clone the github repo to /root/as follows:
cd /root
git clone https://github.com/dell-esg/openshift-bare-metal/
3. From the bastion node, run the Ansible script that creates iPXE, dnsmasq, and
the tftp containerized resources used to install Red Hat Enterprise Linux 7.6 on
the remaining servers:
cd /root/openshift-bare-metal
ansible-playbook src/ipxe-deployer/ipxe.yml
4. Run the docker ps command to verify that two Docker containers are running,
which indicates that the setup was successful.
5. Edit /etc/ansible/hosts file, remove the variable
openshift_hostname=xxxx, and then run the remaining Ansible scripts for the
remainder of the deployment.
Note: The Ansible script ipxe.yml requires the variable <openshift_hostname>.
However, the remaining Ansible scripts do not accept the <openshift_hostname>
variable in [NODES] section of the /etc/ansible/hosts file.
Chapter 5: Preparing Cluster Servers for OpenShift Installation
26 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Chapter 5 Preparing Cluster Servers for OpenShift Installation
This chapter presents the following topics:
Introduction ....................................................................................................... 27
Preparing the cluster nodes ............................................................................. 27
Chapter 5: Preparing Cluster Servers for OpenShift Installation
27 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Introduction
Afer you have configured the bastion node, you can prepare to install Red Hat Enterprise
Linux L 7.6 on all the cluster nodes and then deploy Red Hat OpenShift Container
Platform to each node.
This section describes how to install Red Hat Enterprise Linux 7.6 by pre-boot execution
(PXE) booting the servers. The process involves opening iDRAC remote consoles and
validating that:
• Each server completes the Red Hat Enterprise Linux 7.6 installation process
• The network interfaces are set up correctly
Preparing the cluster nodes
Prepare the cluster nodes as follows:
1. For each server, log in to the iDRAC website and open an iDRAC remote console.
2. Set Next boot to PXE.
3. Power on each server and monitor the progress of the automated Red Hat
Enterprise Linux 7.6 PXE installation on each server.
After the PXE installation is complete, the server reboots. Each server returns to
the login prompt and all network interfaces are configured, as shown in the
following figure.
Figure 2. Red Hat Enterprise Linux 7.6 post-installation page
Chapter 5: Preparing Cluster Servers for OpenShift Installation
28 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
4. After the Red Hat Enterprise Linux 7.6 installation is complete, ping each server
on the internal IP address from the bastion node to verify that the installation was
successful.
1. If a server fails to complete the PXE installation, reboot the server and retry the
installation.
2. If failure persists, check your settings.
3. If you find mistakes in your hosts file, re-run ipxe.yml to update the kickstart
files on the bastion node.
You can find the kickstart files for each node on the bastion node at
/tftp/rhel/media/.
Troubleshooting
Chapter 6: Deploying the OpenShift Cluster
29 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Chapter 6 Deploying the OpenShift Cluster
This chapter presents the following topics:
Introduction ....................................................................................................... 30
Preparing for deployment ................................................................................. 30
Deploying the cluster ........................................................................................ 32
Chapter 6: Deploying the OpenShift Cluster
30 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Introduction
This chapter describes how to prepare your systems for the OpenShift Platform
deployment and start deploying the cluster.
Preparing for deployment
Prepare for the deployment as follows:
1. From the bastion node, run the Ansible nodes_setup.yaml script:
cd /root/openshift-bare-metal/
ansible-playbook src/prerequisites/nodes_setup.yaml -k
2. When prompted, enter the root password for the nodes in your
/etc/ansible/hosts file.
The following figure shows the resulting output.
Figure 3. Ansible nodes_setup.yaml script start
3. From the bastion node, to enable the SSH prompt on all nodes, run the
sshyestoall.sh command:
cd /root/openshift-bare-metal/
chmod 755 *.sh
./sshyestoall.sh
You can now perform SSH from the bastion node to all nodes without SSH
prompting.
Chapter 6: Deploying the OpenShift Cluster
31 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Red Hat recommends using an external load balancer such as F5. For simplicity, the Dell
EMC Ready Architecture uses keepalived or haproxy on infra nodes. Consult Dell
EMC and Red Hat about any additional considerations affecting your environment.
To set up HA:
1. On the bastion node, in the Red Hat Ansible Automation inventory file at
/etc/ansible/hosts, define the following variables:
external_interface=bond0
external_vlan=421
internal_interface=bond0
openshift_master_cluster_ip=192.168.20.99
openshift_master_cluster_public_ip=100.82.42.99
2. From the bastion node, run the Ansible script keepalived.yaml as follows:
cd /root/openshift-bare-metal/ansible-playbook
src/keepalived-multimaster/keepalived.yaml
The following figure shows the completed keepalived.yaml with putty to infra1 tail
and -f /var/log/messages.
Figure 4. Completed keepalived.yaml script
The expected output is:
keepalived.sh exited with status 7
Note: The keepalived daemon will start working during cluster deployment.
Setting up high
availability
Chapter 6: Deploying the OpenShift Cluster
32 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
3. From the bastion node, run prerequisites.yml to install OpenShift
prerequisites to all the nodes:
cd /root/openshift-bare-metal/
ansible-playbook /usr/share/ansible/openshift-
ansible/playbooks/prerequisites.yml
Deploying the cluster
From the bastion node, run deploy_cluster.yml to automatically deploy Red Hat
OpenShift Container Platform:
cd /root/openshift-bare-metal/
ansible-playbook /usr/share/ansible/openshift-
ansible/playbooks/deploy_cluster.yml
As shown in the following figure, the scripts create OpenShift 3.11 resources using the Ansible playbooks and values from your /etc/ansible/hosts file.
Figure 5. OpenShift 3.11 deployment script
Chapter 7: Creating OpenShift User Accounts
33 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Chapter 7 Creating OpenShift User Accounts
This chapter presents the following topics:
Introduction ....................................................................................................... 34
Logging in to OpenShift ................................................................................... 34
Creating user accounts and setting cluster-admin roles ............................... 34
Chapter 7: Creating OpenShift User Accounts
34 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Introduction
Default login files are created on the master nodes at /root/.kube/config during the
installation. You can log in and create new OpenShift users and then add the new users to
groups or cluster roles using the system: admin account.
Logging in to OpenShift
From the bastion node, start an SSH session to any master node and log in to OpenShift:
ssh master1
cd /root/.kube/
oc login -u system:admin
The following output appears:
Logged into "https://r5a.local:8443" as "system:admin" using
existing credentials.
You have access to the following projects and can switch
between them with 'oc project <projectname>':
app-storage
* default
infra-storage
kube-public
kube-service-catalog
kube-system
management-infra
openshift
openshift-ansible-service-broker
openshift-console
openshift-infra
openshift-logging
openshift-metrics-server
openshift-monitoring
openshift-node
openshift-sdn
openshift-template-service-broker
openshift-web-console
Using project "default".
Creating user accounts and setting cluster-admin roles
Create user accounts as follows. The examples in these steps show how to create
accounts for dellemc1, kubeflow, alice, mike, and john. You can substitute any valid
OpenShift user names.
1. Run the following command:
oc create user dellemc1
oc create user kubeflow
oc create user alice
oc create user mike
Chapter 7: Creating OpenShift User Accounts
35 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
oc create user john
The output displays the user accounts that you created:
oc create user alice
user.user.openshift.io/alice created
create user dellemc1
user.user.openshift.io/dellemc1 created
2. To obtain a list of OpenShift users, run the following command:
oc get users
The output displays the user details:
NAME UID FULL NAME
alice 9b20286-48d0-11e9-95d5-98039b58ae12
dellemc1 5c95e3a9-48d0-11e9-95d5-98039b58ae12
john 6d23c350-48d0-11e9-95d5-98039b58ae12
kubeflow 6a59b12d-48d0-11e9-95d5-98039b58ae12
mike 6049b237-48d0-11e9-95d5-98039b58ae12
3. To designate some users as administrators, run the following command.
CAUTION: This command assigns the cluster-admin role to all the accounts included in
the command. Use it with care.
kubectl create clusterrolebinding permissive-binding --
clusterrole=cluster-admin --user=alice --user=kubeflow --
user=dellemc1 --user=mike --group=system:serviceaccounts
In this example, John remains a regular user while the other users are assigned a
cluster-admin role.
The following output appears:
clusterrolebinding.rbac.authorization.k8s.io/permissive-
binding created
4. Set a password for the new users by running htpasswd on the three masters:
cd /etc/origin/master/
htpasswd -b htpasswd dellemc1 Password1
htpasswd -b htpasswd alice Password2
htpasswd -b htpasswd kubeflow Password3
htpasswd -b htpasswd mike Password4
htpasswd -b htpasswd john Password5
The following output appears from master1:
Adding password for user dellemc1
Adding password for user alice
Adding password for user kubeflow
Adding password for user mike
Chapter 7: Creating OpenShift User Accounts
36 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Adding password for user john
5. From the bastion node, log in as a new user with the cluster-admin role:
oc login https://ocp.example.com:8443 -u dellemc1 -p
Password1 --insecure-skip-tls-verify=true
6. To view the status of all OpenShift nodes, run the following command:
oc get nodes
The following output appears:
NAME STATUS ROLES AGE VERSION
app1.r5a.local Ready compute 33m v1.11.0+d4cacc0
app2.r5a.local Ready compute 33m v1.11.0+d4cacc0
app3.r5a.local Ready compute 33m v1.11.0+d4cacc0
app4.r5a.local Ready compute 33m v1.11.0+d4cacc0
infra1.r5a.local Ready infra 33m v1.11.0+d4cacc0
infra2.r5a.local Ready infra 33m v1.11.0+d4cacc0
infra3.r5a.local Ready infra 33m v1.11.0+d4cacc0
master1.r5a.local Ready master 41m v1.11.0+d4cacc0
master2.r5a.local Ready master 41m v1.11.0+d4cacc0
master3.r5a.local Ready master 41m v1.11.0+d4cacc0
stor1.r5a.local Ready compute 33m v1.11.0+d4cacc0
stor2.r5a.local Ready compute 33m v1.11.0+d4cacc0
stor3.r5a.local Ready compute 33m v1.11.0+d4cacc0
stor4.r5a.local Ready compute 33m v1.11.0+d4cacc0
Note: To prevent containers from running on the storage nodes, storage nodes 1–4 can be
configured for applications as do not schedule.
7. To view all pods in all namespaces, run the following command:
oc get pods --all-namespaces
8. To view all routes, run the following command:
oc get routes --all-namespaces
Chapter 8: Creating the Default Storage Class
37 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Chapter 8 Creating the Default Storage Class
This chapter presents the following topics:
Introduction ....................................................................................................... 38
Creating the default storage class ................................................................... 38
Chapter 8: Creating the Default Storage Class
38 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Introduction
To create a default storage class, run this procedure from the bastion node with a user
that has been assigned the “cluster-admin” role. Because the Dell EMC ready architecture
for Red Hat OpenShift Container Platform uses NVMe SSD drives, we named the storage
volumes glusterfs-nvme.
Creating the default storage class
To create the default storage class:
1. From the bastion node, log in to OpenShift as a user that has been assigned the
cluster-admin role.
2. Run the following command:
oc login https://ocp.example.com:8443 -u dellemc1 -p
Password1 --insecure-skip-tls-verify=true
cd /root/openshift-bare-metal/
ansible-playbook src/ocs-pool/storage-class.yml
3. Run the following command:
oc get sc
The output shows the new default storage class:
NAME PROVISIONER AGE
glusterfs-nvme (default) kubernetes.io/glusterfs 2m
glusterfs-registry-block gluster.org/glusterblock 34m
Chapter 9: Validating the OpenShift Container Platform Deployment
39 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Chapter 9 Validating the OpenShift Container Platform Deployment
This chapter presents the following topics:
Introduction ....................................................................................................... 40
Validating the deployment ................................................................................ 40
Chapter 9: Validating the OpenShift Container Platform Deployment
40 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Introduction
This chapter provides guidelines for validating that the OpenShift Container Platform
software is properly deployed by showing examples of the OpenShift Web Console user
interface and explaining how to verify that resources are being displayed as expected.
Note: ocp.example.com is not publicly available on the Internet.
Validating the deployment
The OpenShift Web Console is the web interface of an OpenShift cluster. Use the Web
Console to validate that the OpenShift Container Platform software was properly
deployed:
1. Log in by entering the following URL into your browser, using the hostname you
generated in your DNS:
https://ocp.example.com:8443/
The Service Catalog appears, as shown in the following figure.
Figure 6. Service Catalog
You can use the Service Catalog to easily download and deploy applications, create a
new project, and view the infrastructure.
Chapter 9: Validating the OpenShift Container Platform Deployment
41 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
2. In the upper right, click Create Project and enter the following information:
Create the project: dellemc1
The following page appears.
Figure 7. Creating a project
3. In the upper left of the Web Console, next to OPENSHIFT CONTAINER
PLATFORM, select Cluster Console from the drop-down menu.
The Cluster Console displays the cluster information that is available to a user
with the cluster-admin role.
4. In the left navigation bar, under Home, click Status.
The following page appears.
Figure 8. Cluster Status page
5. Return to the OpenShift Cluster Console and, from the left navigation bar, select
Storage > Storage Classes.
The following page appears.
Chapter 9: Validating the OpenShift Container Platform Deployment
42 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Figure 9. OpenShift Storage Classes
6. In the left navigation bar of the Cluster Console, select Networking > Routes to
view the OpenShift service routes.
The following figure shows the service routes.
Figure 10. OpenShift service routes
7. Select Monitoring > Log in with OpenShift to log in to Prometheus.
Prometheus is the open source cluster monitoring stack that is included with
OpenShift Container Platform.
The following page appears.
Chapter 9: Validating the OpenShift Container Platform Deployment
43 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Figure 11. Authorize Access page
8. Click Allow Selected permissions.
Prometheus status information appears, as shown in the following figure.
Figure 12. Prometheus status view
Chapter 9: Validating the OpenShift Container Platform Deployment
44 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Grafana is an open source package that works with Prometheus for the graphical
display of time series analytics. To configure a visualization of OpenShift
performance, follow the instructions in Grafana Support for Prometheus.
The following figure shows the Web Console view of the Grafana graph of the
OpenShift cluster’s CPU usage.
Figure 13. Grafana for the Web Console
9. Return to the Prometheus page and click Alerts Manager.
The following page appears.
Chapter 9: Validating the OpenShift Container Platform Deployment
45 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Figure 14. Alerts Manager
10. Click the Alerts link beneath Altermanager.
The Alerts page appears, as shown in the following figure. The
DEADMansSwitch test is a default method to validate that the alerts are working.
Chapter 9: Validating the OpenShift Container Platform Deployment
46 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Figure 15. Default alerts
Chapter 10: Validating OpenShift Operation
47 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Chapter 10 Validating OpenShift Operation
This chapter presents the following topics:
Introduction ....................................................................................................... 48
Deploying a sample application: ruby-ex ........................................................ 48
Deploying a sample application: Lab-Guide-Workshop ................................. 51
Chapter 10: Validating OpenShift Operation
48 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Introduction
This chapter describes how to validate that the installed platform is operating correctly.
The validation process involves deploying one or more sample applications to the
OpenShift cluster and running the applications to confirm that key cluster functions are
working.
Deploying a sample application: ruby-ex
This procedure involves downloading a sample website application (ruby-ex), exposing
the service, and validating that you can reach the route (website address).
Follow these steps to deploy the ruby-ex sample application:
1. Log in to OpenShift and run the following command:
oc login https://ocp.example.com:8443 -u dellemc1 -p
Password1 --insecure-skip-tls-verify=true
2. Create the ruby-ex demo project using the CLI, deploy ruby-ex, and then expose
the website by running the following command:
oc new-project demo
The following output appears:
Now using project "demo" on server
"https://ocp.example.com:8443".
You can add applications to this project with the 'new-app'
command. For example, try:
oc new-app centos/ruby-25-
centos7~https://github.com/sclorg/ruby-ex.git
to build a new example application in Ruby.
3. Run the following command:
oc new-app centos/ruby-25-
centos7~https://github.com/sclorg/ruby-ex.git
--> Found Docker image c4310a5 (11 days old) from Docker Hub
for "centos/ruby-25-centos7"
Ruby 2.5
--------
Ruby 2.5 available as container is a base platform for
building and running various Ruby 2.5 applications and
frameworks. Ruby is the interpreted scripting language for
quick and easy object-oriented programming. It has many
features to process text files and to do system management
tasks (as in Perl). It is simple, straight-forward, and
extensible.
Tags: builder, ruby, ruby25, rh-ruby25
Chapter 10: Validating OpenShift Operation
49 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
* An image stream tag will be created as "ruby-25-
centos7:latest" that will track the source image
* A source build using source code from
https://github.com/sclorg/ruby-ex.git will be created
* The resulting image will be pushed to image stream
tag "ruby-ex:latest"
* Every time "ruby-25-centos7:latest" changes a new
build will be triggered
* This image will be deployed in deployment config
"ruby-ex"
* Port 8080/tcp will be load balanced by service "ruby-
ex"
* Other containers can access this service through the
hostname "ruby-ex"
--> Creating resources ...
imagestream.image.openshift.io "ruby-25-centos7" created
imagestream.image.openshift.io "ruby-ex" created
buildconfig.build.openshift.io "ruby-ex" created
deploymentconfig.apps.openshift.io "ruby-ex" created
service "ruby-ex" created
--> Success
Build scheduled, use 'oc logs -f bc/ruby-ex' to track
its progress.
Application is not exposed. You can expose services to
the outside world by executing one or more of the commands
below:
'oc expose svc/ruby-ex'
Run 'oc status' to view your app.
4. Run the following command:
oc expose svc/ruby-ex
route.route.openshift.io/ruby-ex exposed
The Web Console view of the ruby-ex demo appears, as shown in the following
figure.
Chapter 10: Validating OpenShift Operation
50 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Figure 16. Web console view of ruby-ex build
5. To run ruby-ex and the exposed service URL, click the URL.
The ruby-ex website appears, as shown in the following figure.
Figure 17. Ruby application welcome page
Chapter 10: Validating OpenShift Operation
51 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Deploying a sample application: Lab-Guide-Workshop
This section describes how to deploy a more complex application and use it to test access
to the resulting website. The application is a sample training workshop environment from
Red Hat called Lab-Guide-Workshop. Lab-Guide-Workshop provides sample applications
to enable users to learn more about OpenShift.
We used the OpenShift Container Platform 3.11 HA with 15 servers, including four
R740xd servers. Each of the servers had 10 NVMe disks that create the OpenShift
Storage Class “glusterfs-nvme,” which is set as the default.
To deploy Lab-Guide-Workshop:
1. Log in to OpenShift from the bastion node by running the following command:
oc login https://ocp.example.com:8443 -u dellemc1 -p
Password1 --insecure-skip-tls-verify=true
2. To export the variables, run the following commands:
export OCP_ROUTING_SUFFIX=apps.ocp.example.com
export OCP_SUBDOMAIN=dellemc-2030.ocp.example.com
export OCP_SUBDOMAIN_INTERNAL=r5a.local
oc new-project workshop
The following output appears:
Now using project "workshop" on server
"https://ocp.example.com:8443". You can add applications to
this project with the 'new-app' command. For example, try:
oc new-app centos/ruby-25-
entos7~https://github.com/sclorg/ruby-ex.git to build a new
example application in Ruby.
3. Deploy the Lab-Guide-Workshop by running the following commands:
oc new-app docker.io/osevg/workshopper:0.2 --name=lab-guide
-e
CONTENT_URL_PREFIX="https://raw.githubusercontent.com/sa-
ne/na-se-openshift-workshop/dell_workshop/labguide" -e
WORKSHOPS_URLS="https://raw.githubusercontent.com/sa-ne/na-
se-openshift-
workshop/dell_workshop/labguide/_ocp_admin_testdrive.yaml"
-e COOLSTORE_PROJECT="user01" -e
OCP_ROUTING_SUFFIX="$OCP_ROUTING_SUFFIX" -e
MASTER_HOSTNAME="master1" -e
MASTER_EXTERNAL_FQDN="master.$OCP_SUBDOMAIN" -e
INFRA_INTERNAL_FQDN="infranode1.$OCP_SUBDOMAIN_INTERNAL" -e
NODE1_INTERNAL_FQDN="node1.$OCP_SUBDOMAIN_INTERNAL" -e
NODE2_INTERNAL_FQDN="node2.$OCP_SUBDOMAIN_INTERNAL"-e
WEB_CONSOLE_URL="https://master.$OCP_SUBDOMAIN_INTERNAL/con
sole" -e SSH_CONSOLE_URL="https://terminal-
workshop.$OCP_ROUTING_SUFFIX" -e
OPENSHIFT_DOCS_BASE="https://docs.openshift.com/container-
platform/3.11" -e KEYNAME="openshift" -e LOG_TO_STDOUT=true
Deploying Lab-
Guide-Workshop
Chapter 10: Validating OpenShift Operation
52 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
--> Found Docker image d6af21e (17 months old) from
docker.io for "docker.io/osevg/workshopper:0.2" * An image
stream tag will be created as "lab-guide:0.2" that will
track this image * This image will be deployed in
deployment config "lab-guide" * Port 8080/tcp will be load
balanced by service "lab-guide" * Other containers can
access this service through the hostname "lab-guide"
--> Creating resources ...imagestream.image.openshift.io
"lab-guide" created deploymentconfig.apps.openshift.io
"lab-guide" created service "lab-guide" created-->
Success application is not exposed. You can expose services
to the outside world by executing one or more of the
commands below: 'oc expose svc/lab-guide' Run 'oc status' to
view your app.
oc expose svc/lab-guide
route.route.openshift.io/lab-guide exposed
The following page appears.
Figure 18. Lab-Guide-Workshop with one lab-guide pod running
4. Click lab-guide, #1.
You can now scale to three pods, as shown in the following figure.
Chapter 10: Validating OpenShift Operation
53 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Figure 19. Scaling the lab-guide to run three pods
5. Select Applications > Routes to view the exposed route to the lab-guide service.
The following page appears.
Chapter 10: Validating OpenShift Operation
54 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Figure 20. Web URL for the exposed lab-guide workshop service
The Lab-Guide-Workshop application is running on sub-domain
apps.ocp.example.com. OpenShift automatically generated lab-guide-
workshop when the route was created.
The wildcard DNS entry *.ocp.example.com permits resolution of all sub-
domains under ocp.example.com and resolves the OpenShift API. In the Ready
Architecture *.ocp.example.com, resolve to 100.82.42.99. This value matches the
values in /etc/ansible/hosts.
Chapter 10: Validating OpenShift Operation
55 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
The following figure shows the exposed route. The route is ready for users to
access Lab-Guide-Workshop.
Figure 21. Exposed route
The following figure shows an OpenShift Web Console view of Lab-Guide-
Workshop with persistent volume claims on the glusterfs-nvme storage class.
Chapter 10: Validating OpenShift Operation
56 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Figure 22. OpenShift Web Console view of Lab-Guide-Workshop
Appendix A: Sample Switch Configuration Files
57 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Appendix A Sample Switch Configuration Files
This appendix presents the following topics:
Introduction ....................................................................................................... 58
Downloading the sample files .......................................................................... 58
Modifying the deployment control file ............................................................. 61
Switch configuration file examples .................................................................. 65
Appendix A: Sample Switch Configuration Files
58 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Introduction
This chapter provides sample configuration files, including a kickstart file for the bastion
node, a sample Ansible hosts file, and sample switch configurations for all three of the
Dell Networking switches in the design.
Downloading the sample files
From the GitHub repository, download the following files:
• bastion.ks: https://github.com/dell-esg/openshift-bare-
metal/blob/master/examples/bastion.ks
• hosts.example.com: https://github.com/dell-esg/openshift-bare-
metal/blob/master/examples/hosts.example.com
• Switch configuration files:
▪ S3048: https://github.com/dell-esg/openshift-bare-
metal/blob/master/examples/S3048
▪ S5232F-1: https://github.com/dell-esg/openshift-bare-
metal/blob/master/examples/S5232F-1
▪ S5232F-2: https://github.com/dell-esg/openshift-bare-
metal/blob/master/examples/S5232F-1
bastion.ks is a sample kickstart file for Red Hat Enterprise Linux 7.6 installation:
text
eula --agreed
install
cdrom
lang en_US.UTF-8
keyboard --vckeymap=us --xlayouts='us'
timezone Etc/UTC --isUtc
rootpw OS_PASSWORD
auth --enableshadow --passalgo=sha512
services --enabled="chronyd"
firstboot --enable
%include /tmp/clearpart
zerombr
%include /tmp/bootdrive
volgroup vg00 pv.01
logvol swap --vgname=vg00 --name=swapvol --fstype=swap --size=4000
logvol / --vgname=vg00 --fstype=xfs --size=80000 --name=lv_root
reboot
%pre
bootdrive=`readlink -f /dev/disk/by-id/*BOSS* | head -n 1`
echo clearpart --all --initlabel --drives=$bootdrive >
/tmp/clearpart
Kickstart
installation file
Appendix A: Sample Switch Configuration Files
59 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
echo part /boot --fstype ext4 --size=500 --ondisk=$bootdrive >
/tmp/bootdrive
echo part /boot/efi --fstype vfat --size=200 --ondisk=$bootdrive
>> /tmp/bootdrive
echo part pv.01 --size=100000 --grow --ondisk=$bootdrive >>
/tmp/bootdrive
mdadm --create -R /dev/md0 --metadata=0.90 --level=1 --raid-
devices=2 /dev/nvme0n1 /dev/nvme1n1 >> /tmp/dockerstorage
sed -i s/"md_component_detection = 1"/"md_component_detection =
0"/ /etc/lvm/lvm.conf >> /tmp/dockerstorage
pvcreate /dev/md0 >> /tmp/dockerstorage
vgcreate docker-vg /dev/md0 >> /tmp/dockerstorage
%end
%packages
@^minimal
@core
chrony
kexec-tools
net-tools
wget
git
tcpdump
%end
%addon com_redhat_kdump --enable --reserve-mb='auto'
%end
%post --erroronfail
cat << EOF > /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
ONBOOT=yes
BOOTPROTO=none
IPADDR=192.168.20.11
NETMASK=255.255.255.0
USERCTL=no
NM_CONTROLLED=yes
PEERDNS=yes
BONDING_OPTS="mode=802.3ad miimon=100 xmit_hash_policy=layer3+4
lacp_rate=1"
EOF
cat << EOF > /etc/sysconfig/network-scripts/ifcfg-bond0.10 DEVICE=bond0.10
IPADDR=192.168.10.10
NETMASK=255.255.255.0
GATEWAY=192.168.10.1
ONBOOT=yes
VLAN=yes
BOOTPROTO=none
Appendix A: Sample Switch Configuration Files
60 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
USERCTL=no
EOF
cat << EOF > /etc/sysconfig/network-scripts/ifcfg-bond0.421
DEVICE=bond0.421
IPADDR=100.82.42.11
NETMASK=255.255.255.0
GATEWAY=100.82.42.1
DNS1=100.82.42.8
ONBOOT=yes
VLAN=yes
BOOTPROTO=none
USERCTL=no
EOF
cat << EOF > /etc/sysconfig/network-scripts/ifcfg-em1
DEVICE=em1
ONBOOT=yes
MASTER=bond0
BOOTPROTO=none
SLAVE=yes
EOF
cat << EOF > /etc/sysconfig/network-scripts/ifcfg-em2
DEVICE=em2
ONBOOT=yes
MASTER=bond0
BOOTPROTO=none
SLAVE=yes
EOF
cat << EOF > /etc/sysconfig/network-scripts/ifcfg-p2p1
DEVICE=p2p1
ONBOOT=yes
MASTER=bond0
BOOTPROTO=none
SLAVE=yes
EOF
cat << EOF > /etc/sysconfig/network-scripts/ifcfg-p2p2
DEVICE=p2p2
ONBOOT=yes
MASTER=bond0
BOOTPROTO=none
SLAVE=yes
EOF
Appendix A: Sample Switch Configuration Files
61 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
cat << EOF > /etc/hostname
bastion.example.com
EOF
%end
Modifying the deployment control file
Modify the hosts.example.com file for Ansible and save it to /etc/ansible/hosts on
the bastion node:
[OSEv3:children]
masters
nodes
etcd
lb
local
glusterfs
glusterfs_registry
[OSEv3:vars]
openshift_release=v3.11
openshift_deployment_type=openshift-enterprise
ansible_ssh_user=root
root_password="OS_PASSWORD"
openshift_storage_glusterfs_image=registry.access.redhat.com/rhgs3
/rhgs-server-rhel7:v3.11
openshift_storage_glusterfs_block_image=registry.access.redhat.com
/rhgs3/rhgs-gluster-block-prov-rhel7:v3.11
openshift_storage_glusterfs_heketi_image=registry.access.redhat.co
m/rhgs3/rhgs-volmanager-rhel7:v3.11
openshift_master_cluster_method=native
openshift_master_cluster_hostname=r5a.local
openshift_master_cluster_public_hostname=ocp.example.com
openshift_master_default_subdomain=apps.ocp.example.com
openshift_master_cluster_ip=192.168.20.99
openshift_master_cluster_public_ip=100.82.42.99
openshift_master_portal_net=10.0.0.0/16
openshift_master_identity_providers=[{'name': 'htpasswd_auth',
'login': 'true', 'challenge': 'true', 'kind':
'HTPasswdPasswordIdentityProvider'}]
openshift_master_htpasswd_users={'admin':
'$apr1$i4wJZDwp$dKhqHNvPCzTcXXXa9Gooz0'}
openshift_clock_enabled=true
os_sdn_network_plugin_name='redhat/openshift-ovs-multitenant'
# logging
openshift_logging_install_logging=true
Appendix A: Sample Switch Configuration Files
62 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
openshift_logging_es_pvc_dynamic=true
openshift_logging_es_pvc_size=200Gi
openshift_logging_es_cluster_size=3
openshift_logging_es_pvc_storage_class_name='glusterfs-registry-
block'
openshift_logging_kibana_nodeselector={"node-
role.kubernetes.io/infra": "true"}
openshift_logging_curator_nodeselector={"node-
role.kubernetes.io/infra": "true"}
openshift_logging_es_nodeselector={"node-
role.kubernetes.io/infra": "true"}
# metrics
openshift_metrics_install_metrics=true
openshift_metrics_storage_kind=dynamic
openshift_metrics_storage_volume_size=200Gi
openshift_metrics_cassandra_pvc_storage_class_name='glusterfs-
registry-block'
openshift_metrics_hawkular_nodeselector={"node-
role.kubernetes.io/infra": "true"}
openshift_metrics_cassandra_nodeselector={"node-
role.kubernetes.io/infra": "true"}
openshift_metrics_heapster_nodeselector={"node-
role.kubernetes.io/infra": "true"}
# registry
openshift_master_dynamic_provisioning_enabled=True
openshift_hosted_registry_storage_kind=glusterfs
openshift_hosted_registry_storage_volume_size=1000Gi
openshift_hosted_registry_selector="node-
role.kubernetes.io/infra=true"
# OCS storage cluster for applications
openshift_storage_glusterfs_namespace=app-storage
openshift_storage_glusterfs_storageclass=true
openshift_storage_glusterfs_storageclass_default=false
openshift_storage_glusterfs_block_deploy=false
# OCS storage for OpenShift infrastructure
openshift_storage_glusterfs_registry_namespace=infra-storage
openshift_storage_glusterfs_registry_storageclass=false
openshift_storage_glusterfs_registry_block_deploy=true
openshift_storage_glusterfs_registry_block_host_vol_create=true
openshift_storage_glusterfs_registry_block_host_vol_size=500
openshift_storage_glusterfs_registry_block_storageclass=true
openshift_storage_glusterfs_registry_block_storageclass_default=fa
lse
# Red Hat Subscription information. Use user and password OR
activation key and organization id.
Appendix A: Sample Switch Configuration Files
63 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
rhel_subscription_user=YourRHaccount
rhel_subscription_pass= YourRHpassword
activationkey=XXXXXXX
org_id=XXXXXXX
# Network settings. Required for OS provisioning.
dns_local=192.168.20.11
dns_upstream=100.82.42.8
external_interface=bond0
external_vlan=421
external_netmask=255.255.255.0
external_gateway=100.82.42.1
internal_interface=bond0
internal_netmask=255.255.255.0
internal_gateway={{ bastion_ip }}
bastion_ip=192.168.20.11
dhcp_first_ip=192.168.20.100
dhcp_last_ip=192.168.20.150
[local]
127.0.0.1
[masters]
master1.r5a.local
master2.r5a.local
master3.r5a.local
[etcd]
master1.r5a.local
master2.r5a.local
master3.r5a.local
[lb]
infra1.r5a.local
infra2.r5a.local
infra3.r5a.local
[glusterfs]
stor1.r5a.local glusterfs_cluster=1 glusterfs_devices='[
"/dev/nvme2n1", "/dev/nvme3n1", "/dev/nvme4n1", "/dev/nvme5n1",
"/dev/nvme6n1", "/dev/nvme7n1", "/dev/nvme8n1", "/dev/nvme9n1",
"/dev/nvme10n1", "/dev/nvme11n1" ]'
stor2.r5a.local glusterfs_cluster=1 glusterfs_devices='[
"/dev/nvme2n1", "/dev/nvme3n1", "/dev/nvme4n1", "/dev/nvme5n1",
"/dev/nvme6n1", "/dev/nvme7n1", "/dev/nvme8n1", "/dev/nvme9n1",
"/dev/nvme10n1", "/dev/nvme11n1" ]'
stor3.r5a.local glusterfs_cluster=1 glusterfs_devices='[
"/dev/nvme2n1", "/dev/nvme3n1", "/dev/nvme4n1", "/dev/nvme5n1",
"/dev/nvme6n1", "/dev/nvme7n1", "/dev/nvme8n1", "/dev/nvme9n1",
"/dev/nvme10n1", "/dev/nvme11n1" ]'
Appendix A: Sample Switch Configuration Files
64 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
stor4.r5a.local glusterfs_cluster=1 glusterfs_devices='[
"/dev/nvme2n1", "/dev/nvme3n1", "/dev/nvme4n1", "/dev/nvme5n1",
"/dev/nvme6n1", "/dev/nvme7n1", "/dev/nvme8n1", "/dev/nvme9n1",
"/dev/nvme10n1", "/dev/nvme11n1" ]'
[glusterfs_registry]
infra1.r5a.local glusterfs_devices='[ "/dev/nvme2n1",
"/dev/nvme3n1" ]'
infra2.r5a.local glusterfs_devices='[ "/dev/nvme2n1",
"/dev/nvme3n1" ]'
infra3.r5a.local glusterfs_devices='[ "/dev/nvme2n1",
"/dev/nvme3n1" ]'
[nodes]
master1.r5a.local openshift_ip=192.168.20.12
idrac_ip=192.168.10.12 serial=DELLSVCTAG
openshift_node_group_name='node-config-master'
master2.r5a.local openshift_ip=192.168.20.13
idrac_ip=192.168.10.13 serial= DELLSVCTAG
openshift_node_group_name='node-config-master'
master3.r5a.local openshift_ip=192.168.20.14
idrac_ip=192.168.10.14 serial= DELLSVCTAG
openshift_node_group_name='node-config-master'
infra1.r5a.local openshift_ip=192.168.20.15 idrac_ip=192.168.10.15
serial= DELLSVCTAG openshift_node_group_name='node-config-infra'
openshift_public_ip=100.82.42.15
infra2.r5a.local openshift_ip=192.168.20.16 idrac_ip=192.168.10.16
serial= DELLSVCTAG openshift_node_group_name='node-config-infra'
openshift_public_ip=100.82.42.16
infra3.r5a.local openshift_ip=192.168.20.17 idrac_ip=192.168.10.17
serial= DELLSVCTAG openshift_node_group_name='node-config-infra'
openshift_public_ip=100.82.42.17
app1.r5a.local openshift_ip=192.168.20.18 idrac_ip=192.168.10.18
serial= DELLSVCTAG openshift_node_group_name='node-config-compute'
app2.r5a.local openshift_ip=192.168.20.19 idrac_ip=192.168.10.19
serial= DELLSVCTAG openshift_node_group_name='node-config-compute'
app3.r5a.local openshift_ip=192.168.20.20 idrac_ip=192.168.10.20
serial= DELLSVCTAG openshift_node_group_name='node-config-compute'
app4.r5a.local openshift_ip=192.168.20.21 idrac_ip=192.168.10.21
serial= DELLSVCTAG openshift_node_group_name='node-config-compute'
stor1.r5a.local openshift_ip=192.168.20.23 idrac_ip=192.168.10.23
serial= DELLSVCTAG openshift_node_group_name='node-config-compute'
interface3='p7p1' interface4='p7p2'
stor2.r5a.local openshift_ip=192.168.20.24 idrac_ip=192.168.10.24
serial= DELLSVCTAG openshift_node_group_name='node-config-compute'
interface3='p7p1' interface4='p7p2'
Appendix A: Sample Switch Configuration Files
65 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
stor3.r5a.local openshift_ip=192.168.20.25 idrac_ip=192.168.10.25
serial= DELLSVCTAG openshift_node_group_name='node-config-compute'
interface3='p7p1' interface4='p7p2'
stor4.r5a.local openshift_ip=192.168.20.26 idrac_ip=192.168.10.26
serial= DELLSVCTAG openshift_node_group_name='node-config-compute'
interface3='p7p1' interface4='p7p2'
[nodes:vars]
idrac_user=root
idrac_password=iDRAC_Password
oreg_auth_user=YourRHaccount
oreg_auth_password=YourRHpassword
oreg_url=registry.redhat.io/openshift3/ose-${component}:${version}
openshift_examples_modify_imagestreams=true
Switch configuration file examples
S5232F-1 example: running-configuration
This file is run on the first of two ToR switches, which is labeled S5232F-1. This file
contains the following code:
hostname S5232F-1
interface breakout 1/1/1 map 25g-4x
interface breakout 1/1/2 map 25g-4x
interface breakout 1/1/3 map 25g-4x
interface breakout 1/1/4 map 25g-4x
interface breakout 1/1/5 map 100g-1x
interface breakout 1/1/6 map 100g-1x
interface breakout 1/1/7 map 100g-1x
interface breakout 1/1/8 map 100g-1x
interface breakout 1/1/9 map 25g-4x
interface breakout 1/1/10 map 25g-4x
interface breakout 1/1/11 map 25g-4x
interface breakout 1/1/12 map 25g-4x
interface breakout 1/1/13 map 100g-1x
interface breakout 1/1/14 map 100g-1x
interface breakout 1/1/15 map 100g-1x
interface breakout 1/1/16 map 100g-1x
interface breakout 1/1/17 map 100g-1x
interface breakout 1/1/18 map 100g-1x
interface breakout 1/1/19 map 100g-1x
interface breakout 1/1/20 map 100g-1x
interface breakout 1/1/21 map 100g-1x
interface breakout 1/1/22 map 100g-1x
interface breakout 1/1/23 map 100g-1x
interface breakout 1/1/24 map 100g-1x
interface breakout 1/1/25 map 100g-1x
Top-of-rack
switch
configuration
Appendix A: Sample Switch Configuration Files
66 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
interface breakout 1/1/26 map 100g-1x
interface breakout 1/1/27 map 100g-1x
interface breakout 1/1/28 map 100g-1x
interface breakout 1/1/29 map 100g-1x
interface breakout 1/1/30 map 100g-1x
interface breakout 1/1/31 map 100g-1x
interface breakout 1/1/32 map 100g-1x
!
iscsi target port 860
iscsi target port 3260
!
spanning-tree rstp priority 24576
aaa authentication login default local
aaa authentication login console local
!
class-map type application class-iscsi
!
policy-map type application policy-iscsi
!
interface vlan1
no shutdown
!
interface vlan10
description iDRAC
no shutdown
ip address 192.168.10.2/24
!
vrrp-group 10
virtual-address 192.168.10.1
!
interface vlan20
description Internal
no shutdown
ip address 192.168.20.2/24
!
vrrp-group 20
virtual-address 192.168.20.1
!
interface vlan421
description Public
shutdown
!
interface port-channel1
description bastion-node
no shutdown
switchport mode trunk
switchport access vlan 20
switchport trunk allowed vlan 10,421
mtu 9216
vlt-port-channel 1
Appendix A: Sample Switch Configuration Files
67 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
spanning-tree port type edge
!
interface port-channel2
description master-nodes
no shutdown
switchport access vlan 20
mtu 9216
lacp fallback enable
vlt-port-channel 2
spanning-tree port type edge
!
interface port-channel3
description master-nodes
no shutdown
switchport access vlan 20
mtu 9216
lacp fallback enable
vlt-port-channel 3
spanning-tree port type edge
!
interface port-channel4
description master-nodes
no shutdown
switchport access vlan 20
mtu 9216
lacp fallback enable
vlt-port-channel 4
spanning-tree port type edge
!
interface port-channel5
description infra-nodes
no shutdown
switchport mode trunk
switchport access vlan 20
switchport trunk allowed vlan 421
mtu 9216
lacp fallback enable
vlt-port-channel 5
spanning-tree port type edge
!
interface port-channel6
description infra-nodes
no shutdown
switchport mode trunk
switchport access vlan 20
switchport trunk allowed vlan 421
mtu 9216
lacp fallback enable
vlt-port-channel 6
spanning-tree port type edge
Appendix A: Sample Switch Configuration Files
68 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
!
interface port-channel7
description infra-nodes
no shutdown
switchport mode trunk
switchport access vlan 20
switchport trunk allowed vlan 421
mtu 9216
lacp fallback enable
vlt-port-channel 7
spanning-tree port type edge
!
interface port-channel8
description app-nodes
no shutdown
switchport access vlan 20
mtu 9216
lacp fallback enable
vlt-port-channel 8
spanning-tree port type edge
!
interface port-channel9
description app-nodes
no shutdown
switchport access vlan 20
mtu 9216
lacp fallback enable
vlt-port-channel 9
spanning-tree port type edge
!
interface port-channel10
description app-nodes
no shutdown
switchport access vlan 20
mtu 9216
lacp fallback enable
vlt-port-channel 10
spanning-tree port type edge
!
interface port-channel11
description app-nodes
no shutdown
switchport access vlan 20
mtu 9216
lacp fallback enable
vlt-port-channel 11
spanning-tree port type edge
!
interface port-channel12
description storage-nodes
Appendix A: Sample Switch Configuration Files
69 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
no shutdown
switchport access vlan 20
mtu 9216
lacp fallback enable
vlt-port-channel 12
spanning-tree port type edge
!
interface port-channel13
description storage-nodes
no shutdown
switchport access vlan 20
mtu 9216
lacp fallback enable
vlt-port-channel 13
spanning-tree port type edge
!
interface port-channel14
description storage-nodes
no shutdown
switchport access vlan 20
mtu 9216
lacp fallback enable
vlt-port-channel 14
spanning-tree port type edge
!
interface port-channel15
description storage-nodes
no shutdown
switchport access vlan 20
mtu 9216
lacp fallback enable
vlt-port-channel 15
spanning-tree port type edge
!
interface port-channel30
description Uplink-Customer-Site
no shutdown
switchport mode trunk
switchport trunk allowed vlan 421
mtu 9216
vlt-port-channel 30
!
interface port-channel34
description Uplink-S3048
no shutdown
switchport mode trunk
switchport trunk allowed vlan 10,421
mtu 9216
vlt-port-channel 34
!
Appendix A: Sample Switch Configuration Files
70 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
interface ethernet1/1/1:1
no shutdown
channel-group 1 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/1:2
no shutdown
channel-group 2 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/1:3
no shutdown
channel-group 3 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/1:4
no shutdown
channel-group 4 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/2:1
no shutdown
channel-group 5 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/2:2
no shutdown
channel-group 6 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/2:3
no shutdown
channel-group 7 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/2:4
Appendix A: Sample Switch Configuration Files
71 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
no shutdown
channel-group 8 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/3:1
no shutdown
channel-group 9 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/3:2
no shutdown
channel-group 10 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/3:3
no shutdown
channel-group 11 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/3:4
no shutdown
channel-group 12 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/4:1
no shutdown
channel-group 13 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/4:2
no shutdown
channel-group 14 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/4:3
no shutdown
Appendix A: Sample Switch Configuration Files
72 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
channel-group 15 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/4:4
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/5
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/6
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/7
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/8
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/9:1
no shutdown
channel-group 1 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/9:2
no shutdown
channel-group 2 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/9:3
no shutdown
channel-group 3 mode active
no switchport
mtu 9216
flowcontrol receive off
Appendix A: Sample Switch Configuration Files
73 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
!
interface ethernet1/1/9:4
no shutdown
channel-group 4 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/10:1
no shutdown
channel-group 5 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/10:2
no shutdown
channel-group 6 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/10:3
no shutdown
channel-group 7 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/10:4
no shutdown
channel-group 8 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/11:1
no shutdown
channel-group 9 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/11:2
no shutdown
channel-group 10 mode active
no switchport
mtu 9216
flowcontrol receive off
!
Appendix A: Sample Switch Configuration Files
74 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
interface ethernet1/1/11:3
no shutdown
channel-group 11 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/11:4
no shutdown
channel-group 12 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/12:1
no shutdown
channel-group 13 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/12:2
no shutdown
channel-group 14 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/12:3
no shutdown
channel-group 15 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/12:4
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/13
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/14
shutdown
switchport access vlan 1
flowcontrol receive off
!
Appendix A: Sample Switch Configuration Files
75 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
interface ethernet1/1/15
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/16
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/17
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/18
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/19
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/20
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/21
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/22
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/23
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/24
shutdown
switchport access vlan 1
flowcontrol receive off
!
Appendix A: Sample Switch Configuration Files
76 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
interface ethernet1/1/25
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/26
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/27
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/28
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/29
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/30
no shutdown
channel-group 30 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/31
no shutdown
no switchport
flowcontrol receive off
!
interface ethernet1/1/32
no shutdown
no switchport
flowcontrol receive off
!
interface ethernet1/1/33
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/34
no shutdown
channel-group 34 mode active
Appendix A: Sample Switch Configuration Files
77 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
no switchport
mtu 9216
flowcontrol receive off
!
interface mgmt1/1/1
no shutdown
no ip address dhcp
ip address 100.82.33.81/24
ipv6 address autoconfig
!
management route 0.0.0.0/0 100.82.33.1
!
support-assist
!
vlt-domain 100
backup destination 100.82.33.82
discovery-interface ethernet1/1/31-1/1/32
peer-routing
vlt-mac 00:11:22:33:55:aa
snmp-server contact "Contact Support"
S5232F-2 example: running-configuration
This file is run on the second of two ToR switches, which is labeled S5232F-2. The file
contains the following code:
hostname S5232F-2
interface breakout 1/1/1 map 25g-4x
interface breakout 1/1/2 map 25g-4x
interface breakout 1/1/3 map 25g-4x
interface breakout 1/1/4 map 25g-4x
interface breakout 1/1/5 map 100g-1x
interface breakout 1/1/6 map 100g-1x
interface breakout 1/1/7 map 100g-1x
interface breakout 1/1/8 map 100g-1x
interface breakout 1/1/9 map 25g-4x
interface breakout 1/1/10 map 25g-4x
interface breakout 1/1/11 map 25g-4x
interface breakout 1/1/12 map 25g-4x
interface breakout 1/1/13 map 100g-1x
interface breakout 1/1/14 map 100g-1x
interface breakout 1/1/15 map 100g-1x
interface breakout 1/1/16 map 100g-1x
interface breakout 1/1/17 map 100g-1x
interface breakout 1/1/18 map 100g-1x
interface breakout 1/1/19 map 100g-1x
interface breakout 1/1/20 map 100g-1x
interface breakout 1/1/21 map 100g-1x
interface breakout 1/1/22 map 100g-1x
interface breakout 1/1/23 map 100g-1x
interface breakout 1/1/24 map 100g-1x
interface breakout 1/1/25 map 100g-1x
Appendix A: Sample Switch Configuration Files
78 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
interface breakout 1/1/26 map 100g-1x
interface breakout 1/1/27 map 100g-1x
interface breakout 1/1/28 map 100g-1x
interface breakout 1/1/29 map 100g-1x
interface breakout 1/1/30 map 100g-1x
interface breakout 1/1/31 map 100g-1x
interface breakout 1/1/32 map 100g-1x
!
iscsi target port 860
iscsi target port 3260
spanning-tree rstp priority 24576
aaa authentication login default local
aaa authentication login console local
!
class-map type application class-iscsi
!
policy-map type application policy-iscsi
!
interface vlan1
no shutdown
!
interface vlan10
description iDRAC
no shutdown
ip address 192.168.10.3/24
!
vrrp-group 10
priority 200
virtual-address 192.168.10.1
!
interface vlan20
description Internal
no shutdown
ip address 192.168.20.3/24
!
vrrp-group 20
priority 200
virtual-address 192.168.20.1
!
interface vlan421
description Public
shutdown
!
interface port-channel1
description bastion-node
no shutdown
switchport mode trunk
switchport access vlan 20
switchport trunk allowed vlan 10,421
mtu 9216
Appendix A: Sample Switch Configuration Files
79 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
lacp fallback enable
vlt-port-channel 1
spanning-tree port type edge
!
interface port-channel2
description master-nodes
no shutdown
switchport access vlan 20
mtu 9216
lacp fallback enable
vlt-port-channel 2
spanning-tree port type edge
!
interface port-channel3
description master-nodes
no shutdown
switchport access vlan 20
mtu 9216
lacp fallback enable
vlt-port-channel 3
spanning-tree port type edge
!
interface port-channel4
description master-nodes
no shutdown
switchport access vlan 20
mtu 9216
lacp fallback enable
vlt-port-channel 4
spanning-tree port type edge
!
interface port-channel5
description infra-nodes
no shutdown
switchport mode trunk
switchport access vlan 20
switchport trunk allowed vlan 421
mtu 9216
lacp fallback enable
vlt-port-channel 5
spanning-tree port type edge
!
interface port-channel6
description infra-nodes
no shutdown
switchport mode trunk
switchport access vlan 20
switchport trunk allowed vlan 421
mtu 9216
lacp fallback enable
Appendix A: Sample Switch Configuration Files
80 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
vlt-port-channel 6
spanning-tree port type edge
!
interface port-channel7
description infra-nodes
no shutdown
switchport mode trunk
switchport access vlan 20
switchport trunk allowed vlan 421
mtu 9216
lacp fallback enable
vlt-port-channel 7
spanning-tree port type edge
!
interface port-channel8
description app-nodes
no shutdown
switchport access vlan 20
mtu 9216
lacp fallback enable
vlt-port-channel 8
spanning-tree port type edge
!
interface port-channel9
description app-nodes
no shutdown
switchport access vlan 20
mtu 9216
lacp fallback enable
vlt-port-channel 9
spanning-tree port type edge
!
interface port-channel10
description app-nodes
no shutdown
switchport access vlan 20
mtu 9216
lacp fallback enable
vlt-port-channel 10
spanning-tree port type edge
!
interface port-channel11
description app-nodes
no shutdown
switchport access vlan 20
mtu 9216
lacp fallback enable
vlt-port-channel 11
spanning-tree port type edge
!
Appendix A: Sample Switch Configuration Files
81 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
interface port-channel12
no shutdown
switchport access vlan 20
mtu 9216
lacp fallback enable
vlt-port-channel 12
spanning-tree port type edge
!
interface port-channel13
no shutdown
switchport access vlan 20
mtu 9216
lacp fallback enable
vlt-port-channel 13
spanning-tree port type edge
!
interface port-channel14
no shutdown
switchport access vlan 20
mtu 9216
lacp fallback enable
vlt-port-channel 14
spanning-tree port type edge
!
interface port-channel15
no shutdown
switchport access vlan 20
mtu 9216
lacp fallback enable
vlt-port-channel 15
spanning-tree port type edge
!
interface port-channel30
description Uplink-Customer-Site
no shutdown
switchport mode trunk
switchport trunk allowed vlan 421
mtu 9216
vlt-port-channel 30
!
interface port-channel34
description Uplink-S3048
no shutdown
switchport mode trunk
switchport trunk allowed vlan 10,421
mtu 9216
vlt-port-channel 34
!
interface ethernet1/1/1:1
no shutdown
Appendix A: Sample Switch Configuration Files
82 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
channel-group 1 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/1:2
no shutdown
channel-group 2 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/1:3
no shutdown
channel-group 3 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/1:4
no shutdown
channel-group 4 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/2:1
no shutdown
channel-group 5 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/2:2
no shutdown
channel-group 6 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/2:3
no shutdown
channel-group 7 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/2:4
no shutdown
channel-group 8 mode active
Appendix A: Sample Switch Configuration Files
83 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/3:1
no shutdown
channel-group 9 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/3:2
no shutdown
channel-group 10 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/3:3
no shutdown
channel-group 11 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/3:4
no shutdown
channel-group 12 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/4:1
no shutdown
channel-group 13 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/4:2
no shutdown
channel-group 14 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/4:3
no shutdown
channel-group 15 mode active
no switchport
Appendix A: Sample Switch Configuration Files
84 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/4:4
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/5
no shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/6
no shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/7
no shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/8
no shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/9:1
no shutdown
channel-group 1 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/9:2
no shutdown
channel-group 2 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/9:3
no shutdown
channel-group 3 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/9:4
Appendix A: Sample Switch Configuration Files
85 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
no shutdown
channel-group 4 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/10:1
no shutdown
channel-group 5 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/10:2
no shutdown
channel-group 6 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/10:3
no shutdown
channel-group 7 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/10:4
no shutdown
channel-group 8 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/11:1
no shutdown
channel-group 9 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/11:2
no shutdown
channel-group 10 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/11:3
no shutdown
Appendix A: Sample Switch Configuration Files
86 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
channel-group 11 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/11:4
no shutdown
channel-group 12 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/12:1
no shutdown
channel-group 13 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/12:2
no shutdown
channel-group 14 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/12:3
no shutdown
channel-group 15 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/12:4
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/13
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/14
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/15
shutdown
Appendix A: Sample Switch Configuration Files
87 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/16
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/17
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/18
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/19
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/20
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/21
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/22
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/23
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/24
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/25
shutdown
Appendix A: Sample Switch Configuration Files
88 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/26
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/27
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/28
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/29
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/30
no shutdown
channel-group 30 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/31
no shutdown
no switchport
flowcontrol receive off
!
interface ethernet1/1/32
no shutdown
no switchport
flowcontrol receive off
!
interface ethernet1/1/33
shutdown
switchport access vlan 1
flowcontrol receive off
!
interface ethernet1/1/34
no shutdown
channel-group 34 mode active
no switchport
mtu 9216
Appendix A: Sample Switch Configuration Files
89 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
flowcontrol receive off
!
interface mgmt1/1/1
no shutdown
no ip address dhcp
ip address 100.82.33.82/24
ipv6 address autoconfig
!
management route 0.0.0.0/0 100.82.33.1
!
support-assist
!
vlt-domain 100
backup destination 100.82.33.81
discovery-interface ethernet1/1/31-1/1/32
peer-routing
primary-priority 65535
vlt-mac 00:11:22:33:55:aa
snmp-server contact "Contact Support"
S3048 example: running-configuration
This file is run on the OOB management switch, which is labeled S3048. The file contains
the following code:
!
boot system stack-unit 1 primary system://A
boot system stack-unit 1 secondary system://B
boot system stack-unit 1 default system://A
!
hostname S3048
!
protocol lldp
advertise dot1-tlv port-protocol-vlan-id
advertise dot3-tlv max-frame-size
advertise management-tlv system-description system-name
advertise med
!
redundancy auto-synchronize full
!
username dellemc1 password CHANGEME privilege 15
!
protocol spanning-tree rstp
no disable
!
stack-unit 1 provision S3048-ON
!
interface GigabitEthernet 1/1
no ip address
mtu 9216
switchport
OOB
management
switch
configuration
Appendix A: Sample Switch Configuration Files
90 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/2
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/3
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/4
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/5
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/6
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/7
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
Appendix A: Sample Switch Configuration Files
91 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
no shutdown
!
interface GigabitEthernet 1/8
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/9
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/10
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/11
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/12
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/13
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
Appendix A: Sample Switch Configuration Files
92 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
interface GigabitEthernet 1/14
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/15
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/16
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/17
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/18
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/19
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/20
no ip address
Appendix A: Sample Switch Configuration Files
93 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/21
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/22
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/23
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/24
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/25
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/26
no ip address
mtu 9216
switchport
Appendix A: Sample Switch Configuration Files
94 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/27
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/28
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/29
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/30
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/31
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/32
no ip address
mtu 9216
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
Appendix A: Sample Switch Configuration Files
95 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
no shutdown
!
interface GigabitEthernet 1/33
no ip address
mtu 9216
portmode hybrid
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/34
no ip address
mtu 9216
portmode hybrid
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/35
no ip address
mtu 9216
portmode hybrid
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/36
no ip address
mtu 9216
portmode hybrid
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/37
no ip address
mtu 9216
portmode hybrid
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/38
no ip address
mtu 9216
Appendix A: Sample Switch Configuration Files
96 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
portmode hybrid
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/39
no ip address
mtu 9216
portmode hybrid
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/40
no ip address
mtu 9216
portmode hybrid
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/41
no ip address
mtu 9216
portmode hybrid
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/42
no ip address
mtu 9216
portmode hybrid
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/43
no ip address
mtu 9216
portmode hybrid
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
Appendix A: Sample Switch Configuration Files
97 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
!
interface GigabitEthernet 1/44
no ip address
mtu 9216
portmode hybrid
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/45
no ip address
mtu 9216
portmode hybrid
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/46
no ip address
mtu 9216
portmode hybrid
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/47
description Public-ClientAccess
no ip address
mtu 9216
portmode hybrid
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface GigabitEthernet 1/48
description Public-ClientAccess
no ip address
mtu 9216
portmode hybrid
switchport
spanning-tree rstp edge-port
spanning-tree 0 portfast
no shutdown
!
interface TenGigabitEthernet 1/49
no ip address
Appendix A: Sample Switch Configuration Files
98 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
mtu 9216
portmode hybrid
switchport
spanning-tree 0 portfast
no shutdown
!
interface TenGigabitEthernet 1/50
no ip address
mtu 9216
portmode hybrid
switchport
spanning-tree 0 portfast
no shutdown
!
interface TenGigabitEthernet 1/51
description Up-S5232F
no ip address
mtu 9216
!
port-channel-protocol LACP
port-channel 51 mode active
no shutdown
!
interface TenGigabitEthernet 1/52
description Up-S5232F
no ip address
mtu 9216
!
port-channel-protocol LACP
port-channel 51 mode active
no shutdown
!
interface ManagementEthernet 1/1
ip address 100.82.33.83/24
no shutdown
!
interface ManagementEthernet 2/1
no shutdown
!
interface ManagementEthernet 3/1
no shutdown
!
interface ManagementEthernet 4/1
no shutdown
!
interface ManagementEthernet 5/1
no shutdown
!
interface ManagementEthernet 6/1
no shutdown
Appendix A: Sample Switch Configuration Files
99 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
!
interface Port-channel 51
description Up-S5232F-te34
no ip address
mtu 9216
portmode hybrid
switchport
no shutdown
!
interface Vlan 1
!untagged GigabitEthernet 1/33-1/46
!untagged TenGigabitEthernet 1/50
!untagged Port-channel 51
!
interface Vlan 10
description iDRAC
no ip address
tagged Port-channel 51
untagged GigabitEthernet 1/1-1/32
shutdown
!
interface Vlan 421
no ip address
tagged Port-channel 51
untagged GigabitEthernet 1/47-1/48
untagged TenGigabitEthernet 1/49
shutdown
!
management route 0.0.0.0/0 100.82.33.1
!
ip ssh server enable
!
line console 0
line vty 0
line vty 1
line vty 2
line vty 3
line vty 4
line vty 5
line vty 6
line vty 7
line vty 8
line vty 9
!
reload-type
boot-type normal-reload
config-scr-download enable
!
end
Appendix B: All-In-One Installation
100 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Appendix B All-In-One Installation
This appendix presents the following topics:
Introduction ..................................................................................................... 101
Deploying OpenShift All-in-One using a single server ................................. 101
Installing Red Hat Enterprise Linux 7.6 ......................................................... 102
Creating Docker storage ................................................................................. 104
Installing OpenShift 3.11 AIO on a PowerEdge R640 server ........................ 105
Deploying a sample Kubernetes application ................................................ 110
Appendix B: All-In-One Installation
101 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Introduction
This section describes how to create a Red Hat OpenShift single-node developer
deployment. Red Hat refers to this deployment as the All-in-One (AIO). For more
information, see the following Red Hat blog post: OpenShift All-in-One (AIO) for Labs and
Fun.
Deploying OpenShift All-in-One using a single server
Deploying a 3.11 All-in-One bare-metal Kubernetes developer environment using one Dell
EMC PowerEdge R640 server requires:
• RHEL-7.6-20181010.0-Server-x86_64-dvd1.iso
• Red Hat Enterprise Linux subscriptions with OpenShift Enterprise 3.11
• One PowerEdge R640 server with iDRAC Enterprise
• From two to four 10 GB or 25 GB NICs using bond mode=4
• Proper switch configurations supporting LACP port-channel architecture with
tagged public network
• DNS wildcard entries for *.ocp.example.com resolving to your OpenShift bare-metal
server
To start installing Red Hat Enterprise Linux 7.6 on the AIO server, choose one of the
following methods. Both options have the same outcome.
Option 1: Prepare and use an image file with the iDRAC remote console
1. From an existing Red Hat Enterprise Linux 7.6 system, create a USB image by
running the following command:
mkfs.vfat -C ks_usb.img 1024
2. Mount the image by running the following command:
mount -o loop ks_usb.img /mnt
3. Place the customized example.ks file into the image by running the following
command:
cp example.ks /mnt
4. Sync and unmount the image by running the following command:
sync; umount /mnt
5. Copy the ks_usb.img file to the machine that will log in to the server’s iDRAC.
During the installation, you will attach this file to Removable Media under Virtual
Media in the iDRAC remote console.
Prerequisites
Preparing the
deployment
Appendix B: All-In-One Installation
102 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Option 2: Prepare a USB key for physical boot
1. From an existing Red Hat Enterprise Linux 7.6 system, format a USB key by
running the following command:
mkfs.ext3 /dev/sdb
2. Mount the USB key by running the following command:
mount /dev/sdb /mnt
3. Place the customized example.ks file into the image by running the following
command:
cp example.ks /mnt
4. Sync and unmount the image by running the following command:
sync; umount /mnt
Installing Red Hat Enterprise Linux 7.6
To install Red Hat Enterprise Linux 7.6, follow these steps:
1. Log in to the iDRAC Web interface of the R640 PowerEdge server and launch a
remote console.
2. On the Virtual Media tab, connect the virtual media, and then select Map
CD/DVD.
3. Browse to the location of the Red Hat Enterprise Linux 7.6 installation media and
select Map Device.
4. Use either ks_usb.img file (Option 1) or the USB key (Option 2).
▪ Option 1: In the iDRAC remote console under the Virtual Media tab, select
Map Removable Disk. Browse to the location of the ks_usb.img file and
select Map Device.
▪ Option 2: Insert the USB key into a USB port on the PowerEdge R640 server
before powering on the server.
Note: If only one physical RAID, SAS, or SATA disk is presented to the server, use the
device name sdb for the kickstart location in the Red Hat Enterprise Linux 7 installer.
sda = BOSS RAID1 enumerates as OS drive
sdb = ks_usb_img mounted to the iDRAC remote console’s removable media
Note: NVMe disk are enumerated as nvme0n1, nvme1n1, nvme2n1, and nvme3n1.
5. Select Next Boot and set it to Virtual CD/DVD/ISO.
6. Power on the system.
The server boots to the Red Hat Enterprise Linux 7.6 ISO.
Installing Red
Hat Enterprise
Linux
Appendix B: All-In-One Installation
103 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
7. From the Red Hat Enterprise Linux 7.6 installation menu, select Install, and press
the [e] key.
Do not press the [Enter] key.
8. Move the cursor to the end of the line that starts with vmlinuz, and append the
following code:
ks=hd:sdb:/example.ks
The Red Hat Linux installation page appears, as shown in the following figure.
Figure 23. Red Hat Linux installation page
9. Press [CTRL+X] to start the installation.
10. Reboot the server and accept the license.
11. Log in and validate that the networks are functioning as expected.
Use SSH to log in to the PowerEdge R640 server. To register with Red Hat and enable
the proper repositories, run the following commands:
subscription-manager register –username YourRHaccount --password
YourRHpassword --force
subscription-manager attach --pool=XXXXXXXXXXXXXXXXXXXXXXXXXXXXX
subscription-manager repos --enable="rhel-7-server-rpms" --
enable="rhel-7-server-extras-rpms" --enable="rhel-7-server-ose-
3.11-rpms" --enable="rhel-7-server-ansible-2.6-rpms"
Registering with
Red Hat
Appendix B: All-In-One Installation
104 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
yum -y install wget git net-tools bind-utils iptables-services
bridge-utils bash-completion kexec-tools sos psacct
yum -y install openshift-ansible
yum -y install docker-1.13.1
Creating Docker storage
Create Docker storage as follows:
1. If partitions with previous data exist on the disks, clean the disks by running the
following command, replacing X with the target drive number:
dd if=/dev/zero of=/dev/nvmeXn1 bs=1M count=1
For example: dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=1
2. Create Docker storage by running the following command:
cd /etc/sysconfig/
vi docker-storage-setup
3. Change to remove the overlay and add the following code to the file:
VG=docker-vg
WIPE_SIGNATURES=true
4. Save the edited file.
5. Verify that the changes you made are in the file by running the following
command:
cat /etc/sysconfig/docker-storage-setup
VG=docker-vg
WIPE_SIGNATURES=true
docker-storage-setup
Rounding up size to full physical extent 1.49 GiB
Thin pool volume with chunk size 512.00 KiB can address
at most 126.50 TiB of data. Logical volume "docker-pool"
created. Logical volume docker-vg/docker-pool changed.
6. Enable and start Docker by running the following command:
systemctl enable docker
systemctl start docker
systemctl is-active docker
7. Log in to the Docker registry and install Atomic by running the following command:
docker login -u YourRHaccount -p YourRHpassword
docker login https://registry.redhat.io
yum -y install atomic
yum -y install atomic-openshift-clients
Appendix B: All-In-One Installation
105 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
8. Install Red Hat Enterprise Linux updates and reboot the server by running the
following command:
yum clean all
yum makecache
yum -y update
reboot
Installing OpenShift 3.11 AIO on a PowerEdge R640 server
To install OpenShift 3.11 AIO on a PowerEdge R640 server, follow these steps:
1. Copy the modified example.hosts file to /etc/ansible/hosts on the
PowerEdge R640 server by running the following commands:
#bare minimum hostfile ocp 3.11.88 ocp.example.com
[OSEv3:children]
masters
nodes
etcd
[OSEv3:vars]
openshift_release=3.11.88
openshift_deployment_type=openshift-enterprise
openshift_portal_net=172.30.0.0/16
# localhost likely doesn't meet the minimum requirements
openshift_disable_check=disk_availability,memory_availabilit
y
openshift_master_identity_providers=[{'name':
'htpasswd_auth', 'login': 'true', 'challenge': 'true',
'kind': 'HTPasswdPasswordIdentityProvider'}]
openshift_master_htpasswd_users={'admin':
'$apr1$8Ffff2Y3$DXZdrtdQFyPvsow0hc2.71'}
openshift_node_groups=[{'name': 'node-config-all-in-one',
'labels': ['node-role.kubernetes.io/master=true', 'node-
role.kubernetes.io/infra=true', 'node-
role.kubernetes.io/compute=true']}]
openshift_master_api_port=8443
openshift_master_console_port=8443
#Default: openshift_master_cluster_method=native
# Internal cluster name
openshift_master_cluster_hostname=ocp.example.com
# NOTE: Default wildcard domain for applications
openshift_master_default_subdomain=apps.ocp.example.com
[masters]
localhost ansible_connection=local
Appendix B: All-In-One Installation
106 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
[etcd]
localhost ansible_connection=local
[nodes]
# openshift_node_group_name should refer to a dictionary
with matching key of name in list openshift_node_groups.
localhost ansible_connection=local
openshift_node_group_name="node-config-all-in-one"
[nodes:vars]
oreg_auth_user=YourRHaccount
oreg_auth_password=YourRHpassword
oreg_url=registry.redhat.io/openshift3/ose-
${component}:${version}
openshift_examples_modify_imagestreams=false
Note: To uninstall the deployment, run ansible-playbook
playbooks/adhoc/uninstall.yml.
2. Change to the openshift-ansible directory and run prerequisites.yml:
cd /usr/share/ansible/openshift-ansible
ansible-playbook playbooks/prerequisites.yml
This command usually finishes in less than 5 minutes with no failures.
3. Change to the openshift-ansible directory and run deploy_cluster.yml:
cd /usr/share/ansible/openshift-ansible
ansible-playbook playbooks/deploy_cluster.yml
This command might run for as long as 30 minutes, depending on your network
speed and hardware.
4. To view the status of the SSH session, run the following command:
tail -f /var/log/messages
watch docker ps
The following ouput appears. Zoom in to view the output.
Figure 24. Cluster deployment script output
New containers are created as the deploy_cluster.yml script runs to
completion. Zoom in to view the output from the script in the following figure.
Appendix B: All-In-One Installation
107 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Figure 25. Output when the deployment script is close to conclusion
When the cluster deployment is complete, the OpenShift Container Platform 3.11.88 is
working and ready for login, as shown in the following output example.
Figure 26. Cluster deployment at completion
Appendix B: All-In-One Installation
108 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
To create OpenShift users, follow these steps:
1. Using SSH, log in to the PowerEdge R640 server as root:
cd /root/.kube
oc login -u system:admin
2. Create OpenShift users by running the following commands:
oc create user alice
The following output appears:
user.user.openshift.io/alice created
oc create user dellemc1
The following output appears:
user.user.openshift.io/dellemc1 created
3. Add accounts to cluster-role cluster-admin by running the following commands:
kubectl create clusterrolebinding permissive-binding --
clusterrole=cluster-admin --user=alice --user=kubelet --
user=dellemc1 --group=system:serviceaccounts
The following output appears:
clusterrolebinding.rbac.authorization.k8s.io/permissive-
binding created
4. Update passwords for the new accounts by using the htpasswd command:
cd /etc/origin/master
htpasswd htpasswd alice
5. When prompted, enter the password for user alice.
6. Confirm the password.
To validate OpenShift Container Platform deployment:
1. Log in as user alice:
oc login https://ocp.example.com:8443 -u alice -p Password1
Login successful
You have access to the following projects and can
switch between them with 'oc project <projectname>':
* default
kube-public
kube-service-catalog
kube-system
management-infra
openshift
Creating
OpenShift users
Validating
deployment
Appendix B: All-In-One Installation
109 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
openshift-ansible-service-broker
openshift-console
openshift-infra
openshift-logging
openshift-monitoring
openshift-node
openshift-sdn
openshift-template-service-broker
openshift-web-console
Using project "default".
2. Create a new OpenShift project:
oc new-project alice
Now using project "alice" on server
"https://ocp.example.com:8443".
3. To list the OpenShift projects, run the following command:
oc get projects
NAME DISPLAY NAME STATUS
default Active
kube-public Active
kube-service-catalog Active
kube-system Active
management-infra Active
openshift Active
openshift-ansible-service-broker Active
openshift-console Active
openshift-infra Active
openshift-logging Active
openshift-monitoring Active
openshift-node Active
openshift-sdn Active
openshift-template-service-broker Active
openshift-web-console Active
4. List all the pods on the OpenShift Container Platform 3.11 AIO by running the
following command:
oc get pods --all-namespaces
NAMESPACE NAME
READY STATUS RESTARTS AGE
default docker-registry-1-gb7kp
1/1 Running 0 19m
default registry-console-1-mqwg7
1/1 Running 0 19m
default router-1-lzrw8
1/1 Running 0 20m
kube-service-catalog apiserver-n89vr
1/1 Running 0 17m
kube-service-catalog controller-manager-t7gjb
1/1 Running 3 17m
Appendix B: All-In-One Installation
110 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
kube-system master-api-
ocp.example.com 1/1 Running 0
24m
kube-system master-controllers-
ocp.example.com 1/1 Running 0
25m
kube-system master-etcd-
ocp.example.com 1/1 Running 0
24m
openshift-ansible-service-broker asb-1-xb657
1/1 Running 0 16m
openshift-console console-588448f455-ttg2p
1/1 Running 0 18m
openshift-monitoring alertmanager-main-0
3/3 Running 0 17m
openshift-monitoring alertmanager-main-1
3/3 Running 0 17m
openshift-monitoring alertmanager-main-2
3/3 Running 0 17m
openshift-monitoring cluster-monitoring-
operator-5fc9b4994d-8sq6b 1/1 Running 0
19m
openshift-monitoring grafana-75c67ffb7-slfxv
2/2 Running 0 18m
openshift-monitoring kube-state-metrics-
59d85c46fb-s52zd 3/3 Running 0
16m
openshift-monitoring node-exporter-d55m2
2/2 Running 0 16m
openshift-monitoring prometheus-k8s-0
4/4 Running 1 18m
openshift-monitoring prometheus-k8s-1
4/4 Running 1 17m
openshift-monitoring prometheus-operator-
7895dddcdd-9lsdr 1/1 Running 0
19m
openshift-node sync-nwflw
1/1 Running 0 24m
openshift-sdn ovs-t2t8h
1/1 Running 0 24m
openshift-sdn sdn-jzp5v
1/1 Running 0 24m
openshift-template-service-broker apiserver-2m6dr
1/1 Running 0 16m
openshift-web-console webconsole-5877dcf66d-
5ks4z 1/1 Running 0 18m
Deploying a sample Kubernetes application
Deploy a sample Kubernetes application as follows:
1. Run the oc project command.
The following output appears:
Using project "alice" on server
"https://ocp.example.com:8443".
Appendix B: All-In-One Installation
111 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
oc new-app centos/ruby-25-
Centos7~https://github.com/sclorg/ruby-ex.git
--> Found Docker image c4310a5 (8 days old) from Docker Hub
for "centos/ruby-25-centos7"
Ruby 2.5 -------- Ruby 2.5 available as container is
a base platform for building and running various Ruby 2.5
applications and frameworks. Ruby is the interpreted
scripting language for quick and easy object-oriented
programming. It has many features to process text files and
to do system management tasks (as in Perl). It is simple,
straight-forward, and extensible.
Tags: builder, ruby, ruby25, rh-ruby25
* An image stream tag will be created as "ruby-25-
centos7:latest" that will track the source image
* A source build using source code from
https://github.com/sclorg/ruby-ex.git will be created
* The resulting image will be pushed to image stream
tag "ruby-ex:latest"
* Every time "ruby-25-centos7:latest" changes a new
build will be triggered
* This image will be deployed in deployment config
"ruby-ex"
* Port 8080/tcp will be load balanced by service "ruby-
ex"
* Other containers can access this service through the
hostname "ruby-ex"
--> Creating resources ...
imagestream.image.openshift.io "ruby-25-centos7" created
imagestream.image.openshift.io "ruby-ex" created
buildconfig.build.openshift.io "ruby-ex" created
deploymentconfig.apps.openshift.io "ruby-ex" created
service "ruby-ex" created
--> Success
Build scheduled, use 'oc logs -f bc/ruby-ex' to track
its progress.
Application is not exposed. You can expose services to
the outside world by executing one or more of the commands
below:
'oc expose svc/ruby-ex'
Run 'oc status' to view your app.
2. To view logs of the creation of the ruby-ex web application, run the following
command:
oc logs -f bc/ruby-ex
Cloning "https://github.com/sclorg/ruby-ex.git" ...
Commit: c00ecd7c762590f1d52c316c7d00141a745ede18
(Merge pull request #25 from pvalena/master)
Author: Honza Horak <[email protected]>
Date: Thu Dec 13 15:35:54 2018 +0100
Using centos/ruby-25-
centos7@sha256:7a931fd172cb6a852ca2d7903dd6ac6b65d808e158223
b5276b51016d283fc1b as the s2i builder image
---> Installing application source ...
---> Building your Ruby application from source ...
Appendix B: All-In-One Installation
112 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
---> Running 'bundle install --retry 2 --deployment --
without development:test' ...
Warning: the running version of Bundler (1.16.1) is older
than the version that created the lockfile (1.16.4). We
suggest you upgrade to the latest version of Bundler by
running `gem install bundler`
.
Fetching gem metadata from
https://rubygems.org/..............
Using bundler 1.16.1
Fetching puma 3.12.0
Installing puma 3.12.0 with native extensions
Fetching rack 2.0.6
Installing rack 2.0.6
Bundle complete! 2 Gemfile dependencies, 3 gems now
installed.
Gems in the groups development and test were not installed.
Bundled gems are installed into `./bundle`
---> Cleaning up unused ruby gems ...
Running `bundle clean --verbose` with bundler 1.16.1
Warning: the running version of Bundler (1.16.1) is older
than the version that created the lockfile (1.16.4). We
suggest you upgrade to the latest version of Bundler by
running `gem install bundler`.
Frozen, using resolution from the lockfile
Pushing image docker-registry.default.svc:5000/alice/ruby-
ex:latest ...
Pushed 0/10 layers, 1% complete
Pushed 10/10 layers, 100% complete
Push successful
3. Create the OpenShift Container Platform route and expose the website for the
sample application by running the following command:
oc expose svc/ruby-ex
route.route.openshift.io/ruby-ex exposed
4. Log in to the OpenShift Web Console and open the Cluster Console.
5. Select Applications > Routes to view the sample application, as shown in the
following figure.
Appendix B: All-In-One Installation
113 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Figure 27. OpenShift Console: Sample application view
6. Explore the diagnostic pages, carefully reviewing any warnings or alert messages.
If you have questions, contact Red Hat Support.
The following three figures show representative Cluster Console pages for:
▪ Health checks
▪ Nodes
▪ Projects
Appendix B: All-In-One Installation
114 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Figure 28. OpenShift Console health check
Appendix B: All-In-One Installation
115 Dell EMC Ready Architecture for Red Hat OpenShift Container Platform v3.11 Deployment Guide
Figure 29. Checking nodes
Figure 30. OpenShift Console projects
Top Related