From Attack Graphs to Automated Configuration Management — An ...
Automated Configuration of Firmware
-
Upload
michael-arnold -
Category
Technology
-
view
1.624 -
download
4
description
Transcript of Automated Configuration of Firmware
![Page 1: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/1.jpg)
APOLLO GROUP
Automated Configuration of Firmware
Michael ArnoldPrincipal Systems Engineer27 September 2012
![Page 2: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/2.jpg)
2APOLLO GROUP
Who
Tools
Example Workflow
Example Code
Links
Questions
Agenda
© 2012 Apollo Group
![Page 3: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/3.jpg)
3APOLLO GROUP
APOLLO GROUP
Who
© 2012 Apollo Group
![Page 4: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/4.jpg)
4APOLLO GROUP
Who is Apollo?
© 2012 Apollo Group
Apollo Group is a leading provider of higher education programs for working adults.
![Page 5: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/5.jpg)
5APOLLO GROUP
Systems Administrator
Automation geek
13 years in IT
Hadoop Cluster Admin
I deal with:
–Server hardware specification/configuration
–Server firmware
–Server operating system
–Hadoop Cluster performance and availability
Who is Michael Arnold?
© 2012 Apollo Group
![Page 6: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/6.jpg)
6APOLLO GROUP
It has nothing to do with Puppet.
–Unless someone writes a module to program firmware from the running OS.
• Which could work for RAID controllers.
How does firmware tie in with Puppet?
© 2012 Apollo Group
![Page 7: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/7.jpg)
7APOLLO GROUP
APOLLO GROUP
Tools
© 2012 Apollo Group
![Page 8: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/8.jpg)
8APOLLO GROUP
HP SmartStart Scripting Toolkit
Dell OpenManage Deployment Toolkit
IBM ServerGuide Scripting Toolkit
Fujitsu ServerView Scripting Toolkit
Hardware Toolkits
© 2012 Apollo Group
![Page 9: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/9.jpg)
9APOLLO GROUP
Configuration of
–BIOS
–BMC (DRAC/iLO/RSA/etc.)
–Hardware RAID
–FC HBA
What do the tools provide?
© 2012 Apollo Group
![Page 10: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/10.jpg)
10APOLLO GROUP
Manually configure reference hardware
Capture the firmware configuration
Clone the configuration to multiple machines
How does the toolkit work?
© 2012 Apollo Group
![Page 11: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/11.jpg)
11APOLLO GROUP
Toolkit Components
© 2012 Apollo Group
Kernel & Initrd
Firmware Tools
Toolkit
![Page 12: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/12.jpg)
12APOLLO GROUP
Toolkit Requirements
© 2012 Apollo Group
Toolkit
DHCP
TFTP
NFS
![Page 13: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/13.jpg)
13APOLLO GROUP
APOLLO GROUP
Example Workflow
© 2012 Apollo Group
![Page 14: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/14.jpg)
14APOLLO GROUP
Hardware is racked and cabled
Switch ports are up
Server is manually powered on
Firmware defaults are to boot from network if no other boot device is found.
–No other boot devices are found
Assumptions
© 2012 Apollo Group
![Page 15: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/15.jpg)
15APOLLO GROUP
All server hardware is from the same vendor (HP)
Constraints
© 2012 Apollo Group
![Page 16: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/16.jpg)
16APOLLO GROUP
Firmware Update
Hardware Discovery
Firmware Configuration
BuildsystemRegistration
The Process
© 2012 Apollo Group
![Page 17: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/17.jpg)
17APOLLO GROUP
Screencast
© 2012 Apollo Group
![Page 18: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/18.jpg)
18APOLLO GROUP
Unknown system netboots
Workflow
© 2012 Apollo Group
![Page 19: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/19.jpg)
19APOLLO GROUP
Unknown system netboots
Boot into HP SmartStart Scripting Toolkit
Workflow
© 2012 Apollo Group
![Page 20: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/20.jpg)
20APOLLO GROUP
Unknown system netboots
Boot into HP SmartStart Scripting Toolkit
Gather hardware information (CPU/RAM/disk/NIC) via hpdiscovery
Workflow
© 2012 Apollo Group
![Page 21: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/21.jpg)
21APOLLO GROUP
Unknown system netboots
Boot into HP SmartStart Scripting Toolkit
Gather hardware information (CPU/RAM/disk/NIC) via hpdiscovery
If this is HP hardware, then
–Update firmware
–Configure BIOS
–Configure iLO
–Configure RAID
Workflow
© 2012 Apollo Group
![Page 22: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/22.jpg)
22APOLLO GROUP
Unknown system netboots
Boot into HP SmartStart Scripting Toolkit
Gather hardware information (CPU/RAM/disk/NIC) via hpdiscovery
If this is HP hardware, then
–Update firmware
–Configure BIOS
–Configure iLO
–Configure RAID
cobbler-register to the hardware profile
Workflow
© 2012 Apollo Group
![Page 23: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/23.jpg)
23APOLLO GROUP
Unknown system netboots
Boot into HP SmartStart Scripting Toolkit
Gather hardware information (CPU/RAM/disk/NIC) via hpdiscovery
If this is HP hardware, then
–Update firmware
–Configure BIOS
–Configure iLO
–Configure RAID
cobbler-register to the hardware profile
System reboots
Workflow
© 2012 Apollo Group
![Page 24: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/24.jpg)
24APOLLO GROUP
Modify Cobbler system profile, giving it an identity (hostname, IP, OS metadata)
Workflow
© 2012 Apollo Group
![Page 25: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/25.jpg)
25APOLLO GROUP
Modify Cobbler system profile, giving it an identity (hostname, IP, OS metadata)
Known system netboots
Workflow
© 2012 Apollo Group
![Page 26: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/26.jpg)
26APOLLO GROUP
Modify Cobbler system profile, giving it an identity (hostname, IP, OS metadata)
Known system netboots
Boot into OS installer
Workflow
© 2012 Apollo Group
![Page 27: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/27.jpg)
27APOLLO GROUP
Modify Cobbler system profile, giving it an identity (hostname, IP, OS metadata)
Known system netboots
Boot into OS installer
Install OS
Workflow
© 2012 Apollo Group
![Page 28: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/28.jpg)
28APOLLO GROUP
Modify Cobbler system profile, giving it an identity (hostname, IP, OS metadata)
Known system netboots
Boot into OS installer
Install OS
Install Puppet
Workflow
© 2012 Apollo Group
![Page 29: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/29.jpg)
29APOLLO GROUP
Modify Cobbler system profile, giving it an identity (hostname, IP, OS metadata)
Known system netboots
Boot into OS installer
Install OS
Install Puppet
…
Workflow
© 2012 Apollo Group
![Page 30: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/30.jpg)
30APOLLO GROUP
Modify Cobbler system profile, giving it an identity (hostname, IP, OS metadata)
Known system netboots
Boot into OS installer
Install OS
Install Puppet
…
Profit!
Workflow
© 2012 Apollo Group
![Page 31: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/31.jpg)
31APOLLO GROUP
APOLLO GROUP
Example Code
© 2012 Apollo Group
![Page 32: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/32.jpg)
32APOLLO GROUP
LABEL dtk-install_hadoop
kernel /images/dtk-3.5.1/SA.1
append initrd=/images/dtk-3.5.1/SA.2
ramdisk_size=156482 Stage3_type=cdrom quiet DEBUG=0
share_location=192.168.1.1:/srv/nfs/ro/dtk-3.5.1
share_type=nfs selinux=0 share_opts=ro,nolock
share_script=install_hadoop.sh cprofile=Hadoop
ipappend 2
Dell TFTP Config
© 2012 Apollo Group
![Page 33: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/33.jpg)
33APOLLO GROUP
LABEL sstk-install_hadoop
kernel /images/sstk-8.50/vmlinuz
append initrd=/images/sstk-8.50/initrd.img
root=/dev/ram0 rw ramdisk_size=184248 ide=nodma
ide=noraid pnpbios=off network=1
sstk_mount=192.168.1.1:/srv/nfs/ro/sstk-8.50
sstk_mount_type=nfs sstk_mount_options=ro,nolock
sstk_script=install_hadoop.sh cprofile=Hadoop
ipappend 2
HP TFTP Config
© 2012 Apollo Group
![Page 34: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/34.jpg)
34APOLLO GROUP
cd /opt/dell/toolkit/template/scripts
/opt/dell/toolkit/template/scripts/syscap.sh /tmp/syscap
/opt/dell/toolkit/template/scripts/raidcap.sh /tmp/raidcap
/opt/dell/toolkit/template/scripts/raccap.sh /tmp/raccap
# MegaCLI does not come with the DTK so "install" it.
cp -a /opt/dell/toolkit/systems/MegaCLI/opt/MegaRAID \
/opt/MegaRAID
/opt/MegaRAID/MegaCli/MegaCli -CfgSave -a0 -f /tmp/mega.0
Dell Config Capture
© 2012 Apollo Group
![Page 35: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/35.jpg)
35APOLLO GROUP
cd /TOOLKIT
./conrep -s -f /tmp/conrep.dat # BIOS
./hpacuscripting -c /tmp/cpqacuxe.dat # RAID
./hplpcfg /s /tmp/hplpcfg.dat # Emulex HBA
./hpqlarep /S /tmp/hpqlarep.dat # Qlogic HBA
if ./ifhw /tmp/hw_discovery.dat allboards.xml \
"PCI:Integrated Lights-Out" 2> /dev/null ; then
./hponcfg -w /tmp/hponcfg.dat # iLO 2/3
else
./lo100cfg -o /tmp/lo100cfg.dat # LO100i
fi
HP Config Capture
© 2012 Apollo Group
![Page 36: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/36.jpg)
36APOLLO GROUP
*** Detecting system type ***
System Information:
Name : ProLiant DL360 G5
Serial : USE123D456
Arch : x86_64
LNAME : ProLiantDL360G5
SNAME : DL360 G5
Asset Tag : 01234567
Platform Detection
© 2012 Apollo Group
![Page 37: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/37.jpg)
37APOLLO GROUP
HP SSSTK has a function library that sets many variables.
–$PRODUCT_NAME =
• "ProLiant SL170z G6"
• "ProLiant DL360 G6"
–$SERIAL_ID = hardware serial number
–$PRODUCT_LNAME =
• "ProLiantSL170zG6"
• "ProLiantDL360G6"
–$PRODUCT_SNAME =
• "SL170z G6"
• "DL360 G6"
–$TOOLKIT_MNTPNT = /mnt/main
–$TOOLKIT_SERVER_IP = NFS server IP
–$HWDISC_FILE = /TOOLKIT/hpdiscovery.xml
Some Available Variables
© 2012 Apollo Group
![Page 38: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/38.jpg)
38APOLLO GROUP
Set Variables Based on Hardware Type
export "`./hwquery ${HWDISC_FILE} allboards.xml SERVERNAME=SystemName`"
case "${SERVERNAME}" in
"ProLiant DL160 G6"* )
CONREPDAT=${TOOLKIT_MNTPNT}/data_files/dl160_conrep.dat
LO100CFGDAT=${TOOLKIT_MNTPNT}/data_files/dl160_lo100cfg.dat
;;
"ProLiant SL170z G6"* )
CONREPDAT="${TOOLKIT_MNTPNT}/data_files/sl170z_conrep.dat -x \
${TOOLKIT_MNTPNT}/conrep_xml/conrep_SL170zg6_20100401.xml"
LO100CFGDAT=${TOOLKIT_MNTPNT}/data_files/sl170z_lo100cfg.dat
;;
#...
Inside install_hadoop.sh
© 2012 Apollo Group
![Page 39: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/39.jpg)
39APOLLO GROUP
#...
"ProLiant DL360 G5"* )
CONREPDAT=${TOOLKIT_MNTPNT}/data_files/dl360g5_conrep.dat
if ./ifhw ${HWDISC_FILE} allboards.xml "PCI:Smart Array E200i Controller" \
2> /dev/null ; then
HPACUDAT=${TOOLKIT_MNTPNT}/data_files/dl360g5_saE200i_cpqacuxe.dat
fi
if ./ifhw ${HWDISC_FILE} allboards.xml "PCI:Smart Array P400i Controller" \
2> /dev/null ; then
HPACUDAT=${TOOLKIT_MNTPNT}/data_files/dl360g5_saP400i_cpqacuxe.dat
fi
HPONCFGDAT=${TOOLKIT_MNTPNT}/data_files/dl360g5_hponcfg.dat
;;
esac
Inside install_hadoop.sh
© 2012 Apollo Group
![Page 40: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/40.jpg)
40APOLLO GROUP
if [ -n "$CONREPDAT" ] ; then
echo "** Applying BIOS Configuration"
./conrep -l -f ${CONREPDAT}
fi
if [ -n "$HPACUDAT" ] ; then
echo "** Clearing RAID Configuration"
./hpacucli/hpacuscripting -i ${HPACUDAT}_clear; sleep 5
echo "** Applying RAID Configuration"
./hpacucli/hpacuscripting -i ${HPACUDAT}
fi
if [ -n "$LO100CFGDAT" ] ; then
echo "** Applying BMC Configuration"; ./lo100cfg -i ${LO100CFGDAT}
elif [ -n "$HPONCFGDAT" ] ; then
echo "** Applying iLO Configuration"; ./hponcfg -f ${HPONCFGDAT}
fi
Inside install_hadoop.sh
© 2012 Apollo Group
![Page 41: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/41.jpg)
41APOLLO GROUP
echo "** Registering with cobbler"
# Koan does not come with SSSTK, so "install" it.
${TOOLKIT_MNTPNT}/koan/install
# cprofile is set via kernel boot parameter and points at part of a
# Cobbler profile name.
# SERIAL_ID and PRODUCT_LNAME are set via SSSTK.
# PRODUCT_LNAME = ProLiantSL170zG6, ProLiantDL360G6, etc.
# SERIAL_ID = hardware serial number
echo "** Registering to profile ${cprofile}-${PRODUCT_LNAME}"
cobbler-register --server=${TOOLKIT_SERVER_IP} --fqdn=${SERIAL_ID} \
--profile=${cprofile}-${PRODUCT_LNAME} --batch
Register with Buildsystem
© 2012 Apollo Group
![Page 42: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/42.jpg)
42APOLLO GROUP
$ cat dl360g5_saE200i_cpqacuxe.dat_clear
; Version: 8.50.6.0
; http://h30499.www3.hp.com/t5/ProLiant-Servers-ML-DL-SL/CPQACUXE-reset-utility-within-WinPE/td-p/3939475
Action= Reconfigure
Method= Custom
Controller= SLOT 0
ClearConfigurationWithDataLoss= Yes
Clear the HP RAID Controller
© 2012 Apollo Group
![Page 43: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/43.jpg)
43APOLLO GROUP
$ cat dl360g5_saE200i_cpqacuxe.dat
; Version: 8.50.6.0
Action= Configure
Method= Custom
; Controller Specifications
; Controller HP Smart Array E200i
; Firmware Version 1.82
Controller= SLOT 0
ReadCache= 100
WriteCache= 0
RebuildPriority= Medium
ExpandPriority= Medium
SurfaceScanDelay= 15
DriveWriteCache= Disabled
; Array Specifications
Array= A
; Array Drive Type is SAS
; 1I:1:1 (72 GB), 1I:1:2 (72 GB)
Drive= 1I:1:1, 1I:1:2
OnlineSpare= No
; Logical Drive Specifications
LogicalDrive= 1
RAID= 1
Size= 69973
Sectors= 32
StripeSize= 128
ArrayAccelerator= Enabled
Configure the HP RAID Controller
© 2012 Apollo Group
![Page 44: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/44.jpg)
44APOLLO GROUP
You may have to test for any failures when applying configurations.
When reusing HP hardware, adding a user to the BMC that already exists will cause a failure to apply all of the configuration.
Newer firmware can have different config options that will cause an older capture to fail.
Error Handling
© 2012 Apollo Group
![Page 45: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/45.jpg)
45APOLLO GROUP
Firmware Update
Hardware Discovery
Firmware Configuration
BuildsystemRegistration
Summary
© 2012 Apollo Group
![Page 46: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/46.jpg)
46APOLLO GROUP 46APOLLO GROUP
HP SmartStart Scripting Toolkit
http://h18013.www1.hp.com/products/servers/management/toolkit/index.html
Dell OpenManage Deployment Toolkit
http://en.community.dell.com/techcenter/systems-management/w/wiki/1772.dell-openmanage-deployment-toolkit.aspx
IBM ServerGuide Scripting Toolkit
http://www-03.ibm.com/systems/be/management/sgstk/
Fujitsu ServerView Scripting Toolkit
http://www.fujitsu.com/fts/products/computing/servers/primergy/management/deploy/
Cobbler - Linux Installation Server
http://cobbler.github.com/
LSI MegaRAID MegaCLI
http://www.lsi.com/downloads/Public/MegaRAID%20Common%20Files/8.02.24_MegaCLI.zip
Links
© 2012 Apollo Group
![Page 47: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/47.jpg)
47APOLLO GROUP 47APOLLO GROUP
https://intelligentsysadmin.wordpress.com/
https://github.com/razorsedge
http://forge.puppetlabs.com/razorsedge
Contact
© 2012 Apollo Group
![Page 48: Automated Configuration of Firmware](https://reader034.fdocuments.us/reader034/viewer/2022052504/54be533c4a795988158b4581/html5/thumbnails/48.jpg)
48APOLLO GROUP
APOLLO GROUP
Questions?
© 2012 Apollo Group