Post on 15-Jul-2015
Your Inner Sysadmin
Chris Tankersley @dragonmantank LonestarPHP 2015
LonestarPHP 2015 1
Who Am I
• PHP Programmer for over 10 years • Sysadmin/DevOps for around 8 years • Using Linux for more than 15 years • hFps://github.com/dragonmantank
LonestarPHP 2015 2
Here Be Dragons
LonestarPHP 2015 3
Traditional Lamp Stack
LonestarPHP 2015 4
Our Players
LonestarPHP 2015 5
And of course…
LonestarPHP 2015 6
The Server
• /bin -‐ EssenPal user executable files • /boot -‐ Stuff that makes the OS boot up! • /dev -‐ Special device stuff you probably won't touch • /etc -‐ ConfiguraPon files • /home -‐ User home directories • /sbin -‐ System binaries • /usr -‐ MulP-‐user apps and uPliPes • /var -‐ Data usually lives here
LonestarPHP 2015 7
Installing Software
• Compile soXware from scratch • Use the package manager (yum/apt)
LonestarPHP 2015 8
Learn to love the Command Line
LonestarPHP 2015 9
Learn a CLI text editor
• vi/vim • emacs • nano
LonestarPHP 2015 10
Authentication and Authorization
LonestarPHP 2015 11
SSH Keys
• SSH generally uses a Username/Password • SSH Keys pass a public key to the server • Can use a single key for mulPple machines, or mulPple keys for mulPple machines • More secure since ‘passwords’ cannot be stolen
LonestarPHP 2015 12
sudo
You can give admin access to users (or groups of users) without giving them root.
LonestarPHP 2015 13
# Add sudo access to a single user to run as root dragonmantank ALL=(ALL) ALL # Add sudo access to a full group %admin ALL=(ALL) ALL
You can even restrict what commands the users can run # Restrict web developers to only restart Apache and MySQL %webdevs 192.168.1.0/255.255.225.0=(root) NOPASSWD:/usr/sbin/service apache2 restart, /usr/sbin/service mysql restart
Jailing Users
Keeps people from geang to things they shouldn't. Protects the users from themselves.
LonestarPHP 2015 14
Jailed Shells
Gives users a full shell but not the enPre file system. You can pick and choose what programs the user can have access too. Jailkit makes this incredibly easy to set up.
LonestarPHP 2015 15
Jailed SFTP
Locks the user to a specific base path, but doesn’t give them a shell, much like FTP. You get the security of SSH though! It does require a system user however.
LonestarPHP 2015 16
Jailing SFTP
# In /etc/ssh/sshd_config Subsystem ftp sftp-‐internal # At the bottom of the file Match User jailedsftp ChrootDirectory /some/path AllowTCPForwarding no X11Forwarding no ForceCommand sftp-‐internal
LonestarPHP 2015 17
Docker
LonestarPHP 2015 18
If you do it the non-‐Docker way
Scripting Languages
LonestarPHP 2015 19
Bash
Most servers use bash as the default shell. Most shells understand bash's syntax. If you find yourself running the same commands over and over, throw it in a bash script.
LonestarPHP 2015 20
Python
Ships with most distros. Great for when you need more power than what bash has.
LonestarPHP 2015 21
PHP!
Leverage your PHP skills to write shell scripts. • Symfony Console Component • Aura CLI
LonestarPHP 2015 22
Locking Down your Code
LonestarPHP 2015 23
Running Apache as a different user
MPM-‐ITK
LonestarPHP 2015 24
MOD_RUID2
<IfModule mpm_itk_module> AssignUserId [user] [group] </IfModule>
RMode config RUidGid myuser mygroup RDocumentChRoot /var/www/vhosts/domain.com/ www/public
PHP-FPM
user = myuser group = mygroup chroot = /path/to/my/chroot
LonestarPHP 2015 25
Logs
LonestarPHP 2015 26
Logrotate
Rotates logs out for organizaPon (or other purposes)
LonestarPHP 2015 27
weekly rotate 4 create include /etc/logrotate.d /var/log/wtmp { monthly minsize 1M create 0664 root utmp rotate 1 }
Logwatch
Script that runs every so oXen and scans a bunch of logs so you get a preFy e-‐mail with a summary of events
LonestarPHP 2015 28
-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ httpd Begin -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 0.17 MB transferred in 792 responses (1xx 0, 2xx 786, 3xx 0, 4xx 6, 5xx 0) 199 Content pages (0.09 MB), 593 Other (0.09 MB) Requests with error response codes 400 Bad Request /w00tw00t.at.ISC.SANS.DFind:): 1 Time(s) 404 Not Found /MyAdmin/scripts/setup.php: 1 Time(s) /phpmyadmin/scripts/setup.php: 1 Time(s) /w00tw00t.at.blackhats.romanian.anti-‐sec:): 1 Time(s) /webdav/: 2 Time(s) -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ httpd End -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐
OSSEC
Actually a Host Intrusion DetecPon system, but it does this by watching logs. Will alert you immediately to problems, and even shut down the aFacks.
LonestarPHP 2015 29
OSSEC HIDS Notification. 2012 Oct 24 11:38:10 Received From: maple-‐>/var/log/auth.log Rule: 5712 fired (level 10) -‐> "SSHD brute force trying to get access to the system." Portion of the log(s): Oct 24 11:38:09 maple sshd[1062]: Failed password for invalid user alias from 199.167.138.44 port 59988 ssh2 Oct 24 11:38:07 maple sshd[1062]: Invalid user alias from 199.167.138.44 Oct 24 11:38:06 maple sshd[1059]: Failed password for invalid user recruit from 199.167.138.44 port 59884 ssh2
Preventing Intruders
LonestarPHP 2015 30
hosts.deny and hosts.allow
Set of files to allow or deny access to the machine or certain apps/ports on the machine
LonestarPHP 2015 31
IPTables
A firewall that is generally available on Linux machines that can be configured many different ways to allow or block or mangle traffic
LonestarPHP 2015 32
OSSEC
IDS that was logs and will use hosts.deny and iptables to block stuff automaPcally for you!
LonestarPHP 2015 33
Configuration Management
LonestarPHP 2015 34
What is Configuration Management?
Process by which you figure out what goes on your servers, how you want them set up, and keeping track of that informaPon. Files are usually stored in source control on one server and pushed to clients.
LonestarPHP 2015 35
Why do you need it?
• Ever needed to keep track of when files get changed? • Ever needed to roll back a change? • Ever needed to push the same change to a bunch of servers • Ever needed to set up a server exactly the same way as another server?
LonestarPHP 2015 36
General CM Workflow
LonestarPHP 2015 37
Write a Manifest file
Client checks and compiles the manifests
Client makes changes based on manifests
Ansible
• hFps://serversforhackers.com/geang-‐started-‐with-‐ansible/
LonestarPHP 2015 38
Puppet
• hFp://www.erikaheidi.com/page/vagrant
LonestarPHP 2015 39
Server Monitoring
LonestarPHP 2015 40
Quick Poll
• Who here knows that their server is up right now? • Are all of the required services running? • Are there enough resources currently available?
LonestarPHP 2015 41
Service Monitoring with Monit
LonestarPHP 2015 42
Host Monitoring with Icinga
LonestarPHP 2015 43
Software Tools
LonestarPHP 2015 44
tmux/screen
Command line mulPplexer
LonestarPHP 2015 45
tail
Look at the newest entries in a log, or even watch log files as they are generated
LonestarPHP 2015 46
curl
Command line program for transferring data via a URL
LonestarPHP 2015 47
iftop
Displays a breakdown of bandwidth usage by host
LonestarPHP 2015 48
htop
Slightly beFer interface for checking memory and CPU usage
LonestarPHP 2015 49
tcpdump
Allows you to view and record data transmiFed over the network. Couple this with wireshark and you can inspect the packets!
LonestarPHP 2015 50
Servers for Hackers
Chris Fidao @fideloper hFp://serversforhackers.com
LonestarPHP 2015 51
Questions?
LonestarPHP 2015 52
Thank You!
http://ctankersley.com chris@ctankersley.com
@dragonmantank
https://joind.in/13537
LonestarPHP 2015 53