Secrets of a linux ninja Software Freedom Day 2013 Johannesburg, South Africa

22
26/09/2013 Secrets of a Linux Ninja - SFD 2013 Johannesburg, South Africa file:///home/mark/Documents/presentations/linux-ninja/index.html#/ 1/22 SECRETS OF A LINUX NINJA Created by / Mark Clarke @mxc4

description

The slides from a hands on lab given on Software Freedom Day on the 31 August 2013 at the University of the Witwatersrand. It was aimed at showing some of the lesser know features of well known commands as well as some lesser known, yet powerful and command line tools

Transcript of Secrets of a linux ninja Software Freedom Day 2013 Johannesburg, South Africa

Page 1: Secrets of a linux ninja  Software Freedom Day 2013 Johannesburg, South Africa

26/09/2013 Secrets of a Linux Ninja - SFD 2013 Johannesburg, South Africa

file:///home/mark/Documents/presentations/linux-ninja/index.html#/ 1/22

SECRETS OF A LINUX NINJA

Created by / Mark Clarke @mxc4

Page 2: Secrets of a linux ninja  Software Freedom Day 2013 Johannesburg, South Africa

26/09/2013 Secrets of a Linux Ninja - SFD 2013 Johannesburg, South Africa

file:///home/mark/Documents/presentations/linux-ninja/index.html#/ 2/22

LESSON 1: SSH! A NINJA IS SILENT!

Page 3: Secrets of a linux ninja  Software Freedom Day 2013 Johannesburg, South Africa

26/09/2013 Secrets of a Linux Ninja - SFD 2013 Johannesburg, South Africa

file:///home/mark/Documents/presentations/linux-ninja/index.html#/ 3/22

Access lan based services via incoming SSH access,Get external SSH access even if its not enabled,Forward X sessions over SSH,Bypass web proxy restrictions

“Take things as they are. Punch when you have to punch. Kickwhen you have to kick.”

Page 4: Secrets of a linux ninja  Software Freedom Day 2013 Johannesburg, South Africa

26/09/2013 Secrets of a Linux Ninja - SFD 2013 Johannesburg, South Africa

file:///home/mark/Documents/presentations/linux-ninja/index.html#/ 4/22

LAB SET UP

Micro-server represents lan. Laptop virtual machine represents external host. Laptop hostmachine is gateway

Lan Net:192.168.1.0/24External Net:192.168.122.0/24Gateway:192.168.1.4/192.168.122.1Lan Host:192.168.1.3External Host:192.168.122.192

Page 5: Secrets of a linux ninja  Software Freedom Day 2013 Johannesburg, South Africa

26/09/2013 Secrets of a Linux Ninja - SFD 2013 Johannesburg, South Africa

file:///home/mark/Documents/presentations/linux-ninja/index.html#/ 5/22

ACCESS A LAN BASED SERVICE WITH INCOMING SSHACCESS

Scenario:

You have ssh access to the lan but no VPN access,You need to access a web application running on the lan.Web App: http://192.168.1.3/ninja.html

Page 6: Secrets of a linux ninja  Software Freedom Day 2013 Johannesburg, South Africa

26/09/2013 Secrets of a Linux Ninja - SFD 2013 Johannesburg, South Africa

file:///home/mark/Documents/presentations/linux-ninja/index.html#/ 6/22

GET EXTERNAL SSH ACCESS EVEN IF ITS NOT ENABLED

Scenario:

You have no external ssh access to networkYou can connect to external ssh servers from the lan

Page 7: Secrets of a linux ninja  Software Freedom Day 2013 Johannesburg, South Africa

26/09/2013 Secrets of a Linux Ninja - SFD 2013 Johannesburg, South Africa

file:///home/mark/Documents/presentations/linux-ninja/index.html#/ 7/22

FORWARD X SESSIONS OVER SSH

Scenario:

XDMCP is disabled,You have SSH access,XForwarding is enabled in sshd config file,

Page 8: Secrets of a linux ninja  Software Freedom Day 2013 Johannesburg, South Africa

26/09/2013 Secrets of a Linux Ninja - SFD 2013 Johannesburg, South Africa

file:///home/mark/Documents/presentations/linux-ninja/index.html#/ 8/22

BYPASS WEB PROXY RESTRICTIONS

Scenario:

Your offices uses a web proxy which restricts traffic to sites with vital information,You have external SSH access,

Page 9: Secrets of a linux ninja  Software Freedom Day 2013 Johannesburg, South Africa

26/09/2013 Secrets of a Linux Ninja - SFD 2013 Johannesburg, South Africa

file:///home/mark/Documents/presentations/linux-ninja/index.html#/ 9/22

LESSON 2: NINJA MOVES

Page 10: Secrets of a linux ninja  Software Freedom Day 2013 Johannesburg, South Africa

26/09/2013 Secrets of a Linux Ninja - SFD 2013 Johannesburg, South Africa

file:///home/mark/Documents/presentations/linux-ninja/index.html#/ 10/22

Transferring files over SSHResuming partial transfers over SSHMounting a share over SSHTransferring files with netcat

“Do not pray for an easy life, pray for the strength to endure adifficult one.”

Page 11: Secrets of a linux ninja  Software Freedom Day 2013 Johannesburg, South Africa

26/09/2013 Secrets of a Linux Ninja - SFD 2013 Johannesburg, South Africa

file:///home/mark/Documents/presentations/linux-ninja/index.html#/ 11/22

TRANSFERRING FILES OVER SSH

Scenario:

You have no ftp/NFS/Samba share access on the server.You need to copy over a file.You have SSH access.

Page 12: Secrets of a linux ninja  Software Freedom Day 2013 Johannesburg, South Africa

26/09/2013 Secrets of a Linux Ninja - SFD 2013 Johannesburg, South Africa

file:///home/mark/Documents/presentations/linux-ninja/index.html#/ 12/22

RESUMING PARTIAL TRANSFERS OVER SSH

Scenario:

You are using scp to copy files,Your download 60 minute download interrupts after 40 minutes and you want toresume.

Page 13: Secrets of a linux ninja  Software Freedom Day 2013 Johannesburg, South Africa

26/09/2013 Secrets of a Linux Ninja - SFD 2013 Johannesburg, South Africa

file:///home/mark/Documents/presentations/linux-ninja/index.html#/ 13/22

MOUNTING A SHARE OVER SSH

Scenario:

You want to mount a remote folder on a server,You have no share access,Only have SSH access

Page 14: Secrets of a linux ninja  Software Freedom Day 2013 Johannesburg, South Africa

26/09/2013 Secrets of a Linux Ninja - SFD 2013 Johannesburg, South Africa

file:///home/mark/Documents/presentations/linux-ninja/index.html#/ 14/22

TRANSFERRING FILES WITH NETCAT

Scenario:

You need to transfer a file and have no SSH access.

Page 15: Secrets of a linux ninja  Software Freedom Day 2013 Johannesburg, South Africa

26/09/2013 Secrets of a Linux Ninja - SFD 2013 Johannesburg, South Africa

file:///home/mark/Documents/presentations/linux-ninja/index.html#/ 15/22

LESSON 3: A NINJA OBSERVES

Page 16: Secrets of a linux ninja  Software Freedom Day 2013 Johannesburg, South Africa

26/09/2013 Secrets of a Linux Ninja - SFD 2013 Johannesburg, South Africa

file:///home/mark/Documents/presentations/linux-ninja/index.html#/ 16/22

Monitoring disks for signs of failureCommand line tools: iotop,vmstat,iostat,psacct

“I fear not the man who has practiced 10,000 kicks once, but Ifear the man who has practiced one kick 10,000 times.”

Page 17: Secrets of a linux ninja  Software Freedom Day 2013 Johannesburg, South Africa

26/09/2013 Secrets of a Linux Ninja - SFD 2013 Johannesburg, South Africa

file:///home/mark/Documents/presentations/linux-ninja/index.html#/ 17/22

MONITORING DISKS FOR SIGNS OF FAILURE

Scenario:

You know your disks will fail. You just don't know when.Use smartctl to monitor your disks.

Page 18: Secrets of a linux ninja  Software Freedom Day 2013 Johannesburg, South Africa

26/09/2013 Secrets of a Linux Ninja - SFD 2013 Johannesburg, South Africa

file:///home/mark/Documents/presentations/linux-ninja/index.html#/ 18/22

COMMAND LINE TOOLS

Scenario:

What process is grinding your disks? - iotopWhere is the bottle neck - disk io, memory or network - vmstatWhich user or system account is using all your resources - psacctYour server crashes and you don't know why - sar

Page 19: Secrets of a linux ninja  Software Freedom Day 2013 Johannesburg, South Africa

26/09/2013 Secrets of a Linux Ninja - SFD 2013 Johannesburg, South Africa

file:///home/mark/Documents/presentations/linux-ninja/index.html#/ 19/22

LESSON 4: A CLEAN KILL

Page 20: Secrets of a linux ninja  Software Freedom Day 2013 Johannesburg, South Africa

26/09/2013 Secrets of a Linux Ninja - SFD 2013 Johannesburg, South Africa

file:///home/mark/Documents/presentations/linux-ninja/index.html#/ 20/22

Using regular expressions to killWho is preventing you from unmounting a block device?

“Take things as they are. Punch when you have to punch. Kickwhen you have to kick.”

Page 21: Secrets of a linux ninja  Software Freedom Day 2013 Johannesburg, South Africa

26/09/2013 Secrets of a Linux Ninja - SFD 2013 Johannesburg, South Africa

file:///home/mark/Documents/presentations/linux-ninja/index.html#/ 21/22

USING REGULAR EXPRESSIONS TO KILL

Scenario:

You have a process to kill often and it a mission to get its pid all the timepkillkillall -r

Page 22: Secrets of a linux ninja  Software Freedom Day 2013 Johannesburg, South Africa

26/09/2013 Secrets of a Linux Ninja - SFD 2013 Johannesburg, South Africa

file:///home/mark/Documents/presentations/linux-ninja/index.html#/ 22/22

WHO IS PREVENTING YOU FROM UNMOUNTING A BLOCKDEVICE?

Scenario:

You trying to unmount a device but you can't because a process/user is accessing itfuser