*nix Command LineKung Foo

DevChatt, March 2010

Who is this guy?

Brian Daileyrealm3 web applicationsNashville

Why the command line?

It is concise.

It is consistent.

It saves time.

What command line?

Posix compliant shell(Preferably bash or dash.)

Not using *nix? Pity.

Mac OS X:
You're good!

Windows:Use cygwin (preferable) or a Unix port such as:http://unxutils.sourceforge.net/

Other: PuTTY + Amazon EC2, Ubuntu Live CD, etc.

Basics:
Keyboard Navigation

Shell Keyboard Navigation Basics

- Scroll through previous commands
Ctrl+U- Clear all before cursor
Ctrl+K- Clear all after cursor
Ctrl+A-
Ctrl+E-
Alt+B- Move cursor back one word (Similar to Ctrl+)
Alt+F- Move cursor forward one word (Ctrl+)
Alt+Bckspc- Delete previous word (Also, Ctrl+W)
Alt+.- Recall last command argument
Tab- Autocomplete commands, files, folders, and more.
Ctrl+L- Clear the screen
Ctrl+_- Undo
CTRL+R- Recall previous commands
Esc, T- Swap last two words

These shortcuts are nearly universal (bash, MySQL) due to use of the GNU Readline library.
Exhaustive list is available here:
http://onlyubuntu.blogspot.com/2007/03/bash-shell-keyboard-shortcuts-for-linux.html

Goodbye, Mouse!

Example:mysql> SELECT user.id, user.username, user.email, user.address, user.city, user.state, user.name, userfile.file_name, userfile.metadata, userfile.other_stuff FOM user JOIN userfiles ON user.id = userfile.user_id AND user.id BETWEEN 1 AND 100 AND user.created BETWEEN '2008-10-10' AND '2009-10-10' AND user.is_active = 1 AND user.name LIKE 'Frodo%' LIMIT 10;[]

n00b:Hold backspace key for 30 seconds. Fix, retype.Good:Press Alt+B to jump backwards one word at a time.Holy Hand Grenade:Press Alt+3,0,B to jump backwards 30 words.

Shell Keyboard Navigation

Kill and yank = Cut and paste

Kill Commands:
Ctrl+U- Kill all before cursor
Ctrl+K- Kill all after cursorAlt+Bckspc Kill previous word

All kill commands place text into a buffer (aka kill ring).

Ctrl+Y YankAlt+Y - Rotate kill ring, yank new top

Technically yanks text from top of kill ring. To rotate through the ring you must first yank, then press Alt+M to rotate.

Shell Keyboard Navigation

Use Tab for auto-completion!
$ svn ci -mFixed issue. fil[tab]

$ svn ci -mFixed issue. filename.php

[Tab][Tab] shows all available matches:
$ svn ci -mFixed issue. f[tab][tab]
filename.php
frodo_baggins.php
friendly_fellow.php

Enable programmable bash completion for further efficiency (compete args).
$ git c[tab][tab]
checkout cherry cherry-pick ci clean clone co commit config

Shell Keyboard Navigation

!!- Run the last command (same as +return)
!my- Runs last cmd starting with my, e.g., mysql -uroot -p db
!my:p- Find last cmd starting with my and print it out
vi !$ - Replaces !$ with last argument in previous command
vi !*- Insert all arguments from previous command^foo^bar- Run previous command but replace first foo with bar!!:gs/foo/bar Run previous command, but replace all instances of foo with barcd -- change to previous working directory

A very useful (and more comprehensive) list is available here:
http://mail.linux.ie/pipermail/ilug/2006-May/087799.html

Also recommended:http://www.catonmat.net/blog/the-definitive-guide-to-bash-command-line-history/

Shell Basics: I/O Redirect

Concept can be thought of as pipes and streams

Negates the need for an application to write to a file, only for another application to read it.

I/O filters

less Interactive text pager with search capability.sort Sort lines (alphabetically, numerically, etc)grep Regular expression searchesuniq Detect (report or omit) repeated linessed Stream EDitorawk powerful pattern scanning and text processing languagexargs build and execute commands from inputcut- divide into columns

There are a lot more of these tools!

I/O

Basic example of standard output:
# format an XML file and feed it out to formatted.xml
$ xmllint feed.xml --format > formatted.xml
# same as above, but append to existing file.
$ xmllint feed.xml --format >> formatted.xml

Standard input:
$ mysql -uuser -p db1 < schema.sql

Or both:
$ mysql -uuser -p db1 < schema.sql > log.txt

Can also redirect stderr (errors).

Use the pipe!
$ history | grep svn

I/O

Examples of pipe chaining...

Colorize Subversion diff output:
$ svn diff | colordiff | less -R

Search files for child classes, do not include test files, sort them alphabetically, and paginate.
$ grep -r -n -H '^class.*extends.*{$' * | grep -v test | sort | more

I/O

Example:

# copy the latest changeset to a git-less server$ git log -1 --name-status | \# only get files added or modifiedgrep '^[AM]\s' | \# strip out status from streamsed 's/^[AM]\s*//' | \# scp to server (one at a time), retaining pathxargs -I {} scp {} user@myserver.com:dir/{}

(Disclaimer: There are better ways to do deploy changes.)

Holy smokes! How will I remember to type all that?

Let's automate this stuff!

Introducing Alias

alias deploy=git log -1 --name-status | \grep '^[AM]\s' | \sed 's/^[AM]\s*//' | \xargs -I {} scp {} user@myserver.com:dir/{}

# Other Examples# List files in friendly format
alias l='ls -l'
# Colorize svn diff, use less -R to accept colors
alias sdv='svn diff | colordiff | less -R'
# Do not allow updates or deletes without a primary key
alias mysql='mysql --i-am-a-dummy'

Alias

Things to know:

With alias, all arguments are appended to the command.

Save your aliases to your .bashrc to automatically use in new sessions.

Alias

# Usage: fun [name]
alias fun='echo $1 is having fun!'

$ fun Brian
is having fun! Brian

Alias is only useful as the beginning of a command.Arguments don't work well, can't do loops, etc.

Shell Functions

Look up PHP command arguments:
phpargs() {
# $1 is the first argument passed
# get php page from manual mirror
curl -s http://us3.php.net/$1 | \
# pull all text in target div
sed -n '//,//p' | \
# strip out HTML tags
sed 's/]*>//g' | \
# strip out line feed
tr -d "\n"
# add ending line
echo
}Then from the command line:
$ phpargs in_array
bool in_array ( mixed $needle, array $haystack [, bool $strict ] )The only way to learn scripts is to write them!

Further Resources

Comprehensive Introduction to the shell
http://www.linuxcommand.org/learning_the_shell.php

Bash keyboard shortcuts
http://onlyubuntu.blogspot.com/2007/03/bash-shell-keyboard-shortcuts-for-linux.html

Bash arguments
http://www.deadman.org/bash.html

Bash programmable completion:
http://www.debian-administration.org/articles/316
http://freshmeat.net/projects/bashcompletion/

IBM Bash By Tutorial Series
http://www.ibm.com/developerworks/library/l-bash.html
http://www.ibm.com/developerworks/library/l-bash2.html
http://www.ibm.com/developerworks/library/l-bash3.html

Cat on Mat:http://catonmat.com/

Thanks!

Any questions?

Brian Daileyrealm3 web applications

Web: http://realm3.com/Twitter:@brian_daileyEmail:brian@realm3.com/Phone: 917-512-3594

slide-bg: http://bit.ly/xc0m1Kudos to: http://asi9.net/