Introduction to sed

Sed : a “SSed : a “Streamtream ED EDitoritor””

What is Sed ? A “non-interactive” text editor

that is called from the unix command line.

Input text flows through the program, is modified, and is directed to standard output.

Sed : a “SSed : a “Streamtream ED EDitoritor””An Example:

The following sentence is input to the sed program:

echo "Instrumental in ruining entire operation for a Midwest chain operation." |sed 's/ruining/running/‘

Instrumental in running entire operation for a Midwest chain operation.

sed: string replacement

Line-oriented tool for pattern matching and replacement (stream editor)

Not really a programming language (cf. awk)

E.g., apply same change to lots of source files

Filter, i.e., does not modify input file

sed description

pattern a text add to output address s /regex/replacement/ address d delete line delete lines 1-10: sed -e '1,10d‘ delete comments: sed -e '/^#/d‘ print only matching: sed -n -e '/regexp/p

convert Unix to DOS: sed -e 's/$/\r/' myunix.txt > mydos.txt

Eliminate the tedium of routine editing tasks! (find, replace, delete, append, insert) … but your word processor can already do that right? Wrong.Sed is extremely powerful AND comes with every Unix system in the world!

Why Use Sed?Why Use Sed?

Sed is designed to be especially useful in three cases: To edit files too large for comfortable

interactive editing; To edit any size file when the sequence of

editing commands is too complicated to be comfortably typed in interactive mode.

To perform multiple `global' editing functions efficiently in one pass through the input.

Why Use Sed?Why Use Sed?

How Sed Works (cont…)How Sed Works (cont…)echo “amy enjoys hiking and ben enjoys skiing” | sed –e ‘s/skiing/hiking/g; s/hiking/biking/g’

1 ) Sed read in the line “amy enjoys hiking and ben enjoys skiing” and executed the first ‘substitute’ command.The resulting line – in the pattern space:

“amy enjoys hiking and ben enjoys hiking”

2) Then the second substitute command is executed on the line in the pattern space, and the result is : “amy enjoys biking and ben enjoys biking”

3) And the result is written to standard out.

Invoking Sed CommandsInvoking Sed Commands$ sed [-e script] [-f script-file] [-n] [files...]

-e an "in-line" script, i.e. a script to sed execute given on the command line. Multiple command line scripts can be given, each with an -e option.

-n by default, sed writes each line to stdout when it reaches the end of the script (being whatever on the line)

this option prevents that. i.e. no output unless there is a command to order SED specifically to do it

-f read scripts from specified file, several -f options can appear

files are the files to read, if a "-" appears, read from stdin,if no files are given, read also from stdin

Invoking Sed CommandsInvoking Sed Commands

Different Ways to Invoke Sed:

sed –e 'command;command;command' input_file see results

sed –e 'command;command;command' input_file > output_file save results

.... | sed –e 'command;command;command' | .... use in a pipeline

sed -f sedcommands input_file > output_file commands are in file somewhere else

Invoking Sed (some notes)Invoking Sed (some notes)

sed commands are usually on one line

if we want more (multi-line commands), then we must end the first line with an `\'

if a command is one line only, it can be separated by a `;‘

if it is a multi-line, then it must contain all of its line (except the first) by themselves

on command line, what follows a `-e' is like a whole line in a sed script

Regular ExpressionsRegular ExpressionsSed uses regular expressions to match patterns in the input text, and thenperform operations on those patterns. ^ matches the beginning of the line $ matches the end of the line . Matches any single character\ Escapes any metacharacter that follows, including itself.(character)* Match arbitrarily many occurences of (character)(character)? Match 0 or 1 instance of (character)(character)+ Match 1 or more instances of (character)[abcdef] Match any character enclosed in [ ] (in this instance, a b c d e or f)[^abcdef] Match any character NOT enclosed in [ ] (character)\{m,n\}Match m-n repetitions of (character)(character)\{m,\} Match m or more repetitions of (character)(character)\{,n\} Match n or less (possibly 0) repetitions of (character)(character)\{n\} Match exactly n repetitions of (character)\{n,m\} range of occurrences, n and m are integers \(expression\) Group operator.expression1|expression2 Matches expression1 or expression 2. () groups regular expressions

Regular Expressions (character classes)

The following character classes are short-hand for matching special characters.

[:alnum:] Printable characters (includes white space)

[:alpha:] Alphabetic characters

[:blank:] Space and tab characters

[:cntrl:] Control characters

[:digit:] Numeric characters

[:graph:] Printable and visible (non-space) characters

[:lower:] Lowercase characters

[:print:] Alphanumeric characters

[:punct:] Punctuation characters

[:space:] Whitespace characters

[:upper:] Uppercase characters

[:xdigit:] Hexadecimal digits

Regular Expressions Regular Expressions (cont…)(cont…)

/^M.*/

/..*/

/^$/

ab|cd

a(b*|c*)d 

[[:space:][:alnum:]] 

Line begins with capital M, 0 or more chars follow

At least 1 character long (/.+/ means the same thing)

The empty line

Either ‘ab’ or ‘cd’

matches any string beginning with a letter a, followed by either zeroor more of the letter b, or zero or more of the letter c, followed by the letter d.

Matches any character that is either a white

space character or alphanumeric. Note:

Sed always tries to find the longest matching pattern in the

input. How would you match a tag in an HTML document?

Line AddressesLine Addresses

Each line read is counted, and one can use this information to absolutely select which lines commands should be applied to.

1 first line 2 second line ... $ last line i,j from i-th to j-th line, inclusive. j can be $

Examples : sed ’53!d’ prints through line number 52sed –n ‘4,9p’ prints only lines 4 through 9

Context AddressesThe second kind of addresses are context, or Regular Expression,

addresses. Commands will be executed on all pattern spaces matched by that RE.

Examples:

sed ‘/^$/d’ will delete all empty linessed ‘/./,$!d’ will delete all leading blank lines at the top of file

Some Rules:

commands may take 0, 1 or 2 addresses if no address is given, a command is applied to all pattern spaces if 1 address is given, then it is applied to all pattern spaces that match

that address if 2 addresses are given, then it is applied to all formed pattern spaces

between the pattern space that matched the first address, and the next pattern space matched by the second address.

If pattern spaces are all the time single lines, this can be said like, if 2 addrs are given, then the command will be executed on all lines between first addr and second (inclusive)

Sed CommandsSed CommandsWe will go over the only some basic sed commands.

a append c change linesd delete lines i insert p print lines s substitute