Special Topics BSC4933/5936Special Topics BSC4933/5936

Florida State UniversityFlorida State University

The Department of Biological ScienceThe Department of Biological Science

http://www.bio.fsu.edu

Sept. 23, 2003Sept. 23, 2003

An Introduction to BioinformaticsAn Introduction to Bioinformatics

How can you search the databases for similar How can you search the databases for similar

sequences, if pair-wise alignments take Nsequences, if pair-wise alignments take N22

time?! Significance and heuristics . . .time?! Significance and heuristics . . .

Database Similarity Database Similarity SearchingSearching

Steven M. ThompsonSteven M. Thompson

Florida State University School of Florida State University School of Computational Science and Computational Science and

Information Technology (Information Technology (CSITCSIT))

But, why even do database searches?But, why even do database searches?

We can imagine screening databases for sequences similar to ours We can imagine screening databases for sequences similar to ours

using the concepts of dynamic programming and log-odds scoring using the concepts of dynamic programming and log-odds scoring

matrices and some yet to be described tricks. But what do database matrices and some yet to be described tricks. But what do database

searches tell us; what can we gain from them? Why even bother?searches tell us; what can we gain from them? Why even bother?

Inference through homology is a fundamental principle of biologyInference through homology is a fundamental principle of biology. .

When a sequence is found to fall into a preexisting family we may be When a sequence is found to fall into a preexisting family we may be

able to infer function, mechanism, evolution, perhaps even structure, able to infer function, mechanism, evolution, perhaps even structure,

based on homology with its neighbors. If no significant similarity can based on homology with its neighbors. If no significant similarity can

be found, the very fact that your sequence is new and different could be found, the very fact that your sequence is new and different could

be very important. Granted, its characterization may prove difficult, be very important. Granted, its characterization may prove difficult,

but it could be well worth it.but it could be well worth it.

Homology and similarity —Homology and similarity —Don’t confuse homology with similarity: there is a huge difference! Don’t confuse homology with similarity: there is a huge difference!

Similarity is a statistic that describes how much two (sub)sequences Similarity is a statistic that describes how much two (sub)sequences

are alike according to some set scoring criteria. It can be normalized are alike according to some set scoring criteria. It can be normalized

to ascertain statistical significance, but it’s still just a number.to ascertain statistical significance, but it’s still just a number.

Homology, in contrast and by definition, implies an evolutionary Homology, in contrast and by definition, implies an evolutionary

relationship — more than just the fact that we’ve all evolved from the relationship — more than just the fact that we’ve all evolved from the

same old primordial ‘ooze.’ To demonstrate homology reconstruct the same old primordial ‘ooze.’ To demonstrate homology reconstruct the

phylogeny of the organisms or genes of interest. Better yet, show phylogeny of the organisms or genes of interest. Better yet, show

experimental evidence — structural, morphological, genetic, or fossil experimental evidence — structural, morphological, genetic, or fossil

— that corroborates your assertion.— that corroborates your assertion.

There is no such thing as percent homology; something is either There is no such thing as percent homology; something is either

homologous or it is not. Walter Fitch is credited with “homology is like homologous or it is not. Walter Fitch is credited with “homology is like

pregnancy — you can’t be 45% pregnant, just like something can’t be pregnancy — you can’t be 45% pregnant, just like something can’t be

45% homologous. You either are or you are not.” Highly significant 45% homologous. You either are or you are not.” Highly significant

similarity can argue for homology, but never the other way around.similarity can argue for homology, but never the other way around.

So, first — So, first — SignificanceSignificance: :

when is any alignment worth when is any alignment worth

anything biologically?anything biologically?

An old statistics trick — An old statistics trick — Monte CarloMonte Carlo simulations: simulations:

Z scoreZ score = [ = [ ( actual score ) - ( mean of randomized scores )( actual score ) - ( mean of randomized scores ) ] ]

( standard deviation of randomized score distribution )( standard deviation of randomized score distribution )

Independent of all that, what is a Independent of all that, what is a

‘good’ alignment?‘good’ alignment?

The The NormalNormal (Abby Normal?)(Abby Normal?) distributiondistribution — —

Many Z scores measure the distance from the mean Many Z scores measure the distance from the mean

using this simplistic Monte Carlo model assuming a using this simplistic Monte Carlo model assuming a

Gaussian distribution, aka the Gaussian distribution, aka the Normal distribution ( (

http://mathworld.wolfram.com/NormalDistribution.html),),

in spite of the fact that ‘sequence-space’ actually in spite of the fact that ‘sequence-space’ actually

follows what is know as the ‘Extreme Value follows what is know as the ‘Extreme Value

distribution.’distribution.’

However, the Monte Carlo method does approximate However, the Monte Carlo method does approximate

significance estimates fairly well.significance estimates fairly well.

< 2

0 6

50

0

:==

< 2

0 6

50

0

:==

2

2 0

0

:2

2 0

0

: 2

4 3

0

:=2

4 3

0

:= 2

6 2

2 8

:*2

6 2

2 8

:* 2

8 9

8 8

7:*

28

9

8 8

7:*

3

0 2

89

5

28

:*3

0 2

89

5

28

:* 3

2 1

71

4 2

04

2:=

==

*3

2 1

71

4 2

04

2:=

==

* 3

4 5

58

5 5

53

9:=

==

==

==

==

*3

4 5

58

5 5

53

9:=

==

==

==

==

* 3

6 1

24

95

1

13

75

:==

==

==

==

==

==

==

==

==

*==

36

1

24

95

1

13

75

:==

==

==

==

==

==

==

==

==

*==

3

8 2

19

57

1

87

99

:==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

=*=

==

==

38

2

19

57

1

87

99

:==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

=*=

==

==

4

0 2

88

75

4

0 2

88

75

2

62

23

:==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

=*=

==

=2

62

23

:==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

=*=

==

= 4

2 3

41

53

4

2 3

41

53

3

20

54

:==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

=*=

32

05

4:=

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

*==

==

= 4

4 3

54

27

4

4 3

54

27

3

53

59

:==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

=3

53

59

:==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

*=

==

* 4

6 3

62

19

4

6 3

62

19

3

60

14

:==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

=3

60

14

:==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

=*

==

==

* 4

8 3

36

99

4

8 3

36

99

3

44

79

:==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

=3

44

79

:==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

*=

* 5

0 3

07

27

5

0 3

07

27

3

14

62

:==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

= *

31

46

2:=

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

* 5

2 2

72

88

2

76

61

:==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

=*

52

2

72

88

2

76

61

:==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

=*

5

4 2

25

38

2

36

27

:==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

*5

4 2

25

38

2

36

27

:==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

* 5

6 1

80

55

1

97

36

:==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

*

56

1

80

55

1

97

36

:==

==

==

==

==

==

==

==

==

==

==

==

==

==

==

*

5

8 1

46

17

1

62

03

:==

==

==

==

==

==

==

==

==

==

==

==

= *

58

1

46

17

1

62

03

:==

==

==

==

==

==

==

==

==

==

==

==

= *

6

0 1

25

95

1

31

25

:==

==

==

==

==

==

==

==

==

==

=*

60

1

25

95

1

31

25

:==

==

==

==

==

==

==

==

==

==

=*

6

2 1

05

63

1

05

22

:==

==

==

==

==

==

==

==

=*

62

1

05

63

1

05

22

:==

==

==

==

==

==

==

==

=*

6

4 8

62

6 8

36

8:=

==

==

==

==

==

==

*=6

4 8

62

6 8

36

8:=

==

==

==

==

==

==

*= 6

6 6

42

6 6

61

4:=

==

==

==

==

=*

66

6

42

6 6

61

4:=

==

==

==

==

=*

6

8 4

77

0 5

20

3:=

==

==

==

=*

68

4

77

0 5

20

3:=

==

==

==

=*

7

0 4

01

7 4

07

7:=

==

==

=*

70

4

01

7 4

07

7:=

==

==

=*

7

2 2

92

0 3

18

6:=

==

==

*7

2 2

92

0 3

18

6:=

==

==

* 7

4 2

44

8 2

48

4:=

==

=*

74

2

44

8 2

48

4:=

==

=*

7

6 1

69

6 1

93

3:=

==

*7

6 1

69

6 1

93

3:=

==

* 7

8 1

17

8 1

50

3:=

=*

78

1

17

8 1

50

3:=

=*

8

0 9

35

1

16

7:=

*8

0 9

35

1

16

7:=

* 8

2 7

22

8

93

:=*

82

7

22

8

93

:=*

8

4 4

54

7

07

:=*

84

4

54

7

07

:=*

8

6 4

38

5

47

:*8

6 4

38

5

47

:* 8

8 3

22

4

23

:*8

8 3

22

4

23

:* 9

0 2

57

3

28

:*9

0 2

57

3

28

:* 9

2 1

75

2

53

:*

92

1

75

2

53

:*

9

4 2

10

1

96

:*

94

2

10

1

96

:*

9

6 1

02

1

52

:*

96

1

02

1

52

:*

9

8 6

3 1

17

:*

98

6

3 1

17

:*

10

0 5

8 9

1:*

1

00

5

8 9

1:*

1

02

4

0 7

0:*

1

02

4

0 7

0:*

1

04

3

0 5

4:*

1

04

3

0 5

4:*

1

06

1

7 4

2:*

1

06

1

7 4

2:*

1

08

1

4 3

3:*

1

08

1

4 3

3:*

1

10

1

4 2

5:*

1

10

1

4 2

5:*

1

12

1

2 2

0:*

1

12

1

2 2

0:*

1

14

9

1

5:*

1

14

9

1

5:*

1

16

6

1

2:*

1

16

6

1

2:*

1

18

8

9

:*

11

8 8

9

:*

>1

20

1

03

0 7

:*=

>1

20

1

03

0 7

:*=

Based on this known statistical Based on this known statistical

distribution, and robust statistical distribution, and robust statistical

methodology, a realistic methodology, a realistic ExpectationExpectation

function, the function, the E ValueE Value, can be , can be

calculated from database searches.calculated from database searches.

The particulars of how this is done The particulars of how this is done

will wait, but the ‘take-home’ will wait, but the ‘take-home’

message is the same . . .message is the same . . .

‘‘Sequence-space’ Sequence-space’ (Huh, what’s that?)(Huh, what’s that?)

actually follows the ‘actually follows the ‘Extreme Value distribution’’((http://mathworld.wolfram.com/ExtremeValueDistribution.html).).

The Expectation Value?The Expectation Value?

The higher the E value is, the more probable The higher the E value is, the more probable

that the observed match is due to chance in a that the observed match is due to chance in a

search of the same size database, and the search of the same size database, and the

lower its Z score will be, i.e. is NOT significant.lower its Z score will be, i.e. is NOT significant.

Therefore, the smaller the E value, i.e. the Therefore, the smaller the E value, i.e. the

closer it is to zero, the more significant it is and closer it is to zero, the more significant it is and

the higher its Z score will be! The E value is the higher its Z score will be! The E value is

the number that really matters.the number that really matters.

Rules of thumb for a protein search —Rules of thumb for a protein search —

The Z score represents the number of standard deviations some The Z score represents the number of standard deviations some

particular alignment is from a distribution of random alignments particular alignment is from a distribution of random alignments

(the normal distribution).(the normal distribution).

They They very roughlyvery roughly correspond to the listed E Values (based on correspond to the listed E Values (based on

the Extreme Value distribution) for a typical protein sequence the Extreme Value distribution) for a typical protein sequence

similarity search through a database with ~125,000 protein similarity search through a database with ~125,000 protein

entries.entries.

On to the searches —On to the searches —But NBut N22 is way too slow! is way too slow!

How can it be done?How can it be done?

Database searching programs use the two concepts of Database searching programs use the two concepts of

dynamic programming and log-odds scoring matrices; dynamic programming and log-odds scoring matrices;

however, dynamic programming takes far too long when however, dynamic programming takes far too long when

used against most sequence databases with a ‘normal’ used against most sequence databases with a ‘normal’

computer. Remember computer. Remember how bighow big the databases are! the databases are!

Therefore, the programs use tricks to make things Therefore, the programs use tricks to make things

happen faster. These tricks fall into two main categories, happen faster. These tricks fall into two main categories,

that of that of hashinghashing, and that of , and that of approximationapproximation..

Corn beef hash? Huh . . .Corn beef hash? Huh . . .Hashing is the process of breaking your sequence into Hashing is the process of breaking your sequence into

small ‘words’ or ‘k-tuples’ (think all chopped up, just like small ‘words’ or ‘k-tuples’ (think all chopped up, just like

corn beef hash) of a set size and creating a ‘look-up’ corn beef hash) of a set size and creating a ‘look-up’

table with those words keyed to position numbers. table with those words keyed to position numbers.

Computers can deal with numbers way faster than they Computers can deal with numbers way faster than they

can deal with strings of letters, and this preprocessing can deal with strings of letters, and this preprocessing

step happens very quickly.step happens very quickly.

Then when any of the word positions match part of an Then when any of the word positions match part of an

entry in the database, that match, the ‘offset,’ is saved. entry in the database, that match, the ‘offset,’ is saved.

In general, hashing reduces the complexity of the search In general, hashing reduces the complexity of the search

problem from Nproblem from N22 for dynamic programming to N, the for dynamic programming to N, the

length of all the sequences in the database.length of all the sequences in the database.

A simple hash table —A simple hash table —(this example is from the Krane and Raymer text p.50)(this example is from the Krane and Raymer text p.50)

The sequence The sequence FAMLGFIKYLPGCMFAMLGFIKYLPGCM aand a word size of one,nd a word size of one,

would produce this query lookup hash table:would produce this query lookup hash table:

wordword AA CC FF GG II KK LL MM PPYY

Pos.Pos. 22 1313 11 55 77 88 44 33 1111 99

66 1212 1010 1414

comparing it to the database sequence comparing it to the database sequence TGFIKYLPGACTTGFIKYLPGACT,,

would yield the following offset table:would yield the following offset table:

11 22 33 44 55 66 77 88 99 1010 1111 1212

TT GG FF II KK YY LL PP GG AA CC TT

33 -2-2 33 33 33 -3-3 33 -4-4 -8-8 22

1010 33 33 33

Hmmm & some interpretation —Hmmm & some interpretation —

The offset numbers come from the difference The offset numbers come from the difference between the positions of the words in the query between the positions of the words in the query sequence and the position of the occurrence of that sequence and the position of the occurrence of that word in the target sequence. Then . . . .word in the target sequence. Then . . . .

Look at all of the offsets equal to three in the previous Look at all of the offsets equal to three in the previous table. Therefore, offset the alignment by three:table. Therefore, offset the alignment by three:

FAMLGFIKYLPGCMFAMLGFIKYLPGCM |||||||||||||||| TGFIKYLPGACTTGFIKYLPGACT

Quick and easy. Computers can compare these Quick and easy. Computers can compare these

sorts of tables very fast. The trick is to ‘know’ how far sorts of tables very fast. The trick is to ‘know’ how far

to attempt to extend the alignment out.to attempt to extend the alignment out.

OK. Heuristics . . . What’s that?OK. Heuristics . . . What’s that?Approximation techniques are collectively known as ‘heuristics.’ Approximation techniques are collectively known as ‘heuristics.’

Webster’s defines heuristic as “serving to guide, discover, or Webster’s defines heuristic as “serving to guide, discover, or

reveal; . . . but unproved or incapable of proof.”reveal; . . . but unproved or incapable of proof.”

In database similarity searching techniques the heuristic usually In database similarity searching techniques the heuristic usually

restricts the necessary search space by calculating some sort of a restricts the necessary search space by calculating some sort of a

statistic that allows the program to decide whether further scrutiny statistic that allows the program to decide whether further scrutiny

of a particular match should be pursued. This statistic may miss of a particular match should be pursued. This statistic may miss

things depending on the parameters set — that’s what makes it things depending on the parameters set — that’s what makes it

heuristic. heuristic. ‘Worthwhile’ results at the end are compiled and the ‘Worthwhile’ results at the end are compiled and the

longest alignment within the program’s restrictions is created.longest alignment within the program’s restrictions is created.

The exact implementation varies between the different programs, The exact implementation varies between the different programs,

but the basic idea follows in most all of them.but the basic idea follows in most all of them.

Two predominant versions exist: BLAST and FastTwo predominant versions exist: BLAST and Fast

Both return local alignments, and are not a single program, but Both return local alignments, and are not a single program, but

rather a family of programs with implementations designed to rather a family of programs with implementations designed to

compare a sequence to a database in about every which way compare a sequence to a database in about every which way

imaginable.imaginable.

These include:These include:

1)1) a DNA sequence against a DNA database (not recommended unless a DNA sequence against a DNA database (not recommended unless

forced to do so because you are dealing with a non-translated region of forced to do so because you are dealing with a non-translated region of

the genome — DNA is just too darn noisy, only identity & four bases!),the genome — DNA is just too darn noisy, only identity & four bases!),

2)2) a translated (where the translation is done ‘on-the-fly’ in all six frames) a translated (where the translation is done ‘on-the-fly’ in all six frames)

version of a DNA sequence against a translated (‘on-the-fly’ six-frame) version of a DNA sequence against a translated (‘on-the-fly’ six-frame)

version of the DNA database (not available in the Fast package),version of the DNA database (not available in the Fast package),

3)3) a translated (‘on-the-fly’ six-frame) version of a DNA sequence against a a translated (‘on-the-fly’ six-frame) version of a DNA sequence against a

protein database,protein database,

4)4) a protein sequence against a translated (‘on-the-fly’ six-frame) version of a protein sequence against a translated (‘on-the-fly’ six-frame) version of

a DNA database,a DNA database,

5)5) or a protein sequence against a protein database.or a protein sequence against a protein database.

Many implementations allow for the possibility of frame shifts in Many implementations allow for the possibility of frame shifts in

translated comparisons and don’t penalize the score for doing so.translated comparisons and don’t penalize the score for doing so.

The BLAST and Fast programs — some generalitiesThe BLAST and Fast programs — some generalities

BLAST — Basic Local Alignment BLAST — Basic Local Alignment

Search Tool, developed at NCBI.Search Tool, developed at NCBI.

1)1) Normally NOT a good idea Normally NOT a good idea

to use for DNA against to use for DNA against

DNA searches w/o DNA searches w/o

translation (not optimized);translation (not optimized);

2)2) Pre-filters repeat and “low Pre-filters repeat and “low

complexity” sequence complexity” sequence

regions;regions;

4)4) Can find more than one Can find more than one

region of gapped similarity;region of gapped similarity;

5)5) Very fast heuristic and Very fast heuristic and

parallel implementation;parallel implementation;

6)6) Restricted to precompiled, Restricted to precompiled,

specially formatted specially formatted

databases;databases;

FastA — and its family of relatives, FastA — and its family of relatives,

developed by Bill Pearson at the developed by Bill Pearson at the

University of Virginia.University of Virginia.

1)1) Works well for DNA against Works well for DNA against

DNA searches (within limits DNA searches (within limits

of possible sensitivity);of possible sensitivity);

2)2) Can find only one gapped Can find only one gapped

region of similarity;region of similarity;

3)3) Relatively slow, should Relatively slow, should

often be run in the often be run in the

background;background;

4)4) Does not require specially Does not require specially

prepared, preformatted prepared, preformatted

databases.databases.

The algorithms, in brief —The algorithms, in brief —

BLAST:BLAST:

Fast:Fast:

Two word hits on the Two word hits on the same diagonal above same diagonal above some some similaritysimilarity threshold triggers threshold triggers ungapped extension ungapped extension until the score isn’t until the score isn’t improved enough above improved enough above another threshold:another threshold:

the HSP.the HSP.

Find all ungapped Find all ungapped exact exact word hits; maximize the word hits; maximize the ten best continuous ten best continuous regions’ scores: regions’ scores: init1init1..

Combine non-Combine non-overlapping init overlapping init regions on different regions on different diagonals:diagonals:initninitn..

Use dynamic Use dynamic programming ‘in a programming ‘in a band’ for all regions band’ for all regions with with initninitn scores scores better than some better than some threshold: threshold: optopt score.score.

Initiate gapped extensions Initiate gapped extensions using dynamic programming for using dynamic programming for those HSP’s above a third those HSP’s above a third threshold up to the point where threshold up to the point where the score starts to drop below a the score starts to drop below a fourth threshold: yields fourth threshold: yields alignment.alignment.

BLAST — the algorithm in more detailBLAST — the algorithm in more detail1)1) After BLAST has sorted its lookup table, it tries to find all double word After BLAST has sorted its lookup table, it tries to find all double word

hits along the same diagonal within some specified distance using what hits along the same diagonal within some specified distance using what

NCBI calls a Discrete Finite Automaton (DFA). These word hits of size NCBI calls a Discrete Finite Automaton (DFA). These word hits of size

WW do not have to be identical; rather, they have to be better than some do not have to be identical; rather, they have to be better than some

threshold value threshold value TT. To identify these double word hits, the DFA scans . To identify these double word hits, the DFA scans

through all strings of words (typically through all strings of words (typically WW=3 for peptides) that score at =3 for peptides) that score at

least least TT (usually 11 for peptides). (usually 11 for peptides).

2)2) Each double word hit that passes this step then triggers a process called Each double word hit that passes this step then triggers a process called

un-gapped extension in both directions, such that each diagonal is un-gapped extension in both directions, such that each diagonal is

extended as far as it can, until the running score starts to drop below a extended as far as it can, until the running score starts to drop below a

pre-defined value pre-defined value XX within a certain range within a certain range AA. The result of this pass is . The result of this pass is

called a High-Scoring segment Pair or HSP.called a High-Scoring segment Pair or HSP.

3)3) Those HSPs that pass this step with a score better than Those HSPs that pass this step with a score better than SS then begin a then begin a

gapped extension step utilizing dynamic programming. Those gapped gapped extension step utilizing dynamic programming. Those gapped

alignments with Expectation values better than the user specified cutoff alignments with Expectation values better than the user specified cutoff

are reported. The extreme value distribution of BLAST Expectation are reported. The extreme value distribution of BLAST Expectation

values is pre-computed against each precompiled database — this is values is pre-computed against each precompiled database — this is

one area that speeds up the algorithm considerably.one area that speeds up the algorithm considerably.

The BLAST algorithm, continuedThe BLAST algorithm, continuedThe math can be generalized thus: for any two sequences of length The math can be generalized thus: for any two sequences of length mm and and

nn, local, best alignments are identified as HSPs. HSPs are stretches of , local, best alignments are identified as HSPs. HSPs are stretches of

sequence pairs that cannot be further improved by extension or trimming, sequence pairs that cannot be further improved by extension or trimming,

as described above. For un-gapped alignments, the number of expected as described above. For un-gapped alignments, the number of expected

HSPs with a score of at least HSPs with a score of at least SS is given by the formula: is given by the formula:

E = KmneE = Kmness

This is called an This is called an EE-value for the score -value for the score SS. In a database search . In a database search nn is the is the

size of the database in residues, so size of the database in residues, so NN==mnmn is the search space size. is the search space size. KK and and

are be supplied by statistical theory, and, as mentioned above, can be are be supplied by statistical theory, and, as mentioned above, can be

calculated by comparison to pre-computed, simulated distributions. These calculated by comparison to pre-computed, simulated distributions. These

two parameters define the statistical significance of an two parameters define the statistical significance of an EE-value.-value.

The The EE-value defines the significance of the search. As mentioned above, -value defines the significance of the search. As mentioned above,

the smaller an the smaller an EE-value is, the more likely it is significant. A value of 0.01 to -value is, the more likely it is significant. A value of 0.01 to

0.001 is a good starting point for significance in most typical searches. In 0.001 is a good starting point for significance in most typical searches. In

other words, in order to assess whether a given alignment constitutes other words, in order to assess whether a given alignment constitutes

evidence for homology, it helps to know how strong an alignment can be evidence for homology, it helps to know how strong an alignment can be

expected from chance alone.expected from chance alone.

The Fast algorithm — in more detailThe Fast algorithm — in more detailFast is an older algorithm than BLAST. The original Fast paper Fast is an older algorithm than BLAST. The original Fast paper

came out in 1988, based on David Lipman’s work in a 1983 paper; came out in 1988, based on David Lipman’s work in a 1983 paper;

the original BLAST paper was published in 1990. Both algorithms the original BLAST paper was published in 1990. Both algorithms

have been upgraded substantially since originally released. have been upgraded substantially since originally released.

Fast was the first widely used, powerful sequence database Fast was the first widely used, powerful sequence database

searching algorithm. Bill Pearson continually refines the programs searching algorithm. Bill Pearson continually refines the programs

such that they remain a viable alternative to BLAST, especially if such that they remain a viable alternative to BLAST, especially if

one is restricted to searching DNA against DNA without translation. one is restricted to searching DNA against DNA without translation.

They are also very helpful in situations where BLAST finds no They are also very helpful in situations where BLAST finds no

significant alignments — arguably, Fast may be more sensitive than significant alignments — arguably, Fast may be more sensitive than

BLAST in these situations.BLAST in these situations.

Fast is also a hashing style algorithm and builds words of a set k-Fast is also a hashing style algorithm and builds words of a set k-

tuple size, by default two for peptides. It then identifies all exact tuple size, by default two for peptides. It then identifies all exact

word matches between the sequence and the database members. word matches between the sequence and the database members.

Note that the word matches must be exact for Fast and only similar, Note that the word matches must be exact for Fast and only similar,

above some threshold, for BLAST.above some threshold, for BLAST.

The Fast algorithm, continuedThe Fast algorithm, continuedFrom these exact word matches:From these exact word matches:

1)1) Scores are assigned to each continuous, ungapped, diagonal by Scores are assigned to each continuous, ungapped, diagonal by

adding all of the exact match BLOSUM values.adding all of the exact match BLOSUM values.

2)2) The ten highest scoring diagonals for each query-database pair The ten highest scoring diagonals for each query-database pair

are then rescored using BLOSUM similarities as well as identities are then rescored using BLOSUM similarities as well as identities

and ends are trimmed to maximize the score. The best of each and ends are trimmed to maximize the score. The best of each

of these is called the of these is called the Init1Init1 score. score.

3)3) Next the program ‘looks’ around to see if nearby off-diagonal Next the program ‘looks’ around to see if nearby off-diagonal Init1Init1

alignments can be combined by incorporating gaps. If so, a new alignments can be combined by incorporating gaps. If so, a new

score, score, InitnInitn, is calculated by summing up all the contributing , is calculated by summing up all the contributing Init1Init1

scores, penalizing gaps with a penalty for each.scores, penalizing gaps with a penalty for each.

4)4) The program then constructs an optimal local alignment for all The program then constructs an optimal local alignment for all

InitnInitn pairs with scores better than some set threshold using a pairs with scores better than some set threshold using a

variation of dynamic programming “in a band.” A sixteen residue variation of dynamic programming “in a band.” A sixteen residue

band centered at the highest band centered at the highest Init1Init1 region is used by default with region is used by default with

peptides. The score generated from this step called peptides. The score generated from this step called optopt..

The Fast algorithm, still continuedThe Fast algorithm, still continued5)5) Next, Fast uses a simple linear regression against the natural Next, Fast uses a simple linear regression against the natural

log of the search set sequence length to calculate a normalized log of the search set sequence length to calculate a normalized

z-score for the sequence pair. Note that this is not the same z-score for the sequence pair. Note that this is not the same

Monte Carlo style Z score described earlier, and can not be Monte Carlo style Z score described earlier, and can not be

directly compared to one. directly compared to one.

6)6) Finally, it compares the distribution of these z-scores to the Finally, it compares the distribution of these z-scores to the

actual extreme-value distribution of the searchactual extreme-value distribution of the search. Using this . Using this

distribution, the program estimates the number of sequences distribution, the program estimates the number of sequences

that would be expected to have, purely by chance, a z-score that would be expected to have, purely by chance, a z-score

greater than or equal to the z-score obtained in the search. This greater than or equal to the z-score obtained in the search. This

is reported as the Expectation value. is reported as the Expectation value.

7)7) If the user requests pair-wise alignments in the output, then the If the user requests pair-wise alignments in the output, then the

program uses full Smith-Waterman local dynamic programming, program uses full Smith-Waterman local dynamic programming,

not ‘restricted to a band,’ to produce its final alignments.not ‘restricted to a band,’ to produce its final alignments.

Let’s see ‘em in actionLet’s see ‘em in actionTo begin we’ll go to the most widely used (and abused!) To begin we’ll go to the most widely used (and abused!) biocomputing program on earth: NCBI’s BLAST —biocomputing program on earth: NCBI’s BLAST —

Connect to NCBI’s BLAST page with any Web browser:Connect to NCBI’s BLAST page with any Web browser:

http://www.ncbi.nlm.nih.gov/BLAST/http://www.ncbi.nlm.nih.gov/BLAST/..

There is a wealth of information there, including a There is a wealth of information there, including a wonderful tutorial and several very good essays for wonderful tutorial and several very good essays for teaching yourself way more about BLAST than this lecture teaching yourself way more about BLAST than this lecture can ever hope for.can ever hope for.

For now I’ll demonstrate with a simple example, one of my For now I’ll demonstrate with a simple example, one of my favorites, the elongation factor 1favorites, the elongation factor 1 protein from protein from Giardia Giardia lamblialamblia, named EF1A_Giala in the Swiss-Prot database, , named EF1A_Giala in the Swiss-Prot database, but we have to use the accession code, Q08046, for but we have to use the accession code, Q08046, for NCBI’s BLAST server to find the sequence. Let’s see NCBI’s BLAST server to find the sequence. Let’s see how it works and how quickly we get results back.how it works and how quickly we get results back.

Let’s contrast that with GCG’s BLAST version.Let’s contrast that with GCG’s BLAST version.

I’ll illustrate with the same molecule and I’ll use GCG’s I’ll illustrate with the same molecule and I’ll use GCG’s

SeqLab GUI to show the difference between the two SeqLab GUI to show the difference between the two

implementations of the program.implementations of the program.

And finally, let’s see how GCG’s FastA version And finally, let’s see how GCG’s FastA version

compares to either BLAST implementation.compares to either BLAST implementation.

Again, I’ll launch the program from SeqLab with the Again, I’ll launch the program from SeqLab with the

same example, but this time I’ll take advantage of same example, but this time I’ll take advantage of

Fast’s flexible database search syntax, being able to Fast’s flexible database search syntax, being able to

use any valid GCG sequence specification. Here I’ll use any valid GCG sequence specification. Here I’ll

search against a precompiled LookUp list file of all of search against a precompiled LookUp list file of all of

the so-called ‘primitive’ eukaryotes in Swiss-Prot.the so-called ‘primitive’ eukaryotes in Swiss-Prot.

Finally — Why do I keep ‘diss’ing’ DNA Finally — Why do I keep ‘diss’ing’ DNA for searches and alignment?for searches and alignment?

All database similarity searching and sequence All database similarity searching and sequence alignment, regardless of the algorithm used, is far more alignment, regardless of the algorithm used, is far more sensitive at the amino acid level than at the DNA level. sensitive at the amino acid level than at the DNA level. This is because proteins have twenty match criteria This is because proteins have twenty match criteria versus DNA’s four, and those four DNA bases can versus DNA’s four, and those four DNA bases can generally only be identical, not similar, to each other; and generally only be identical, not similar, to each other; and many DNA base changes (especially third position many DNA base changes (especially third position changes) do not change the encoded protein.changes) do not change the encoded protein.

All of these factors drastically increase the ‘noise’ level of All of these factors drastically increase the ‘noise’ level of a DNA against DNA search, and give protein searches a a DNA against DNA search, and give protein searches a much greater ‘look-back’ time, typically doubling it. much greater ‘look-back’ time, typically doubling it.

Therefore, whenever dealing with coding sequence, it is Therefore, whenever dealing with coding sequence, it is always prudent to search at the protein level!always prudent to search at the protein level!

Altschul, S. F., Gish, W., Miller, W., Myers, E. W., and Lipman, D. J. (1990) Basic Local Alignment Tool. Altschul, S. F., Gish, W., Miller, W., Myers, E. W., and Lipman, D. J. (1990) Basic Local Alignment Tool. Journal of Molecular BiologyJournal of Molecular Biology 215, 403-410. 215, 403-410.

Altschul, S.F., Madden, T.L., Schaffer, A.A., Zhang, J., Zhang, Z., Miller, W., and Lipman, D.J. (1997) Gapped Altschul, S.F., Madden, T.L., Schaffer, A.A., Zhang, J., Zhang, Z., Miller, W., and Lipman, D.J. (1997) Gapped BLAST and PSI-BLAST: a New Generation of Protein Database Search Programs. BLAST and PSI-BLAST: a New Generation of Protein Database Search Programs. Nucleic Acids Nucleic Acids ResearchResearch 25, 3389-3402. 25, 3389-3402.

Genetics Computer Group (GCG) (Copyright 1982-2002) Genetics Computer Group (GCG) (Copyright 1982-2002) Program Manual for the Wisconsin PackageProgram Manual for the Wisconsin Package , , Version 10.3, Accelrys, Inc. A Pharmocopeia Company, San Diego, California, U.S.A.Version 10.3, Accelrys, Inc. A Pharmocopeia Company, San Diego, California, U.S.A.

Gribskov, M. and Devereux, J., editors (1992) Gribskov, M. and Devereux, J., editors (1992) Sequence Analysis PrimerSequence Analysis Primer. W.H. Freeman and Company, New . W.H. Freeman and Company, New York, New York, U.S.A.York, New York, U.S.A.

Henikoff, S. and Henikoff, J.G. (1992) Amino Acid Substitution Matrices from Protein Blocks. Henikoff, S. and Henikoff, J.G. (1992) Amino Acid Substitution Matrices from Protein Blocks. Proceedings Proceedings of the National Academy of Sciences U.S.A.of the National Academy of Sciences U.S.A. 89, 10915-10919. 89, 10915-10919.

Needleman, S.B. and Wunsch, C.D. (1970) A General Method Applicable to the Search for Similarities in the Needleman, S.B. and Wunsch, C.D. (1970) A General Method Applicable to the Search for Similarities in the Amino Acid Sequence of Two Proteins. Amino Acid Sequence of Two Proteins. Journal of Molecular BiologyJournal of Molecular Biology 48, 443-453. 48, 443-453.

Pearson, W.R. and Lipman, D.J. (1988) Improved Tools for Biological Sequence Analysis. Pearson, W.R. and Lipman, D.J. (1988) Improved Tools for Biological Sequence Analysis. Proceedings of Proceedings of the National Academy of Sciences U.S.A.the National Academy of Sciences U.S.A. 85, 2444-2448. 85, 2444-2448.

Schwartz, R.M. and Dayhoff, M.O. (1979) Matrices for Detecting Distant Relationships. In Schwartz, R.M. and Dayhoff, M.O. (1979) Matrices for Detecting Distant Relationships. In Atlas of Protein Atlas of Protein Sequences and StructureSequences and Structure, (M.O. Dayhoff editor) 5, Suppl. 3, 353-358, National Biomedical Research , (M.O. Dayhoff editor) 5, Suppl. 3, 353-358, National Biomedical Research Foundation, Washington D.C., U.S.A.Foundation, Washington D.C., U.S.A.

Smith, T.F. and Waterman, M.S. (1981) Comparison of Bio-Sequences. Smith, T.F. and Waterman, M.S. (1981) Comparison of Bio-Sequences. Advances in Applied MathematicsAdvances in Applied Mathematics 2, 2, 482-489.482-489.

Wilbur, W.J. and Lipman, D.J. (1983) Rapid Similarity Searches of Nucleic Acid and Protein Data Banks. Wilbur, W.J. and Lipman, D.J. (1983) Rapid Similarity Searches of Nucleic Acid and Protein Data Banks. Proceedings of the National Academy of Sciences U.S.A.Proceedings of the National Academy of Sciences U.S.A. 80, 726-730. 80, 726-730.

References and a Comment:References and a Comment:The better you understand the chemical, physical, and biological systems The better you understand the chemical, physical, and biological systems involved, the better your chance of success in analyzing them. Certain involved, the better your chance of success in analyzing them. Certain strategies are inherently more appropriate to others in certain circumstances. strategies are inherently more appropriate to others in certain circumstances. Making these types of subjective, discriminatory decisions and utilizing all of the Making these types of subjective, discriminatory decisions and utilizing all of the available options so that you can generate the most practical data for evaluation available options so that you can generate the most practical data for evaluation are two of the most important ‘take-home’ messages that I can offer!are two of the most important ‘take-home’ messages that I can offer!