Implementation of the BLAST A lgorithm U sing Hadoop MapReduce

Implementation of the BLAST Algorithm Using Hadoop MapReduce

Siniša Ivković, Goran Rakočević, Prof. Veljko Milutinovic

University of BelgradeSchool of Electrical Engineering

Siniša Ivković - [email protected]

Introduction-Sequence alignment• way of arranging sequences of DNK, RNK or protein

to identify regions of similarity

• functional• structural• evolutionary relationships between sequences

2/13

- How to know that two genes, often in different organizams, in fact two versions of the same gene?

Similarity!


• There are a number of algorithms that solve problems of aligning the sequences and guarantee the best solutions

• By increasing amount of data that need to be processed execution speed of these algorithms becomes unacceptable

• Therefore, we must turn to heuristic methods - BLAST

Introduction

3/13


BLAST - Basic Local Alignment Search Tool

•Fast local sequence alignment algorithm

•BLAST efficiency lies in the fact that it tends to find regions ofhigh similarity, not necessarily trying to find and check all local alignment.

4/13

KRKLQRNRTSFTQEQIEALEKEFERTHYPDVFARERLAAKIDLPEARIQVWFSNRRAKWRREEKLKKKHRRNRTTFTTYQLHQLERAFEASHYPDVYSREELAAKVHLPEVRVQVWFQNRRAKWRRQERLKKKHRRNRTTFTTYQLHQLERAFEASHYPDVYSREELAAKVHLPEVRVQVWFQNRRAKWRRQERL


Parallel BLAST- Most bioinformatics algorithms are designed as a sequential

• The very nature of bioinformatics processing• The rapid spread of knowledge in biology causes

constant emergence of new concepts, andsignificant changes to already known

- Declining price of genome sequencing requires increasing the speed of execution of these algorithms

- Implementations of Parallel BLAST• PThread• MPI

5/13


ETF Hadoop BLAST- Big Data – collection of data sets so large and complex that it becomes difficult to process using standard database tools or traditional data processing applications- Parallel computing – a form of computation in which many calculations are carried out simultaneously

• communication and synchronization between processes • hardware failure

- MapReduce – programming model that frees programmers of thinking about these problems- Apache Hadoop – free implementation of the MapReduce paradigm

6/13


MapReduce

MAP

MAP

MAP

SORT

VALUE

VALUE

VALUE

VALUE

VALUE

VALUE

REDUCE

REDUCE

VALUE

VALUE

7/13


mySequence{q1}

{db1}

{db2}

{db3}

{q1} {db1} {q1} {db2} {q1} {db3}

MAP MAP MAP

{db1}{hit1}

{db1}{hit2}

{db2}{hit3}

{db2}{hit4}

{db3}{hit5}

{db3}{hit6}

ETF Hadoop BLAST - Implementation

8/13


mySequence{q1}

{db1}

{db2}

{db3}

{q1} {db1} {q1} {db2} {q1} {db3}

MAP MAP MAP

{db1}{hit1}

{db1}{hit2}

{db2}{hit3}

{db2}{hit4}

{db3}{hit5}

{db3}{hit6}

REDUCE REDUCE

{db1} {db2}

{db3}

{hit1} {hit3}

{hit6}

ETF Hadoop BLAST - Implementation

9/13


ETF Hadoop BLAST

>GENSCAN00000000013 pep:genscan chromosome:GRCh37:18:4755977:4807982:1 transcript:GENSCAN00000000013 transcript_biotype:protein_codingTANTGLLAVKVEVIILVSLTHAQLSRAGQHAGCTTCLQDECAVAAGEEEETQQGELADVIYPSLLAASTSSVLEDGAGPHKGLQKLSRLIRFVDVVGGFRREKGYMAWIKPRYSEFPKVNSWTESSFPFGTANTGLLAVKVEVIILVSLTHAQLSRAGQHAGCTTCLQDECAVAAGEEEETQQGELADVIYPSLLAASTSSVLEDGAGPHKGLQKLSRLIRFVDVVGGFRREKGYMAWIKPRYSEFPKVNSWTESSFPFGHSP: 661E-value: 0.001446314485823671

10/13


Conclusion- Bioinformatics has become an important part of many areas of biology

• Sequencing and annotating genomes and their observed mutations

• Datamining of biological literature and the development of gene ontologies

• Understanding of evolutionary aspects of molecular biology

- Personalized medicine• Medical model that proposes the customization of healthcare• We need to consider whole spectar of clinical information

• Electronic health care records• Clinical trials• etc.

11/13


Conclusion

- We need to collect information from real world- Develop analytics that can actually extract causal relationships

and generate predictive models

- Future steps:- Specialized hardvare (FPGA)

12/13

13/13

Siniša Ivković[email protected]

Thank you for your attention

Implementation of the BLAST A lgorithm U sing Hadoop MapReduce

Documents

Transcript of Implementation of the BLAST A lgorithm U sing Hadoop MapReduce