Linux shell & shell scripting - II -...
of 56
/56
-
Author
vuongkhanh -
Category
Documents
-
view
246 -
download
4
Embed Size (px)
Transcript of Linux shell & shell scripting - II -...
IBS574-ComputationalBiology&Bioinformatics
Spring2018,Tuesday(02/01),2:00-4:00PM
Linuxshell&shellscripting-II
AshokR.DinasarapuPh.DScientist,Bioinformatics
Dept.ofHumanGenetics,EmoryUniversity,Atlanta
102/01/2018,2-4PM
WhatisShellscripting?
2
Ascriptisacollectionofcommandsstoredinafile.
Shellisacommand-lineinterpreter. Toexecuteashellscript,ex.hello.sh./hello.sh ../hello.sh/home/user_name/script/hello.shbashhello.sh(orshhello.sh)
02/01/2018,2-4PM
3
Easiestwaytodothisis
Starttyping!
02/01/2018,2-4PM
SSHallowsyoutoconnecttoyourserversecurelyandperformLinuxcommand-lineoperations.
Alternatively,https://blnx1.emory.edu:22443/
~meansyourhomedirectory(/home/user_name) Windows:downloadPuTTY
https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html
4
ConnectingServerviaSSH
1/30/2018,2-4PM
mkdir&directorystructure
Createdirectoriesfromyourhomedirectory(i.e/home/user_name)
Usage:mkdir-pproject/{data,script,out,log} project/data project/script project/out project/log
Usage:cdproject/script502/01/2018,2-4PM
Chooseatexteditor:emacs,Vim
Usage:vihello.sh
INSERTmode:presskeyslikeiORa&starttyping."i"willletyouinserttextjustbeforethecursor."I"insertstextatthebeginningofthecurrentline."a"willletyouinserttextjustafterthecursor,and"A"willletyoutypeattheendofthecurrentline.
602/01/2018,2-4PM
HowTo:Create/Edittextfiles
Typethefollowingtext:
#!/bin/sh
#Myfirstscript
echo"HelloWorld!"
7
#!/bin/shaspecialcluegiventotheshellindicatingwhatprogramisusedtointerpretthescript.
02/01/2018,2-4PM
HowTo:Create/Edittextfiles
Typethefollowingtext:
HowTo:Create/Edittextfiles
#!/bin/sh
#Myfirstscript
echo"HelloWorld!"
8
#!/bin/bash#!/usr/bin/perl#!/usr/bin/Rscript#!/usr/bin/envpython
#!/bin/shaspecialcluegiventotheshellindicatingwhatprogramisusedtointerpretthescript.
02/01/2018,2-4PM
Viewsymboliclinkusinglsla/bin/sh
SAVEmode:pressesckeyAND:q!fornottosaveOR
:xtosavealltypedcontent.
902/01/2018,2-4PM
Alternatively,use:wqforwriteandquit(saveandexit)
HowTo:SavefileinViTextEditor
Changethepermissionsoffiles
Read(r),write(w),andexecute(x) 3typesofusers(user,group&other)
Usage:ls-l
chmod
1002/01/2018,2-4PM
11
makefileexecutable
TomakeitexecutableUsage:chmod+rwxr-xr-xhello.sh
Usage:chmod+xhello.sh
Usage:./hello.sh
Tomakeitun-executable Usage:chmod-xhello.sh
02/01/2018,2-4PM
12
makefileexecutable
TomakeitexecutableUsage:chmod+rwxr-xr-xhello.sh
Usage:chmod+xhello.sh
Usage:./hello.sh
Tomakeitun-executable Usage:chmod-xhello.sh
+755
02/01/2018,2-4PM
VariablesinLinux/Shell
#!/bin/bash
#Myfirstscript
i=22
echoHello,Iam$i#echoHello,Iam$i
13
Usage:vihello.sh
Usage:./hello.sh
02/01/2018,2-4PM
VariablesinLinux/Shell
#!/bin/bash
#Myfirstscript
i=22
j="Hello,Iamecho$j$i
14
Usage:vihello.sh
Usage:./hello.sh
02/01/2018,2-4PM
VariableexpansioninLinux/Shell
#!/bin/bash#Myfirstscript
i=22j=${i}0K=${i}1echo$i$j$k
15
Usage:vihello.sh
Usage:./hello.sh
02/01/2018,2-4PM
the{}in${}areusefulifyouwanttoexpandthevariable
#!/bin/bash#Myfirstscript
PASS=test1234
if[$PASS==test1234];thenechoCorrectpassword!!
fi
16
ConditionsinLinux/Shell
Rememberthatthespacingisveryimportantintheifstatement.
Usage:vihello.sh
Usage:./hello.sh
02/01/2018,2-4PM
#!/bin/bash#Myfirstscript
PASS=test123
if[$PASS==test1234];thenechoCorrectpassword!!
elseechoentercorrectpassword!!
fi
17
ConditionsinLinux/Shell
Usage:vihello.sh
Usage:./hello.sh02/01/2018,2-4PM
#!/bin/bash#Myfirstscript
foriin{1..10};do echo$i#echo${i}a
done
18
forloopinLinux/Shell
Usage:vihello.sh
Usage:./hello.sh
02/01/2018,2-4PM
#!/bin/bash#Myfirstscript
foriin{1..10};do echo$i#echo${i}a
done
19
forloopinLinux/Shell
Usage:vihello.sh
Usage:./hello.sh
02/01/2018,2-4PM
Alternatively,$(seq0110)$(seq0210)
12345678910
#!/bin/basharr=(ABCDE)foriin{0..4};doecho$iecho${arr[$i]}
done
20
Usage:vihello.sh
Usage:./hello.sh
02/01/2018,2-4PM
ArrayinLinux/Shell
#!/bin/sh#Myfirstscript
foriin123xyzdo echo$idone
21
forloopinLinux/Shell
Usage:vihello.sh
Usage:./hello.sh
loop indices doesn't have to be just numbers!!!
02/01/2018,2-4PM
{1..3}xyzbashsupports
#!/bin/bash#Myfirstscript
i=0while[$i-le5];do#echobefore$ii=$(($i+1))echoafter$i
done
22
Usage:vihello.sh
whileloopinLinux/Shell
02/01/2018,2-4PM
Usage:./hello.sh
#!/bin/bash#functiondefinitionadd_user(){
USER=$1PASS=$2echoPasswd$PASScreatedfor$USERon$(date)
}#functioncallecho$(add_userbobletmein)
23
FunctionsinLinux/Shell
Usage:vihello.sh
Usage:./hello.sh
02/01/2018,2-4PM
#!/bin/bash#functiondefinitionadd_user(){
#USER=$1#PASS=$2echoPasswd$1createdfor$2on$3
}#functioncallecho$(add_userbobletmein$(date))
24
FunctionsinLinux/Shell
Usage:vihello.sh
02/01/2018,2-4PM
Usage:./hello.sh
downloadafastqfile
Usage:cdproject/datawgethttps://github.com/CGATOxford/UMI-tools/releases/download/v0.2.3/example.fastq.gz ViewUsage:zcatexample.fastq.gz|head
CheckfilesizeUsage:ls-lhexample.fastq.gz
CountnumberofsequencesinafastqfileUsage:grep-c"^@"example.fastq.gz
2502/01/2018,2-4PM
02/01/2018,2-4PM 26
zcatexample.fastq.gz|[email protected][email protected]TGCCCTCTTCTGGTGCATCTGAAGACAGCTACAGTGTACTTAGATATAATAAATAAATCTT+SRR2057595.94=DFDBDHHFH[email protected]@SRR2057595.14TGGGTTAATGCGGCCCCGGGTTCCTCCCGGGGCTACGCCTGTCTGAGCGTCGCT
SequenceID
Sequence
AFASTQfilenormallyusesfourlinespersequence
#!/bin/bash#cdproject/log(torunthisscriptfromlogdirectory)
zcat../data/example.fastq.gz|\
awk{if(NR%4==2)printlength($1)}>../out/length.txt
27
Calculatethelengthofreads
Usage:vifastq.sh
Usage:../script/fastq.sh
Createthefollowingexecutablefileatproject/script
02/01/2018,2-4PM
28
Usage:vihist.R
02/01/2018,2-4PM
Plot-HistogramCreatethefollowingexecutablefileatproject/script
#!/usr/bin/Rscript#cdproject/log(torunthisscriptfromlogdir)t.dat
#!/bin/bash#cdproject/log(torunthisscriptfromlogdirectory)
zcat../data/example.fastq.gz|\
awk{if(NR%4==2)printlength($1)}>../out/length.txt
#includeRscriptandrun
../script/hist.R
29
Calculatethelengthofreads
Usage:vifastq.sh
Usage:../script/fastq.sh
Createthefollowingexecutablefileatproject/script
02/01/2018,2-4PM
#!/bin/bash#cdproject/log(torunthisscriptfromlogdirectory)
zcat../data/example.fastq.gz|\
awk{if(NR%4==2)printlength($1)}>../out/length.txt
#includeRscriptandrun
../script/hist.R
30
Calculatethelengthofreads
Usage:vifastq.sh
Usage:../script/fastq.sh2>error.log
Createthefollowingexecutablefileatproject/script
02/01/2018,2-4PM
Redirecterroroutputtoafile
Open/ViewImage
02/01/2018,2-4PM 31
LoginasinteractivemodeUsage:[email protected]
Usage:xdg-openrplot.jpg
-XEnablesX11forwarding-YEnablestrustedX11forwardingX11istheXWindowSystem
[email protected]:~/project/out/rplot.jpg/Users/adinasarapu/
Todownloadfromserver:
AWK
02/01/2018,2-4PM 32
AWKisaninterpretedprogramminglanguage&designedfortextprocessing.
Theword"AWK"isderivedfromtheinitialsofthelanguage'sthreedevelopers:A.Aho,B.W.KernighanandP.Weinberger.
FollowingarethevariantsofAWK AWK-the(veryold)originalfromAT&T NAWK-Anewer,improvedversionfromAT&T GAWKItisGNUAWK.TheFreeSoftware
foundation'sversion
BasicStructureofAWK
02/01/2018,2-4PM 33
pattern{action}
Thepatternspecifieswhentheactionisperformed.
Bydefault,AWKexecutecommandsoneveryline.
AWKreadsalinefromtheinputstream(file,pipe,orSTDIN)
Thisprocessrepeatsuntilthefilereachesitsend.
BasicStructureofAWK
02/01/2018,2-4PM 34
BEGIN{print"START"}pattern{print}END{print"STOP"}
BEGINblockexecutesonlyonce.Thisisgoodplacetoinitializevariables.
ThebodyblockappliesAWKcommandsoneveryinputline. TheENDblockexecutesattheendoftheprogram.
Twootherimportantpatternsarespecifiedbythekeywords"BEGIN"and"END
pattern{action}
Chooseatexteditor:emacs,Vim
Usage:vimarks.txt
Fileprocessing/Printingrecords
35
1 Amit Physics 882 Ralf Maths 903 ShyamBiology 874 Kedar English 855 Ashok History 89
awk'{print}' IsGreaterThan(ASCIIcomparison) if["$1">"$2"]>= IsGreaterThanOrEqualTo if["$1">="$2"]< IsLessThan if["$1"=$2))
IntegerComparisonOperators
02/01/2018,2-4PM
