Rolando V. RaqueñoWednesday, June 10, 2015 Statistics Function Implementation Traditional...
-
date post
19-Dec-2015 -
Category
Documents
-
view
214 -
download
0
Transcript of Rolando V. RaqueñoWednesday, June 10, 2015 Statistics Function Implementation Traditional...
Rolando V. Raqueño Tuesday, April 18, 2023
Statistics Function Implementation
Traditional Programming Approach
Rolando V. Raqueño Tuesday, April 18, 2023
minimum.pro
function minimum, xn = n_elements(x)answer = x(0)for i=1L, n-1 do begin
if( answer gt x(i) ) then $answer = x(i)
endforreturn, answer
end
Rolando V. Raqueño Tuesday, April 18, 2023
sum.pro
function sum, x
n = n_elements( x )
answer = 0
for i = 0, n-1 do begin
answer = answer + x(i)
endfor
return, answer
end
Rolando V. Raqueño Tuesday, April 18, 2023
Overflow Problems
• What if the array is an integer array?
• Sums will have a maximum limit based on the number of bits specified by the integer type.
Rolando V. Raqueño Tuesday, April 18, 2023
Corrected sum.pro
function sum, x
n = n_elements( x )
answer = 0.0D
for i = 0L, n-1 do begin
answer = answer + x(i)
endfor
return, answer
end
Rolando V. Raqueño Tuesday, April 18, 2023
mean.pro
function mean, x
n = n_elements(x)
answer = sum(x) / n
return, answer
end
Rolando V. Raqueño Tuesday, April 18, 2023
Integer Problems
IDL> b=9/5
IDL> print,b
1
Rolando V. Raqueño Tuesday, April 18, 2023
Integer Problems
IDL> b=9/double(5)
IDL> print,b
1.8000000
Rolando V. Raqueño Tuesday, April 18, 2023
Corrected mean.pro
function mean, x
n = n_elements(x)
answer = sum(x) / double(n)
return, answer
end
Rolando V. Raqueño Tuesday, April 18, 2023
sum_squares.pro
function sum_squares, x
squares = x^2.0
answer= sum( squares )
return, answer
end
Rolando V. Raqueño Tuesday, April 18, 2023
variance.pro
function variance, x
n = double(n_elements(x))
answer=(sum_squares(x)-(sum(x)^2.0/n))/(n-1)
return, answer
end
Rolando V. Raqueño Tuesday, April 18, 2023
sd.pro
function sd, x
answer = sqrt(variance(x))
return, answer
end
Rolando V. Raqueño Tuesday, April 18, 2023
Shell Programming
Rolando V. Raqueño Tuesday, April 18, 2023
UNIX Tip of the Day
• Directory maneuvering commands pushd, popd, and, dirs
% cd /usr/tmp% pwd/usr/tmp% pushd ~rvrpci% pwd/cis/staff/rvrpci
Rolando V. Raqueño Tuesday, April 18, 2023
UNIX Tip of the Day
% dirs
/cis/staff/rvrpci /usr/tmp
% pushd
% pwd
/usr/tmp
% dirs
/usr/tmp /cis/staff/rvrpci
% pushd /usr/local/bin
Rolando V. Raqueño Tuesday, April 18, 2023
UNIX Tip of the Day
% dirs
/usr/local/bin /usr/tmp
/cis/staff/rvrpci
% pwd
/usr/local/bin
% pushd
% dirs
/usr/tmp /usr/local/bin
/cis/staff/rvrpci
Rolando V. Raqueño Tuesday, April 18, 2023
UNIX Tip of the Day% dirs
/usr/tmp /usr/local/bin
/cis/staff/rvrpci
% pwd
/usr/tmp
% pushd +2
% pwd
/cis/staff/rvrpci
Rolando V. Raqueño Tuesday, April 18, 2023
UNIX Tip of the Day% dirs
/cis/staff/rvrpci /usr/tmp
/usr/local/bin
% popd
% dirs
/usr/tmp /usr/local/bin
Rolando V. Raqueño Tuesday, April 18, 2023
IMPORTANT UNIX Concepts• Environment and Shell Variables
– These allow you to customize your UNIX environment
– They are different in terms of their SCOPE• SCOPE determines the visibility of a variable
Rolando V. Raqueño Tuesday, April 18, 2023
Other IMPORTANT UNIX Concepts• Environment Variable
– Examples are TERM and DISPLAY– Set a particular variable to a value by using the setenv command
– You can print the value of a particular variable or all the environment variable using the printenv command
Rolando V. Raqueño Tuesday, April 18, 2023
% Environment Variables %• Examples
– To set environment variables
% setenv TERM vt100% setenv DOG Goofy– print out the terminal type
% printenv TERM vt100– print out all environment variables
% printenv
Rolando V. Raqueño Tuesday, April 18, 2023
Shell Variables
• Shell variables are similar to Environment variables except they have a limited scope, i.e., they exist only in the shell which they are defined.
• Environment variables on the other hand, exist in all its children shells
• To illustrate this concept, let us look at the following example
Rolando V. Raqueño Tuesday, April 18, 2023
Environment vs. Shell Variables% set prompt = "Parent Shell > "Parent Shell > setenv DOG GoofyParent Shell > set mouse=MickeyParent Shell > printenv DOGGoofyParent Shell > echo $mouseMickeyParent Shell > xterm &
(YOU SHOULD NOW HAVE A NEW xterm WINDOW)THIS IS KNOWN AS
“SPAWNING A NEW (OR CHILD) PROCESS”
Rolando V. Raqueño Tuesday, April 18, 2023
Environment vs. Shell Variables(IN THE NEW xterm WINDOW, DO THE FOLLOWING)
% set prompt = "Child Shell > "Child Shell > printenv DOGGoofyChild Shell > echo $mousemouse: Undefined variable.
Rolando V. Raqueño Tuesday, April 18, 2023
Environment vs. Shell VariablesChild Shell > setenv DOG PlutoChild Shell > set mouse=MinnieChild Shell > printenv DOGPlutoChild Shell > echo $mouseMinnieChild Shell > exit
(THE xterm WINDOW SHOULD NOW GO AWAY - THIS PROCESS HAS NOW BEEN KILLED)
Rolando V. Raqueño Tuesday, April 18, 2023
Environment vs. Shell VariablesParent Shell >Parent Shell > printenv DOGGoofyParent Shell > echo $mouseMickeyParent Shell >
Rolando V. Raqueño Tuesday, April 18, 2023
Environment & Shell Variables
• Why is this important?– UNIX uses Environment and Shell Variables
control a number of processes– Customizes your working environment– Variables used for UNIX Scripts
• They are typically defined and initialized in your .login and .cshrc files
Rolando V. Raqueño Tuesday, April 18, 2023
Useful Shell Variables
filec#Allows file completionpath #List of command
directoriescdpath #List of candidate
directories to cd intohistory #Number of commands to
remember
Rolando V. Raqueño Tuesday, April 18, 2023
What is shell programming?
• Shell programming – automate a set of UNIX commands.– Just like any programming language– “wrappers”
• black box a customized collection of UNIX commands.
– Example of shell programs.login.cshrc
Rolando V. Raqueño Tuesday, April 18, 2023
.login file
set path=($HOME/bin /usr/local/bin \/usr/ucb /usr/sbin /bin /usr/bin \/usr/bin/X11 .)stty dec newtset -I -Qset mail=/usr/spool/mail/$USERset editmode = emacsumask 077biff ndate
Rolando V. Raqueño Tuesday, April 18, 2023
.cshrc file
if ($?prompt) then
set notify
set history = 100
set savehist = 100
alias pd pushd
alias pop popd
alias vt100 "set term = vt100"
endif
Rolando V. Raqueño Tuesday, April 18, 2023
When these files are executed?
.cshrc – is automatically executed when you start a new
shell
.login – only gets executed once when you first login
Can be re-executed by giving the source command% source .cshrc
Rolando V. Raqueño Tuesday, April 18, 2023
Other useful .login and .cshrc entries
set filec
set cdpath=(~ ~rvrpci/pub ~/mythesis)
Other common entries
set path=( $path /usr/local/bin)
set path=(/usr/local/bin $path)
Rolando V. Raqueño Tuesday, April 18, 2023
User defined shell program
• Determine name of command
• Determine input, output, and option arguments
• Determine UNIX commands to execute
• Establish error trapping
• Make shell program executable
Rolando V. Raqueño Tuesday, April 18, 2023
A simple shell program
• dd command to swap bytes
% dd if=input.dat of=output.dat bs=2 conv=swab
• Very difficult to remember• Very little utility to non-UNIX geeks
(normal people)
Rolando V. Raqueño Tuesday, April 18, 2023
We would rather see...
% swap_bytes input.dat output.dat
Rolando V. Raqueño Tuesday, April 18, 2023
Special Shell Variables Set
% swap_bytes input.dat output.dat
$0 $1 $2$argv[1] $argv[2]command
Rolando V. Raqueño Tuesday, April 18, 2023
Another Special Shell Variables
% swap_bytes input.dat output.dat
$#argv
Indicates how many arguments are present
In this case, 2
Rolando V. Raqueño Tuesday, April 18, 2023
shell program swap_bytes
#!/bin/csh -f
dd if=$1 of=$2 bs=2 conv=swab
Rolando V. Raqueño Tuesday, April 18, 2023
Making swap_bytes shell script executable
% ls -l swap_bytes
-rw------- ... swap_bytes
% chmod u+x swap_bytes
% ls -l swap_bytes
-rwx------ ... swap_bytes
Rolando V. Raqueño Tuesday, April 18, 2023
To run swap_bytes
• swap_bytes becomes just another unix command!
% swap_bytes input.dat output.dat
Rolando V. Raqueño Tuesday, April 18, 2023
Limitation of swap_bytes
• No error trapping
• Should give usage when typing command
% swap_bytes
usage: swap_bytes input_file output_file
Rolando V. Raqueño Tuesday, April 18, 2023
Improvement to swap_bytes
#!/bin/csh -f if ( $#argv != 2 ) then
echo "usage: $0 input_file output_file"
exit 1
endif
dd if=$1 of=$2 bs=2 conv=swab
Rolando V. Raqueño Tuesday, April 18, 2023
Commad exit status
• By convention
exit 0
Indicates successful command completion
exit 1 (or non-zero)
Indicates some error condition
Rolando V. Raqueño Tuesday, April 18, 2023
Informational message from swap_bytes
• UNIX style informational message
% swap_bytes
usage: swap_bytes input_file output_file
Rolando V. Raqueño Tuesday, April 18, 2023
Interactive swap_bytes
• If you want a “friendlier” shell program– Have it query the user for the inputs
• Another special shell variable can be used
$<
Rolando V. Raqueño Tuesday, April 18, 2023
Interactive swap_bytes
#!/bin/csh -f
if ( $#argv != 2 ) then
echo -n "Please enter the input file> "
set input=$<
echo -n "Please enter the output file> "
set output=$<
endif
dd if=$input of=$output bs=2 conv=swab
Rolando V. Raqueño Tuesday, April 18, 2023
Interactive swap_bytes example
• User simply types the command
% swap_bytes
Please enter the input file> input.dat
Please enter the output file> output.dat
Rolando V. Raqueño Tuesday, April 18, 2023
UNIX Quotes
Rolando V. Raqueño Tuesday, April 18, 2023
A note about quotes in UNIX
% set a=ls
% echo a
% echo $a
% set b=“$a”
% echo $b
% set b=‘$a’
% echo $b
% set b=`$a`
% echo $b
Rolando V. Raqueño Tuesday, April 18, 2023
A note about shell variables
• Shell variables can also double up as arrays
• Using the previous example,
% echo $b
% echo $b[1]
% echo $#b
% echo $b[$#b]
Rolando V. Raqueño Tuesday, April 18, 2023
A more complex shell program
• In pbmplus utilities, rawtopgm conversion existspgmtoraw conversion does not
• A version of pgmtoraw in a programming language like C– Time consuming– Program will likely be used infrequently
• Solution: shell program
Rolando V. Raqueño Tuesday, April 18, 2023
pgmtoraw shell script design
• Define input and output files
• Figure out dimensions of input image
• Determine number of bytes for input image
• Determine number of bytes for header
• Need to strip out the header bytes
• Write out headerless image data
Rolando V. Raqueño Tuesday, April 18, 2023
Define input and output files pgmtoraw
#!/bin/csh -fset command_name=$0set number_args=$#argvif( $number_args != 1 ) then echo “usage:$command_name input_file_name” exit 1endif...
Rolando V. Raqueño Tuesday, April 18, 2023
Dimensions of input image ( pnmfile)
% more test_data.pgmP23 32551 2 34 5 67 8 9
% pnmfile test_data.pgmtest_data.pgm: PGM plain, 3 by 3 maxval 255
Rolando V. Raqueño Tuesday, April 18, 2023
pgmtoraw (continued)
set input_file=$1
set pnm_info = `pnmfile $input_file`
set image_type = $pnm_info[2]
set data_type = $pnm_info[3]
set width = $pnm_info[4]
set height = $pnm_info[6]
set maxval = $pnm_info[8]
set pixel_bytes = 1
@ image_bytes = $width * $height
Rolando V. Raqueño Tuesday, April 18, 2023
pgmtoraw (continued)
set file_info=`wc -c $input_file`
set bytes_in_file = $file_info[1]
@ header = $bytes_in_file - $image_bytes
dd if=$input_file bs=$pixel_bytes skip=$header
Rolando V. Raqueño Tuesday, April 18, 2023
Resulting pgmtoraw utility
• Uses existing routines to obtain informationpnmfile
wc
dd
• Functional tool written in 20 command lines
Rolando V. Raqueño Tuesday, April 18, 2023
Current Limitations of Resulting pgmtoraw utility
• No check between “ASCII” vs. “RAW” pgmif( data_type == ‘plain,’) ...
• No provisions for multibyte per pixel case– Use pnmfile results to check for above cases– endian case needs to be addressed for multibyte
case
• Above conditions can be addressed by suite of UNIX commands
Rolando V. Raqueño Tuesday, April 18, 2023
Shell Scripts Wrappers and IDL
• Another utility formerly missing in pbmplusjpegtopnm or pnmtojpeg
• IDL has jpeg read/write capability– Create a “wrapper” that makes an idl
program pbmplus-like
Rolando V. Raqueño Tuesday, April 18, 2023
pnmtojpeg.pro
pro pnmtojpeg, input_file, output_file
read_ppm, input_file, image
write_jpeg, output_file, image
end
Rolando V. Raqueño Tuesday, April 18, 2023
Usage of pnmtojpeg.pro in IDL
IDL> pnmtojpeg,‘image.pnm’,’image.jpg’
Rolando V. Raqueño Tuesday, April 18, 2023
Usage of pnmtojpeg.pro in IDL
IDL> pnmtojpeg,‘image.pnm’,’image.jpg’
• For IDL to automatically find pnmtojpeg.pro
– It must be in the current working directory
– Directory containing pnmtojpeg.pro must be defined in the ENVIRONMENT VARIABLE•IDL_PATH
Rolando V. Raqueño Tuesday, April 18, 2023
IDL_PATH environment variable
setenv IDL_DIR /cis/common/rsi/idl_5
setenv IDL_PATH \+$IDL_DIR/lib:\+$IDL_DIR/examples:
\+/dirs/common/rsi/idl_5:\+/dirs/common/lib/idl:\+~/lib/idl
Rolando V. Raqueño Tuesday, April 18, 2023
pnmtojpeg.csh
#!/bin/csh -f
echo pnmtojpeg “,” “’”$1”’” “,” “’”$2”’” | idl
Rolando V. Raqueño Tuesday, April 18, 2023
Usage of pnmtojpeg.csh
% pnmtojpeg.csh image.pnm image.jpg
Rolando V. Raqueño Tuesday, April 18, 2023
Limitation of pnmtojpeg.csh
• Does not conform to pbmplus piping, i.e.,% tifftopnm file.tif | pnmscale 2.0 > new_file.pnm
• No error trapping
Rolando V. Raqueño Tuesday, April 18, 2023
Usage cases of pnmtojpeg(no command line arguments)
% tifftopnm file.tif | pnmscale 2.0 | pnmtojpeg > new_file.jpg
Rolando V. Raqueño Tuesday, April 18, 2023
Usage cases of pnmtojpeg(1 command line argument)
% pnmtojpeg image.pnm > image.jpg
Rolando V. Raqueño Tuesday, April 18, 2023
Usage cases of pnmtojpeg(2 command line arguments)
% pnmtojpeg image.pnm image.jpg
Rolando V. Raqueño Tuesday, April 18, 2023
Yet another wrapper pnmtojpeg
#!/bin/csh -f
# If user interrupts process, jump to stop
onintr stop
# $0 is the command name
# $#argv is the number of arguments
# $$ is the process id
Rolando V. Raqueño Tuesday, April 18, 2023
Code for no argument case
if($#argv == 0) then
set input_file = /usr/tmp/$0_input_$$
set output_file = /usr/tmp/$0_output_$$
cat > $input_file
pnmtojpeg.csh $input_file $output_file
cat $output_file
.
.
.
Rolando V. Raqueño Tuesday, April 18, 2023
Code for 1 argument case
else if($#argv ==1) then
set input_file = $1
set output_file = /usr/tmp/$0_output_$$
pnmtojpeg.csh $input_file $output_file
cat $output_file
.
.
.
Rolando V. Raqueño Tuesday, April 18, 2023
Code for 2 argument caseelse set input_file = $1 set output_file = $2 pnmtojpeg.csh $input_file $output_fileendif
#clean up when finishedstop: rm -f /usr/tmp/$0_input_$$ rm -f /usr/tmp/$0_output_$$
Rolando V. Raqueño Tuesday, April 18, 2023
pnmtojpeg summary
• Produced a “new” pbmplus utility• Used UNIX shell scripting
– Argument handling– Scratch space /usr/tmp– Process id handling– Clean up
• Integrated IDL program and commands• 21 lines of UNIX commands
Rolando V. Raqueño Tuesday, April 18, 2023
Summary
• The “dot” files
• Basics of Shell Scripting
• Special Shell Variables
• Seamless integration of UNIX to other utilities (IDL)
Rolando V. Raqueño Tuesday, April 18, 2023
Other Shell Constructs to keep in mind
• foreach• while• case