Python scriptingSlide title 70 pt CAPITALS Slide subtitle minimum 30 pt
Python
Why Python?
Tutorial Outline
interactive "shell"
variables
Try It Out!
Feel free to play along with your shell during the class
Download Python from www.python.org
By and large they are all mutually compatible
Recommended version: 2.7.3
Use IDLE if you can
Python scripting
Python scripting
Interactive “Shell”
Great for experimenting with the library
Great for testing your own modules
Two variations: IDLE (GUI),
>>> print "Hello, world"
Numbers
Integers
Examples: 0, 1, 1234, -56
Note: dividing an integer by another integer will return only the integer part of the quotient, e.g. typing 7/2 will yield 3
Long integers
Example: 999999999999999999999L/999999999999999999999l
Python will convert automatically to long integer (e.g: 2**100)
Octal constants
Hex constants
Complex numbers
x=complex(3,4) -> x.real, x.imag,x.conjugate
Operations on Numbers
Basic algebraic operations
Exponentiation: a**b
Comparison operators
Greater than, less than, etc.: a < b, a > b, a <= b, a >= b
Identity tests: a == b, a != b
Bitwise operators
Bitwise exclusive or: a ^ b # Don't confuse this with exponentiation
Bitwise and: a & b
Shift a left or right by b bits: a << b, a >> b
Python scripting
Python scripting
Strings & OPERATIONS
“Line one \n Line two” #Escape sequences
“He said\”I am the one\”” #Escaping chars
'single quotes'
"""triple quotes"""
r"raw strings"
Python scripting
Python scripting
Lists
Lists can contain numbers, strings, nested sublists…
Examples: L1 = [0,1,2,3], L2 = ['zero', 'one'],
L3 = [0,1,[2,3],'three',['four,one']]
Lists are mutable:
Elements can be removed
List Operations
Dictionaries
Lookup:
Delete, insert, overwrite:
Python scripting
Python scripting
More Dictionary Ops
Keys, values, items:
d.keys() -> ["duck", "back"]
d.values() -> ["duik", "rug"]
d.items() -> [("duck","duik"), ("back","rug")]
{"name":"Guido", "age":43, ("hello","world"):1,
42:"yes", "flag": ["red","white","blue"]}
Dictionary Details
these cannot be changed after creation
reason is hashing (fast lookup technique)
not lists or other dictionaries
these types of objects can be changed "in place"
no restrictions on values
again, because of hashing
Tuples
Tuples can contain numbers, strings, nested sub-tuples, or nothing
Examples: t1 = (0,1,2,3), t2 = ('zero', 'one'),
t3 = (0,1,(2,3),'three',('four,one')), t4 = ()
As long as you're not nesting tuples, you can omit the parentheses
Example: t1 = 0,1,2,3 is the same as t1 = (0,1,2,3)
Tuple indexing works just like string and list indexing
Tuples are immutable: individual elements cannot be reassigned in place (difference with lists)
Concatenation:
Python scripting
Python scripting
Variables
Not typed
else: greeting = 12**2
Reference Semantics
References for mutable types (dictionaries, lists, arrays)
Values for immutable types (numbers, strings, tuples)
Example (mutable):
a
1
2
3
b
4
a
1
b
a
1
b
a
1
Control Structures
if condition:
Operators: <, <=, >, >=, ==, !=, in
Grouping Indentation
In Python:
{
---
---
In C and other languages (Java, C++, C#...) indentation is only a coding convention for readability purposes….
In Python indentation replaces braces!
Python scripting
Python scripting
Functions
def func(args):
May be simple one-to-one mappings
def f1(x):
def f2(x,y):
Functions
The user can set arguments to default values in function definitions. These arguments will be optional when calling the function
def f4(x,a=1,b=1):
When calling the function, arguments can be implicit/explicit
f4(2,1,1)
Functions can take multiple types as inputs and/or outputs.
Anything calculated inside a function but not specified as an output quantity (either with return or global) will be deleted once the function stops running
Python scripting
Python scripting
Function EXAMPLE
return b
SCOPE RULES
Python employs the following scoping hierarchy, in decreasing order of breadth:
Built-in (Python)
Global (module)
Names assigned at the top level of a module, or directly in the interpreter
Names declared global in a function
Local (function)
Names assigned inside a function definition or loop
If a module file is read into the interpreter via execfile, any vars/functions defined in the top level of the module file will be promoted to the top level of the program
If we had imported script.py instead, the list x would not be defined on the top level. To call x, we would need to explicitly tell Python its scope, or context.
Example -> script.py: x=[0,1,2]
import script -> reference to x is now script.x
SEARCH ORDER
Python scripting
Python scripting
Scope rules
Modules may well contain sub-modules. if we have a file named module.py which, in its definition, imports a sub module named sub-module, which in turn contains some variable named x that we need to reference…
import module -> module.submodule.x
The same name can be used in different scopes
Examples:
>>> a = 2
>>> def f5(x,y):
a = x+y # this a has no knowledge of the global a, and vice-versa
b = x-y
return a**2,b**2 # local value is deleted as soon as the function ends
>>> f5(3,3)
36
>>> a
2 #executing f5 doesn’t affect the value of the “a” var
Python scripting
Python scripting
Classes
...
Statements may also be class variable assignments
class My_class(object):
def name(self, arg1, arg2, ...): # method definition
..
Class EXAMPLE
class Ftp(object):
self.data = None
self.ftp_server=ftp_server
self.username=username
""" Deletes files from an active ftp connection #comments
Args:
"""
#If a single file is provided, enclose it in a list
if isinstance(files, str) or isinstance(files,unicode): files = [files]
for file_ in files:
Delete method
Using Classes
ftp = Ftp(“127.0.0.1”,user,password,……)
To use methods of the instance, call using dot notation:
ftp.delete(files_to_delete)
ftp.username
Modules
functions, classes, variables
Import with rename:
Python scripting
Python scripting
Packages
Must have __init__.py file
import P.Q.M; print P.Q.M.foo()
Python scripting
Python scripting
Catching Exceptions
def foo(x):
return 1/x
def bar(x):
bar(0)
Try-finally: Cleanup
f = open(file)
Python scripting
Python scripting
Raising Exceptions
raise IndexError
raise IndexError, "k out of range"
try:
something
File Objects
f = open(filename[, mode[, buffersize])
mode can be "r", "w", "a" (like C stdio); default "r"
append "b" for binary mode
append "+" for read/write open
methods:
Standard Library
Regular expressions:
re module
Function
os.path.split(path)
Split the pathname path into a pair, (head, tail) where tail is the last pathname component and head is everything leading up to that
str.split([sep[, maxsplit]])
Return a list of the words in the string, using sep as the delimiter string. If maxsplit is given, at most maxsplit splits are done
glob.glob1(path,mask)
Returns files and folders matching the specified mask in the selected path
sys.path.append(path)
Adds the path to the list of paths where our script will look modules into
Python scripting
Python scripting
Jython / PYthon
Jython is an implementation of the Python language for the Java platform, that runs on the JVM
Jython 2.5 implements the same language as Python 2.5
Jython 2.7 currently in alpha stage
ODG integrates Jython 2.2.1 (13.0) and below and will integrate Jython 2.5.3 from version 13.1
With Jython, any java package can be imported (e.g: Jsch package is used for the SFTP operations)
PYTHON
JYTHON
C
EDOS_Util module
Its goal is to provide all the frequently used operations needed when writing preprocess/postprocess functions
Contains common operations for: file handling, Ftp, Sftp, date and time, compressing and uncompressing…
2 versions: util legacy (ODG <=13.0) and util (ODG from 13.1)
Implements multithreading
Useful to take advantage of multicore capabilities
Interleaving is also used in some cases (e.g: get files from FTP and extract them) to improve performance
FTP
EDOS_Util module
File operations
move_multiple
Moves multiple files from a source to a target directory
rename
Renames a file/folder. Target has to be just the filename (no path)
rename_multiple
mkdir
Deletes a file or a folder (and all its files/subfolders).
clean_or_create_dir
Creates a directory, or delete its contents if it already exists
Python scripting
Python scripting
EDOS_util module
Time functions
now
Returns the current date and time in the specified format. If not specified, returned in %Y-%m-%d_%H-%M-%S (i.e:"2008-11-21_22-08-54")
today
Returns the current date in the specified format. "hour_offset" allows to specify and optional offset in hours (+/-) to the current time (E.g. "21_11_2008")
yesterday
previous_day_as_yyyymmdd
previous_hour
Returns the date and time of a previous hour E.g. "21_11_2008_09_45_00".
get_week_first_day_monday
Returns the current week, considering monday as the first day
Python scripting
Python scripting
EDOS_util module
Compression functions
Miscellaneous functions
pack
This function will compress the source file or folder into 'archiveName' output file (full path must be provided)
extract
This function will extract the file(s) passed in the first argument to 'target_folder'
Function
Python scripting
Python scripting
EDOS_util module
Call this functions several times
Use Ftp/Sftp class directly
ftp_send sftp_send
Send files from a single dir to an ftp/sftp directory
Python scripting
Python scripting
EDOS_UTIL EXAMPLE
Example: I need to download and extract all the gzip files located inside CM folder (no subfolders), containing today’s date (“yyyymmdd”). The server is located at ftp.ericsson.net and the server port is 12000. The destination folder will be the data folder for ODG in Ericsson2G. user name is “user”, and password “pwd”
from util import today_as_yyyymmdddd,ftp_get
td=today_as_yyyymmdddd
download_folder=r“C:\Program Files (x86)\Ericsson\OSS Data Gateway\OSS Data Gateway\data\Ericsson2G”
ftp_get( “ftp.ericsson.net ”, “user”, “pwd”, “/CM”, download_folder, file_patterns=“*”+td+”*.gzip”,
port=12000,plain_structure=True, extract=True, extract_folder=download_folder,delete_compressed=True)
Python scripting
Python scripting
GENERIC PREPROCESS
Its goal is to provide a preprocess base with the more frequent use cases:
Makes use of the util/edos_util library
Versions 1.x use the util module
Versions 2.x use the edos_util/edos_util_legacy modules
Download implemented for 3 different sources:
Shared Folder
Python scripting
Python scripting
GENERIC PREPROCESS
What if generic preprocess doesn’t fulfill our needs?
Sometimes a quick change in the code (not the configuration) is enough to adapt it to our needs
Use case: CM download from 2 different FTP servers
Not possible yet using the script configuration! But if we navigate through the code…
… we can just add another line to download from a second server
Generic postprocess also available (backups operdb)
Python scripting
Python scripting
Plugins available
Fast and good for reading python scripts, but lacks of debugging, code completion…
Pyscripter
Nice and complete editor
online Python shell, graphical debugger with step by step, showing variables values in popups…
No direct Jython support (no debugging, programs can be run through the external run)
Eclipse+Pydev
Supports Jython (shell, debugger, run…)
Jython grammar and interpreter version can be configured for each project
Con: it is a heavier environment
Python scripting
Python scripting
RESOURCES
Python scripting
Python scripting