Turbocharging FME: How to Improve the Performance of Your FME Workspaces
Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide...
-
Upload
gabriella-priscilla-dixon -
Category
Documents
-
view
217 -
download
0
Transcript of Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide...
![Page 1: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/1.jpg)
Sep. 21-22, 2006v FME Worldwide User Conference - Vancouver
Using FME Objects Python API to Provide Platform Independent TranslationsAndrew Smith, Lagen Spatial, Australia
![Page 2: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/2.jpg)
FME Objects Python API 2
Using FME Objects Python API to Provide Platform Independent
Translations
Andrew SmithSpatial Technical Consultant
Lagen Spatial - Australia
![Page 3: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/3.jpg)
FME Objects Python API 3
Using FME Objects Python API to Provide
Platform Independent Translations
Introduction Who am I? Project Overview
Python and FMEObjects Method for Development Pitfalls Tips and Tricks
Output ExamplesQuestions
![Page 4: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/4.jpg)
FME Objects Python API 4
Using FME Objects Python API to Provide
Platform Independent Translations
Who am I?
Name: Andrew Smith
Company: Lagen Spatial Pty Ltd
Official Title: Spatial Technical Consultant
Email:[email protected]
![Page 5: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/5.jpg)
FME Objects Python API 5
Using FME Objects Python API to Provide
Platform Independent Translations
Project Overview
Geoscience Australia
plays a critical role by producing first-class geoscientific information and knowledge. This can enable the government and the community to make informed decisions about the exploration of resources, the management of the environment, the safety of critical infrastructure and the resultant wellbeing of all Australians.
www.ga.gov.au
![Page 6: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/6.jpg)
FME Objects Python API 6
Using FME Objects Python API to Provide
Platform Independent Translations
Project Overview
Geographic Data Management Division Solution Name - Geoflex Project Leader - Tony Hunter
Formats Supported
ESRI Shape File (.shp, .shx, .dxf)
Personal Geodatabase
Mapinfo Native Format (.tab), Mapinfo Data Interchange Format (.mif), and
Geographic Mark-up Language (.gml)
![Page 7: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/7.jpg)
FME Objects Python API 7
Using FME Objects Python API to Provide
Platform Independent Translations
Project Overview
Requirements
Be platform independent i.e. the core extraction process is able to execute on both Windows and Unix Operating Systems
Allow a User to Select the Feature Dataset to be extracted
Allow a User to Extract the datasets to on 3 scales National State/Province Map Tile
Custom Symbology Developed by GA to be applied to the Output Datasets
Dynamically update the list of Feature Datasets that are available for extraction through interrogating the underlining Spatial DBMS schema
![Page 8: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/8.jpg)
FME Objects Python API 8
Using FME Objects Python API to Provide
Platform Independent Translations
Platform Independent Translations with FMEObject for Python
Method
FME Workbench to Develop the Translation Process
![Page 9: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/9.jpg)
FME Objects Python API 9
Using FME Objects Python API to Provide
Platform Independent Translations
![Page 10: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/10.jpg)
FME Objects Python API 10
Using FME Objects Python API to Provide
Platform Independent Translations
Copy the Transformers into a text editor to get the FME Script that defines Translation
![Page 11: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/11.jpg)
FME Objects Python API 11
ArcSDEQuerier
MACRO wheremac SDE_SEARCH_METHOD_FIELD __SEARCH_METHOD
INCLUDE [ if { {SDE_AREA_INTERSECT} == {SDE_NONE} } { puts {MACRO wheremac}} ]
FACTORY_DEF * TeeFactory \ FACTORY_NAME ARCSDEQUERIER_Globber \ INPUT FEATURE_TYPE clip_area_0 \ OUTPUT FEATURE_TYPE __GLOBBED__
FACTORY_DEF * SDE30QueryFactory \FACTORY_NAME ARCSDEQUERIER_Querier \ INPUT FEATURE_TYPE __GLOBBED__ \ @SupplyAttributes(__SERVER,lagen-pc-
1,__INSTANCE,esri_sde,__DATASET,sde,__USERID,sde,__PASSWORD,sdeuser,__TARGET_TABLE,STONNING_CAD,__SEARCH_METHOD,SDE_AREA_INTERSECT,__WHERE_CLAUSE,"",__VERSION,SDE.DEFAULT) \
SDE_SERVER_FIELD __SERVER \ SDE_INSTANCE_FIELD __INSTANCE \ SDE_DATASET_FIELD __DATASET \ SDE_USERID_FIELD __USERID \ SDE_PASSWORD_FIELD __PASSWORD \ SDE_VERSION_NAME_FIELD __VERSION \ SDE_TARGET_TABLE_FIELD __TARGET_TABLE \ $(wheremac) \ WHERE_CLAUSE_FIELD __WHERE_CLAUSE \ QUERY_MODE QUERY \ SEARCH_ORDER SPATIAL_FIRST \ COMBINE_ATTRIBUTES ADD \ OUTPUT_DUPLICATES NO \ GET_SPATIAL_RELATIONS \ OUTPUT RESULT FEATURE_TYPE * \
@RemoveAttributes(__SERVER,__INSTANCE,__DATASET,__USERID,__PASSWORD,__TARGET_TABLE,__SEARCH_METHOD,__WHERE_CLAUSE,__VERSION) \
@SupplyAttributes(_table_name,@FeatureType()) \ @FeatureType(@Concatenate(ARCSDEQUERIER,"_",@FeatureType())) \ @Transform(SDE30,FME_GENERIC)
![Page 12: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/12.jpg)
FME Objects Python API 12
ClippingFactory
FACTORY_DEF * ClippingFactory \ FACTORY_NAME CLIPPER \ INPUT CLIPPER FEATURE_TYPE clip_area_1 \ INPUT CLIPPEE FEATURE_TYPE ARCSDEQUERIER_STONNING_CAD \ MULTICLIP YES \ CLIPPEE_ON_BOUNDARY INSIDE \ MERGE_CLIPPER_ATTRIBUTES No \ DO_NOT_AGGREGATE NO \ CLIPPER_ATTR_PREFIX "clipper_" \ OUTPUT INSIDE FEATURE_TYPE CLIPPER_INSIDE \ OUTPUT CLIPPED_INSIDE FEATURE_TYPE CLIPPER_CLIPPED_INSIDE \ OUTPUT EXTRA_CLIPPER FEATURE_TYPE CLIPPER__delete__
@Tcl2("FME_LogMessage fme_warn {Deleting extra clipper feature encountered which entered transformer CLIPPER}") \
OUTPUT NONPOLY_CLIPPER FEATURE_TYPE CLIPPER__delete__ @Tcl2("FME_LogMessage fme_warn {Deleting non-polygonal clipper feature which entered transformer CLIPPER}")
FACTORY_DEF * TeeFactory \ FACTORY_NAME CLIPPER_Nuker \ INPUT FEATURE_TYPE CLIPPER__delete__ \ @Log("Offending feature was:")
![Page 13: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/13.jpg)
FME Objects Python API 13
Joiner (Relate) ‘Factory’
Relate CACHE_SIZE "JOINER:all_symbology_lookup_table" 5000
Relate TABLE_LOCATION "JOINER:all_symbology_lookup_table" "d:\User Documents\LagenSpatial\Projects\GA - FME Data Migration\Symbology Library (new)\version1.1 20060419\all_symbology_lookup_table.csv"
Relate TABLE_DEF "JOINER:all_symbology_lookup_table" CSV \
FACTORY_DEF * TeeFactory \ FACTORY_NAME JOINER \ INPUT FEATURE_TYPE CLIPPER_INSIDE \ INPUT FEATURE_TYPE CLIPPER_CLIPPED_INSIDE \ OUTPUT FEATURE_TYPE JOINER_JOINED \ @Relate(JOINER,Read) \ @RenameAttributes(_matched_records,matched_records)
![Page 14: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/14.jpg)
FME Objects Python API 14
Using FME Objects Python API to Provide
Platform Independent Translations
Save the pasted output as one or more .fmi files
Use the FMEFactoryPipeline.addFactoriesFromFile() method to include your translation pipeline .fmi files into your PyFMEObjects Script.
![Page 15: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/15.jpg)
FME Objects Python API 15
Using FME Objects Python API to Provide
Platform Independent Translations
WHY???
Reduce possible errors in your script Special Characters
\n – New line\r – Carriage Return\w – White Space\t – Tab !!!
Special characters MEAN something in both Python and FME Mapping Files
![Page 16: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/16.jpg)
FME Objects Python API 16
Using FME Objects Python API to Provide
Platform Independent Translations
Allows you to solve other bugs Logical Programmatic Data Flow
Logical – If…Then…Else Do…Loop Switch…Case
![Page 17: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/17.jpg)
FME Objects Python API 17
Using FME Objects Python API to Provide
Platform Independent Translations
Allows you to solve other bugs Logical Programmatic Data Flow
Programmatic – Incorrect Method use String Manipulation Reading Multiple Source Datasets
![Page 18: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/18.jpg)
FME Objects Python API 18
Using FME Objects Python API to Provide
Platform Independent Translations
Allows you to solve other bugs Logical Programmatic Data Flow
Data Flow – Combination of Logical and Programmatic Errors Unexpected output
![Page 19: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/19.jpg)
FME Objects Python API 19
Using FME Objects Python API to Provide
Platform Independent Translations
Once you have your Python script reproducing the same output as Workbench, its time to Embed your Factories
…which is also where the fun begins!!
![Page 20: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/20.jpg)
FME Objects Python API 20
Using FME Objects Python API to Provide
Platform Independent Translations
First of all…
Use the Python Lists i.e. stringListVariable = [‘green’, ‘eggs’, ‘and’, ‘ham’]
Cuts out the possibility of naughty special characters sneaking their way into your Factory Definition
i.e. DON’T USE THE “””green eggs and ham“”” strings – it tells Python to use the string as it is, Naughty Special Characters and all
![Page 21: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/21.jpg)
FME Objects Python API 21
GOOD
fme_session.config(['Relate', 'RELATION_DEF', 'JOINER', '1:0..1+', 'TABLE', 'JOINER:all_symbology_lookup_table', 'UNIQUE(Esri_No,feature_type)', 'JOIN', 'Esri_No', 'TO', 'SYMBOL', 'JOIN', 'feature_type', 'TO', 'fme_type', 'MATCHED_RECORDS_ATTR', 'matched_records', 'TRIM_TRAILING', 'No', 'TRANSFER', 'mapinfo_brush_background', 'TO', 'mapinfo_brush_background', 'TRANSFER', 'mapinfo_brush_foreground', 'TO', 'mapinfo_brush_foreground', 'TRANSFER', 'mapinfo_brush_pattern', 'TO', 'mapinfo_brush_pattern', 'TRANSFER', 'mapinfo_pen_color', 'TO', 'mapinfo_pen_color', 'TRANSFER', 'mapinfo_pen_pattern', 'TO', 'mapinfo_pen_pattern', 'TRANSFER', 'mapinfo_pen_width', 'TO', 'mapinfo_pen_width', 'TRANSFER', 'mapinfo_symbol_angle', 'TO', 'mapinfo_symbol_angle', 'TRANSFER', 'mapinfo_symbol_color', 'TO', 'mapinfo_symbol_color', 'TRANSFER', 'mapinfo_symbol_file_name', 'TO', 'mapinfo_symbol_file_name', 'TRANSFER', 'mapinfo_symbol_font', 'TO', 'mapinfo_symbol_font', 'TRANSFER', 'mapinfo_symbol_shape', 'TO', 'mapinfo_symbol_shape', 'TRANSFER', 'mapinfo_symbol_size', 'TO', 'mapinfo_symbol_size', 'TRANSFER', 'mif_brush_background', 'TO', 'mif_brush_background', 'TRANSFER', 'mif_brush_foreground', 'TO', 'mif_brush_foreground', 'TRANSFER', 'mif_brush_pattern', 'TO', 'mif_brush_pattern', 'TRANSFER', 'mif_pen_color', 'TO', 'mif_pen_color', 'TRANSFER', 'mif_pen_pattern', 'TO', 'mif_pen_pattern', 'TRANSFER', 'mif_pen_width', 'TO', 'mif_pen_width', 'TRANSFER', 'mif_symbol_angle', 'TO', 'mif_symbol_angle', 'TRANSFER', 'mif_symbol_color', 'TO', 'mif_symbol_color', 'TRANSFER', 'mif_symbol_file_name', 'TO', 'mif_symbol_file_name', 'TRANSFER', 'mif_symbol_font', 'TO', 'mif_symbol_font', 'TRANSFER', 'mif_symbol_shape', 'TO', 'mif_symbol_shape', 'TRANSFER', 'mif_symbol_size', 'TO', 'mif_symbol_size',])
![Page 22: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/22.jpg)
FME Objects Python API 22
BAD
fme_pipeline.addFactoryFromString( """FACTORY_DEF * SDE30QueryFactory \ FACTORY_NAME ARCSDEQUERIER_Querier \ INPUT FEATURE_TYPE __GLOBBED__ \ @SupplyAttributes(__SERVER,""" + server + """,__INSTANCE,port:""" + instance
+""",__DATASET,""" + _dataset + """,__USERID,""" + username + """,__PASSWORD,""" + password + """,__TARGET_TABLE,""" + _owner + "." + featuredataset.upper() + """,__SEARCH_METHOD,SDE_ENVELOPE,__VERSION,SDE.DEFAULT) \
SDE_SERVER_FIELD __SERVER \ SDE_INSTANCE_FIELD __INSTANCE \ SDE_DATASET_FIELD __DATASET \ SDE_USERID_FIELD __USERID \ SDE_PASSWORD_FIELD __PASSWORD \ SDE_VERSION_NAME_FIELD __VERSION \ SDE_TARGET_TABLE_FIELD __TARGET_TABLE \ SDE_SEARCH_METHOD_FIELD __SEARCH_METHOD QUERY_MODE QUERY \ SEARCH_ORDER OPTIMIZE \ COMBINE_ATTRIBUTES MERGE \ GET_SPATIAL_RELATIONS \ OUTPUT RESULT FEATURE_TYPE ARCSDEQUERIER_OUTPUT \
@RemoveAttributes(__SERVER,__INSTANCE,__DATASET,__USERID,__PASSWORD,__TARGET_TABLE,__SEARCH_METHOD,__WHERE_CLAUSE,__VERSION) \
@SupplyAttributes(_table_name,@FeatureType()) \ @Transform(SDE30,FME_GENERIC)""" )
![Page 23: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/23.jpg)
FME Objects Python API 23
Using FME Objects Python API to Provide
Platform Independent Translations
Why Use Python?
Great ‘Whipitupability’
It is an ‘interpreted’ language
Its free and powerful!!! With a great open source community
Supported by ESRI and FME
You can write your own transformers for FME
![Page 24: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/24.jpg)
FME Objects Python API 24
Using FME Objects Python API to Provide
Platform Independent Translations
Example - Sydney
![Page 25: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/25.jpg)
FME Objects Python API 25
Using FME Objects Python API to Provide
Platform Independent Translations
Example - Darwin
![Page 26: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/26.jpg)
FME Objects Python API 26
Using FME Objects Python API to Provide
Platform Independent Translations
Example - Perth
![Page 27: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/27.jpg)
FME Objects Python API 27
Using FME Objects Python API to Provide
Platform Independent Translations
Example – Pt. Lincon
![Page 28: Sep. 21-22, 2006 v FME Worldwide User Conference - Vancouver Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen.](https://reader036.fdocuments.us/reader036/viewer/2022062321/56649f2b5503460f94c461c9/html5/thumbnails/28.jpg)
FME Objects Python API 28
Using FME Objects Python API to Provide
Platform Independent Translations
QUESTIONS???
Name: Andrew SmithEmail: [email protected]: +61 2 9545 7740