Porting A Perl/Sybase Application to Model 204 Chuck Gait CSC.
-
Upload
christine-battin -
Category
Documents
-
view
218 -
download
0
Transcript of Porting A Perl/Sybase Application to Model 204 Chuck Gait CSC.
Porting A Perl/Sybase Application to Model 204
Chuck GaitCSC
Agenda
• Goal of the Task
• Perl
• The Process
• Lessons Learned
Goal of the Task
• Move the existing Sybase database part of the application to M-204, AND
• Maintain the current Sybase code base, AND
• Change as little of the code as possible.
Goal, cont'd
• Why?– Task broken up into 2 parts (short term and near
long term); we're in Phase 1– Phase 2 will involve a complete rewrite (to get
rid of duplicate and spaghetti code)– Access Restriction:
• Two separate sets of users, on two separate networks need access to the same data.
Goal, cont'd
• So– we need to have a single code base to be able to
access both databases.
• But, why have two separate databases since both contain the same data?– Because the security people say so, that's why.
Application Specs
• User GUI– 28,000 lines of Perl code (packages and
scripts)– Approx. 1000 lines of Java code– About 10,000 lines of shell programs– About 200 lines of Sybase code
• ISQL• Dblib
– Not much static HTML (100-200 lines)
Application Specs, cont'd
• Input Side– About 10,000 lines of Perl
• Processes incoming message traffic:– Scans the data, auto-correcting where it can– Reformats certain text portions into formatted data– Stores the messages via Sybase
– 200 or so lines of Sybase
• If we get to this prior to July, great! Or, not.
Perl
• Phil's Eclectric Rubbish Lister
• AKA CGI programming
• The workhorse behind most web processing
• Very Fast and Powerful
• Many different ways to do the same thing (we saw this all over)
• Can be obfuscated, uh, obtuse
More Perl
• @name means an array (context)– $name is a list context for above array
• $name[0], $name[1], ..., $name[n]
• %a is a hash context (i.e, an associative array)– $a{b} is the b-th element in the “a” hash
• Perl is “way cool”
The Process
• Looked at the application– Pulled out “all” of the Sybase code– Had a meeting with the M-204 shop
• Tasked them with writing a SQL parser– Our job was done!
The Process, 2
• M-204 shop revolted!
• Had another meeting
• Result:– Pseudo UL interface for M-204– 3 Perl packages on the front end:
• SYBX• DBX• SWAPI
SYBX
• A Perl package
• Replaced all ISQL with Dblib calls
• Returned data in the agreed-upon format
DBX
• A Perl package
• Software switch:– If server is SYBASE
• Send SQL request to SYBX
– Otherwise• Send SQL request to SWAPI
SWAPI
• Steve's WRANGLER API
• A Perl package– Translates SQL into Pseudo UL
• Uses regular expressions
– Sends PUL to M204– Retrieves output
• Postprocesses it, if necessary (aggregates, etc)
– Returns data in agreed-upon format
Perl and Regexp
$reSelect = "select\\s+(?:(distinct)\\s+)?"
. "($reSelColList|\\*)"
. "(?:\\s+from\\s+($reIdList))"
. "(?:\\s+where\\s+($reComplex))?"
. "(?:\\s+($reGroupBy))?"
. "(?:\\s+($reOrderBy))?";
Perl & Regexp, part 2
select\s+(?:(distinct)\s+)?((?:(?:|)\s*,\s*)*(?:|)|\*)(?:\s+from\s+((:(?:\b[A-Za-z]\w*(?:\.[A-Za-z]\w*)?\s*,\s*)*\b[A-Za-z]\w*(?:\.[A-Zaz]\w*)?)))(?:\s+where\s+((?:\s+(?:and|or)s+*(?:nt\s+)?(?:|(?:"(?:.*?[^\\])?"|'(?:.*?[^\\])?'|\b-?\d+}\b-?\d+(?:\.\d*)?|\b[A-Za-z]\w*(?:\.[A-Za-z]\w*)?|\*)\s+(?:not\s+)?between\s+(?:"(?:.*?[^\\])?"|'(?:.*?[^\\])?'|\b-?\d+}\b-?\d+(?:\.\d*)?|\b[A-Za-z]\w*(?:\.[A-Za
-z]\w*)?|\*)\s+and\s+(?:"(?:.*?[^\\])?"|'(?:.*?[^\\])?'|\b-?\d+}\b-?\d+(?:\.\d*)?|\b[A-Za-z]\w*(?:\.[A-Za-z]\w*)?|\*)|(?:"(?:.*?[^\\])?"|'(?:.*?[^\\])?'|\b-?\d+
}\b-?\d+(?:\.\d*)?|\b[A-Za-z]\w*(?:\.[A-Za-z]\w*)?|\*)(?:\s+not\s+)?\s*(?:like|<>|<|<=|!=|=|>=|>)\s)*(?:"(?:.*?[^\\])?"|'(?:.*?[^\\])?'|\b-?\d+}\b-?\d+(?:\.\d*)
?|\b[A-Za-z]\w*(?:\.[A-Za-z]\w*)?|\*)|(?:"(?:.*?[^\\])?"|'(?:.*?[^\\])?'|\b-?\d+
}\b-?\d+(?:\.\d*)?|\b[A-Za-z]\w*(?:\.[A-Za-z]\w*)?|\*)\s+(?:not\s+)?in\s*\(\s*(?
:(?:"(?:.*?[^\\])?"|'(?:.*?[^\\])?'|\b-?\d+}\b-?\d+(?:\.\d*)?|\b[A-Za-z]\w*(?:\.
[A-Za-z]\w*)?|\*)\s*,\s*)*(?:"(?:.*?[^\\])?"|'(?:.*?[^\\])?'|\b-?\d+}\b-?\d+(?:\
.\d*)?|\b[A-Za-z]\w*(?:\.[A-Za-z]\w*)?|\*)\s*\)|(?:"(?:.*?[^\\])?"|'(?:.*?[^\\])
?'|\b-?\d+}\b-?\d+(?:\.\d*)?|\b[A-Za-z]\w*(?:\.[A-Za-z]\w*)?|\*) \s+is\s+(?:not\
s+)?null)))?(?:\s+((?:\bgroup\s+by\s+(?:(?:\b[A-Za-z]\w*(?:\.[A-Za-z]\w*)?\s*,\s
*)*\b[A-Za-z]\w*(?:\.[A-Za-z]\w*)?))))?(?:\s+((?:\border\s+by\s+(?:(?:\b[A-Za-z]
\w*(?:\.[A-Za-z]\w*)?|\b-?\d+)(?:\s+(?:asc|desc))?\s*,\s*)*(?:\b[A-Za-z]\w*(?:\.
[A-Za-z]\w*)?|\b-?\d+)(?:\s+(?:asc|desc))?)))?
Pseudo User Language
• A select turns into GET
• An update turns into RPL
• A delete turns into DEL
PUL
• select fld1,fld2 from tbl1 where fld1=10– becomes
• GET~TBL1~FLD1|FLD2~FLD1=10~
• select * from tbl1,tbl2 where fld1=10 and tbl1.x=tbl2.x– Becomes
• XREF~TBL1|TBL2~FLD1|FLD2~FLD1=10~x~
The Returned Data
• Data coming back from both databases had to be identical
• That's why we converted all ISQL calls to Dblib– Some code changes required.
The Result
• No change for the user– Application looks identical for both databases– Great for the user since no retraining is needed.
Lessons Learned
• Was a lot more difficult than originally estimated.
• Missed some SQL that was generated on the fly
• Should have just bit the bullet and did the complete rewrite.