Moving at the Speed of Change Multitheaded Service...
Transcript of Moving at the Speed of Change Multitheaded Service...
Sponsored by Powered by
Moving at the Speed of Change May 2015
Charlotte PowerBuilder Conference
Multitheaded Service
Applications For PowerBuilder Classic!
Charlotte PowerBuilder Conference Moving at the Speed of Change May 2015
• Email: [email protected] • Blog: http://chrispollach.blogspot.ca • PBDJ: http://chrispollach.sys-con.com • LinkedIn: http://ca.linkedin.com/in/chrispollach • SourceForge: http://sourceforge.net/projects/stdfndclass • TaeKwonDo: http://www.syeoh.com
2 Maximizing your PowerBuilder Application Horsepower!
Based on Actual Production Implementations!
By Chris Pollach – President: Software Tool & Die Inc.
Ottawa, Canada
Charlotte PowerBuilder Conference Moving at the Speed of Change May 2015
The Problem
3 PCI – Payment Card Industry Compliance
Approach:
Automated: OCR + Redact + Audit Trail
- 75 Form types
- Number format
- Image Quality
-Millions images!
Charlotte PowerBuilder Conference Moving at the Speed of Change May 2015
4 Volumetrics!
Charlotte PowerBuilder Conference Moving at the Speed of Change May 2015
5
Charlotte PowerBuilder Conference Moving at the Speed of Change May 2015
6 1 Worker = 4-5 Years! CI-2
PCI Repository
DataBase
Ima
ging
Imag
ing
Ima
ging
* Archive
Seg . 99+
* Archive
Segment3
* Archive
Segment2
* Archive
Segment1
C.I. Database
(217
Terabytes)
Production
Worker
Archive 1
Archive 1
Qualifier Redaction
Worker#2
Update
Archive 1
Redaction
Worker#1
Redaction
Worker#nn
Redaction Work Queue
IM1
Solution … Multithreading (48 workers => 6 months)
1-48
worker
Threads
Original
Credit Card
Information
IM1 IM1
IM1
DBServices Database
RWQ
Update
All Archives
processed
Sequentially
Trigger OCR
OCR
OCR
Charlotte PowerBuilder Conference Moving at the Speed of Change May 2015
7 Candidate for Multithreading!
DB Services Database C.I.
Database
App_Form
Image
Supporting
Doc_Image
Photo
Image
Signature
Image
DEC
Image
ICWQ Image_ID
Archive_DB_Nam
e
Table_Name
...
Image
Conversion
Worker
Image
MagicK
Bitmap DML
JPEG
Bitmap * Av: => 849,787
JPEG * Av: 76,159
12M-14M!
Charlotte PowerBuilder Conference Moving at the Speed of Change May 2015
8 Another candidate for Multithreading!
Windows Thread
Outbound
(Message) Worker
Windows Thread
Outbound
(Message) Worker
Windows Thread
Inbound
(Transaction)
Worker
Windows Thread
Inbound
(Transaction)
Worker
GCMS WIP Database
RQ Request-1
Request-2
Request-3
Request-4
Request-5
...
Windows Service (Main Thread)
Queue Manager
Request Datum
SubThread
Inbound
(Request) Worker
GCMS
S
P
MQ Msg-1
Msg-2
Msg-3
Msg-4
Msg-5
...
Queue
ID
W
S
SubThread
Outbound
(Message) Worker
Msg + XML
...
...
1
3
2
4 4
5
5
6
5
6
5
XML
...
ePPS
IEC 7
Table TR +
SP
S
P
S
P
Table S
P Table
Table
7
Folder
Clean
S
P 8
F.R.
Charlotte PowerBuilder Conference Moving at the Speed of Change May 2015
9
Charlotte PowerBuilder Conference Moving at the Speed of Change May 2015
10
Debug
Common Modules
Windows
Service
Application (* included both Main
Thread and Sub-
Threads
implementation)
File
Text Msg
PowerShell
Log
MainThread
ErrorHandler
Database
WebService
Notification
XML
Connectivity,Pooling
Trans. Mgnt ...
Start,Stop,Query ...
Connectivity, Encryption,
Security ...
Trapping, Controlled,
Shutdown, Continue, Stop
...
Event, File, Database
Debug ...
SMTP, SMS, MAPI
...
Run, O/S Feature
...
Parsing, Schema
Validation, Escapement ...
Console Query, Stop
Report ...
Message
O/S App.
Web
Service
Sub-
Thread
HTTP/HTTPS
NetTCP
Format, Parameter parsing,
Date Time stamping ...
Process,Quiesce
Activate ... SubThread *
Service O/S
M
M
O/S
Charlotte PowerBuilder Conference Moving at the Speed of Change May 2015
11 Cla
ss
Co
ns
tru
cti
on
! Common Classes
In/Out
MainThread
BASE
Thread XML Notification Message
SubThread
...
Windows Service Application(s)
Base Layer
Abstract Layers
Concrete Layer
In/Out
SubThread As Required
As Required As Required
ABSTRACT
MainThread
Charlotte PowerBuilder Conference Moving at the Speed of Change May 2015
12 Thread Interaction
Main Thread Class Access Privilege Sub Thread Class
Window/CVUO Window/CVUO
Window/CVUO NVUO
NVUO Window/CVUO
NVUO NVUO
Charlotte PowerBuilder Conference Moving at the Speed of Change May 2015
13 Implementation
SUB-Thread(s)
MAIN Thread
Application (Main Thread)
Real
Sub-Thread
#01
Config
Legend - No wait method calls (posted)
Sub-Thread
Handler #nn
Sub-Thread
Handler #02
Sub-Thread
Handler #01
Sub-Thread
Handler #03
Handler Array [ ]
Main
Process
Method (Handle,
Parameters)
...
Method
(Completion Code,
Parameters)
Process
Process
Process
Real
Sub-Thread
#02
Main
Process
Process
Process
Process
Real
Sub-Thread
#nn
Main
Process
Process
Process
Process
Real
Sub-Thread
#03
Main
Process
Process
Process
Process
Sleep (no work)
...
StartThread (#nn)
GetStatus (#nn)
StopThread (#nn) ProcessQueue (QID)
Alive?
Alive?
Charlotte PowerBuilder Conference Moving at the Speed of Change May 2015
14 Load Distribution!
MS-Windows based server
Processor0
Processor1
Processor2
Processor3
In/Out bound
Sub-Thread 1
In/Out bound
Sub-Thread 2
In/Out bound
Sub-Thread 3
In/Out bound
Sub-Thread 4
Affinity (0)
Affinity (1)
Affinity (2)
Affinity (3)
Processor
* - “best performance”
Charlotte PowerBuilder Conference Moving at the Speed of Change May 2015
15 Designing for 24/7 operations!
CPU
Load
Stability
Memory
Charlotte PowerBuilder Conference Moving at the Speed of Change May 2015
16 Multiple Servers
PCI
Controller
Server #1
PCI
Controller
Server #2
PCI
Controller
Server #3
Statsus
Table
TCP/IP
Used TCP/IP Peer-to-Peer
?Distributed PowerBuilder?
Charlotte PowerBuilder Conference Moving at the Speed of Change May 2015
17
Co
ns
ole
s!
Sub-thread Console
Main Thread Console
Charlotte PowerBuilder Conference Moving at the Speed of Change May 2015
18 eMail Alerts!
Main Thread Alert
Sub-thread Alert
Charlotte PowerBuilder Conference Moving at the Speed of Change May 2015
• Both PowerBuilder Classic and PB.Net do not support an MS-Windows Service console application like Visual Studio.
• Must employ 3rd party tools in order to transpose a PB application into an MS-Windows Service.
• Many commercial & open source software tools for MS-Windows Service support.
19 MS-Windows Service Dilemma.
Charlotte PowerBuilder Conference Moving at the Speed of Change May 2015
• Service Runner (open source - $0) • N.S.S.M. (open source - $0) • FireDaemon (commercial - $49) • AlwaysUp (commercial - $49) • Application as Service (commercial - $79) • Turbo Service Manager (open source - $0) • Winserv (open source - $0) • Launcher Service (open source - $0) • PBNIServ (commercial - $250) [PB only]
20 Service software for PowerBuilder!
Charlotte PowerBuilder Conference Moving at the Speed of Change May 2015
All systems mentioned built from …
Multithreaded Frameworks Details
• http://sourceforge.net/projects/stdfndclass
• Free
• Open Source
• Examples
• Documentation
• Support
21
Charlotte PowerBuilder Conference Moving at the Speed of Change May 2015
Questions?
22 Q&A Session
Charlotte PowerBuilder Conference Moving at the Speed of Change May 2015
Have you hugged your DataWindow today? 23