Handling large file with managed file transfer and oracle's cloud services
-
Upload
robert-van-moelken -
Category
Technology
-
view
548 -
download
1
Transcript of Handling large file with managed file transfer and oracle's cloud services
Robert van Mölken - AMIS Services
Handling Large File with Managed File Transfer and Oracle's Cloud Services
Who Am I
• Robert van Mölken• Oracle Integration Specialist since ‘07• Beta tester, blogger, SOA SIG Lead
for the Dutch User Group and author of “Implementing Integration Cloud Service”
• Linkedin: linkedin.com/in/rvmolken • Blog: technology.amis.nl • Twitter: @robertvanmolken
Middleware Partner of the Year 2011, 2013, 2014 & 2015
EMEA Partner of the Year 2014
3
What will we discuss today?
Agenda
File Transfer Challenges
File Transfer Patterns
Real World Use Case with Cloud Integrations
Demo of On-Premises 2 Cloud Integration
4
The growing challenge with Managing File Transfers
Cloud introduces a new dimension of complexity…
On-premises File Transfers
Cloud File Transfers
5
File Transfer Challenges
• Multiple ways and entry points for receiving files• Uncontrolled proliferation (spread) of FTP servers or File shares• Not integrated with enterprise security standards• Single points of failure, not HA or reliable delivery• Complex and costly to maintain
• Handling large files – How do we not clog our Middleware environment?– How do we know a file is processed successfully?
• Different styles of transfers with On-Premises and Cloud applications• Varied protocol – (s)FTP, file, SOAP WS, REST API, Cloud Adapter
6
Managed File Transfer Overview
• Simple and Secure End-to-End Managed File Gateway– Support large files, PGP encryption, auditing, monitoring – pass-by-reference, extensible with custom call-outs
• Lightweight Web based Design Time Interface– Easily build, edit and deploy end-to-end transfers
• Standards Based Middleware & Cloud Integrations– (s)FTP, File, SOA, B2B, Webcenter Content, Storage Cloud
• Additional Characteristics– Scheduling, Embedded sFTP server, FTP proxy, no
transformations– Advanced Management: Pause, Resume, Resubmit
7
Oracle SOA Cloud Overview
Oracle Applications, SAP, customer, legacy…
ON-PREMISES
3rd PartyORACLE CLOUD
Salesforce
OracleSOA Cloud Service
Oracle SaaS Applications
OracleSOA Suite
• SOA Suite in the Cloud– Platform for Orchestration, application integration, API
management, virtualization, orchestration, streaming analytics, B2B, MFT, connectivity, business monitoring dashboards
– Fully portable, lift and shift from/to on-premise– Identical components to Oracle SOA Suite for faster
deployment portability
• Complete access to product surface area – Access to Weblogic console, Enterprise Manager & Filesystem– Rapid and fully automated provisioning– Secure, Highly Available with Clustering– Fully Managed i.e. easy patching– Build anywhere, deploy anywhere
8
Oracle Storage Cloud Overview
Oracle SOA Suite, Oracle Apps, customer, legacy…
ON-PREMISES
ORACLE CLOUD
Oracle PaaS & SaaS Applications
• Secure file storage in the Cloud– Secure, elastic, reliable, and cost-effective cloud storage– Access anywhere, 24/7, from any device– Replicated on multiple nodes
• Easy-to-use solution– Store, manage & consume large amounts of unstructered data– OpenStack Swift-compatible REST API or Java API– Web- based console for monitoring & management
• Role-based access control– Very granular level, folders of on individual files– Make data accessible publicly
OracleStorage Cloud Service
3rd Party
Cloudberry
OracleManaged File Transfer
9
Oracle Document Cloud Overview
Oracle SOA Suite, Oracle Apps, customer, legacy…
ON-PREMISES
ORACLE CLOUD
Oracle PaaS & SaaS Applications
• Content Collaboration in the Cloud– Intuitive & user-friendly web-based interface– Collaborate within or outside the company
• Anytime, Anywhere, Any Device– 24 X 7 access to files, folders, people and conversations– Access content from mobile devices, sync with Mac or PC or work off
web browser (and offline access).
• Engage and communicate– Presenter App to display presentations, PDFs, etc.– Social conversations about Cloud assets
• Application extensions– Sync data between cloud and existing content server
OracleDocument Cloud Service
3rd Party
Office 365
OracleManaged File Transfer
10
Overview Summary
• Managed File Transfer
• SOA Suite Cloud Service
• Storage Cloud Service
• Document Cloud Service
File Transfer Patterns
11
12
Stand-alone MFT Patterns
Source TargetMFT
• Direct – source to target
• Fan-out – source to multiple targets
• Chained – multiple MFT transfers
Source TargetMFT
Target
Target
Source MFT MFT Target
13
SOA Cloud Patterns
• SOA web services
• SOA based appplication
Source TargetMFT SOA
Source TargetSOA MFT
Source
Target
MFT Flow 1
SOAApplication
MFTFlow 2Source
Initiate instance
Mid-process receive
14
Storage Cloud Patterns
• Cloud File Back-up
• Automatic File Encryption
Source StorageCloudMFT
StorageCloud
StorageCloud
MFTPGP Encryption
Source
StorageCloud
MFT
Target
15
Document Cloud Patterns
• Cloud Collaboration
• Sync changes to back-end
Source DocumentCloudMFT
DocumentCloud TargetMFT
Web
Mobile
Social
Mobile
16
Hybrid Integration Patterns
• Hybrid Integration
Source MFT Target
MFT
Target
DocumentCloud
SOA
StorageCloud
Web
Mobile
Social
Real World Use Case
17
18
Use Case Overview
Combining on-premises File Handling with Cloud Collaboration & Back-up
• Files are supplied by on-premises CRM on File share
• MFT transfers files to SOA Cloud Service
• SOA Cloud Service composite App combines files to single ZIP
• MFT transfers ZIP file to Document CS and back-ups file to Storage CS
• Files can be accessed by anytime, anywhere.
19
Use Case Overview
Web
Mobile
Social
Source MFT
Source MFT
SOA MFT
DocumentCloud
StorageCloud
Web
App
On-Premises 2 Cloud IntegrationDefining MFT transfers for Batch
20
21
Check availability network share
Source MFT
Source MFT
TRG
IN
22
Check availability network share
Source MFT
Source MFT
TRG
IN
23
Check availability network share
Source MFT
Source MFT
TRG
IN
24
Define trigger source
Source MFT
Source MFT
TRG
IN
25
Define trigger source
Source MFT
Source MFT
TRG
IN
26
Define trigger source
Source MFT
Source MFT
TRG
IN
27
Define file source
Source MFT
Source MFT
TRG
IN
28
Define file source
Source MFT
Source MFT
TRG
IN
29
Define file source
Source MFT
Source MFT
TRG
IN
30
Define trigger target
MFT
MFT
SOA
31
Define trigger target
MFT
MFT
SOA
IN
TRG
32
Define trigger target
MFT
MFT
SOA
IN
http://cloudhost:8001/soa-infra/services/default/ProcessBatchService/receiveTrigger
TRG
33
Define file target
MFT
MFT
SOA
TRG
IN
34
Define file target
MFT
MFT
SOA
TRG
IN
35
Define file target
MFT
MFT
SOA
TRG
IN
http://cloudhost:8001/soa-infra/services/default/ProcessBatchService/receiveFile
36
Define & deploy trigger transfer
MFT
MFT
SOA
37
Define & deploy trigger transfer
MFT
MFT
SOA
TRG
IN
38
Define & deploy trigger transfer
MFT
MFT
SOA
TRG
IN
39
Define & deploy trigger transfer
MFT
MFT
SOA
TRG
IN
40
Define & deploy trigger transfer
MFT
MFT
SOA
TRG
IN
41
Define & deploy file transfer
MFT
MFT
SOA
TRG
IN
42
Define & deploy file transfer
MFT
MFT
SOA
TRG
IN
43
Define & deploy file transfer
MFT
MFT
SOA
TRG
IN
44
Define & deploy file transfer
MFT
MFT
SOA
TRG
IN
On-Premises 2 Cloud IntegrationSOA application for MFT batching
45
46
Inspecting SOA application
MFT
MFT
SOA
47
Inspecting SOA application
MFT
MFT
SOA
48
Inspecting SOA application
MFT
MFT
SOA
49
Inspecting SOA application
MFT
MFT
SOA
50
Inspecting SOA application
MFT
MFT
SOA
On-Premises 2 Cloud IntegrationDefining MFT transfers for Cloud integration
51
52
Define FTP source
SOA MFT
DocumentCloud
StorageCloud
53
Define FTP source
SOA MFT
DocumentCloud
StorageCloud
FTP
54
Define FTP source
SOA MFT
DocumentCloud
StorageCloud
FTP
55
Define FTP source
SOA MFT
DocumentCloud
StorageCloud
FTP
56
Define Storage Cloud target
SOA MFT
DocumentCloud
StorageCloud
57
Define Storage Cloud target
SOA MFT
DocumentCloud
StorageCloud
58
Define Storage Cloud target
SOA MFT
DocumentCloud
StorageCloud
One thing to keep in mind!
Import certificate of StorageCloud domain in keystore
59
Define Document Cloud target
SOA MFT
DocumentCloud
StorageCloud
60
Define Document Cloud target
SOA MFT
DocumentCloud
StorageCloud
• NO adapter available in MFT 12.2 yet!• Document Cloud provides REST API
Solution: • Build custom Java Callout that calls REST API
• REST API upload example:– POST https://
documents-amiscloud04.documents.us2.oraclecloud.com/documents/api/1.1/files/data
– Multipart HTTP request, one part JSON object with the parentID and the other part is the content of the file itself (Mime Attachment).
– { “parentID”: “GUID-of-folder”, “primaryFile”: “name-of-file.extenstion” }
61
Define Document Cloud target
SOA MFT
DocumentCloud
StorageCloud
• GUID of folder can be found when sharing the folder
62
Define Cloud Transfer
SOA MFT
DocumentCloud
StorageCloud
63
Define Cloud Transfer
SOA MFT
DocumentCloud
StorageCloud
64
Define Cloud Transfer
SOA MFT
DocumentCloud
StorageCloud
65
Add Pre-processing Actions
SOA MFT
DocumentCloud
StorageCloud
66
Add Pre-processing Actions
SOA MFT
DocumentCloud
StorageCloud
67
Deploy Cloud Transfer
SOA MFT
DocumentCloud
StorageCloud
On-Premises 2 Cloud IntegrationTesting the Integration
68
69
Determine state of deployments
70
Transferring ZIP file to CloudSOA puts file on FTP folder
SOA MFT
DocumentCloud
StorageCloud
FTP
71
Transferring ZIP file to CloudSOA puts file on FTP folder
SOA MFT
DocumentCloud
StorageCloud
FTP
72
Transferring ZIP file to CloudMFT picks up file and processes
SOA MFT
DocumentCloud
StorageCloud
73
Transferring ZIP file to CloudMFT picks up file and processes
SOA MFT
DocumentCloud
StorageCloud
74
Transferring ZIP file to CloudInspect MFT instance
SOA MFT
DocumentCloud
StorageCloud
75
Transferring ZIP file to CloudInspect MFT instance
SOA MFT
DocumentCloud
StorageCloud
76
Transferring ZIP file to CloudInspect MFT instance
SOA MFT
DocumentCloud
StorageCloud
77
Transferring ZIP file to CloudInspect MFT instance
SOA MFT
DocumentCloud
StorageCloud
78
Transferring ZIP file to CloudInspect Storage Cloud Container
SOA MFT
DocumentCloud
StorageCloud
79
Transferring ZIP file to CloudInspect Storage Cloud Container
SOA MFT
DocumentCloud
StorageCloud
Summary
80
81
Summary
File Transfer Challenges
File Transfer Patterns
Real World Use Cases and Cloud Integrations
Demo of On-Premises 2 Cloud Integration
Pre-order: October 2016
82
Questions?
83