SCRIBE WHITE PAPER HOW SCRIBE ONLINE WORKS · PDF fileHOW SCRIBE ONLINE WORKS. Scribe Software...
Transcript of SCRIBE WHITE PAPER HOW SCRIBE ONLINE WORKS · PDF fileHOW SCRIBE ONLINE WORKS. Scribe Software...
© Scribe Software Corporation | All rights reserved | 1.603.622.5109 scribesoft.com | 1
WHITE PAPER : HOW SCRIBE ONLINE WORKS
Introduction Scribe Online is a leading integration platform as a service (iPaaS) from Scribe
Software. It has a multi-tenant architecture that scales by distributing workloads
across agents that belong to each tenant. The cloud handles administrative
tasks such as user management, connection management, integration design,
scheduling and monitoring. Agents carry out instructions they receive from the
cloud and report results back to the cloud.
Scribe OnlineUser Interface
Third PartySaaS
Application
Tenant Specific Components
Connections SolutionsMaps ExecutionHistory
ScribeOnlineCloud
Customer Site
On-premiseAgents
Applications
Cloud AgentsScribe Ecosystem
Shared Components
APIConnectorsConnectorDevelopment
Kit
Scribe Ecosystem
© Scribe Software Corporation | All rights reserved | 1.603.622.5109 scribesoft.com | 2
WHITE PAPER : HOW SCRIBE ONLINE WORKS
The platform uses connectors to communicate with specific applications,
databases and file systems. Connectors address the specifics of each type of
endpoint, including authentication, discovery of metadata, query formation and
target operations. They interact with the platform through an open interface that
allows anyone with a Scribe Online account to create a connector.
A comprehensive security model encompasses all aspects of the platform. The
user interface, agents, and third-party applications communicate with the cloud
through secure channels. Connectors use secure protocols to communicate
with the APIs for the respective applications. Credentials are stored in encrypted
containers, and IP safe lists restrict access to specific tenants.
The functionality of the platform is available through a modern REST API, which
allows partners to embed the platform’s capabilities into their applications.
Continue reading to learn the details of how the components work.
© Scribe Software Corporation | All rights reserved | 1.603.622.5109 scribesoft.com | 3
WHITE PAPER : HOW SCRIBE ONLINE WORKS
Organizations and Users
Each tenant in the Scribe Online cloud service is called an organization. It
represents the company using the service, and there are three types:
1. A normal organization stands alone and represents a company that buys
the Scribe Online service directly from Scribe or through a reseller. Normal
organizations often begin with a trial subscription initiated through the
Scribe website.
2. A parent organization belongs to a company that purchases the Scribe
Online iPaaS on a wholesale basis with the ability to create and manage child
organizations for its customers.
3. A child organization falls under a specific parent organization and
represents a company that is a customer of the owner of the
parent organization.
Most cloud applications restrict each user to a single organization. Scribe
Online allows users to belong to more than one organization. The user interface
provides users who belong to more than one organization with a picklist to
switch from one organization to another.
An administrator in an organization can invite other users to that organization
and remove users from the organization. Child organizations automatically
inherit the users of the parent organization. Each child organization can also
have its own users in addition to the users from the parent organization.
Each organization has one or more subscriptions to Scribe Online services.
© Scribe Software Corporation | All rights reserved | 1.603.622.5109 scribesoft.com | 4
WHITE PAPER : HOW SCRIBE ONLINE WORKS
Services
Scribe Online offers three different services:
1. The Integration Service performs bidirectional integration between
two or more endpoints with scheduling, support for complex logic and
transformations, and the ability to recognize records that are new or
changed since the last integration job ran.
2. The Replication Service creates and maintains a copy of the data in a
cloud application such as Salesforce.com or Microsoft Dynamics CRM in a
database. The database can be on premise or in the cloud.
3. The Migration Service enables Scribe partners to migrate data from
one application to another. It offers most of the features of the Integration
Service, with the exception of scheduling and detection of changed records.
The Migration Service is only available as a wholesale service that the partner
manages through a parent organization.
Each service is available in multiple tiers with different features and options.
© Scribe Software Corporation | All rights reserved | 1.603.622.5109 scribesoft.com | 5
WHITE PAPER : HOW SCRIBE ONLINE WORKS
Connectors
A connector links an application, database or file system to the Scribe Online
platform. Most connectors support two-way communication, and they have several
basic functions, including:
• Authentication with a set of user credentials
• Discovery of metadata, often including custom objects and custom fields
• Sending queries, usually with filters, and receiving the results for processing
• Accepting inbound messages for processing
• Translation of Scribe Online platform operations such as Create, Lookup, Update/
Upsert and Delete into specific API calls that the application understands
Connectors have the ability to process large datasets using paging, batches or bulk
loads. They can process hundreds or even thousands of records at a time in one API
call, improving performance and reducing the number of API hits.
The Scribe Online user interface includes a Marketplace of connectors. Through the
Marketplace, users enable the connectors they need in their organizations. Scribe
provides many of the connectors, and third parties have created many others.
© Scribe Software Corporation | All rights reserved | 1.603.622.5109 scribesoft.com | 6
WHITE PAPER : HOW SCRIBE ONLINE WORKS
Connections
Once a connector is enabled in a Scribe Online organization, it can be used
to make a connection to a specific instance of an application, database or file
system. In most cases, a connection includes user credentials for the application
or database and a URL or other location of the application.
Scribe Online tests the credentials before creating the connection and stores
the connection details in an encrypted container for later use when running
maps. Connections are reusable -- an existing connection can be used again
and again in different maps.
© Scribe Software Corporation | All rights reserved | 1.603.622.5109 scribesoft.com | 7
WHITE PAPER : HOW SCRIBE ONLINE WORKS
Agents
The Scribe Online architecture includes separate design and run-time
components. The cloud platform handles administration, design and monitoring,
while delegating execution of replication and integration jobs to its agents. The
agent retrieves its commands from the cloud through Microsoft’s Azure Service
Bus, carries out the instructions and reports the results to the cloud.
Scribe Online offers two types of agents:
• Cloud agents handle integrations between cloud-based endpoints such as
SaaS applications, cloud databases and cloud filesystems like Dropbox. For
example, a cloud agent can be used to integrate leads between Marketo and
Microsoft Dynamics CRM Online. Scribe hosts the agents in its private cloud.
• On-premise agents handle integrations where one or more endpoints exist
behind a firewall, inaccessible to a cloud agent. Users install an on-premise
agent behind the firewall where the agent can communicate with the local
endpoint as well as any cloud endpoints. For example, one would use an on-
premise agent to replicate Salesforce.com data to a local database.
All agents run on the same codebase and operate semi-autonomously, increas-
ing the resilience and availability of the overall system. Even if an agent loses
communication with the cloud, it continues to run the latest compiled versions
of its maps according to the schedule. Once connectivity is reestablished, it
reports the results to the cloud. This means agents continue to run even during
scheduled maintenance of the cloud platform.
© Scribe Software Corporation | All rights reserved | 1.603.622.5109 scribesoft.com | 8
WHITE PAPER : HOW SCRIBE ONLINE WORKS
When Scribe releases a new version of Scribe Online or a connector, the agent
finishes any work it is doing, then downloads and runs the new code. This
ensures that new software does not interrupt integration jobs.
Maps
The Scribe Online cloud provides a graphical design environment where users
create and maintain integration maps. Each map begins with one or more
connections from the set defined in an organization.
Scribe Online offers three types of maps:
1. Query maps use one of the connections associated with the map to get data
for processing. The query can include filters to restrict the results. In most
cases, this will include a net change filter, which Scribe Online uses to retrieve
only records that are new or changed since the last time the map ran.
2. Message maps allow a cloud application like Salesforce.com to push
messages to the map. Each message map has a unique URL. Users configure
the cloud application to deliver records to the URL. When records arrive,
the agent executes the map and uses one of the connections in the map to
interpret the message payload. Message maps process asynchronously.
3. Request/Reply maps allow an external application to send records to a
map in JSON format through a REST API. Like message maps, each request/
reply map has its own unique URL. Unlike the other two map types, the
application sending the request does not need a connection in Scribe Online.
As the agent executes the map, it builds a response, which it sends back to
the calling application. Request/reply maps process synchronously.
© Scribe Software Corporation | All rights reserved | 1.603.622.5109 scribesoft.com | 9
WHITE PAPER : HOW SCRIBE ONLINE WORKS
The Scribe Online user builds the map with the visual designer by dragging
blocks onto a canvas to define a logic sequence. Control blocks direct the logic
flow. The main control block, For Each Result, creates a loop that executes once
for each record the query produces. Others provide if-then-else and other
conditional tests.
Operation blocks take action on one of the connections in the map, such as
creating, reading, updating, upserting or deleting a record. They also handle
field mapping and data transformation with a library of dozens of Excel-like
functions. For update, upsert and delete operations, users configure match
fields to identify the correct record in the target connection.
Scribe Online also provides a map debugger that allows a user to execute a
map one step at a time or to set break points when certain conditions occur.
This aids in troubleshooting map design.
© Scribe Software Corporation | All rights reserved | 1.603.622.5109 scribesoft.com | 10
WHITE PAPER : HOW SCRIBE ONLINE WORKS
Solutions
In many cases, an integration process involves more than one map. A solution
is a set of ordered maps that are assigned to run on a specific agent. Each time
a solution runs, it executes the maps in order until the final map completes.
Solutions can run according to a pre-set schedule, on-demand through the user
interface or via an API call.
Solutions can be exported and imported. The exported maps contain the
definition of the connections used, data flow, and mappings/formulas used.
Export and import allow you to move solutions between test and production
environments or from one customer’s organization to another.
REST API
Scribe Online offers a REST API that provides application programmers with
access to much of the functionality of the platform. This allows you to embed
the functionality of Scribe Online in another SaaS application. Developers can
automate processes for:
• Creating users and organizations
• Installing agents
• Adding connectors and creating connections
• Creating a solution with a set of maps
• Scheduling solutions
• Monitoring the results of integration tasks
Scribe Online enforces a safe list for the API based on the IP address making the
API call. Users configure one or more IP address ranges that can make API calls
for an organization.
© Scribe Software Corporation | All rights reserved | 1.603.622.5109 scribesoft.com | 11
WHITE PAPER : HOW SCRIBE ONLINE WORKS
New Connector Development
SaaS companies and integration consulting firms can create their own
connectors for Scribe Online. The scope of a connector development project
includes authentication, metadata discovery, queries and target operations like
create, update and delete.
There are three options available for building Scribe Online connectors.
• The Messaging Framework works with applications that send outgoing
webhooks in either JSON or XML format. It allows you to quickly build a
connector from a sample message in Microsoft Visual Studio. You can use
the connector as the source in a message map but target operations are not
supported.
• The Fast Connector Framework is a wizard-driven tool that helps you
build a connector to an HTTP web service by completing a Microsoft Visual
Studio Solution with the specific details of the application you’re connecting
to. This approach allows you to build connectors that support both source
and target operations.
• The Connector Development Kit (CDK) offers more flexibility and access
to additional functionality, such as complex filtering in queries, picklist values
in metadata, additional target operations beyond create and update, and
support for related entities.
Scribe offers a certification process for those who wish to make a connector
available to others. Once certified, your connector can be listed in the Scribe
Online Marketplace.
© Scribe Software Corporation | All rights reserved | 1.603.622.5109 scribesoft.com | 12
WHITE PAPER : HOW SCRIBE ONLINE WORKS
Monitoring and Notifications
Each time an agent runs an integration job, it reports the results back to the
Scribe Online cloud. Results include how long the job ran, how many records
were processed, how many failed, and any errors that occurred. Users can view
the Execution History related to a solution in the Scribe Online user interface.
Scribe Online can reprocess failed records for up to 45 days.
Users can also configure Scribe Online to send notifications via email when
certain events occur. Notifications are available for solution and record failures,
system updates, connector installation and lack of agent communication with
the cloud. These settings are specific to each user in each organization because
users can belong to multiple organizations and might want different settings for
each one.
© Scribe Software Corporation | All rights reserved | 1.603.622.5109 scribesoft.com | 13
WHITE PAPER : HOW SCRIBE ONLINE WORKS
About Scribe Software
Scribe is an established global provider of application and data integration
software. The company’s easy to use platforms, comprehensive integration lifecycle
management capabilities and open APIs offer the fastest way to create, run, and
adapt integrations. More than 12,000 customers and 1,200 partners – including
enterprises, software as a service (SaaS) providers, systems integrators (SIs) – rely
on Scribe’s integration platform as a service (iPaaS) and on-premises platform
to accelerate integration projects and time-to-value and to meet the complex
connectivity demands of today’s cloud and hybrid environments. Scribe and its
solution development partners serve customers in every industry, including financial
services, life sciences, manufacturing, public sector, retail, utilities, higher education
and media and entertainment.
For a free trial of Scribe’s software platforms go to:
www.scribesoft.com/products/trial-software/.
CORPORATE HEADQUARTERS
150 Dow Street Manchester, NH 03101 USA 1.603.622.5109
EMEA OFFICE
Bezuidenhoutseweg 161, 2594 AG Den Haag, The Netherlands +31 (0) 70 8200322