Workflow ETL workflow Complex event processing(CEP ) Mona Alnahari

29
Workflow ETL workflow Complex event processing(CEP) Mona Alnahari

description

Workflow ETL workflow Complex event processing(CEP ) Mona Alnahari. Workflow is the definition, execution and automation of business processes where tasks, information or documents are passed from one participant to another for action, according to a set of procedural rules. - PowerPoint PPT Presentation

Transcript of Workflow ETL workflow Complex event processing(CEP ) Mona Alnahari

Page 1: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

Workflow ETL workflow Complex event processing(CEP)

Mona Alnahari

Page 2: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

WHAT IS A WORKFLOW Workflow is the definition, execution

and automation of business processes where tasks, information or documents are passed from one participant to another for action, according to a set of procedural rules.

Page 3: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

During the years, workflow technology has evolved from just workflow automation, which has been focused on automating human centric processes, through Enterprise Application Integration, which is focused on the exchange of information between heterogeneous systems, through Business Process Modeling and Analysis,

Page 4: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

which is focused on gaining a detailed understanding of business processes and the potential impact of changes to those processes, through Business Activity Monitoring, which is focused on analyzing the efficiency and effectiveness of business processes and activities, to BPM that supports the complete process improvement cycle, from definition to implementation, monitoring and analysis, and through ongoing optimization.

Page 5: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

WORKFLOW SOFTWARE HISTORY

The workflow software solutions introduced in the 90’s were based on workflow engines and business rules. Their main objective was to replace the paper-based task-routing activities with automated electronic form processes. By the late 1990s, the workflow software market integrated advanced functionality, including business rules engines, policy management, modeling tools, process monitoring, and optimization.

 

Page 6: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

WORKFLOW SOFTWARE HISTORY in 1993, the Workflow Management

Coalition (WfMC) is a global organization of adopters, developers, consultants, analysts, as well as university and research groups engaged in workflow and BPM.  The WfMC creates and contributes to process related standards, educates the market on related issues, and is the only standards organization that concentrates purely on process. 

Page 7: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

WORKFLOW SOFTWARE HISTORY The WfMC created Wf-XML and XPDL,

the leading process definition language used today in over 80 known solutions to store and exchange process models. XPDL is a process design format for storing the visual diagram and all design time attributes.

Page 8: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

WORKFLOW SOFTWARE HISTORY

In 2005, Microsoft deployed the Windows Workflow Foundation which replaced the basic workflow engines with sophisticated workflow functionality.

  Today workflow software needs to comply

with the full BPM discipline of modeling, execution, simulation, optimization and monitoring functionality.

Page 9: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

BPM – BUSINESS PROCESS MANAGEMENT

BPM is an integrated collection of critical software technologies that enables the control and management of business processes

BPM software is a model-driven workflow environment that makes the model executable, while keeping the model as the central focus for future process changes.

Page 10: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

A workflow application is a software application which automates, at least to some degree, a process or processes. The processes are usually business-related, but it may be any process that requires a series of steps that can be automated via software.

Page 11: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

WORKFLOW EXAMPLES

In machine shops, particularly job shops and flow shops, the flow of a part through the various processing stations is a work flow.

Insurance claims processing is an example of an information-intensive, document-driven workflow.

Wikipedia editing is an example of a stochastic workflow.

Page 12: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

Some steps of the process may require human intervention, such as an approval or the development of custom text, but functions that can be automated should be handled by the application. Advanced applications allow users to introduce new components into the operation.

Page 13: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

WORKFLOW EXAMPLES The Getting Things Done system is a

model of personal workflow management for information workers.

In global software development, the concept of follow- the – sun describes a process of passing unfinished work across time zones.

Page 14: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

WORKFLOW EXAMPLES In Service- oriented architectures an

application can be represented through an executable workflow, where different, possibly geographically distributed, service components interact to provide the corresponding functionality, under the control of a Workflow Management System.

Page 15: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

THE BENEFIT OF WORKFLOW SOFTWARE

Workflow software provides organizations with the technology functionality, enabling them to innovate and create new working environments that not only define the business process and tasks, but execute and run them through common day-today applications like Outlook and Word.

Page 16: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

THE BENEFIT OF WORKFLOW SOFTWARE

Improves productivity – Automated workflows reduce the time spent on manual tasks.

Visibility – Workflows enable managers to see what is happening with business critical processes at every point along the process, and at any moment in time.

  Faster business reaction to change – Workflow software

enables organizations to react quickly and smoothly to market changes through process modifications.

 

Page 17: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

THE BENEFIT OF WORKFLOW SOFTWARE

Improves Accountability – Monitoring time to execute business functions creates productivity measurement and continual process improvement

Page 18: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

THREE TYPES OF WORKFLOW  Sequential Workflow (Typically Flow Chart based,

progresses from one stage to next and does not step back)

State Machine Workflow (Progress from ‘State’ to ‘State’, these workflows are more complex and return to a previous point if required)

  Rules-driven Workflow (Implemented based on a

sequential workflow. The rules dictate the progress of the workflow)

Page 19: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

ETL WORKFLOWS An ETL workflow is a design blueprint

for the ETL process. The designer constructs a workflow of activities (or operations), usually in the form of a graph, to specify the

order of cleansing and transformation operations that should be applied to the source data, before being loaded to the data warehouse.

Page 20: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

Data warehousing is a technology that enables decision-making and data analysis in

large organizations

Page 21: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

To populate a data warehouse with up-to-date records extracted from operational sources, special tools are employed, called Extraction – Transform – Load (ETL) tools, which organize the steps of the whole process as a workflow

Page 22: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

ETL WORKFLOW TASKS1. The identification of relevant

information at the source side. 2. The extraction of this information. 3. The transportation of this information

to the Data Staging Area (DSA), where most of the transformation usually take place.

Page 23: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

4. The transformation of the information coming from multiple sources into a common format.

5. The cleansing of the resulting data set, on

the basis of database and business rules.

6. the propagation and loading of the data to the data warehouse and the

refreshment of data marts.

Page 24: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

ETL tools constitute a multi-million dollar market Due to their importance and complexity.

The traditional database vendors provide ETL solutions along with the DBMS’s: IBM with

InfoSphere Information Server , Microsoft with SQL Server Integration Services (SSIS) , and Oracle with Oracle Warehouse Builder .

There also exist independent vendors that cover a large part of the market.

Page 25: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

Nevertheless, an in-house development of the ETL workflow is preferred in many data warehouse projects, due to the significant cost of purchasing and maintaining an ETL tool.

The spread of existing solutions comes with a major drawback. Each one of them follows a different design approach, offers a different set of transformations, and provides a different internal language to represent essentially similar functions.

Page 26: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

Although Extract-Transform-Load (ETL) tools are available in the market for more than a decade, only in the last few years have researchers and practitioners started to

realize the importance that the integration process has in the success of a data warehouse project.

Page 27: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

COMPLEX EVENT PROCESSING Event processing is a method of tracking and

analyzing (processing) streams of data about things that happen (events), and deriving a conclusion from them.

Complex event processing, or CEP, is event processing that combines data from multiple sources to infer events or patterns that suggest more complicated circumstances. The goal of complex event processing is to identify meaningful events (such as opportunities or threats) and respond to them as quickly as possible.

Page 28: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

CEP TYPES A computation-oriented CEP solution is focused

on executing on-line algorithms as a response to event data entering the system. A simple example is to continuously calculate an average based in data on the inbound events.

Detection-oriented CEP is focused on detecting combinations of events called events patterns or situations. A simple example of detecting a situation is to look for a specific sequence of events.

Page 29: Workflow ETL workflow Complex  event processing(CEP ) Mona Alnahari

SUMMARY Workflows automate the flow of employee tasks and activities,

reducing the time the process took to complete as well as potential errors caused by human interaction.

Workflows makes processes more efficient, compliant, agile, and visible by ensuring that every process step is explicitly defined, monitored over time, and optimized for maximum productivity. Given optimal, up-to-the-minute process data, managers and employees can take quicker action and make smarter decisions.

Workflows empower business users and IT to work together to rapidly modify systems and processes to reflect changes in the business.