Quick Start Guide

Kapo

w M

ashu

p Se

rver

6.4

Quick Start Guide

Quick Start Guide

CONTENTS iii

Contents

WELCOME TO KAPOW MASHUP SERVER .......................................................... 1 Installation ........................................................................................1 About the Guide .................................................................................1 Contact Information ............................................................................1

Support ..........................................................................................1 Kapow Developer Connection.............................................................1

OVERVIEW OF KAPOW MASHUP SERVER ......................................................... 2 Web Integration .................................................................................2 Enter Kapow Mashup Server.................................................................2 How It Works .....................................................................................3

Robots ...........................................................................................4 Objects ..........................................................................................4

Summary ..........................................................................................4 KAPOW MASHUP SERVER APPLICATIONS ........................................................ 5

Creating Robots..................................................................................5 Running Robots ..................................................................................6 Managing Robots ................................................................................7

DOCUMENTATION OVERVIEW....................................................................... 8 User Guides .......................................................................................8 Programmer Guides ............................................................................8 Administrator Guides...........................................................................9 Recommended Reading .......................................................................9

Kapow Mashup Server Basics.............................................................9 Data Collection .............................................................................. 10 Clipping........................................................................................ 10 Integration into Web Applications or Portals ....................................... 10 Integration into other Java Applications............................................. 11 Integration with Microsoft .NET Application ........................................ 11 Java Web Services for BEA WebLogic 8.1........................................... 12 Web Services for Microsoft .NET....................................................... 12

QUICK START TUTORIAL 1: DATA COLLECTION .............................................. 13 Overview......................................................................................... 13

Notes ........................................................................................... 13 Before You Start ............................................................................ 13

Defining the Tip Object ...................................................................... 13 Step 1 – Start up ModelMaker.......................................................... 15 Step 2 – Create a New Domain Model ............................................... 16

iv QUICK START GUIDE

Step 3 – Create the Tip Object ......................................................... 17 Step 4 – Define Attributes for the Tip Object ...................................... 17 Step 5 – Validate the Domain Model ................................................. 20 Step 6 – Save the Model ................................................................. 20

Creating the Robot............................................................................ 21 Step 7 – Start RoboMaker ............................................................... 21 Step 8 – Create a new Robot ........................................................... 22 Step 9 – Choose Robot Type............................................................ 22 Step 10 – Enter the URL to Start From.............................................. 23 Step 11 – Choose Output Object ...................................................... 24 Step 12 – Loop Through Table Rows ................................................. 26 Step 13 – Extract Tip Number.......................................................... 27 Step 14 – Extract Tip URL ............................................................... 29 Step 15 – Load the Page with the Tip................................................ 30 Step 16 – Extract Tip Title............................................................... 31 Step 17 – Extract Description .......................................................... 32 Step 18 – Return Object ................................................................. 33 Step 19 – Save the Robot ............................................................... 33

Running the Robot ............................................................................ 33 Step 20 – Run in RoboMaker’s Debugger ........................................... 34 Step 21 – Run Using RoboRunner..................................................... 34 Step 22 – Start RoboServer............................................................. 35 Step 23 – Start RoboClient .............................................................. 36 Step 24 – Run the Robot from RoboClient.......................................... 36 Step 25 – Run the Robot from the RoboSuite Java API ........................ 37 Step 26 – Run the Robot from the RoboSuite .NET API ........................ 38

Summary ........................................................................................ 39 QUICK START TUTORIAL 2: CLIPPING.......................................................... 40

Overview......................................................................................... 40 Creating the Clipping Robot................................................................ 40

Step 1 – Start RoboMaker ............................................................... 40 Step 2 – Create a New Robot........................................................... 41 Step 3 – Choose Robot Type............................................................ 42 Step 4 – Enter the URL to Start From................................................ 43 Step 5 – Configure the URLs to Clip .................................................. 44 Step 6 – Add a New Clip Branch ....................................................... 46 Step 7 – Enter Name of New Clip Branch ........................................... 47 Step 8 – Hide a Button ................................................................... 48 Step 9 – Select the Logo ................................................................. 49 Step 10 – Select the Search Form .................................................... 50 Step 11 – Show the Clip.................................................................. 51 Step 12 – Make a Search ................................................................ 52

CONTENTS v

Step 13 – Ensure the Correct Clip Branch is Used ............................... 53 Step 14 – Save the Robot ............................................................... 55

Installing Tomcat.............................................................................. 56 Step 16 – Download Apache Tomcat ................................................. 56 Step 17 – Install Apache Tomcat ...................................................... 56 Step 18 – Install the RoboSuite Tag Library Demo Web Application on Apache Tomcat.............................................................................. 58 Step 19 – Test that Tomcat is Running Correctly ................................ 58

Deploying the Clipping Robot.............................................................. 59 Step 20 – Start the Java Servlet Clip Wizard ...................................... 59 Step 21 – Choose a Name for the Deployment Descriptor File............... 60 Step 22 – Finish the Wizard............................................................. 60

Running the Clipping Robot ................................................................ 61 Step 23 – Restart Tomcat................................................................ 61 Step 24 – Start RoboServer............................................................. 61 Step 25 – Try out Clipping............................................................... 61

Summary ........................................................................................ 62 Next Steps....................................................................................... 62

INDEX.................................................................................................. 63

WELCOME TO KAPOW MASHUP SERVER 1

Welcome to Kapow Mashup Server Welcome to the Kapow Mashup Server product suite. In this guide you will get a brief overview of the entire product and its manuals.

Installation Read the Installation Guide for a description of how to install the product. This is not covered in this overview.

About the Guide This guide is intended as a quick and not too technical introduction to Kapow Mashup Server. It contains a short introduction to the Kapow Mashup Server product, a description of the applications in the suite, and an overview of the available manuals. We also present a list of recommended reading, necessary to get you started with the product. Finally, the guide contains two simple tutorials, one describing data collection and one describing web clipping. The tutorials will step by step guide you through the two simple cases, introducing you to all the applications of Kapow Mashup Server and how they are interconnected.

Contact Information

Support Support information is available at this page:

http://support.kapowtech.com

Kapow Developer Connection The Kapow Developer Connection provides access to many resources for developers. It contains the latest downloads, as well as a problem tracking system, and a discussion forum. It is located at this URL:

http://kdc.kapowtech.com/

http://support.kapowtech.com/

http://kdc.kapowtech.com/

2 QUICK START GUIDE

Overview of Kapow Mashup Server Kapow Mashup Server is a platform for Web Integration. In this chapter, we will describe the meaning of that term, and the role of Kapow Mashup Server in bringing about this integration.

Web Integration Web Integration is the process of leveraging the informational and functional resources of the Web in an application of your choice.

There are a number of compelling reasons why such an approach would be interesting. The first and most obvious is the sheer quantity of resources. Second, the Web integration approach allows you to take advantage of existing enterprise information systems (EIS) that provide Web interfaces.

The challenges in implementing such a solution from scratch should also be obvious. The resources are there, but they are notoriously heterogeneous, and are often dispersed among several pages. Another complication is the language in which the resources are available – Hypertext Markup Language (HTML). The very thing that has contributed to its success (its loose and forgiving syntax) makes it very difficult to impose structure on HTML. Add to this the complexity of client-side JavaScript, user validation and session handling, and the solution becomes non-trivial.

Enter Kapow Mashup Server Kapow Mashup Server helps transform the resources of the Web into well-defined nuggets of information and functionality. In effect, Kapow Mashup Server transforms a web site into one or more services available to a client application. Most importantly, it does this in a non-intrusive way. Figure 1 illustrates how Kapow Mashup Server accomplishes Web Integration.

YourApplication RoboSuite

Website

Website

Website

Figure 1: Web Integration with Kapow Mashup Server

OVERVIEW OF KAPOW MASHUP SERVER 3

How It Works This next figure goes into more detail on the architecture of the Kapow Mashup Server based solution. The central application in executing the integration is called RoboServer. RoboServer receives a request from your application and executes what is called a robot. We will give a more detailed description of what a robot is shortly. For now it is sufficient to know that to integrate with a web site you write a robot that when executed interacts with that web site. This is illustrated in Figure 2 below. Figure 2 also illustrates how different applications interact with RoboServer. A web application may use the RoboSuite JSP Tag Library to call robots from JSP pages. This tag library uses the RoboSuite Java API to contact RoboServer. A Java or .NET application may use the RoboSuite Java API or RoboSuite .NET API to contact RoboServer, and other applications, e.g. a C++ program, may contact RoboServer directly over a socket connection.

RoboSuite JSP Tag Library

RoboSuite Java/.NET APIs

RoboServer

Website Website Website Website

Robot Robot Robot Robot

JSP WebApplication

OtherApplications

Java/.NETApplication

Figure 2: How Kapow Mashup Server Works

4 QUICK START GUIDE

Robots Robots are the fundamental concept in Kapow Mashup Server. A robot can be thought of as a small program that performs some sort of interaction or data-extraction from the Web.

The programming analogy is not wholly accurate, however. Robots can be written and maintained by non-programmers using a set of intuitive visual builder tools. By navigating the target web pages in a built-in browser, the user defines the actions that the robot will take. In this regard, robots have much in common with macros, and can also be described as macros for the Web.

Objects Robots are invoked at the request of a client application or as part of a batch job. They can be passed a set of objects as input, and when running they return a set of objects back to the user.

The robot developer decides what kind of objects a robot takes as input, and which objects it may return. The objects are modeled to fit real-world objects, like cars, houses, stock quotes, and airplane reservations. After the modeling of objects is done, the robot developer can create the robots that actually perform the tasks, e.g. searching for products (e.g. for a price comparison site), booking airplane tickets, retrieving stock quotes or accessing customer information in a CRM system.

Summary In this chapter, we explained the nature of Web Integration, and how Kapow Mashup Server fits as the solution for this.

KAPOW MASHUP SERVER APPLICATIONS 5

Kapow Mashup Server Applications Kapow Mashup Server is made up of several applications. Figure 3 shows the relationship between the different applications, and how they are used to write, run and manage a collection of robots.

Robot

Robo-Maker

creates

ObjectModel

uses

LogDatabase

Log Message

Robo-Manager

works on

Robo-Server

(or Robo-Runner)

YourApplication

Model-Maker

executes

creates

interactswith

generates

Websiteinteracts with

Website

Website

Website

RoboSuiteControlCenter

monitors

Figure 3: Kapow Mashup Server Architecture Overview

Creating Robots The most essential role in a solution based on Kapow Mashup Server is that of the robot developer, the person responsible for writing and maintaining robots. The main tools of a robot developer are ModelMaker and RoboMaker:

ModelMaker: ModelMaker is an application for modeling the objects used by a robot. These include the objects a robot extracts (and returns to the caller), and the objects it requires as input parameters.

RoboMaker: The central tool in Kapow Mashup Server, RoboMaker allows you to create and debug robots.

As a user, you should be familiar with these two central applications.

6 QUICK START GUIDE

Running Robots Kapow Mashup Server offers several ways to execute robots. They can be run as batch jobs using RoboRunner, or from a client application using RoboServer and an appropriate API or communication protocol.

RoboRunner: RoboRunner allows you to run robots from the command line. It provides almost as much functionality as RoboServer, except that it does not allow you to pass objects to the robot.

RoboServer: RoboServer allows remote clients to execute robots. It accepts requests and sends back replies to client applications.

Control Center: Control Center allows you to remotely monitor Robo-Servers and their robots.

RoboClient: RoboClient is a simple client to RoboServer that allows you to test various requests, and to get a feel for RoboServer.

The choice of using RoboRunner or RoboServer depends on what you wish to accomplish. For data collection running at certain intervals, you can use RoboRunner, together with a scheduling tool of your own choice. If you want to do web integration initiated by an end-user, you will need RoboServer.

To communicate with RoboServer from your client applications, you may use one of the APIs. Currently, we provide application programming interfaces for the Microsoft .NET platform, the Java™ platform and the Java Server Pages™ (JSP) technology.

RoboSuite Java API: Provides a complete Java API for accessing every capability of RoboServer.

RoboSuite JSP Tag Library: Built on top of the RoboSuite Java API, the tag library provides support for using robots in JSP pages.

RoboSuite .NET API: Provides a Microsoft .NET API for accessing every capability of RoboServer.

Additionally, for BEA customers, we provide a custom control for BEA Workshop 8.1:

RoboSuite Control for BEA WebLogic Workshop: This is an extensible control that allows you to integrate Kapow Mashup Server into the BEA platform through WebLogic Workshop.

KAPOW MASHUP SERVER APPLICATIONS 7

Managing Robots Managing a single robot or even a couple of robots is quite easy. Keeping track of hundreds or thousands is a challenge. RoboManager is a tool that helps you deal with this complexity.

RoboManager: A tool for organizing and maintaining a large number of robots. RoboManager allows you to monitor how each robot is performing in order to detect robot failures or other problems.

8 QUICK START GUIDE

Documentation Overview This section describes each manual in turn and finally provides a list of recommended reading.

User Guides The user’s guides describe the applications that make up the suite, one by one.

ModelMaker User's Guide: Gives an introduction to ModelMaker, as well as an overview of the object model.

RoboMaker User's Guide: A comprehensive guide to writing robots in RoboMaker, and debugging robots in the integrated debugging tool RoboDebugger.

RoboManager User's Guide: Describes how to organize and manage a large number of robots using RoboManager.

RoboServer User's Guide: Describes RoboServer and RoboClient. The guide also describes the Robot Query Language (RQL) that is used to communicate with RoboServer.

Control Center User's Guide: Describes how to monitor RoboServers and their robots using the Control Center.

RoboRunner User's Guide: Describes how to use RoboRunner for executing robots.

Code Generation Guide: Describes the various wizards for code generation, e.g. Java, C#, and clipping.

Programmer Guides The programming APIs come with a set of programming guides to complement the user’s guides.

Java Programmer's Guide: Explains how to use the RoboSuite Java API to execute robots on a RoboServer. The target audience for this book is Java programmers of varying skill levels.

JSP Tag Library Programmer's Guide: Gives a complete description of the RoboSuite JSP Tag Library that is used to run robots from JSP pages. This book requires basic knowledge of HTML, and some experience with writing JSP pages is recommended.

.NET Programmer's Guide: Describes how to use the .NET API for RoboSuite. The target audience for this book is .NET programmers of varying skill levels.

BEA Workshop Control Developer's Guide: Provides a description of the RoboSuite Control for BEA Workshop 8.1.

DOCUMENTATION OVERVIEW 9

In addition to these programming guides, both APIs come with a set of reference documentation, which is installed as part of the installation.

Administrator Guides Kapow Mashup Server comes with an installation guide that describes how to install and set up the application suite.

Installation Guide: Describes how to install Kapow Mashup Server, how to install database connections and how to install certificates. It also details the Settings application. It does not go into detail on how to set up Kapow Mashup Server for particular needs, such as data collection or web integration.

RQL Service User's Guide: An advanced guide to configuring RoboServer, and setting up the communication protocols it uses.

Recommended Reading In this section, we take a look at which parts of the manuals you need to read depending on what tasks you want to perform with Kapow Mashup Server. We present recommended reading for several areas of interest, mainly data collection and different integration approaches. Each category lists the manuals and chapters to read. The rest of the documentation can serve as reference material and as a source of additional information when needed.

It is recommended that you start reading the material listed under Kapow Mashup Server Basics below. This will give you a solid foundation for reading further and your first attempts at creating a robot. Depending on the role Kapow Mashup Server will play in your solution, you should then go on to read one of the next three categories of information.

Kapow Mashup Server Basics This is the required reading for those developing robots. It describes the basic working of RoboMaker and ModelMaker, and is basic knowledge every user of the product should be familiar with.

• RoboMaker User's Guide:

Introduction

RoboMaker Basics

Getting Started

Tutorial Case 1: Essentials

Tutorial Case 2: Form Submission

• ModelMaker User's Guide

Introduction

ModelMaker Basics

10 QUICK START GUIDE

Tutorial: Creating a Domain Model

This should be enough to help you get started.

Data Collection If you plan to use robots for large-scale data collection, you should familiarize yourself with RoboRunner and RoboManager.

• RoboRunner User's Guide (The entire guide)

• RoboManager User's Guide (The entire guide)

Clipping If you want to use Kapow Mashup Server for clipping web sites and reusing them in another context, you should read the following:

• RoboMaker User's Guide:

How to Clip

• RoboServer User's Guide

Introduction

Starting RoboServer

Getting a Feel for RoboServer

Understanding RQL (optional)

• Code Generation Guide

(The relevant clipping portlet generation wizards.)

Integration into Web Applications or Portals This is what you should read if you wish to use Kapow Mashup Server for integrating functionality from existing Web resources into your own web application:


Introduction

Starting RoboServer



• JSP Tag Library Programmer's Guide

Introduction

Overview

After reading these chapters, you should experiment with the Demo Web Application (located as part of the installation in API/robosuite-jsp-taglib/examples).

DOCUMENTATION OVERVIEW 11

You may also wish to look through the Clipping documentation (the previous category), to see another approach for providing portlet content.

Integration into other Java Applications The RoboSuite Java API allows you to use robots from any Java application. Needless to say, you should have some Java programming experience before you attempt to use the supplied API. Before programming directly against the Java API you should try to use the Java Code Generation Wizard and see whether or not it is capable of fulfilling your needs.


Introduction

Starting RoboServer




Java Code Generation Wizard

• Java Programmer's Guide

Introduction

Core Functionality

After this, you should try out the examples that are provided as part of the installation (in the API/robosuite-java-api/examples folder).

Integration with Microsoft .NET Application The RoboSuite .NET API allows you to use robots from any .NET application. Its main target audience is Microsoft .NET developers. Before programming directly against the .NET API you should try to use the C# Code Generation Wizard and see whether or not it is capable of fulfilling your needs.


Introduction

Starting RoboServer




C# Code Generation Wizard

• RoboSuite .NET Programmer's Guide

Introduction

Core Functionality

After this, you should try out the examples that are provided as part of the installation (in the API/robosuite-dotnet-api/examples folder).


Java Web Services for BEA WebLogic 8.1 This is what you should read if you wish to use Kapow Mashup Server for creating web services for BEA WebLogic 8.1:


Introduction

Starting RoboServer




Introduction

BEA Java Web Service Wizard

Web Services for Microsoft .NET If you wish to use Kapow Mashup Server for creating web services running under Microsoft .NET you should read the following:


Introduction

Starting RoboServer




.NET Web Service Wizard

QUICK START TUTORIAL 1: DATA COLLECTION 13

Quick Start Tutorial 1: Data Collection This chapter contains a short tour of Kapow Mashup Server. It will guide you through the creation and execution of your first simple robot. It should take about 30 minutes to complete.

The goal of the tour is to take you through the core Kapow Mashup Server applications, and show how they function together to help you create and run robots.

Overview We will be creating a simple robot that goes to the Kapow website and fetches a list of “RoboSuite tips of the week”. This demonstrates simple data collection.

The first thing we do is modeling the type of object that will be returned from the robot. We will call this object Tip, and it will contain attributes that describe the tip. Then we go on to create the robot. It will be a small to medium sized robot that reads the various parts of the tip from the web page.

After the robot is finished, we run the robot. First we run it in RoboMaker’s debugger, then with RoboRunner, then by calling RoboServer from RoboClient, and finally programmatically using the Java API to call RoboServer.

The tutorial is divided into a number of small steps, each describing a single action to take. Most steps start out with a short description of what we will do, and two subsections: Action, and Effect.

Notes During the tour, we make references to the installation folder. By this we mean the directory that Kapow Mashup Server is installed in. On a Windows system, this would be C:\Program Files\Kapow Mashup Server 6.4 if you have chosen the default value at installation. Since the precise location of the folder will vary among users, depending on where you selected to install, we refer to it instead as the installation folder.

Before You Start Before you can go through this guide, you need to have Kapow Mashup Server installed on your system. If you have not installed Kapow Mashup Server already, now would be a good time. You can find help on how to do this in the Installation Guide.

Defining the Tip Object The first step in defining the Tip object is to take a first look at the target site and consider what the robot should do.


In your browser, go to the page http://www.kapowtech.com/tutorial/quickstart. This site is constructed with the sole purpose of acting as an example site for this tutorial and is therefore kept extremely simple. The main page is a table of all the tips, containing the numbers and titles of these. The titles link to a full description of the tips. Take a few minutes to familiarize yourself with the structure of the site.

We want the robot to send back information describing the tip of the day.

We will choose the name Tip to describe it, and it will have four attributes:

title: A short string describing what the tip is about.

number: What number tip it is. All tips are numbered incrementally, starting with number 1.

url: The location of a page describing the tip. This can be used if we want to present the tip in our own web page and link to the original location of the tip.

description: The actual description of the tip, often just a short paragraph of text.

About Domain Models

In a real application, we would often need more than a single object type to describe the data. We would perhaps need an object type to describe information sent to the robot, as well as several different object types that could be returned by the robots.

These related objects make up a domain model, modeling the problem domain. Even if you only have a single object (as in this case), it still constitutes a domain model.

http://www.kapowtech.com/tutorial/quickstart


Step 1 – Start up ModelMaker Now that we have designed the object model to use, the time has come to create it in ModelMaker.

Action

Start ModelMaker, e.g. by selecting it from the Start Menu (Windows systems).

Effect

ModelMaker will start and show the welcome screen shown in Figure 4.

Figure 4: ModelMaker Welcome Screen


Step 2 – Create a New Domain Model Now create a new domain model.

Action

In the welcome screen, select Create a new domain model… and click OK.

Effect

The ModelMaker window will appear as shown in Figure 5.

Object List

Attribute Table

Add, Remove, and Configure Attributes

Add, Remove, and Preview Objects

Figure 5: ModelMaker Main Window


Step 3 – Create the Tip Object Now we need to add an object to our domain model. To do this, click on the

button in the lower left corner of the window. Select Output Object from the list and click OK.

Figure 6: Creating a New Object

The new object now appears in the list of objects to the left. Note that it is currently unnamed. Set the name of the object to Tip by editing the Name field.

Step 4 – Define Attributes for the Tip Object Below the name of the object is an empty list of attributes in the object. Now we will add the four attributes to hold information about the tip.


Press the button just below the list of attributes.

Figure 7: Adding an Attribute

Write title as the Name of this attribute, and select Short Text as its Type. When you have done this the dialog should as shown in Figure 7. Then press OK. You will see the new attribute appearing in the list of attributes.


Now add the three other attributes. They should be declared so the Tip object type contains the four attributes shown in Figure 8.

Attribute Name Type

title Short Text

number Integer

url Short Text

description Long Text

Figure 8: Attributes of the Tip Object

When you have added all four attributes, your ModelMaker window should look like this:

Figure 9: ModelMaker after Addition of Attributes


Step 5 – Validate the Domain Model Now that the model is complete, we should make sure that the Domain Model is valid. Press the button in the toolbar. The following message box should appear:

Figure 10: Validating the Domain Model

If you do not get this dialog, but instead an error dialog, you need to go back and find out where things went wrong before you continue the tutorial.

Step 6 – Save the Model Now store the model to disk. Press the button to save the model. Choose the name Tip for the domain model.


Creating the Robot Now the time has come to write the robot itself, using the RoboMaker application.

Step 7 – Start RoboMaker Start RoboMaker, e.g. by selecting it from the Start Menu (Windows systems). When RoboMaker has started, a welcome screen like the one shown in Figure 11 is opened.

Figure 11: RoboMaker Welcome Screen


Step 8 – Create a new Robot We select Create a new robot… and click OK. This will open the New Robot Wizard, which will assist us in writing the robot.

Step 9 – Choose Robot Type In the first wizard step, we select Data collection robot and click Next.

Figure 12: Choosing the Robot Type


Step 10 – Enter the URL to Start From In the new wizard step, enter the URL to start from:

www.kapowtech.com/tutorial/quickstart/index.html

and click Next.

Figure 13: Entering the URL to Start From

http://www.kapowtech.com/tutorial/quickstart/index.html


Step 11 – Choose Output Object In the new wizard step, we choose the output objects. The list of objects should look familiar to you by now. It uses the same basic design as all other lists in Kapow Mashup Server. You can add, remove, and change the order. Click to add a new object, and select an object type from the drop-down list, which says “Select an Object”. Choose the one named Tip, as shown in Figure 14.

Figure 14: Choosing Output Object

Click Finish when you are done. The RoboMaker Main Window should now look similar to the one shown in Figure 15.


State View

Robot View

Step View

Objects View

Figure 15: The Robot

Notice that the robot already contains two steps. The first step loads the page, and the second step, which is the current step, has not yet been configured.


Step 12 – Loop Through Table Rows We will now loop through all the table rows in the table in the middle of the screen.

Action

The loaded page contains a table. We want to loop through the rows of this, that is, look at them one by one and thereby extract each tip in the table. To do this, place the mouse at some cell in the table, e.g. Number, and right click. Choose “Loops” | “For Each Table Row” | “Exclude First Row”. The first row should be excluded because it contains the headers of the columns.

Figure 16: Looping Through Table Rows

Notice the number (1) in the For Each Tag Path step (see Figure 17). This indicates that we are currently looking at the first row that the robot will iterate over.


Step 13 – Extract Tip Number The next thing we want to do is to insert a step to extract the number of each tip.

Action

In the Browser View, right-click on the tip number, and select the menu item “Extraction” | “Extract Number” | “Tip.number”.

Figure 17: Extracting Tip Number


A dialog for extracting a number now appears. Make sure the Convert to Integer checkbox is checked.

Figure 18: Extract Number Configuration

Click OK to close the dialog. Have a look at the Objects View (in the lower right corner). In the “Output Objects” tab, you can see the output objects and the values of their attributes. You will now see that the number attribute of the Tip object, as expected, has the value 4.


Step 14 – Extract Tip URL Similarly, we also want to extract the URL of the tip.

Action

In the Browser View, right-click on the title of the tip and select the menu item “Extraction” | “Extract URL” | “Tip.url”.

Figure 19: Extracting Tip URL


Step 15 – Load the Page with the Tip To get further information about the current tip, we need to load the page containing the tip itself. So we insert a step to do this.

Action

In the Browser View, right-click on the title of the tip and select the menu item “Click”.

Figure 20: Loading the Page with the Tip


Step 16 – Extract Tip Title Now, we want to extract the title and the description of the tip. We start by extracting the title.

Action

In the Browser View, right-click on the title of the tip and select the menu item “Extraction” | “Extract Text” | “Tip.title”.

Figure 21: Extracting Tip Title


Step 17 – Extract Description Finally, we will extract the description of the tip.

Action

In the Browser View, right-click on the tip text in the Browser View, and select the menu item “Extraction” | “Extract Text” | “Tip.description”.

Figure 22: Extracting Tip Description


Step 18 – Return Object The last thing we need to do to complete the robot is to tell it that the object we have extracted is complete, that is, we have extracted all its attributes. This is done by inserting a Return Object step.:

Action

From the Action tab in the Step View in the right part of the RoboMaker window, choose Return Object as the step action. Choose the Tip object as the object to return.

Figure 23: Returning Tip Object

Step 19 – Save the Robot Now the robot is complete. Save it by pressing the button in the toolbar. Name it TipOfTheWeek.robot.

It is important that you save the robot in the default location suggested by RoboMaker, as we will try to access it from other applications later.

Running the Robot In the following steps, we will show various examples of how to run our TipOfTheWeek robot.


Step 20 – Run in RoboMaker’s Debugger Open the RoboDebugger in RoboMaker by clicking on the button on the toolbar. When the debugger opens, run the robot by pressing . You will see a list of returned objects appearing as shown in the figure below. These are the objects that your robot has extracted. You may click on an object in the list and see its content in the panel below the list.

Figure 24: Running in RoboDebugger

Step 21 – Run Using RoboRunner Now, let us run the robot using RoboRunner. Open a command prompt and go to a folder where you would like the output files from RoboRunner to be placed (this can be any folder, and these sample output files can safely be deleted afterwards).

In the command prompt, enter the following:

RoboRunner -SimpleRobotInfoEnvironment -FileStorageEnvironment fileName=objects.txt -FileMessageEnvironment fileName=messages.txt -file "C:/Program Files/Kapow Mashup Server 6.4/Project/Library/TipOfTheWeek.robot"


This runs the robot and places the extracted objects in the file objects.txt and messages in the file messages.txt. Open these files, e.g. with a text editor, to see the contents of the objects that were extracted and the messages given.

Step 22 – Start RoboServer Now, let us run our robot using RoboServer. To do this, you must first start RoboServer. On Windows, RoboServer can be started from the Start menu. On Linux and UNIX variants, RoboServer is started with the command:

RoboServer –service socket:50000

This will start RoboServer listening on a socket at port 50000.


Step 23 – Start RoboClient First, let us try to communicate with RoboServer using RoboClient, so start RoboClient.

Step 24 – Run the Robot from RoboClient We first select a protocol for communication with RoboServer. In the “RQL Protocol” property, select Socket, and in the “RQL Request” property, select the Execute Request. Then set the Robot URL to library:/TipOfTheWeek.robot and press the button to send the request. The responses will appear in the list to the right as shown in Figure 25.

Figure 25: Running from RoboClient


Figure 26 shows the received responses as a list. As you can see the robot sends back four extracted objects containing the tips.

Figure 26: RQL Responses

Step 25 – Run the Robot from the RoboSuite Java API This part of the guide shows how to run the robot using the RoboSuite Java API. It assumes that you have a Java SDK installed. Feel free to skip this step if you are not planning to write Java applications accessing RoboServer.


The code example is written for a Windows platform. If you are running Linux or a UNIX variant, you need to replace all semi-colons with colons in the text below.

Open a command prompt in the installation folder.

Then go to the subfolder API/robosuite-java-api/examples by typing the following two commands:

cd API cd robosuite-java-api cd examples

Compile the RunRobot.java example as follows (on UNIX you will have to replace the semi-colons with colons in the classpath below):

javac -classpath ../lib/robosuite-api.jar;../lib/crimson.jar RunRobot.java

Now run the example (on UNIX you will have to replace the semi-colons with colons in the classpath below):

java -cp .;../lib/robosuite-api.jar;../lib/crimson.jar RunRobot library:/TipOfTheWeek.robot

This will run the robot and output the results to the screen.

Take a look at the Java code in RunRobot.java to learn how the API was used to run the robot.

Step 26 – Run the Robot from the RoboSuite .NET API This part of the guide shows how to run the robot using the RoboSuite .NET API. It assumes that you have the Microsoft .NET Framework SDK version 2.0 installed on your computer.1

Open a command prompt in the API/robosuite-dotnet-api/examples directory of your installation directory. Run the build.bat file to compile the C# source files in this directory. On successful compilation you will get a number of .exe files ready to run.

Now type: RunRobot.exe library:/TipOfTheWeek.robot

This will run the robot and output the result to the screen.

If you look at the code in RunRobot.cs you can see how the API is used to run the robot.

1 The Microsoft .NET Framework SDK version 2.0 can be downloaded from Microsoft’s web site.


Summary This tutorial described how to create a simple model and robot, and how to run the robot in a variety of ways. Through RoboClient you have gotten your first view of RQL requests and replies, and perhaps you have run your first robot through the RoboSuite Java API or .NET API.


Quick Start Tutorial 2: Clipping This chapter contains a short tutorial on clipping. It will guide you through the creation and execution of a simple clipping robot for use as a portlet in a portal. It should take about 30 minutes to complete.

The goal of the tour is to introduce you to the basic clipping concepts and show you an example of how to deploy a clipping robot.

Overview We will be creating a simple clipping robot that is used for transferring the search functionality of Google into a portlet.

Note that Google may change over time, so you may not experience exactly the same results as shown in the figures.

Clipping is enabled in collaboration between RoboServer and a web server. In order to get full benefit from this tutorial, you will therefore need to have a web server available that is supported by Kapow Mashup Server.

This tutorial includes descriptions of how to install and set up a web server and how to deploy the clipping robot.

Creating the Clipping Robot We start by creating the clipping robot, using the RoboMaker application.

Step 1 – Start RoboMaker If RoboMaker is not already running, start it now, e.g. by selecting it from the Start Menu (on Windows systems).

QUICK START TUTORIAL 2: CLIPPING 41

Step 2 – Create a New Robot If you have just started RoboMaker, select Create a new robot… in the welcome screen and click OK. Otherwise, press the icon to open the New Robot Wizard.

Figure 27: Creating a New Robot


Step 3 – Choose Robot Type In the first wizard step, select “Clipping robot” and click Next.

Figure 28: Choosing Robot Type


Step 4 – Enter the URL to Start From In the next wizard step, enter the URL to start from:

www.google.com

and then click Finish to create the robot.

Figure 29: Entering the URL to Start From

http://www.google.com/


The RoboMaker Main Window should now look similar to the one shown in Figure 30.

Figure 30: The Clipping Robot

A simple robot has been created, consisting of four steps. The first step is called Begin Clip, and the remaining steps form the default clip branch of the robot. The default clip branch is the branch that will be used for clipping all pages that you do not define specific clip branches for.

Also notice that the Portlet View has opened. The Portlet View shows your clips as they will appear in the portlet. You can interact with the clips in the same way as a user interacts with the portlet.

By default, the Portlet View opens and closes automatically as needed. However, you can also open and close the Portlet View yourself by clicking the

icon in the Main Window of RoboMaker.

As a rule of thumb, you should always use the Portlet View when navigating a clipping robot. Within the current clip branch, however, you can change steps as usual.

Step 5 – Configure the URLs to Clip The next thing we want to do is to configure which URLs the robot should clip. We want all pages on Google itself to be clipped and be shown inside the


portlet, but all links away from Google – such as when the user clicks a link in a search result – should open in a new window and not be clipped. This way, the users of your portal will stay in your portal as long as they navigate on Google, but any links away from Google will open in new windows separately from your portal.

To do this, click the button in the Portlet View to open the Robot Configuration Window.

Figure 31: Robot Configuration Window

In the “Clipping Restrictions” property, select “Include Only URLs in these Domains”. In the “Domains” property that appears, enter “*.google.*” to include all links on Google. Make sure the “Excluded Links” property is set to “Open in New Window”, so that other links will open in new windows.

Click OK to close the Robot Configuration Window.

The clipping robot that you have now is a fully functional clipping robot, and you can skip forward to step 12 if you want to try the robot as it is.


Alternatively, you can continue the tutorial and see how you can make modifications to the Google pages as part of the clipping.

Step 6 – Add a New Clip Branch We want to modify how the start page of Google looks in the portlet. To do this, first add a new clip branch specifically for this page. In the Portlet View, click the icon to open the Edit Clip Wizard:

Figure 32: Adding a New Clip Branch

Make sure the “Add new clipping rules” option is selected, and click Next to continue.


Step 7 – Enter Name of New Clip Branch On the next wizard page, enter a name for the new clip branch, or use the suggested name.

Figure 33: Entering the Name of the New Clip Branch

Click Finish to add the new clip branch. You will now be placed at the location in the robot where you can configure the new clip branch. The Portlet View will close automatically to allow you to use the Main Window for this.


Step 8 – Hide a Button The first modification is to hide the search button called “I’m Feeling Lucky”. In the Browser View, right-click on the “I’m Feeling Lucky” button and choose “Hide Tag”, as shown in Figure 34.

Figure 34: Hiding a Button


Step 9 – Select the Logo The next modification is to select the parts of the page to clip. This is done by configuring the Clip step, so ensure that you are on the Clip step – click on it if not.

In the Browser View, click on the Google logo, so that a green box appears around it. If the green box covers only part of the logo, click the icon in the toolbar above the Browser View to expand the box until it covers the logo. If the box becomes too big, click the icon to reduce it. When the green box covers the logo, right-click in the box and select “Use only this Tag”, as shown in Figure 35.

Figure 35: Selecting the Logo

This will select the Google logo as one of the tags to clip from the page.


Step 10 – Select the Search Form Then, in the Browser View, click the text input field (the search field). In the toolbar above the Browser view, click the icon to expand the green box until it covers the search form and the links above it (see Figure 36 below). If the box becomes too big, click the icon to reduce it. When the box covers the form and the links, right-click in the box and select “Use this Tag” (not “Use only this Tag”), as shown below:

Figure 36: Selecting the Search Form

Now, you have selected the Google logo and the search form as the two parts of the page to clip, leaving out the rest of the page.


Step 11 – Show the Clip Click the icon to the right in the toolbar at the top of the RoboMaker Main Window. This will open the Portlet View and show the clip that you have defined:

Figure 37: The Modified Clip


Step 12 – Make a Search Now, try out your portlet in the Portlet View by entering some text (e.g. “Kapow Technologies”) in the search field and clicking the “Google Search” button or hitting Enter.

Figure 38: Entering a Search Text


The Portlet View shows the search results as they will appear in your portlet:

Figure 39: The Search Results

Try performing a new search by clicking the icon to restart the clipping, entering another text, and clicking the search button again. Notice how the robot is being executed in the Main Window of RoboMaker as you are navigating in the Portlet View. For the first page, the clip branch that you have added will be executed. For the search results, the default clip branch will be executed.

Step 13 – Ensure the Correct Clip Branch is Used Make sure you are on the search results page in the Portlet View, by making a new search if necessary. Click on the small Google logo in the upper left corner to go back to the front page. Check whether the front page is clipped as you wanted it to be, with only the logo and the search form included.

If not, this happens because the link back to the front page uses a different URL than the standard one (www.google.com) to get to the front page. To fix


this, click the icon in the toolbar of the Portlet View. In the Edit Clip Wizard, select “Clip using other rules”:

Figure 40: Clipping Using another Clip Branch


Click the Next button. On the next page, select the clip branch that you want to use for this page – this is the clip branch that you added, and which is already selected by default:

Figure 41: Selecting the Clip Branch to Use

Click Finish. The Edit Clip Wizard will now configure the robot to ensure that the selected clip branch will be used for the page. As you can see in the Portlet View, the front page is now clipped as desired.

If you want to, verify this by making another search in the Portlet View, and clicking on the Google logo again on the results page to get back to the front page.

Step 14 – Save the Robot The robot is now complete, so save it by pressing the button in the toolbar. Name it GoogleClip.robot. Do not close RoboMaker yet.

The rest of this tutorial will show you how to deploy and run your clipping robot.


Installing Tomcat Next, we install the Apache Tomcat web server. You can skip this section if you already have Apache Tomcat installed. If you do not want to deploy your robot, or want to deploy it on another web server, go to the “Summary” section at the end of this chapter.

Step 16 – Download Apache Tomcat The Kapow Developer Connection contains a link to the binary distribution of the Apache Jakarta Tomcat web server. Windows users should use this link to get the latest Tomcat installer:

http://kdc.kapowtech.com/tomcat.html

Users of other systems can go to the Apache Jakarta Projects homepage for a list of downloads.

http://jakarta.apache.org

Step 17 – Install Apache Tomcat Choose to install the program in C:\apache-tomcat folder on a Windows system, or /apache-tomcat folder on Unix-like systems. Press Next after choosing a folder.

Figure 42: Selecting the Tomcat Installation Folder

http://kdc.kapowtech.com/tomcat.html

http://jakarta.apache.org/


Then choose an administrator password for Tomcat. This isn't really important for this tutorial, so just choose an arbitrary password, and click Next.

Figure 43: Choosing Administrator Password


After choosing the password, press Next, followed by Install, and Tomcat will complete the installation. Press Finish to finish the installer.

Figure 44: Tomcat Installation Completed

Step 18 – Install the RoboSuite Tag Library Demo Web Application on Apache Tomcat To create the clipping portlet, we need a web application to place it in. The RoboSuite JSP Tag Library Demo Web Application is an excellent candidate for this. Copy the file API\robosuite-jsp-taglib\examples\rstl-demo.war located in the installation folder to the C:\apache-tomcat\webapps folder.

When you start Tomcat in the next step, the application will be deployed automatically.

Step 19 – Test that Tomcat is Running Correctly Start Apache Tomcat. The installation provides you with a shortcut to start Tomcat. On Windows systems double click on the quick launch icon and press Start to start a Tomcat server. On non-Windows systems, you should use the startup.sh script in the bin folder where you installed Tomcat.

Go to the URL http://localhost:8080/rstl-demo/. This should show you the introduction page to the RoboSuite Tag Library Demo Web Application.

http://localhost:8080/rstl-demo/index.jsp


Deploying the Clipping Robot Now we can deploy the clipping robot, so go back to RoboMaker (if RoboMaker is no longer running, restart it and load the GoogleClip.robot).

Step 20 – Start the Java Servlet Clip Wizard In the Clip menu of the RoboMaker main window, choose Standard Java | Java Servlet Clip Wizard. This will open the following window.

Figure 45: JSP Clipping Wizard

Browse to the root folder of the web application. In our case, the path is C:\apache-tomcat\webapps\rstl-demo. Click the Next button.


Step 21 – Choose a Name for the Deployment Descriptor File In the next screen, you choose the name for the Deployment Descriptor file (including its path from the root of the web application) that the wizard will generate. The wizard will install a servlet and the URL pattern *.clip is mapped to this, so a HTTP request that hits a Deployment Descriptor file will be redirected to this servlet which will send back the clip. Just use the default file name suggested by the wizard and click the Next button.

Figure 46: Choose a Name for the JSP File

Step 22 – Finish the Wizard The three next pages contain advanced settings related to deployment, and other further configuration properties, which we do not need right now. The only thing you need to know is that the protocol that the clip will use when contacting the RoboServer is a socket-based protocol at port 50000 on the local machine. Just click Finish to make the wizard perform the requested operations.


When the wizard has completed, you can view a short summary of what the wizard has done by clicking Details.

Figure 47: Details of the Wizard Execution

Click OK twice to close the two dialogs.

Running the Clipping Robot

Step 23 – Restart Tomcat The wizard has added some library files and updated the web.xml of the web application, so we need to restart Tomcat. Some web servers can auto-detect such changes, but not Tomcat.

In Windows, you restart Tomcat by double-clicking on the icon and selecting Stop followed by Start. On non-Windows systems, you shut down Tomcat by closing its process, or invoking the shutdown.sh script in the bin folder of the Tomcat installation folder. Restart Tomcat by invoking the startup.sh script.

Step 24 – Start RoboServer Start RoboServer. This can be done from the Start Menu on Windows Systems, or by invoking the RoboServer executable file in the bin subfolder of the installation.

Step 25 – Try out Clipping Go to the following page, to try out your clipping robot:

http://localhost:8080/rstl-demo/GoogleClip.clip.

http://localhost:8080/rstl-demo/GoogleClip.clip


Summary This tutorial introduced you to clipping by building and running a simple clipping robot.

Next Steps If you want to deploy your clipping robot on another web server than Apache Tomcat, you can use the portlet generation wizards available in the Clip menu in the RoboMaker Main Window. For more information on using the wizards, refer to the online documentation for the wizards, or the Code Generation Guide. If there is no portlet generation wizard available for your target web server, please contact Kapow Technologies for further information.

For more information about clipping robots, see the “How to Clip” chapter in the RoboMaker User’s Guide.

You can also go on and try the various tutorials found in the ModelMaker and RoboMaker User’s Guides, or start on the recommended reading.

INDEX 63

Index

.

.NET, 7

A administrator guides, 10

Apache Jakarta Tomcat, 59

attribute, 19

extracting, 33

type, 20

B BEA, 7

BEA Java Web Service Wizard, 13

BEA Workshop Control Developer's Guide, 10

C C#, 7

Click step, 32

clipping, 42

Edit Branch Wizard, 49

hiding tags, 51

Portlet View, 47

restricting links, 48

selecting tags to clip, 52

start URL, 45

using another clip branch, 56

Code Generation Guide, 9

Control Center, 7

Control Center User's Guide, 9

converting to integer, 30

creating robots, 6

D Data Collection, 15

deploying a clipping robot, 62

Deployment Descriptor file, 63

domain model, 16, 22

domain model validating a, 22

E extracting attributes, 29, 33

extracting numbers, 29

extracting text, 33

H HTML, 2

I installation, 1

J Java, 7

Java Server Pages, 7

JavaScript, 2

JSP, 7

K Kapow Developer Connection, 1

Kapow Mashup Server, 2

KDC, 1

L looping, 28


M managing robots, 8

ModelMaker, 6, 17

ModelMaker User's Guide, 9

O object

returning, 35

object model, 5

objects, 5

output object, 19

P Portlet View, 47

programmer guides, 9

R recommended reading, 10

clipping, 11

data collection, 11

integration

.NET, 13

Java, 12

web applications or portals, 12

Kapow Mashup Server basics, 11

web services

Java, 13

return object, 35

returning object, 35

RoboClient, 7, 38

RoboDebugger, 36

RoboMaker, 6, 23, 42

RoboMaker User's Guide, 9

RoboManager, 8

RoboManager User's Guide, 9

RoboRunner, 7, 36

running a robot in, 36

RoboRunner User's Guide, 9

RoboServer, 4, 7, 37, 64

RoboServer User's Guide, 9

RoboSuite .NET API, 7, 40

RoboSuite .NET Programmer's Guide, 10

RoboSuite Control, 7

RoboSuite installation folder, 15

RoboSuite Installation Guide, 10

RoboSuite Java API, 4, 7, 40

RoboSuite Java Programmer's Guide, 9

RoboSuite JSP Tag Library, 7

RoboSuite JSP Tag Library Demo, 61

RoboSuite JSP Tag Library Programmer's Guide, 9

RoboSuite Tag Library, 4

robot

clipping

creating a, 42

configuration, 48

creating a, 23

running a, 36

saving a, 35, 58

type, 44

Robot Configuration Window, 48

robots, 5

creating, 6

running, 7

RQL Service User's Guide, 10

Running a robot from RoboClient, 38

running a robot from RoboSuite .NET API, 40

INDEX 65

running a robot from RoboSuite Java API, 40

running a robot in RoboRunner, 36

running robots, 7

running robots in RoboDebugger, 36

T Tomcat, 59

U URL extraction, 31

user guides, 9

W Web Integration, 2

Web Service Wizard, 13

WebLogic, 7

Workshop, 7

Quick Start Guide

Documents

Transcript of Quick Start Guide