Use of kbmMW components in a server application by Benno Evers

I explained in a previous issue of Blaise how the free kbmMW Codegear edition can be acquired and installed. In this article I will explain what components need to be used to setup a minimal server. We will use a Firebird database for this example,

The server applicationThe server we will use in this article was derived from the sample kbmMW demo server. This server is a normal Windows application and the components in that sample will be placed on the main form. The advantage of this is that it is easy to debug the application and tasks like logging can easily be added.In actual environments the application server will often be a Windows service. The techniques and components used in this article will be identical for a service application except that the components placed on the main form in the application will often be placed on a data-module in the case of a service. Also a service does not contain a GUI.The demo server application can be analysed according to its component usage:• Components used for database access• Components used for the server and its transports• One or more service modules (services) that contain the

functionality the application server implements.

Components for database accessTo gain access to the Firebird database used in our sample server, we use the IBX components provided with Delphi. The kbmMW part uses the kbmMWIBX components provided in the free Codegear edition. We start by placing an IBDatabase and IBTransaction component, which can be found on the Interbase tab inside Delphi. Ibdatabase1 will use Ibtransaction1 as its default transaction, by setting up the defaulttransaction property of IBDatabase1. We also need to add a database in the Databasename property of IBDatabase1. In our example we use an absolute path (so without a servername) to the database so the demo application can run out of the box. In a real life situation you would use a full connection string, constructed on the pattern

servername local database path on the server . We also provide a username and password to get access to our database. That can be done using the Params property. In this property we add:

By setting loginprompt to False we can prevent the application asking for a username and password at startup.To the right of the IBDatabase and IBTransaction we can see three kbmMW components that are important for database access. These are the kbmMWIBXConnectionpool, kbmMWInterbaseMetaData and kbmMWPooledSession.

kbmMW uses a so called “offline” model. That means that access to the database will always be momentary (the duration of a query and fetch of the results) and no cursors will be left open in the database. That is very different to the way we acces data in typical client server applications, where in most cases we will have open cursors inside the database.

< >:< >

user_name=sysdba / password=masterkey

KbmMWIBX5ConnectionPool

Because setting up a connection to a database is relatively time consuming we provide component which pools all connections. The services inside the applicationserver (described later) need access to the database. The connectionpool will create these connections when needed and will keep open a pool of previously used connections. This way uses resources efficiently.

Linking the connectionpoolTo use the connectionpool, we need to link it to our database and transaction. We can do this by setting up the Database and Transaction property of the connectionpool. We set them up respectively to IBDatabase1 and IBTransaction1. At this point we also link the connectionpool to our metadata component by settingthe Metadata property of the connectionpool.

In our connectionpool component we can set the minimum number of connections to the database, by providing a value in the minconnections property. By default this will be 0. However, in some situations it is useful to keep several database connections open. If this Minconnections property differs from 0, the applicationserver will open the specified number of connections and will keep them open at all times. Like the minconnections property, we also have a maxconnections property. By default this is -1, meaning there is no limit on the number of open connections to the database. This property is becomes useful when you use a database connection that has a limitation on the number of concurrent connections. Most commercial databases have such a limit. In our example server maxconnections is set to 1 for a specific reason. Because in our demo we use an absolute path to the database, we need to limit the number of concurrent users, to prevent problems using Firebird.

Minimum and maximum number of database connections

Caching inside our connectionpool

One of the features of our connectionpool is a smart cache. If this is activated, all queries and their results will be temporary . If exactly the same query with exactly the same paramaters are requested from the database, the connectionpool will provide the saved result instead of activating a new query to the database. This will obviously improve performance.Using MaxCacheAge, the programmer can specify how long results will be kept in the cache. This is used per entry (or in fact query). Because the cache impacts available resources, we avoid keeping very old results, or queries that have not been requested recently.To limit the application server's resource usage we can also limit the maximum number of queries in the cache and the maximum number of result records. These can be specified using the properties MaxCacheEntries and MaxCacheRecordCount.Using caching you risk acquiring old data. In practice however this risk is very limited if you always use the application server to access the database. The resolver components (explained later) will invalidate a cache if any of the tables contained in the cache are updated. Also the programmer has an option to overrule the cache in his service modules.

saved

kbmMWInterbaseMetaDataThe kbmMWInterbaseMetaData component is used to specify various system-dependent settings for the database you are using.. This component is available for all supported databases. We use the Interbase component here because it is the closest match to our Firebird database (Firebird was derived from Interbase).The settings we can provide using the metadata component are things like time or date formats, or the use of fieldnames. In Firebird fieldnames containing quotes are handled differently from fieldnames without quotes.

expertstarter DELPHI 7 and above (Win32)

Figure 1:

Page 18 COMPONENTSDEVELOPERS 4 September 2010 BLAISE PASCAL MAGAZINE 12

Can´t wait for our Database Special?This issue will be published in September

Page 19

KbmMWPooledSessionThe pooled session acts as a 'glue' component. The services running on the application server will connect to the Connectionpool using the pooled session component. The pooled session ensures that all requests to the database will be handled in the correct order.The pooled session can be given a unique name using the sessionname property. It is this name we will use later on to link our services to the correct pooled session. We set up the sessionpool by linking it to the connectionpool via its connectionpool property.

Server and Transport componentsTransport components

To allow our application server to communicate with clients a transport component is used. The free Codegear edition of kbmMW has three transports bundled, Local transport, TCPIP Indy transport and ISAPI transport. It is possible to use multiple transports simulatuneously, even if they are of the same type. This allows for very powerful setups, e.g. having a local LAN networks and slower internet connections each using their own connection.

The local transport is exactly what its name implies, a communication layer without using any network connection. It is very useful when designing an application. Both server and client are bundled in one big application, using local transport as a connection. Local transport also is very useful if a client prefers a classic client server (2-tier) application. You can use the services you already have, and design the entire application as if it were an application server and a split client. Using local transport will give the client 1 big client directly connecting to the database. You can also take advantage of kbmMW's powerful caching in this instance to enhance performance.

If the architecture of the above client is chosen correctly, it is very easy to split later to make it a 3-tier application. All that needs to be done is to make an application server and include the services. Change local transport to one of the other transports (e.g. TCP/IP) and you are up and running again.

TCP/IP transport is one of the most versatile transports in kbmMW. It allows connections both to local networks, and also over the internet.

Local transport

TCP/IP transport

By changing the four zeros for a valid IP number of the server-machine, we can set up the TCP/IP transport to respond only to requests coming in on this specific IP address. This is a very powerful feature. As written earlier, we can use more than 1 TCP/IP transport simultanuously. We might have more than 1 network card in our server-machine. By using the bindings property we set up a connection for each of these network cards, e.g. one serving the local LAN and another serving the WAN connections. Linking a transport component to a server is very easy by linking the server property. Another important property to set up is the streamformat property.

The metadata component also provides an API to get access to some database dependent functions. One of the functions we use in our sample server is the use of a generator to get unique Ids for our primary keys. A generator is the Firebird way of providing a unique number outside of a transaction context. In our service example later on we will provide an example of using the metadata component to get a generated value. To use the metadata component, it must be linked to our connectionpool by setting the Metadata property of the connectionpool.

The kbmMWTCPIPIndystransport will provide a listener at a specific IP port. It will respond to incoming client requests at this specific port number. The port that is used can be set up using the bindings property. In our example the TCP/IP transport will respond to incoming connections on port 3000. In this case the IP is not relevant, thus it is 0.0.0.0. So this transport will respond to all incoming connections on port 3000.

Figure 2 A minimal server application based on in this article has at least a kbmMWServer component and a transport component, in our example the kbmMWTCPIPIndyServerTransport. :

Figure 3:

Figure 4

The free kbmMW Codegear edition bundles 4 transports, HTTP, SOAP, STANDARD and XML.

The HTTP stream format can be used to create e.g. a REST interface. A client can send a request in the form of a URL, and the server will respond by sending an HTTP response, e.g. a webpage. The use of HTTP stream format is outside the scope of this article. Additional information can be found in the kbmMW on the knowledge base components4developers website

HTTP Stream format

(www.components4developers.com ).

September 2010 BLAISE PASCAL MAGAZINE 12 COMPONENTSDEVELOPERS 4

Use of kbmMW components in a server application (continuation 1)

SOAP Stream format

The transport also provides a SOAP stream format. SOAP is a generic transport format based on XML and often used for webservices.To use SOAP with the kbmMW servers a WSDL can be downloaded at the website of Componets4Devlopers. This WSDL is generic and can be used with all application servers based on kbmMW.

Because it is based on XML and has sort of a wrapper around it, SOAP is not a very effcient transport layer. Also because of the generic way the WSDL is set up, the implemantation of SOAP is a bit spartan.

Personally if I ever give clients the option to use SOAP, I create an intermediate layer e.g. an ISAPI running on a webserver. This layer exposes the SOAP functionality needed and communicates with to the underlying application server.There is an extensive article at in the kbmMW knowledgebase about the use of SOAP. It can be found here:http://www.components4programmers.com/downloads/kbmmw/documentation/SOAP_with_kbmMW.PDF

Standard stream format

XML stream format

The standard stream format is mostly used in TCP/IP transports. It is an optimized binary transport for client/server communications.

The XML stream format uses standard XML messages for requests and responses between server and clients.The XML messages contain an envelope encapsulating the actual message data. Using XML format, specific functions at the server can be called or datasets requested.

Using XML creates a lot of transport overhead due to the verbose syntax of XML. However XML is more or less a standard and can be an ideal option to have non-Delphi clients communicate with your Delphi application server. Imagine of a webshop front-end (the website) transferring orders to your Delphi backoffice using an XML transport.

There is an extensive white paper about using the XML format that can be found in the kbmMW knowledgebase. It can be found here:http://www.components4programmers.com/downloads/kbmmw/documentation/kbmMW_XML_transport_format.pdf

procedurevar begin

end

. ( : );: ;

:= . ( , ); := . ( ,

, );;

TForm1 FormCreate Sender TObjectsd TkbmMWCustomServiceDefinition

sd kbmMWServer1 RegisterService TkbmMWInventoryService falsesd kbmMWServer1 RegisterServiceByName

TTIMESERVICE False

// Register here the services which this server application should provide.// RegisterService registers the service with the name the service prefers to be known as.// Using RegisterServiceByName you can register services under other custom names.// This can be of interest if you want to provide a standard service under a custom name.//// By setting the Enabled parameter to true, the service will be enabled.//// The MaxCount is used to define how many concurrent instances of the service is allowed.// If MaxCount<0 there is no limit, and new service instances are started when needed.// If MaxCount>=0 then a max of MaxCount services can be running at the same time.// If, for example, MaxCount=10 and there are already ten instances serving requests, then the eleventh // request will be put into a queue waiting for a vacant service from the service pool.// The timeout parameter controls how long a service can stay idle (not being used)// before its garbage collected or in other words before that specific service instance is// destroyed and removed from the pool of available service instances. Set timeout to 0 to disable garbage collection // of instances of that particular service.//// RegisterService(AServiceClass:TkbmMWCustomServiceClass; Default:boolean);// RegisterServiceByName(AServiceName:string; //AServiceClass:TkbmMWCustomServiceClass; // Default:boolean);// Set the properties of variable sd to control other service definition settings.

'TIMESERVICE'

If we examine this code, we can see that initialising the server is as simple as letting it know what services are available. Services are registered in lines beginning sd := kbmMWServer1.RegisterService ... In our server, the first service being initialised is the kbmMWInventoryService. The kbmMWInventoryService is a standard service in the kbmMW framework and will almost always be available for clients. The kbmMWInventoryService can provide information about available services at this application server, including available versions of that service. A client can request an inventory and will get a list of available services and the parameters needed to use the service. As with almost everything inside kbmMW, the programmer can use this very flexibly and decide per service what information should be shared via the inventory service. A programmer also can decide to create a service without exposing any information about that service through the inventory. The second service we see being initialized is called TimeService. This will provide selected functionality from our demo application to clients. We will show later in this article how we can create our own service, creating the TimeService Service. In a real application server the list of services will be a lot longer. In general you will try to keep functionality that belongs together in one service. That way you will create server side building blocks that can be reused in the future. Business rules that apply to certain types of data will typically be implemented on these services. The kbmMWServer component also provides functionality so that a certain service can use the functionality of another service by calling it. Suppose you have a service that provides address data of clients which can be called specifying a customerID. Now suppose you create a new service creating bills. Instead of creating a new query requesting customer data on this billing service, you could use a call to the address service to provide you with the customer address for this bill. This way all the business rules you inplemented in the Address service will apply to the address data you get from the service.

Creation of your own servicesThe service-modules in kbmMW are from a coders view comparable to datamodules in Delphi. You can place all kinds of non-visual components on them. KbmMW uses these modules as a sort of template to create instances of them when needed, running in their own thread. The services themselves are typical stateless. This means a service has no knowledge of previous actions and can not be used to store values, e.g. an ID used in a previous call. Big advantage of this is that the service could run on every machine. The concept of being stateless and not having a memory is an important concept to remember when defining and using services. Of course kbmMW also provides features to create a statefull service, but this is outside the scope of this article. Statefull service are only used in very specific cases. To create your own service, kbmMW provides a wizard that creates a template for your service. The wizard can be found using the menu, File → New → Other.

The kbmMW server component The kbmMW server component is the most important component of our application server. It is the traffic marshaller controlling all communications with our transport component(s), authenticating users,creating services and then disposing of them after use.

The basis of an application server are so called services that provide functionality to clients. Because multiple clients will be connected to a server most of the time, multiple instances of a service need to be available, typically one for every client requesting some functionality. These services each work in their own thread.. The kbmMWserver component will create these service-threads when needed and dispose of them when done. The server component handles this in a very efficient manner by keeping used services in a pool for some time after use.

For every service a programmer can specify a minimal number of service instances that always need to be active, because instantiating a service thread takes a long time. Similarly a user can set up a maximum number of simultaneous service-instances that can be active. This would be done to reduce demand for resources at the server. If the maximum number of service-threads are in use, client calls for that service will be placed in a queue and handled when one of the services becomes available again.

The basic use of a kbmMW server component is fairly simple. You place a kbmMWServer component on your basic module, in our case a form. After this the server needs to be initialised at startup. In our example server we use the FormCreate event for this.

Page 20 COMPONENTSDEVELOPERS 4 September 2010 BLAISE PASCAL MAGAZINE 12

Use of kbmMW components in a server application (continuation 2)

Page 21

Figure 5 After choosing Other you will get a popup screen where the kbmMW service wizard can be selected.

The Wizard provides features for different kinds of services. Because in our example we want to build a service using data from a database extended with some RPC calls we choose the Query service inside the wizard.

Having chosen this we click the big arrow at the bottom. The wizard presents a new page showing a combobox. This combobox which allows us to choose a database. In our example project we use Firebird, so we choose Interbase express (IBX) for our database connection. Now click the big arrow again. The page shows some extra options.

The bottom part shows another combobox which tells the service wizard the location of our connectionpool. In our example we placed this on our Mainform (Form1), so that is what we choose in the combobox. Below the combobox are two checkboxes. These are options for components that can be placed on our new service module by the wizard. We won't use any stored procedures in our example, so that checkbox can be unchecked. After this we click the big next arrow again, to get to the next step of the wizard.

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

This step gives us an option to give our service a name. We already talked about the service TimeService, so we type in TIMESERVICE to use as our name.We can also provide a service version. As a programmer you have an option to have more than 1 service version running using the same name. Sometimes that is useful to support both old and newer service functionality.. In our case we leave this field empty.Now click a number of times on the big next arrow at the bottom of the screen until we get to the Choose code output screen.In this Choose code output screen we check the Delphi checkbox and click the next arrow again. Now you will see a screen summarising all the settings you have made. If that is correct click the big green V, so the code is actually generated. The result of this action is a new Delphi unit, populated with the components we chose earlier.

On this module we see the 3 basic components we will almost always use with databases. The kbmMWIBX5Resolver is a very powerful component capable of creating dynamic SQL. The resolver is linked to a query component and will create correct SQL for any an insert, update or delete. The most important properties that need to be changed are InsertKeyFields and UpdateKeyfields. Generally both should be set to True.. By setting these properties the resolver will be able to create the correct SQL.The resolver also provides events to allow the programmer to change or view SQL or to change field values. One of these events is OnGetValue. This event provides a way to check the contents of a field and if needed provide it with a different value. In our example we will use this event to modify a primary key, just before it is written to a database.

September 2010 BLAISE PASCAL MAGAZINE 12 COMPONENTSDEVELOPERS 4

Use of kbmMW components in a server application (continuation 3)

About the author: Benno Evers is an independent developer with a background in electronics and embedded software.Software running on windows has been developed using Delphi, starting with Delphi 1. Since 2003 kbmMW has been a big part of most applications requiring data-access. The database platform used is often Firebird, using kbmMW and UIB. Remarks and questions can be emailed to Benno@EversCT.nl

procedure

varbegin

if thenbegin

if then

begin

endend

end

. ( : ; : ;

: ); : ;

( . )= =

. . (, , );

:= ; ;

TTIMESERVICE kbmMWIBX5Resolver1GetValueADeltaHandler TkbmCustomDeltaHandler AField TField

AValue Variant Sleutel Variant

UpperCase Afield Fieldname

Avalue NULL

Form1 kbmMWIBX5ConnectionPool1 MetaValuemwmdtSequence SleutelAValue Sleutel

var

{ Resolver.Getvalue will be called during resolve (writing) of data to the database. By implementing OnGetValue we can change the contents of any field. In our example, we will check if our keyfield is filled. If it is

NULL we will create a value using a Generator. } 'PROJECTID'

{ Through the Meta component we can call on the metadata of the database. in this case: generators }

'PROJECTIDINC'

;;

In the above code we can see that we check the field ProjectID. This is the project table's primary key, and must contain a value. In our sample project we want this field to get an autoincremented ID. In Firebird we use generators for that, so when creating our database we also created a generator called PROJECTIDINC. Earlier I explained that we can use the Metadata component to access data specific functions. Querying and updating are examples of such functions.. Using MetaValue() we can call the PROJECTIDINC generator. It will return the newly generated value in our variable Sleutel. By assigning this to aValue, the NULL will be replaced by the autoincremented value for the ID. Of course this technique can be used for other fields too, e.g. replacing a boolean field by 'Y' or 'N' just before writing to the database. The kbmMWIBX5Query is basically is a normal query component. We can provide a normal SQL query in its SQL property. Most of the time this will be a select query, because inserts, deletes and updates are handled using the resolver component. That is why the query component and resolver component need to be linked using the resolver property. The resolver component can be linked to more than one query at the same time. The kbmMWIBX5Query also needs to communicate with our database. As I wrote earlier, on our main module (form1 In our example) we placed a connectionpool and a sessioncomponent. This is where we use them. We will use the session property of our kbmMWIBX5Query to link the query to our sessioncomponent and connectionpool. Just type the Sessionname we defined in the session component into the sessionproperty. This has to be done for all query components whenever there is more than one query component in a service module.To enable our resolvercomponent to create the correct SQL we need to define what table is being used and what its keyfields are. We can do this by setting up the TableName and Keyfieldnames property of our kbmMWIBX5Query.The queries we use in our example application are also called named queries, meaning that a client can call them using their componentname. The query itself will be executed at the server. The only SQL commands that can be seen on the network will be between server and databaseserver. The client then receives the modified data back from the server. The great advantage of this is the database structure can be hidden from the clients. There is no SQL between client and server so sniffing the network will reveals nothing about the underlying database. To make sure our clients can be seen from the client, we also need to set the published property of our kbmMWIBX5Query component to true. The service itself also has a number of properties that need to be set up. They specify how the service should behave on the server.

These properties can be shown by clicking the service module. The transportstreamformatter can be linked here if the wizard did not do that. It will take care of formatting communication within the framework. Most of the time the kbmMWBinaryStreamFormatter will be used. The property AllowClientNamedQuery needs to be set to true. By doing this we tell the servicethat it is OK for a client to call the queries (named queries) by their component names. The property AllowClientKeyFields also needs to be true. It specifies that we are allowed to send new records to the server, including a filled keyfield. This is most useful when setting up master-detail relationships. The components in this article describe a minimum configuration for a kbmMW application server. In a subsequent article I will explain how we can extend our query service with extra functionality that can be called from a client (so called RPC calls). Also I will describe what is needed to create a kbmMW client that can use this application server.Questions can be sent to b or you can visit the kbmMW newsgroups at .

enno@eversct.nlnews.computers4developers.com

Figure 11

New Features:

Additionally there is now support for AnyDAC, UniDAC and IBDAC data API's

- clientdataset.AsyncOpen allows asynchronous response to query/dataset requests from the server. Together with the client side method SetQueryResult, this feature can be used with new server side features which allow you to send incremental resultsets from the

server asynchronously using a new function SendPartialResultDataset-

-

-

New virtual database adapter (kbmMWMD) which provides access to a virtual database/table or memorytable.

Fielddefs/parameters/records can be provided via a memory table, defined at design time or at runtime, or via on the fly events

OnPerformFielddefs and OnPerformQuery. to allow

for virtual definition of dataset definitions and data: OnGetDefinitions, OnGetData, OnGetMoreData, OnExecute and OnGetMetaData

which provides a simple to use single file virtual file system for developers. In fact it can be used with

any TStream descendant to access streams of data in a very simple key/value way

Provided more events on TkbmMWCustomQuery service

New TkbmMWLookupStorage class

: ; : ;

:= . ( , / ); := . ( ); . ( , );

. ( , );

. ( , )

. ( );

. ( )

. : ( );

. ; . ;

;

fs TFileStream

ss TkbmMWLookupStorage

fs TFileStream Create somefile read write options

ss TkbmMWLookupStorage Create fs

ss Add someotherstream

ss Add anotherstream

ss Get yetanotheremptystream

ss Delete

ss

ss Count

ss GetIdentifiers somestringlist

ss Free

fs Free

begin

if not

then

if Contains then

end

'someidentifier'

// Adds the otherstream contents to the virtual file system.

'c:\somepath\somefilename.txt'

// Adds another stream to the virtual file system under the given key.

'someidentifier'

// Didnt find the data under that identifier. Else the stored stream will be available in the yetanotheremptystream stream.

'someidentifier'

// Removes the data associated with the someidentifier key.

'someotheridentifier'

// Does the virtual filesystem contain a stream with the given identifier

// Returns the identifiers in the virtual filesystem

// Returns the number of entries in the virtual filesystem.

Page 22 COMPONENTSDEVELOPERS 4 September 2010 BLAISE PASCAL MAGAZINE 12

Use of kbmMW components in a server application (continuation 4)

ONLY FOR SUBSCRIBERS

COMPONENTSDEVELOPERS 4

COMPONENTSDEVELOPERS 4

SPECIAL OFFER:

KBMMW PROFESSIONAL

ENTERPRISE INCLUDING COMPETITIVE

UPGRADES

DISCOUNT

AT ORDERING A NEW LICENSE OF

OR

40%

THE COUPONCODE TO REFER TO IS: .

THIS OFFER IS ONLY VALID THROUGH THE PERIOD STARTING AT SEPTEMBER 18/2010 UNTIL OCTOBER

18/2010

KBMMWBLAISEPASCAL2010