Having fun with Delphi and AMQP - C4D -...

Having fun with Delphi and AMQP

In last issue of Blaise Pascal there was a great article by Fikret Hasovic, about AMQP (Advanced Message Queue Protocol), explaining its structure and some of the historical background about it.This article is meant to follow up on that article, by showing in praxis, how to install one of the more famous AMQP servers (RabbitMQ), and connect to it from a Delphi application using the extensive middleware framework kbmMW Enterprise Edition.

So why AMQP? Because AMQP is one of the only widely supported methods of doing messaging between different architectures and platforms. As kbmMW is a swiss knife of features, providing extensive connectivity for Delphi, C++Builder and FreePascal, its only logical to also support AMQP as one of the many ways to integrate kbmMW based software with the world.

For now kbmMW supports AMQP v. 0.91 as a client fully, but in fact knows about all AMQP 0.91 definitions so it potentially could also act as an AMQP server. Components4Developers are considering also supporting AMQP v. 1.0, which is actually in most ways a subset of AMQP 0.91 but with a few changes to the transport format. Please check the previous article by Fikret Hasovic to learn about the differences between the two versions.

RabbitMQ

Installing RabbitMQFirst step in anything to do with AMQP, is to install an AMQP 0.91 aware server. There are multiple to choose from, but one of the most popular ones, which is both fast and fairly easy

to use, is RabbitMQ.For this example I only show how to install

on Windows. RabbitMQ is also running on Linux and other platforms.

As RabbitMQ is written using the Erlang language, we need to install the Erlang runtime executables first.

Choose, depending on your Windows version, to download either the 32 bit or 64 bit version of Erlang:

http://www.erlang.org/download.html

Run the downloaded file and install Erlang.When installed, update your Windows system environment variables to include the variable ERLANG_HOME.

It must point to the directory containing the Erlang bin directory. (Start > Settings > Control Panel > System >

Advanced > Environment Variables)

PrewordSome may pose the question if it isn't the case that AMQP is a competitor to kbmMW's own Wide Information Bus. The short answer is yes:- Both frameworks have been designed for

distributing information from publishers to subscribers.

- Both are built around temporary or persistent queue based storage for the messages while they are in transit to the end subscriber.

- Both have been designed with performance and stability in mind.

However they also differ in areas:- AMQP is 100% content agnostic. There is NO

defined way or standard for how to interpret the contents of an AMQP message. It has to be 100% agreed between the sender and receiver. kbmMW have defined types of messages for different purposes and with defined structure, but also supports transferring generic content.

- AMQP is only a messaging framework. Nothing more, nothing less. kbmMW WIB is the messaging framework of kbmMW, but kbmMW also contains an application server, database connectivity, native JSON and XML stacks and more, and is probably more comparable (in functionality) to a combo of J2EE, Corba, RMI and AMQP.

- AMQP is always doing copy/store of messages to relevant queues, while kbmMW supports reference counting when multiple endpoints needs to receive a message.

- AMQP natively builds on persistent named queues, although it supports temporary queues as well. kbmMW natively builds on shared in/out queues, and optional persistency in named queues and thus avoids message copying as much as possible.

Nr 5 / 6 2014 BLAISE PASCAL MAGAZINE4COMPONENTSDEVELOPERS

112

4COMPONENTSDEVELOPERS

Delphi

expertstarter

COMPONENTS

DEVELOPE

RS4

http://www.components4developers.com

Then add a new system variable. Provide the correct path to the Erlang installation as the variable value:

Now Erlang is ready for use.Then download RabbitMQ from here:

Click the download figure

http://www.rabbitmq.com

Then click the quick 5mb Windows download

And run the downloaded installer. It will quickly install RabbitMQ.If you have a regular Windows start menu, you will now see RabbitMQ in the start menu.


4COMPONENTSDEVELOPERSHaving fun with Delphi and AMQP (Continuation 1)

Nr 5 / 6 2014 BLAISE PASCAL MAGAZINE


If you are using Windows 8/8.1 without Classic Shell, you can instead find the shortcuts on the Windows 8 application window.

As can be seen in the shortcuts, it's possible to start RabbitMQ as a service, which means it will stay running after you log of your machine, which is how you will want it, the moment you use AMQP in a production environment.

Ports and numerous other configurations can be made, either by defining a configuration file, or on Windows by setting a number of environment variables, before starting RabbitMQ. For now we will run with the default configuration. However you can read more about how to configure it here http://www.rabbitmq.com/configure.html.

rabbitmq-plugins enable rabbitmq_management

For the ease of use, RabbitMQ supports a management console which can be reached via a Web browser, after having enabled it. To do that, start the RabbitMQ command prompt via the start menu in Windows.In the command prompt type:


114



Then start RabbitMQ, by installing it as a service via the start menu. Start it, also via the start menu.After it has been started, then RabbitMQ will be listening on the default TCP port 5672. You can

check that by typing:

Somewhere in the list you will see TCP 0.0.0.0:5672 if the RabbitMQ server has

been started using the default settings.As we have enabled a management console, you will also find RabbitMQ listening on TCP port 15672.

We can access that port via a web browser.

netstat –an | more

http://localhost:15672

The default username and password is guest/guest.

By logging in you will get to a console where you can manage channels, connections, queues, exchanges and users. This can be very handy to use, to check the status of your queues while you are developing. You can even pop off messages and save them to a binary file which you can check using a fitting editor (perhaps a hex editor if your message bodies are binary).

Now we have RabbitMQ (in its default configuration) running.





Preparing for a kbmMW client

https://www.rabbitmq.com/access-

control.html

It's advisable to change password on the guest user or completely disable it, but only after you have created a new user with a password which is “secret” to the world, except for to those that needs to access your AMQP server.

Further, RabbitMQ supports dividing the AMQP server up in “departments” or virtual hosts, which are not able to see each other. So when a AMQP client is allowed to access one virtual hosts, it does not automatically have access to other virtual hosts. One default virtual host / (slash) is default available.

As it's a good practice to define a custom virtual host, and users with strong passwords and not rely on the default ones, we will do that now, using the RabbitMQ command prompt.

To create a user with the username user1 and password pwd123 (please choose better passwords and user names) type the following in the RabbitMQ command prompt:

Then we need to create a virtual host. Let's call it kbmmw. Type the following:

And finally we need to add rights for the user user1 to access the virtual host kbmmw. For now we grant full rights (including create and destroy queues, exchanges) within the virtual host.

You can read more about access control here:

Now we are ready to build the

kbmMW AMQP client.

rabbitmqctl add_user user1 pwd123

rabbitmqctl add_vhost kbmmw

rabbitmqctl set_permissions -p kbmmw user1 ".*" ".*" ".*"

The kbmMW AMQP clientI'll show an example Delphi based AMQP client using kbmMW. First lets create the user interface:

The actual AMQP client implementation is available via the class TkbmMWAMQPClient. So we need an instance of that, for example created in the OnCreate event of the form and deleted in the OnDestroy event.uses

private

procedurebegin

nil

end

procedurebegin

end

…, , …… : ;…

. ( : );

:= . ( );;

. ( : );

. ;;

kbmMWAMQP kbmMWAMQPClient

FClient TkbmMWAMQPClient

TForm1 FormCreate Sender TObject

FClient TkbmMWAMQPClient Create

TForm1 FormDestroy Sender TObject

FClient Free

{ Private declarations }

Next step is to be able to connect that to an AMQP server. Let's write some code for the Connect and Disconnect buttons.

procedurebegin

end

procedurebegin

end

. ( : );

. (, , , );

;

. ( : );

. ;;

TForm1 btnConnectClick Sender TObject

FClient Connect

TForm1 btnDisconnectClick Sender TObject

FClient Disconnect

'192.168.1.103' 'user1' 'pwd123' 'kbmmw'

The circled buttons and the TMemo are required for the sample, the remaining is optional.


116



The IP address can be changed to match the address

of your RabbitMQ server (perhaps 127.0.0.1) if its

running on the same machine as your kbmMW

AMQP client.

Having a connection is great.

However we also need to define a channel.

AMQP allows for multiple parallel streams of data

being passed via separate channels concurrently.

At the same time the AMQP protocol guarantees that

messages pushed via a single channel is always sent

in the order that the data has been pushed.

You will normally want to have a channel per thread,

in a multithreaded environment, to ensure that your

communication is running asynchronously.

So that's the reason for the Open channel button.

Lets add some code for it.

What is a channel?

… : ; : ;

( : ; : );

… . ( : );

:= . ; . := ;

;

. ( : );

. ( );;

. ( : ; : );

. ( );;

private

procedure

procedure

begin

end

procedure

begin

end

procedure

begin

end


FChannel IkbmMWAMQPChannel

OnContent Channel IkbmMWAMQPChannel

AContent TkbmMWMemoryStream

TForm1 btnOpenChannelClick Sender

TObject

FChannel FClient OpenChannel

FChannel OnContent OnContent

TForm1 btnCloseChannelClick Sender

TObject

FClient CloseChannel FChannel

TForm1 OnContent Channel IkbmMWAMQPChannel


TkbmMWInterlocked Increment FReceived

//var // s:string;

// The following will be executed in a thread, and should // thus be synchronized to be threadsafe. // However for simplicity of the sample //we don't do this now. // However NEVER run code in production that updates // the GUI from another thread without using //Synchronize because it WILL fail! // s := TkbmMWPlatformMarshal.UTF8Decode( // AContent.Memory,AContent.Size); // mLog.Lines.Add( //'Got'+ IntToStr(AContent.Size) // +' Bytes of content: '+ s);

In our case we simply update a counter, counting how many messages we have received. We could also have extracted the received data (as the commented code shows).

Next step is to declare at least one queue, which will be used for temporary storage of data in the AMQP server. The sample shows how to declare multiple queues, and wait for acknowledgement from the server that the queues has been formed. As DeclareQueue is an async function, it will not wait for the queue actually having been declared on the server, however the kbmMW AMQP client does keep track of acknowledgements or failures from the server when they turn up. By calling WaitAcks, the client program stalls until all 3 queues have been confirmed to be created on the server.

Even though a queue may already have been declared and created by another AMQP client (or manually via the RabbitMQ management interface), you still need to declare it in your own client. If the queue do not exist on the server, it will default be created. If it does exist, then it will be used as is, but only if you declare it with the exact same options (eg. mwaqoAutoDelete etc) as when the queue was declared first time. If it has been declared with different options, then you will receive an error and be disconnected from the AMQP server.

In fact the AMQP protocol states that any error happening due to wrong/illegal request from an AMQP client MUST result in disconnection from the server, immediately after the server have sent an asynchronous notice about the reason. It can be handy to look in the RabbitMQ log directory to check up on reasons for disconnections, if you do not understand why you are being disconnected.

… : ; : ; , , : ; ( : ;

: );…

. (: );

:= . (, ,[ ]);

:= . (, ,[ ]);

:= . (, ,[ ]);

. ;;

private

procedure

procedure

begin

end

{ Private declarations }FClient TkbmMWAMQPClient


FQueue1

FQueue2

FQueue3 IkbmMWAMQPQueue



TForm1 btnDeclareNamedQueueClick

Sender TObject

FQueue1 FClient DeclareQueue

FChannel mwaqoAutoDelete





FChannel WaitAcks

'QUEUE1'

'QUEUE2'

'QUEUE3'

We will need to hold on to the returned channel as we will need it. Further we have defined a callback method which will be called whenever data is received for the kbmMW AMQP client on the channel.





Now 3 queues have been declared. However we can't directly push messages to a queue in AMQP. We instead communicate with a message exchange which acts like a post office, and decides which queues are to receive which messages. There are a number of different exchange types to choose between - direct, fanout and topic - to name a couple.

The direct exchange type is used for delivering a message directly to a specifically named queue. The fanout exchange type is used for delivering a message to all queues directly bound to that exchange, and the topic exchange type is used for delivering a message to all queues that is bound via a subscription that match with the topic.This leads to exchange / queue binding. After declaring an exchange (which will create it if it does not already exist on the server, in the same way as with queues), queues needs to be bound to the exchange. So lets write some code to do that.… : ; : ; , , : ; : ;

( : ; : );

… . (: );

:= . (, ,

, [ ]);

. ( , , );;

private

procedure

procedure

begin

end


'MYEXCHANGE'

''



FQueue1

FQueue2


FExchange IkbmMWAMQPExchange



TForm1 btnDeclareExchangeClick

Sender TObject

FExchange FClient DeclareExchange

FChannel

KBMMW_AMQP_EXCHANGETYPE_FANOUT

mwaeoAutoDelete

FClient BindQueue FQueue2 FExchange

In this code, we have declared a fanout type exchange, and bound our Queue2 to it. So whenever we post something to the exchange MYEXCHANGE, the data will automatically be put into the queue named Queue2. If other queues has been bound to the exchange, they will also receive a copy of the message. But in our case we also want to receive messages. To do that we need to define a consumer.

A consumer is always bound to a specific queue. Its allowed to declare multiple consumers, one for each queue we want to receive messages from. The consumer name is only used for if we want to explicitly delete the consumer again later.

However we don't need to do that, because the moment we close the connection, the server will automatically delete all consumers we have declared. Notice though that the queues are not deleted (unless we have setup options to do so, when we declared the queue), and thus will continue to receive messages, which the client can consume next time it connects, and declares a consumer for those queues.

… : ; : ; , , : ; : ; : ;

( : ; : );

… . (: );

:= . . (, );

;

private

procedure

procedure

begin

end


'ACONSUMER'



FQueue1

FQueue2


FExchange IkbmMWAMQPExchange

FConsumer1 IkbmMWAMQPConsumer



TForm1 btnDeclareConsumerClick

Sender TObject

FConsumer1 FChannel Consumers DeclareConsumer

FQueue2

Now all plumbing is done, and we can publish our first message to the AMQP server.procedure

begin

end

. (: );

. . (, ,[], );

;

TForm1 btnPublish1Click

Sender TObject

FChannel Publisher Publish

FExchange '' 'HELLO'

AMQP basically only supports sending binary data as message data, but to simplify things, the kbmMW AMQP client provides a couple of overloaded Publish methods for sending a string, a memory stream and an array of bytes.

The string will automatically be UTF8 encoded

before being sent so the receiver should UTF8

decode it before use.

When clicking the Publish 1 button, a message is published via the exchange we have declared. The message will end up on the queue Queue2, which we have declared a consumer for, why we will receive the message again in the OnContent event.

It's allowed to declare a queue without a name. What happens is that the server will

automatically create a unique name and assign that name to the queue.

kbmMW AMQP will get to know about the chosen

name and will update the queue variable with the new name. If you want to ensure that the queue has been given a name, before you start to do other operations (like binding it to an exchange), you should call the channel.WaitArgs method.

After it returns you are guaranteed to have been handed a unique name for the queue and thus is able to bind it to an exchange and/or a consumer. You can click the Show Declared queues after

WaitArgs and see the server generated queue name.The TTimer we have on the form is only used for

displaying the value of the received number of messages.


118



Benno Evers is our specialist for questions about kbmMW. He can help you with basic questions regarding kbmMW as well as with turnkey Development and Consultancy. He’s a specialist in netwoks, internet and hardware.

Specialist help and consultancy for kbmMW

[email protected]

CUSTOM TECHNOLOGY

COMPONENTSDEVELOPERS4

procedurevar

begin

end

procedurebegin

nil

end

procedurebegin

end

procedurevar

beginfor to dobegin

end

end

. ( : );

: ;

:= . ( , ); . . ( ( ));

;

:

. ( : );

. . ();

:= ;;

. ( : );

:= . (, ,[ ]);

;

. ( : );

: ; : ;

:= . . - := . . [ ]; . . ( + ( )+ + . ); ;

;

TForm1 Timer1Timer Sender TObject

i integer

i TkbmMWInterlocked Exchange FReceived

mLog Lines Add inttostr i

The remaining optional buttons couldcontain code like this

TForm1 btnDeleteConsumerClick Sender TObject

FChannel Consumers DeleteConsumer

FConsumer1

FConsumer1

TForm1 btnDeclareNoNameQueueClick Sender TObject

FQueueNoName FClient DeclareQueue


TForm1 btnShowDeclaredQueuesClick Sender TObject

i integer

q IkbmMWAMQPQueue

i FClient Queues Count

q FClient Queues Queue i

mLog Lines Add inttostr i q QueueName

0

0 1

''

'Queue ' '='

Now you are all set for integrating your favorite Delphi/C++Builder/FreePascal based application with any AMQP 0.91 server in the world.


4COMPONENTSDEVELOPERSHaving fun with Delphi and AMQP (Continuation 7-end)



Supports Delphi/C++Builder/RAD Studio 2009 to XE7 (32 bit, 64 bit and OSX where applicable). kbmMW for XE5 to XE7 includes full support for Android and IOS (client and server).

kbmMemTable and kbmMW are highly addictive! Once used, and you are hooked for life!

!

kbmMemTable is the fastest and most feature rich in memory table for Embarcadero products.

- Easily supports large datasets with millions of records

- Easy data streaming support- Optional to use native SQL engine- Supports nested transactions and undo- Native and fast build in M/D,

aggregation /grouping, range selection features

- Advanced indexing features for extreme performance

Warning!

KBMMW V. 4.60 AMQP support ( Advanced Message Queuing Protocol)

- Added AMQP 0.91 client side gateway

support and sample.

- Updated StreamSec TLS transport plugin

component (by StreamSec).

- Improved performance on Indy TCP/IP

Client messaging transport for large number

of inbound messages.

Full FastCGI hosting support.

Host PHP/Ruby/Perl/Python applications in

kbmMW!

- Native high performance 100% developer

defined application server with support for

loadbalancing and failover

- Native high performance JSON and XML

(DOM and SAX) for easy integration with

external systems

- Native support for RTTI assisted object

marshalling to and from XML/JSON, now also

with new fullfeatured XML schema

(XSD) import

- High speed, unified database access

(35+ supported database APIs) with

connection pooling, metadata and

data caching on all tiers

- Multi head access to the application server,

via AJAX, native binary, Publish/Subscribe,

SOAP, XML, RTMP from web browsers,

embedded devices, linked application

servers, PCs, mobile devices, Java systems

and many more clients

-

EESB, SOA,MoM, EAI TOOLS FOR INTELLIGENT SOLUTIONS. kbmMW IS THE PREMIERE N-TIER PRODUCT FOR DELPHI /C++BUILDER BDS DEVELOPMENT FRAMEWORK FOR WIN 32 / 64, .NET AND LINUX WITH CLIENTS RESIDING ON WIN32 / 64, .NET, LINUX, UNIX MAINFRAMES, MINIS, EMBEDDED DEVICES, SMART PHONES AND TABLETS.

COMPONENTSDEVELOPERS4

IN A NUTSHELL : ONE + ONE = THREE


Having fun with Delphi and AMQP - C4D -...

Documents

Transcript of Having fun with Delphi and AMQP - C4D -...