RabbitMQ for .NET Developers P2-Notes
-
Upload
chinh-nguyen -
Category
Technology
-
view
324 -
download
3
Transcript of RabbitMQ for .NET Developers P2-Notes
![Page 1: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/1.jpg)
RabbitMQ for .NET Developers P2: Notes
Ngo Nguyen Chinh
Ha Noi 2016
(*) Here are my notes after completing a course of study offered by Pluralsight
1. Messages and Data
1.1 Introduction
Agenda
● Serialization & Real Data ● Message Format & Serialization ● Messages & Message Identification
1.2 Serialization & Real Data
What & Why
● Real world != “Hello World” ● Real World
○ Applications work with objects ○ Messaging systems work with messages ○ Objects are converted to messages ○ Messages are converted back to objects
![Page 2: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/2.jpg)
Application Message Processing
1.3 Serialization Demo
PowerShell script
![Page 3: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/3.jpg)
Client application
![Page 4: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/4.jpg)
Server application
Contracts
![Page 5: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/5.jpg)
1.4 Message Format and Serialization
What & Why
● Types of serialization ○ XML ○ JSON ○ Binary
● How do I communicate what the format of the message body is?
ContentType property!
1.5 Message Format Demo
PowerShell script
![Page 6: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/6.jpg)
Client application
![Page 7: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/7.jpg)
![Page 8: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/8.jpg)
![Page 9: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/9.jpg)
Server application
![Page 10: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/10.jpg)
![Page 11: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/11.jpg)
Contracts
1.6 Message Identification
What & Why
● Real world ○ Many different types of messages ○ Data structures can use inheritance or polymorphism ○ Messages aren’t always sent by the same technology as processes it
● To process a message the receiver needs to workout what the message is
○ From the content? ○ The sender indicates the message type
BasicProperties has a Type property intended for the message type
![Page 12: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/12.jpg)
Application Message Processing
How Can We Do It
● Message Type = BasicProperties.Type ○ Custom String? ○ .net Fully Qualified Name ○ XSD namespace + Root Element
● Considerations ○ Should be application agnostic (if you want Interoperability) ○ What about versioning?
![Page 13: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/13.jpg)
1.7 Shared Type Demo
PowerShell script
Client application
![Page 14: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/14.jpg)
![Page 15: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/15.jpg)
![Page 16: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/16.jpg)
Server application
![Page 17: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/17.jpg)
Contracts
![Page 18: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/18.jpg)
1.8 Interoperable Demo
Using Xml Schema
1.9 Summary
● Messaging Challenges ○ Serialization & Real Data ○ Message Format & Serialization ○ Messages & Message Identification
This is one of the keys to interoperability
![Page 19: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/19.jpg)
2. Large Messages
2.1 Introduction
Agenda
● Large Messages ○ Buffered ○ Chunked
2.2 Large Messages
Background
● What if I need to push large messages through Rabbit? ● Messaging systems
○ General recommendation = small messages ● AMQP
○ Supports very large messages ● RabbitMQ
○ You can use large messages ○ But you need to be careful
How Can We Do It
● Buffered ○ Small / medium files
● Chunked ○ Larger files
Buffered Sample
(1) Client reads entire file
![Page 20: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/20.jpg)
(2) Then, this message is sent to a message queue
(3) Server receives this message from the message queue
(4) Server writes entire file
![Page 21: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/21.jpg)
Chunked Sample
(1) Client reads a chunk file
(2) Then, the Client sends this chunk to a message queue
(3) Next step, the Client reads other chunks from file and send them to the message queue
![Page 22: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/22.jpg)
(4) Server receives a chunk from the message queue
(5) Then, the Server write this chunk to file
(6) The Server received an other chunk from the message queue. After receiving new chunk, the Server writes this chunk to file.
(7) Repeat until write last chunk to file.
![Page 23: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/23.jpg)
2.3 Buffered Demo
PowerShell script
![Page 24: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/24.jpg)
Client application
![Page 25: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/25.jpg)
Server application
![Page 26: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/26.jpg)
2.4 Chunked Demo
PowerShell script
![Page 27: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/27.jpg)
Client application
![Page 28: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/28.jpg)
![Page 29: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/29.jpg)
Server application
2.5 Considerations for Chunking
● A group of related messages ● Multiple Receivers ● Ordered Delivery and Resequencer
2.6 Summary
● Challenges of real world messaging solutions ○ Large Messages
● Solutions ○ Chunked ○ Buffered ○ Separation from other messages
![Page 30: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/30.jpg)
3. Dealing with Errors
3.1 Introduction
Agenda
● Errors & Retries ● Dead Letter Queue ● Routing Failures
3.2 Basic Errors and Retry
Background
● Error Scenarios ○ Exception processing a message ○ Poison Message ○ A message we don’t understand
● Basic Error Handling ○ Reject and Retry ○ Reject and Discard
Retry Sample
(1) Client sends a message to a message queue
(2) Server receives this message from the queue
(3) There is an error processing. The message is rejected
![Page 31: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/31.jpg)
(4) For RETRY mode. Server will return this message to queue
(5) The message is Retried
Reject Sample
(1) Client sends a message to a message queue
(2) Server receives this message from the queue
(3) There is an error processing. The message is rejected
(4) For DISCARD mode. The message will Discard
![Page 32: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/32.jpg)
How Can We Do It
Demo Scenarios
![Page 33: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/33.jpg)
3.3 Basic Retry Demo
PowerShell script
![Page 34: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/34.jpg)
Client application
![Page 35: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/35.jpg)
Server application
![Page 36: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/36.jpg)
Run demo: DISCARD
(1) Client sends a message “1”
(2) Queue has an unackledged message
![Page 37: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/37.jpg)
(3) Server can’t process this message “1”, so the server discards the message
Run demo: RETRY
(1) Client sends a message “2”
![Page 38: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/38.jpg)
(2) The message is rejected but it is retried forever
(3) And queue still has an unack message
![Page 39: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/39.jpg)
3.4 Advanced Errors and Retry
Background
● We can’t retry forever! ● We should retry n times and then reject ● We can decide if to reject sooner or later depending on the type of error
Scenarios Sample
How Can We Do It?
● Reject = BasicReject (no requeue) ● Retry = Custom Solution
○ Custom Property to Track no. Retries ○ No retries incremented with each attempt ○ Instead of rejecting we republish a new message with updated
properties ○ Application decides how many times it will allow a message to retry
![Page 40: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/40.jpg)
3.5 Advanced Errors Demo
PowerShell script
![Page 41: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/41.jpg)
Client application
![Page 42: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/42.jpg)
Server application
![Page 43: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/43.jpg)
![Page 44: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/44.jpg)
3.6 Dead Letter Queue
Background
● When we reject messages we don’t just want to throw them away ● Store errors for review by Ops ● Advanced error handling? ● Repair & Resubmit
Dead Letter Sample
(1) Client sends a message to Queue
(2) Server receives this message from Queue
(3) The message is rejected and not requeuing message. This message is moved to DeadLetterQueue
![Page 45: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/45.jpg)
(4) A DeadLetterQueueProcessor receives this message from the DeadLetterQueue
(5) The DeadLetterQueueProcessor repair and resubmit message
(6) Server receives the repaired message from Queue
![Page 46: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/46.jpg)
How Can We Do It?
● Declare an Exchange for receiving messages ● On queue set the xdeadletterexchange property
● Routing Rules ○ Normal exchange routing rules can apply ○ You can setup a dead letter routing key
3.7 Dead Letter Queue Demo
PowerShell script
![Page 47: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/47.jpg)
Client application
![Page 48: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/48.jpg)
Server application
![Page 49: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/49.jpg)
DeadLetterQueueProcessor
![Page 50: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/50.jpg)
Run Demo
● Queues & Exchanges
![Page 51: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/51.jpg)
● Send message “1”
![Page 52: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/52.jpg)
● Send message “2”
![Page 53: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/53.jpg)
3.8 Routing Failures
Background
● When processing messages sometimes a message will not be subscribed to
● Routing failure ○ Could be an error ○ Could be a malicious message ○ Could be fixable
Alternative Exchange!
Routing Failures
(1) Client sends a message with routing key ‘bananas’
(2) No one is interested!
![Page 54: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/54.jpg)
(3) This message is moved to Alternative Exchange. Then it is moved to Backup Queue
How Can We Do It?
![Page 55: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/55.jpg)
3.8 Routing Failures Demo
PowerShell script
![Page 56: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/56.jpg)
Client application
![Page 57: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/57.jpg)
Run Demo
● Exchanges and Queues
![Page 58: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/58.jpg)
● Send correct routing (routing key: ‘apples’)
![Page 59: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/59.jpg)
● Send correct routing (routing key: ‘oranges’)
● Send routing failures (routing key: ‘bananas’)
3.9 Summary
● Challenges of real world messaging solutions ○ Basic Error Handling ○ Advanced Error Handling ○ Dead Letter Queue ○ Routing Failures
![Page 60: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/60.jpg)
4. Advanced Topics
4.1 Introduction
Agenda
● Scheduled Delivery ● Virtual Hosts
4.2 Scheduled Delivery
Background
● How can I publish a message but delay the processing of the message?
Holding Queue?
Scheduled Delivery
(1) Client sends a message to HoldingQueue
(2) Message schedule reached. It is moved to Exchange
![Page 61: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/61.jpg)
(3) Then this message is moved to ProcessingQueue
How Can We Do It
4.3 Scheduled Delivery Demo
PowerShell script
![Page 62: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/62.jpg)
Client application
![Page 63: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/63.jpg)
Run Demo
● Exchanges & Queues
![Page 64: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/64.jpg)
![Page 65: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/65.jpg)
● Send messages (1) Client sends a message
(2) First, the HoldingQueue receives this message
(3) Then, after waiting the timeout (5000), this message is moved to the NormalQueue
![Page 66: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/66.jpg)
4.4 Virtual Hosts
Background
● Can I sandbox messaging in RabbitMQ? ● Why
○ Application isolation ○ Data protection ○ Support separation ○ SLA
Virtual Host!
Virtual Host
![Page 67: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/67.jpg)
How Can We Do It?
![Page 68: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/68.jpg)
4.5 Virtual Hosts Demo
PowerShell script
![Page 69: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/69.jpg)
Client application
Run Demo
● Exchanges & Queues
![Page 70: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/70.jpg)
● Send a message to Host1
● Send a message to Host2
4.6 Summary
● Challenges of real world messaging solutions ○ Scheduled Delivery ○ Virtual Hosts
![Page 71: RabbitMQ for .NET Developers P2-Notes](https://reader034.fdocuments.us/reader034/viewer/2022042600/587815061a28ab971e8b7cb5/html5/thumbnails/71.jpg)
5. References
https://app.pluralsight.com/library/courses/rabbitmqdotnetdeveloperspt2