Thursday, July 21, 2016

Microsoft Service bus: Queue, Topic and Event Hub

Selecting between Queue, Topic and Event Hub:


Usage:

Queue:

The scenario for queue is almost clear. Each message will be receive only by 1 receiver.
Of course it is possible to have multiple senders and multiple receivers, but the key point is that each message will receive only to one of those receivers.

A simple example of using queues is POS (point of sale) systems. POS terminals produce some data with different load/times and the inventory management system has to process all of them. The inventory system and the Terminals are loosely coupled there might be different softwares in each terminal, but there is only 1 receiver on the other end.


Topic-subscription:

This type of queue will be used when a single message has to be received by multiple receivers. It is also possible to differentiate between receivers so a group of receivers will only receive special types of messages.
The same as queues, there can be multiple instances of sender/receivers (multiple receiver per subscription) is available, but when a receiver reads data from 1 subscription, the others cannot read it again from the same subscription.
A simple example for this scenario is logging/Search. You want to have an instance of your data to use in a logging (search) system, but there has to be no relation between your realtime system with the logging system rationally.

The second scenario is also possible when you want to divide your messages into different lines. Like when you want to create a priority queue, or when a set of producers produce data for different systems. 
A simple example for this would be when you create data in a way and process them in different manners. For instance if there is an ordering system registers orders for different companies, then each company only needs to know about its orders.

Event-Hub:

Event Hub is simply a big stream of data. In a way, Event hub is the answer for the same issues that we addressed in the Topics. There are multiple consumers for each message, but the big difference is that Event-hub has been implemented for maximizing the throughput.
Each Topic in event hub will multiple partitions (normally between 4 to 16) and 1 instance of the client will read from each partition. If any of the clients goes down or there are less clients that the partitions, clients will compete against each other to access to more partitions.

A simple scenario for this solution is logging players actions in a popular game. 

How they Access data

Queue:

There is only 1 line of messages. Clients wait for a message and when a new message received, only one of them will process it.

Topic-subscription:

There is only multiple lines of subscriptions. Clients of a subscription wait for data and the same scenario as queues happens here. It is important to note that there is no relation between data in different subscriptions.

Event-Hub:

Clients are responsible for the data that they read. Meaning that it is possible for different clients to read a message multiple time. Messages would not remove from system until a reasonable time (some days), or depending on size of data. 
Each client will ask the hub for data after special message (check point) so it is very possible to see some messages has been read multiple time when a client restarts.
There is a concept of Consumer Groups which is almost the same as subscription in the topic base service bus.

How to manage failure

Topic-subscription:

There is 2 ways of receiving a message:

1- PeekLock (default)

Data will be read by the client and it will be locked for some time. When the client finished its work, it simply tells to the broker that and the broker will remove the item from queue.
If an error happens in the process client can abandon the message, therefore another client would read the same message later on in case that the error was temporary.
Also, if the client crashes in the middle of the process, the lock will be time outed and the message will be readable for another client.

2-ReceiveAndDelete

The broker doesn't care about the probable errors. It simply say the message will be read 1 times max!

Event-Hub:

Messages will be in the hub for a long time, so we can be sure that data will be read by clients at least 1 time
Each client will set a check point after a portion of time (or any rule that the clients decides) and if it restarts, it will continue reading from that checkpoint again. So the client has to take care of duplicates it self.

Installation Environment

Queue and Topic-subscription are available on both Microsoft Azure (cloud) and on premise but Event-Hub is only available on the Azure.

Read More
Azure Event Hubs overview

18 comments:

  1. This was truly awesome. thanks so much for this!! Azure Online Training

    ReplyDelete
  2. Very very nice article. Thanks for sharing. Please keep sharing...

    games

    ReplyDelete
  3. This is Very very nice article. Everyone should read. Thanks for sharing. Don't miss WORLD'S BEST BusGames

    ReplyDelete
  4. VERY USEFUL BLOG, IF YOU NEED MORE ISSUES
    CALL OUR TOLLFREE NUMBER +1-845-481-1722
    CAN VISIT OUR SITE : https://setup-microsoft-office.com/

    ReplyDelete
  5. thanks for a few adjunct informative weblog. in which else ought to I gather that form of auspices written in such a really perfect manner? i have a task that Im simply now running regarding, and i have been at the see out for such recommendation. 30088-4

    ReplyDelete
  6. i have entre most of the articles re your website now, and i basically following your fashion of running a blog. I bonus it to my favorites weblog web page list and might be checking sponsorship soon. Please test out my web site as ably and take me know what you think. Windows 10 Pro Keys Office 2019 Professional Plus CD Keys.

    ReplyDelete
  7. Thanks for sharing this blog.This article gives lot of information.
    Azure development online Training

    ReplyDelete
  8. Quickly this site will indisputably be famous among all blogging people, because of its fastidious articles or reviews.
    mu online

    ReplyDelete
  9. نحن نمتلك مجموعة من الخبراء والمتخصصين فى شركة تنظيف بابها ونحن نستطيع التعامل مع كافة المساحات المختلفة فلا يهم ان كنت تمتلك منزل او فيلا فأن لدينا خبرات كبيرة تمكنا من تقديم خدماتنا على اكمل وجه ولدينا عمال وفنيين محترفين ولهم خبرات مختلفة نقدم ايضآ تنظيف لواجهات الشركات والفنادق. فكل ما تحتاجه من معدات واجهزة ومنظفات ذات جودة عالمية موجودة بشركتنا فنحن نسعى فقط لارضاء العميل أولآ واخيرآ

    شركة عزل خزانات بخميس مشيط
    شركة مكافحة حشرات بخميس مشيط
    شركة غسيل مجالس بخميس مشيط
    شركة غسيل خزانات المياه بخميس مشيط
    شركة غسيل خزانات بخميس مشيط
    شركة تنظيف شقق بابها

    ReplyDelete
  10. Montre Rolex Suisse Pas Cher, combining elegant style and cutting-edge technology, a variety of styles of Montre omega Suisse Pas Cher, the pointer walks between your exclusive taste style.

    ReplyDelete
  11. This blog is really great. The information here will surely be of some help to me. Thanks!. free spins for house of fun gift exchange

    ReplyDelete
  12. Look at this as an investment in your future if you like. The professional writing services are used to constructing articles, papers and essays that are engaging and that hit the spot. Once you have a draft of your essay the adjustments make sure that it is top-notch and helps you to achieve your objective.australian essay writing service

    ReplyDelete
  13. wow, great, I was wondering how to cure acne naturally. and found your site by google, learned a lot, now i’m a bit clear. I’ve bookmark your site and also add rss. keep us updated. playboy888 apk

    ReplyDelete
  14. https://best-usa-casinos-online.blogspot.com/2013/07/slot-dunk-slots-game-best-new-sport.html?showComment=1576413288585#c8516662397045719615

    ReplyDelete
  15. i was simply scanning along and happened upon your site. simply needed to say great site and this article truly helped me.qq online


    ReplyDelete
  16. Nice and good article. It is very useful for me to learn and understand easily. Thanks for sharing your valuable information and time. Please keep updating.

    Digital Marketing Training in Chennai

    Digital Marketing Course in Chennai

    ReplyDelete