Multi Service Bus Core 2.0

Hi, today I want to present to you Multi Service Bus Core 2.0. The new version has a lot of small bugs fixed. I recently bought to myself very good tool to draw diagrams. Below I will want first starting with showing the concept diagram. Multi Service Bus Core 2.0 offers Server to Client communication. It is important point so I described it first with this but let me start from the basics principles of the library solution. Whole library exposes only 3 kind of generic C# components named Requester, Receiver, Responder. Requester is a server in client-server pattern, it is responsible to create messages in the queue ready to be taken by Receiver or Responder. The main difference between the last two is that Receive do not have to crate response and Responder have to. That is the main difference on them. Now let me quickly describe components inside whole solution. For exchanging messages you have to serialise them and for that I am using binary serialisation based on Protbuf for .NET written by Mark Gravell. For communication I am using simple contract created in Apache Thrift written by some very smart guys. And all mentioned libraries when I downloaded them where shared on Apache License 2.0 as an open source. And that is why I am also sharing Multi Service Bus Core 2.0 on the same Apache License 2.0. Below you can see how overall solution is looks like from construction of component used to build it perspective.

MultiServiceBusCore

Now, when all details about the internal design was described I want to explain possible communication layers inside your own implementations. First that is very important you can create inside one process communication. That may sound silly why should I make inside one the same process communication and use those MSBC2.0 for that you may asking? The reason for that is that behind the scenes it uses the Publisher-Consumer pattern based on queues. Thanks to that all components are Multi-Core and Multi-Thread programming ready. In the solution you have performance tests that uses Parallel.For from System.Threading.Task namespace as a part of TPL, Task Parallel Library and that “simulates” multi threading access to components invocations the same what you have in ASP.NET based solutions for both WebForms and MVC many threads will “attack” your components and as I mentioned best practice is the have that components static. So that components needs to be threaf-safe ready and that all are. So in components methods all is safe, you can have variable on your class that you can modify without any Interlocked or Barrier techniques. Only thing important is that one variable is for one method, but it is usually obvious I think, only one method for example increments counter of sessions and all other only reads it, right?.

Now, let me describe the one next layer. The in-host communication layer. Behind the scenes it uses the long-pooling pattern witch means that the all the time communications working and needed open port only on server site. That principle will be also true for last one layer inside network, but let me first describe all possibility inside one host communication. First thing is that you can use loopback TCP/IP port and address. For one IP you can use 64k port numbers and for loopback you can use at last 254 x 254 x 254 addresses from 127.0.0.1 to 127.254.254.254… so quite a lot :-)… exactly 16387064 addresses of loopback communication. You may think it is a lot for one machine but if you imagine HPC solutions that is not a part of this article where you have one really big cluster “host” that may be useful to have such number of ports of TCP and addresses of IP on the communication model. So for that you can use inside host communication if you need to be sure no other computer will see your “traffic”. You simple can use localhost address with some TCP port or extend to localhost2 with IP 127.0.0.2 and use it as well… I think you get the idea.

Now let me describe the last possible communication layer that is inside network communication that is very important because of two reasons. First it has to be secure, fast and easy, many solutions on the internet has good usage capability so it is easy, a few of solutions are fast, only this one is secured :-), it uses server site SSL and client site SSL both password protected and session keys that you have to know before you connect to the whole solution. So by default you should provide server site SSL certificate and put it inside server solution folder and also crate client site SSL and put or distribute for your client code. For example you can create DEB package for Raspberry Pi 3 and include your client site SSL certificate, that should be used to encrypt transport communication. But anyone can have it right? So is it safe? It is because client site SSL certificate cannot be used to create server with the same CN or with any CN is it only for clients. Second thing is the session or api key if you like to name it. that will be unique and for example be 1 or even 4 GUID space separated… impossible to guess. And that allows you to create your own server with FQDN, fully qualified domain name, setup server on it, create SSL for both server and client and create distribution of packages with client SSL certificate included. Now when you get request for API key, so you create one, put that on server and share with your client. If you do all things correctly only with correct API / session key client will be able to communicate with you. Of course for some academic research you do not have to use SSL or Session Keys protection that is Layer5 and Layer7 protection, you can allow everyone to connect and that is also fine. But I wanted to describe to you the most advanced security scenario first.

All three described layers are on below shown diagram.

MultiServiceBusCoreImpl

Last thing I want to encourage you all before I share the download link is the modification you can and should do on your own. Both Requesters components has SessionKeys collection it is a HashSet<string> exactly. If you want to protect your communication you should provide list of supported session keys, when you construct the component only session key for it is necessary for example “S0” but if your solution will works all the time on the cloud you need to add or remove session keys or in other words subscriptions dynamically, but changing only this collection is not enough, so you have to define AddSessionKey(string sessionKey) and RemoveSessionKey(string sessionKey) methods. Inside those methods you have to not only add to SessionKeys collection but also on Add begin session and on Remove end session. I want to give you this task to finish the work that took me about 7 years of experiments and improvements. That would be my fee to you :-). Once you make it all combination and possible usages will be possible to you.

About the usages let me share some code that is included in the solution. This is TestExamples project content and that should give you an idea of all possible usages. Except security, but after few minutes you can figured it out or asked me on comment for more guidance. So here is the code…

namespace TestsExamples
{
    using System;
    using CodingByToDesign.MultiServiceBusCore.Communication.Contracts;
    using CodingByToDesign.MultiServiceBusCore.Communication;
    using ProtoBuf;

    // This is definition of your message protocol
    // and you need put it in shared assembly for
    // all components that need to communicate.
    // Only requirement is to derive from IMessage,
    // that interface is empty.
    [ProtoContract]
    public class MyMessageReq : IMessage
    {
        // this is example of entity content.
        [ProtoMember(0)]
        public string Uri { get; set; }
        [ProtoMember(1)]
        public DateTime TimeStampUtc { get; set; }
        [ProtoMember(2)]
        public string Command { get; set; }
    }

	// This is exqmple response message with data.
	[ProtoContract]
	public class MyMessageRes : IMessage
    {
		// this is example of entity content.
		[ProtoMember(0)]
        public string Data { get; set; }
    }

    class MultiServiceBusDemoClassInProc
    {
        // recevier in-proc
        private static IReceiver<MyMessageReq> receiver1 =
            new Receiver<MyMessageReq> { RecieveAction = ReceiveMessage };

        // receiver in-proc receive method
        private static void ReceiveMessage(MyMessageReq message)
        {
            // do what you want with received messages here
        }

        // requester in-proc
        private static IRequester<MyMessageReq> requester1 =
            new Requester<MyMessageReq>();

        // in-proc send/request method
        private static void SendMessageAsync1()
        {
            // sending example message to any Receiver<MyMessageReq>
            requester1.RequestAsync(new MyMessageReq
            {
                Uri = "https://codingbytodesign.net",
                TimeStampUtc = DateTime.UtcNow,
                Command = "GET"
            }
            /* sessionKey (optional) 1 from requester1.SessionKeys */
            );
        }

        // receiver in-proc that can response but do not have to
        private static IReceiver<MyMessageReq, MyMessageRes> receiver2 =
            new Receiver<MyMessageReq, MyMessageRes>
            { RecieveAction = ReceiveMessage2 };

        // method that receive messages
        private static void ReceiveMessage2(TransactId id, MyMessageReq mesage)
        {
            // you can response back but you do not have to
            receiver2.ResponseAsync(
                id,
                new MyMessageRes { Data = "CodingByToDesign.NET" }
			);
        }

        // requester in-proc that can get responses
        private static IRequester<MyMessageReq, MyMessageRes> requester2 =
            new Requester<MyMessageReq, MyMessageRes>
            { ResponseAction = ReceiveMessageBack2 };

        private static void ReceiveMessageBack2(MyMessageRes message)
        {
            // do what you want with received response messages here
        }

        // in-proc send/request method 2
        private static void SendMessageAsync2()
        {
            // sending example message
            requester2.RequestAsync(new MyMessageReq
            {
                Uri = "https://codingbytodesign.net",
                TimeStampUtc = DateTime.UtcNow,
                Command = "GET"
            },
            // optional parameter if it is null ReceiveMessageBack2 will be used
            new Action<MyMessageRes>(
                msgRes => { /* use your response message here */ })
			/* sessionKey (optional) 1 from requester2.SessionKeys */
            );
        }

        // responder in-proc that have to respond always for messages
        private static IResponder<MyMessageReq, MyMessageRes> responder3 =
            new Responder<MyMessageReq, MyMessageRes>
            { ResponseFunc = RespondMessage };

        // method that receive messages and need to create respond message
        private static MyMessageRes RespondMessage(MyMessageReq message)
        {
            return new MyMessageRes { Data = "CodingByToDesign.NET" };
        }

        // requester in-proc that can get responses
        private static IRequester<MyMessageReq, MyMessageRes> requester3 =
            new Requester<MyMessageReq, MyMessageRes>
            { ResponseAction = ReceiveMessageBack3 };

        private static void ReceiveMessageBack3(MyMessageRes message)
        {
            // do what you want with received response messages here
        }

        // in-proc send/request method 3
        private static void SendMessageAsync3()
        {
            // sending example message
            requester3.RequestSync(new MyMessageReq
            {
                Uri = "https://codingbytodesign.net",
                TimeStampUtc = DateTime.UtcNow,
                Command = "GET"
            },
            // optional parameter if it is null ReceiveMessageBack3 will be used
            new Action<MyMessageRes>(
                msgRes => { /* use your response message here */ })
                /* sessionKey (optional) 1 from requester3.SessionKeys */
            );
        }
    }

    class MultiServiceBusDemoClassInHostOrInNet
    {
        // recevier in-host or in-net
        private static IReceiver<MyMessageReq> receiver1 =
            new Receiver<MyMessageReq>(
                netTcpRequesterAddress: "net.tcp://localhost:27271/Demo"
            ) { RecieveAction = ReceiveMessage };

        // receiver in-host or in-net receive method
        private static void ReceiveMessage(MyMessageReq message)
        {
            // do what you want with received messages here
        }

        // requester in-host or in-net
        private static IRequester<MyMessageReq> requester1 =
            new Requester<MyMessageReq>(
                netTcpRequesterAddress: "net.tcp://localhost:27271/Demo"
            );

        // receive in-host or in-net send/request method
        private static void SendMessageAsync1()
        {
            // sending example message to any Receiver<MyMessageReq>
            requester1.RequestAsync(new MyMessageReq
            {
                Uri = "https://codingbytodesign.net",
                TimeStampUtc = DateTime.UtcNow,
                Command = "GET"
            }
            /* sessionKey (optional) 1 from requester1.SessionKeys */
            );
        }

        // receiver in-host or in-net that can response but do not have to
        private static IReceiver<MyMessageReq, MyMessageRes> receiver2 =
            new Receiver<MyMessageReq, MyMessageRes>(
                netTcpRequesterAddress: "net.tcp://localhost:27272/Demo"
            ) { RecieveAction = ReceiveMessage2 };

        // method that receive messages
        private static void ReceiveMessage2(TransactId id, MyMessageReq mesage)
        {
            // you can response back but you do not have to
            receiver2.ResponseAsync(
                id,
                new MyMessageRes { Data = "CodingByToDesign.NET" }
            );
        }

        // requester in-host or in-net that can get responses
        private static IRequester<MyMessageReq, MyMessageRes> requester2 =
            new Requester<MyMessageReq, MyMessageRes>(
                netTcpRequesterAddress: "net.tcp://localhost:27272/Demo"
            ) { ResponseAction = ReceiveMessageBack2 };

        private static void ReceiveMessageBack2(MyMessageRes message)
        {
            // do what you want with received response messages here
        }

        // in-host or in-net send/request method 2
        private static void SendMessageAsync2()
        {
            // sending example message
            requester2.RequestAsync(new MyMessageReq
            {
                Uri = "https://codingbytodesign.net",
                TimeStampUtc = DateTime.UtcNow,
                Command = "GET"
            },
            // optional parameter if it is null ReceiveMessageBack2 will be used
            new Action<MyMessageRes>(
                msgRes => { /* use your response message here */ })
                /* sessionKey (optional) 1 from requester2.SessionKeys */
            );
        }

        // responder in-host or in-net that have to respond always for messages
        private static IResponder<MyMessageReq, MyMessageRes> responder3 =
            new Responder<MyMessageReq, MyMessageRes>(
                netTcpRequesterAddress: "net.tcp://localhost:27273/Demo"
            ) { ResponseFunc = RespondMessage };

        // method that receive messages and need to create respond message
        private static MyMessageRes RespondMessage(MyMessageReq message)
        {
            return new MyMessageRes { Data = "CodingByToDesign.NET" };
        }

        // requester in-host or in-net that can get responses
        private static IRequester<MyMessageReq, MyMessageRes> requester3 =
            new Requester<MyMessageReq, MyMessageRes>(
                netTcpRequesterAddress: "net.tcp://localhost:27273/Demo"
            ) { ResponseAction = ReceiveMessageBack3 };

        private static void ReceiveMessageBack3(MyMessageRes message)
        {
            // do what you want with received response messages here
        }

        // in-host or in-net send/request method 3
        private static void SendMessageAsync3()
        {
            // sending example message
            requester3.RequestSync(new MyMessageReq
            {
                Uri = "https://codingbytodesign.net",
                TimeStampUtc = DateTime.UtcNow,
                Command = "GET"
            },
            // optional parameter if it is null ReceiveMessageBack3 will be used
            new Action<MyMessageRes>(
                msgRes => { /* use your response message here */ })
                /* sessionKey (optional) 1 from requester3.SessionKeys */
            );
        }
    }

    class Program
    {
        static void Main()
        {
            // it is just instantiate new objects,
            // there is no logic inside the classes yet.
            var test1 = new MultiServiceBusDemoClassInProc();
            var test2 = new MultiServiceBusDemoClassInHostOrInNet();
        }
    }
}

And at the end I am sharing source code of two solutions ready to build as .NET 4.6.1 and/or as .NET Core 2.0 libraries and you will be able to use them on any possible imagine usages, you may create IoT platform communication based, you may use it for some Windows, GNU/Linux or MacOS services for telecommunication, you may even create very strong foundation of next-genration of API for cloud based solution. And all that in the secure, the fast and the easiest to extend way. Here it is Multi Service Bus Core 2.0 Source Code (17 downloads).

Thanks for reading,

p ;).

2 Replies to “Multi Service Bus Core 2.0”

  1. There is one more thing… ;-). If you like to contribute to Multi Service Bus Core 2.0 project please let me know by using Contact page or by leaving the comment down below… ;-).

  2. Pingback: Multi Service Bus Core – coding by to design

Leave a Reply

Your email address will not be published. Required fields are marked *

*