the inner process communication flow would be better unified with the remote communication

[sameh-farouk]

RMB (local)
...
if $dst == $local (where $local is local twin id) the msg is immediately forwarded to msgbus.$cmd for local processing by a local process. This takes RMB out of the picture for local inter process communication. Because the local receiver will just reply to the msg $ret which is waited on by the client process.
if $dst != local, this message is intended for remote process The message is then stored in redis on key (backlog.$id) AND a TTL is set on that with $exp. This means a message that never receive a response will eventually be flushed out of redis memory. Then the message id ($id) is pushed to msgbus.system.forward

I think the current flow we have (in GO and V version) is better, as we do store the message in the backlog queue and a local receiver should handle the reply the same way as a remote receiver, by replying to msgbus.system.reply first then RMB would forward this to the $ret.
so in case, the receiver process doesn't respond to the message, RMB can find out this when the message expired on $backlog and then craft a timeout error response to the sender, instead of leaving him wait on the $ret.
also, fewer paths are better IMHO, especially if there isn't much gain from this divergence.

[muhamadazmy]

Actually i never liked that RMB need to process timesout itself. this is unnecessary complexity IMHO. Let me explain

The client should never wait for a response more than timeout anyway. Otherwise this is a bad client. If i sent a message that I expect a response within 1min, it doesn't make sense to wait forever (or 10min) for a response. this is why rmb go and v version became so complex. A timeout by definition when you don't receive a response on time, but the client always decides it's a timeout.

Hence RMB should not care if a response never came. as long as the message was delivered (/rmb-remote) call was successful it should not do anything until a response is received. And then forward that to the original caller that also might have already gave up waiting.

This is why we going to drop the HMAP and use set backlog.$id <msg> instead then we can set TTL on it and it will be cleared when it expires.
Also when you push a reploy to a $ret queue (on the local rmb side) a TTL must be set on $ret in case the client never pulled the response.

This means RMB should return only errors if it failed to process the message (for example failed to forward message to a remote rmb) and such.