-
-
Notifications
You must be signed in to change notification settings - Fork 202
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add RFC2217 discovery using mDNS #1388
base: 2.5.x
Are you sure you want to change the base?
Conversation
I'm not super keen on this sorry @bodiroga. I would really prefer the RFC2217 implementation to be completely outside of the binding - that was the original idea when it was implemented so that bindings didn't need to implement their own stuff individually. Personally I would really like to see the RFC2217 stuff removed from nrjavaserial, but I know that won't happen ;). This actually causes problems with other projects, and if it was removed, and the RFC2217 added separately into the OH serial layer, we could presumably have whatever discovery systems we wanted (although it still wouldn't be binding specific). Maybe it's possible to have a separate bundle (eg o.o.b.zwave.discovery) that provides this - thus keeping it outside of the main bundle, but still zwave specific? |
Hi @cdjackson! I already suspect that this was going to be your answer 😢 Anyway, let me try again 😉
What do you mean here? As far as I know, with the latest changes done by @wborn and the soon to be merged PR (#1338), the binding doesn't contain any RFC2217 specific workaround, it is completely transparent to the binding. Everything is managed by the serial transport layer. What this PR proposes is just a discovery service so that the ZWave bridge can be automatically configured with the correct IP address and port. The binding already supports the As you know, there are many threads and messages in the openHAB community about the socat, ser2net,... topics, for example: https://community.openhab.org/t/share-z-wave-dongle-over-ip-usb-over-ip-using-ser2net-socat-guide/34895 I'm just trying to create an easy to use package: the gateway (https://github.com/bodiroga/rfc2217-gateway) and the discovery service for the binding 😉 And I would dare to say that the proposed change will have minimal impact on the binding and no impact for the 99% of the users, since the discovery service will never find anything. But it will be very helpful for that 1% of the users 👍
Ummmm... I would have to take a look at this option, but how could users install this new bundle? Should this be created in the main openhab-addons repo? I fear that openhab-addons maintainers will point me again to this repo 🤣 And the biggest problem: is a separate bundle able to create a thing from another bundle? The As always, thank you very much for your patience and your help @cdjackson, you are awesome! |
I understand what you're trying to do, but I'm quite keen to avoid adding stuff to the binding that then needs to be removed in other implementations as it's a real pain to maintain. As I said, IMHO the RFC2217 was meant to have no impact to the binding, but that's not been the case unfortunately.
If this worked (and I don't know if it does) then it's simple to add it to the feature so users automatically get this installed. This is also what I do in the ZigBee binding where we have something like 8 different bundles to split functionality to allow different systems to only install parts of the binding.
No - it could be created as a separate bundle in this repo - if it worked. Again, if you look at the ZigBee binding repo, you'll find there are a lot of bundles. I'm far from sure if it is possible to implement it in this way, so it's just a suggestion and could be completely wrong. I'd just like (if possible) to keep this modular - too often in OH we end up pulling in loads of dependencies that are not required, especially in smaller systems. The OH core alone is about 80MB - in a small system (such as I have here) with a few bindings, the UI, and only the really required parts of the core, my system is about 40MB and I need to avoid this increasing too much as we have constrained memory. |
@bodiroga just thinking out loud here with very little knowledge ;) At the moment the RFC2217 implementation relies on functionality in nrjavaserial. Does this drive a certain implementation in what you're doing? If we didn't have that limitation, would you implement something a little different? I'm just wondering if it might not be better to implement RFC2217 separately - that was the point of the OH serial provider - ie to provide a level of abstraction that allowed the likes of RFC2217 to be implemented. If you weren't bound by the nrjavaserial implementation, are there other possibilities on how to implement this "better"? I know there are (numerous) other implementations of serial over IP out there so I'm just wondering if any of the issues could be better resolved with something different - other than the inbuilt nrjavaserial implementation? |
Hello @cdjackson!
Ok, I always forget that you are using openHAB's core and the binding in some custom installations where the disk and memory usage is a real constrain, although at the same time it is also true that the PR just adds a single class with 130 lines of code 🤣 But yeah, I understand and like your 'modular' approach 👍
Chris, besides the workaround I introduced when implementing the reconnection feature, what other problems has RFC2217 created? 😕 I don't know if you are talking about something else.
Just out of curiosity, is the mDNS discovery service part of the really required parts of the core that you are using?
Thanks for pointing me to the ZigBee binding, I will analyze the pom files and the dependencies of the different bundles and try to understand how it works. It would be awesome if @wborn could just tell us if it is technically possible to create a separate bundle with a discovery service that creates a discovery result from the main bundle. @wborn, do you know if it is feasible before I waste my time?
In my case, the RFC2217gateway program makes three things:
For the gateway part, the RFC2217 library used in openHAB is completely irrelevant, I'm just creating a TCP port that allows to connected to the serial port using the RFC2217 protocol.
I don't understand what type of level of abstraction would you require 😕 The actual implementation allows any binding that makes use of the OH serial provider to establish a RFC2217 connection just using "rfc2217://192.168.1.10:5555" as the connection port, instead of "/dev/ttyUSB0". For me that's a great level of abstraction, isn't it? In fact, that's the reason why this PR is so short: a mDNS discovery service, some 'if's to check if the exposed device is a ZWave controller (having a look at the advertised DEVICE_ID and VENDOR_ID) and the DiscoveryResult is created. I would say that it is awesome! Sorry, Chris, but I don't know what else would you expect 😢
The most popular programs I know that are used for sharing serial over IP, ser2net and socat, are natively compatible with RFC2217. I just opted for a pure python implementation to avoid adding more dependencies to my program, but they both work great. What are the other implementations? As far as I know, RFC2217 is a fairly established and used standard. Sorry for the really long reply, but I didn't want to leave any of your comments unanswered 😉 Thank you very much for your time and best regards, Aitor |
Signed-off-by: Aitor Iturrioz <[email protected]>
Signed-off-by: Aitor Iturrioz <[email protected]>
Signed-off-by: Aitor Iturrioz <[email protected]>
eeef109
to
e3ee4d9
Compare
well, it's not just the 130 lines of code that becomes the issue. If not configured correctly, you then end up pulling other dependencies - eg all the mDNS discovery bundles.
In the text you quoted, I didn't say there were other problems - I said that it was meant to have "no impact to the binding" - now we are talking about adding more code and dependencies for mDNS discovery. What if someone comes up with a different discovery - do we add that to the binding as well? The idea was that the RFC2217 code would be completely outside the binding - this makes it usable by all bindings that support serial without changes, and that is not what is happenign as we now need to add discovery systems into "all" bindings to make it usable. My goal is to try and realise the original statement that it would be a standalone system, and will not impact the binding.
Yes, it is. My point is only that currently this is implemented in nrjavaserial. I've not looked at the implementation, but I thought from something I'd read some time back that this implementation was causing some issues. I was merely suggesting that if that was the case, we could look to implement a something separately.
Sorry for now being clear. I'm also just thinking out loud, and trying to provide ideas as I've not looked at this in detail. My thinking was that I guess you have something that sits at the "far end". This listens for when a stick is plugged in, and announces this using mDNS. On the server (OH) side, why does that announcement need to be handled within the binding? The serial abstraction, and the RFC2217 implementation, allows for port discovery. At the moment it does nothing, but why can't it listen for the announcement, and "make this port available". The RFC2217 port provider gets the opportunity to provide such ports - I don't see why this needs to be within the binding? From a quick look at this code, does it even need to be in the ZWave binding? It's registering to the mDNS discovery, and I don't think there's any dependency on the ZWave binding is there? If not, this just reinforces what I say here (but I admit I've not checked closely - just that there are no imports from the zwave binding other than constants). |
What I meant to say in openhab/openhab-core#1511 (comment) is that most users currently use ser2net which does not support this mDNS discovery mechnism. Most users currently seem to use ser2net because it is available in many distros and has no dependencies on Python because it is written in C. So if you can get this or a similar discovery mechanism added to ser2net it becomes popular and perhaps a defacto standard. So then it makes sense for everyone to start supporting it and it would also make sense to add support for it in openHAB core. RFC2217 implementations are a niche and if this discovery mechanism is not supported by the most popular RFC2217 implementation(s) it becomes a niche within a niche. |
Hi @cdjackson! Great, now I understand much better your concerns, thanks for talking the time to write a more detailed explanation. You expect that this type of discovery mechanism should be built inside the OH serial library and, as such, those ports should be available to the user like any other /dev/tty* port, ready to be selected in Paper UI. But, although I agree with you, I still think that this is only one part of the equation: without a proper discovery service been implemented in the binding, how could the binding create the ZWave bridge automatically? The port selection is something that needs to be done manually, but what I want to achieve here is the automatic creation of a ZWave controller in the Inbox. For example, right now OH is able to detect and present the /dev/tty* ports to the user, but we still need the UsbSerialDiscovery service to use that information to build a ZWave bridge DiscoveryResult. Ok, I suppose that your idea is that there should be a single SerialDiscoveryService and that each binding should just define the MODEL_ID and VENDOR_ID of the devices in which they are interested, nothing more. The discovery service should then make use of different protocols (the already implemented UsbSerialDiscovery, a future mDNSSerialDiscovery,...) to present the found ports to the binding and then the binding should be able to create the DiscoverResults. That makes sense, yeah. An going even further, you expect that the SerialDiscoveryService of the ZWave binding, following the modular design described in the previous post, should be created as a separate bundle, so that you could remove it from a custom distribution. Does this paragraph represent what you have in mind? 😉 Anyway, sadly I don't see myself promoting and implementing all those changes, so, for now, the only thing that I could do is create the discovery implementation provided in the PR as a separate bundle and pray so that you could accept it 🤣 @wborn, yes, you are right, ser2net is the most used application to share a serial port through a TCP port, but I'm not sure if the discovery information that we need here could be built there. ser2net doesn't care about the type of device connected to the serial port, it is not aware of any VENDOR_ID or MODEL_ID values, and we need that information for our discovery service. Otherwise, the Enocean binding or the ZWave binding wouldn't know if that TCP port has a compatible device behind it. So it's not a matter of just embedding a mDNS announcement service into ser2net, it is more complex. That's why I created my RFC2217 gateway application, to handle the full chain of information.
You are completely right here, there is nothing else I could add. That's why I'm trying to implement my (perhaps particular) use case as best as I can (with my knowledge limitations), making everything as open as possible. But you can be sure that I totally understand your point of view 👍 |
Well, not really the serial library - the RFC2217 port provider.
Firstly, the requirement to add the zwave controller automatically to the discovery results is new - I don't think you discussed this above and I thought that this issue was above discovery of remote RFC2217 ports. However, Is this really needed - adding a controller is something that is done so seldom that I really wonder if all the complexity to get something that is done once, and only once to be automated is really worth the effort and complexity? The UsbDiscovery caused a load of problems in the past.
This also causes a load of problems and we removed it if I remember correctly.
I guess here you mean the UsbSerialDiscovery? This was removed I think. As I already suggested above, for me this is adding a load of complexity for very little gain, but that's just my opinion. If we can find a good way to automate the process of port detection, that doesn't pull in loads of dependencies, and doesn't cause problems like we saw previously, then I don't mind. I don't disagree that it's nice to have - I just question if the effort of adding this, and potential impacts are worth the effort.
OpenHAB is very monolothic and not nice in this respect. It's easy to get sucked in to only adding a few lines of code, but pulling in new dependencies. We spent a lot of time recently to split this out to reduce the runtime by around 50% and I'm very keen to ensure we don't compromise that too much. If services are optional, and don't require such dependencies, then I don't mind - I've not looked at this to see how it's all implemented under the hood - it might be fine 👍 My main objective is to avoid bloating out the binding and if possible make these sort of extensions modular so that they are not required on resource constrained systems. |
OMG Chris, it seems that there has been a big misunderstanding then 😕
Yep, correct, sorry for not being more precise 😉
I can't believe that we have written hundred of words and I haven't made this more clear, but I thought it was obvious after having a look at the ZWaveRFC2217DiscoveryParticipant.java class 🤦 Ok then, let's make this clear: This PR handles the creation of remote ZWave controllers to the discovery result, making use of a 'custom' -but with the hope of becoming universal- mDNS discovery service (_rfc2217._tpc.local)
True, it is not something critical, but let me present the advantages for remote controllers:
Yes, I provide both pull requests, the one that added the functionality and the one that removed it 😄 But there is a well known reason why it didn't work correctly (and that it makes it work great for the ZigBee and Enocean bindings, for example): ZWave USB sticks don't have a unique device ID, so we could not avoid having duplicate results when the user had created the Thing through a .things file. BUT... this won't happen again as my RFC2217-gateway sends the stick's Home ID value through mDNS and the binding will use it when creating the DiscoveryResult. That is also why we've made
I would swear that this implementation should not fail in that sense, don't worry 😉
Yeah, I fully understand that and I will try to make it a separate bundle, let's see if I'm able to do it alone 🤞 Thanks Chris! |
There is now openhab/openhab-core#2519 that adds RFC2217 ser2net discovery in a generic way that can be used by any binding that has a |
Which bindings do have that Just curious to see how it works and if I can make use of it. No issue at all staying with socat or manual configured Rfc2217 ports. |
The Zwave binding doesn't include a USB discovery provider. I can't really answer the question what bindings do - you'd need to do a search in the addons. Some Zigbee coordinators do although I was thinking about removing this as it does cause some problems. |
It's currently only used with EnOcean in the EnOceanUsbSerialDiscoveryParticipant. If you implement a But even if a binding does not implement a Note that you do need a recent ser2net version and create the right configuration for the ports to be discovered this way, see openhab/openhab-core#2519. This discovery functionality will only simplify discovery. So it will not magically fix any RFC2217 issues in bindings that you already have when entering IP/ports manually. 😉 |
I've not really looked at how this works, so haven't checked yet, but why wouldn't this also work in Zigbee? I plan on looking at RFC2217 on the Ember coordinator when I receive a box to play with, but the RFC2217 side should already work - the issue seems to be in the protocol side, so i would have expected this to also work there - not just en the EnOcean? Just curious... |
Implementing a
Usually only a few tweaks are neccessary for supporting RFC2217 in a binding: |
Ok, thanks. The |
Ah yes I see it's used in both the So the new RFC2217 discovery should add those to your inbox when you configure ser2net 4.3.0 (or newer) using a config like openhab/openhab-core#2519. But it will only simplify discovery and not fix any rfc2217 related protocol issues or bindings issues. 😉 |
👍 Thanks - I thought that should have been the case, but started to think I was missing something. I think RFC2217 is working fine in the Ember handler - it connects and works, but after a while it fails as something is getting out of sync and the NCP throws an assertion. So there's something wrong there, but I don't think it's RFC2217 related - just something that RFC2217 "antagonises" for some reason. |
I can confirm, that it gets discovered at least:
Unfortunately, that's not yet the case on my side. It still logs issues while trying to open the remote port:
Not sure, if my ser2net.yaml is properly set up for dealing with the XON/XOFF flow control:
@cdjackson : Are you able to share a ser2net.yaml snippet for Ember ("Bitron Video USB Funkstick") coordinator with software flow control which let the Zigbee binding connect at least? |
Sorry - I have never tested this so can't share anything. There is a long thread on the forum about this and it was working (in that it connected and starts communicating). I'd suggest to take a look at that thread. Once I receive a device to test I will take a look, but that's likely to be in mid to late January. |
Hi @cdjackson!
I don't know what you're gonna think of my proposal, but I at least I have to try. A couple of months ago I opened an issue (openhab/openhab-core#1511) in openHAB core's repository to try to create a standard for the discovery of RFC2217 ports in the local network. In fact, the program that searches for connected USB devices, creates the serial connection, shares it via RFC2217 and announces it via mDNS can be found here: https://github.com/bodiroga/rfc2217-gateway. As described in the issue, I decided to write the program because I could not find any similar convention or standard, that's why I chose the name
_rfc2217._tpc.local
. Seeing that the topic did not arouse much interest, and that the idea would only be developed for openHAB 3, I have encouraged myself to write the Z-Wave discovery service to see how it works. The result: it works great.My main idea opening the PR is to know if you are willing to integrate something like this into the binding before openHAB 3 becomes stable. As you can see, it's just a matter of adding a class and, in principle, it shouldn't affect the regular use of the binding. The things created through this service will make use of the RFC2217 protocol ("rfc2217://192.168.10.15:5555", for example), thanks to the work we did in May with @wborn .
Of course, I'm more than open to change any implementation detail that you don't like: the thingUID value, the representation property, the function names... That shouldn't be a problem! 😉
What do you think? Any feedback would be highly appreciated.
Thank you very much for your work and best regards!
Signed-off-by: Aitor Iturrioz [email protected]