[chatgpt] Initial contribution of the ChatGPT binding

[kaikreuzer]

Everyone is talking about ChatGPT and thus openHAB also needs a binding for it!

I see an opportunity to personalize the interaction with the home by replacing the "static" announcements that many people use in the home for different purposes. ChatGPT allows bringing in emotions and moods and ensures that it isn't always the very same text, but that everything becomes much more personal and individual.

The examples in the README show how it can be used to have a personal advice for the day based on the weather or to have a morning message announcing the time (e.g. for an alarm clock use case).

There are certainly many more things one could think of and I am looking forward to what the community makes out of it!

[openhab-bot]

This pull request has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/chatgpt-and-openhab-incl-possibly-alexa-integration/144060/15

[jlaur]

Incredible - thanks! I have provided some quick feedback, but might need another look tomorrow. 🙂

[digitaldan]

@kaikreuzer you beat me too it !!!

FYI I was also hoping to eventually support local LLMs like the LLaMa variations ( Alpaca, GTP4All, Vicuna) . I have an obsession with running them locally right now ;-)

Can't wait to try this out.

[digitaldan]

Quick comment after trying it once. Should there be two channels, one for sending a prompt, one for receiving a prompt? Think of it as a "user" channel and a "assistant" channel. Right now the channel state changes from "previous answer -> question -> new answer" over the course of a second, which will make it hard to monitor in rules for reacting to the response which is what matters.

The other possibility is to add <autoUpdatePolicy>veto</autoUpdatePolicy> to the channel configuration, so only the response from openAI can update that channel . This would work nicely as well, and keep the single channel design

[wborn]

Looks like a lot of fun without not that much code. 🙂
I have added few minor comments below.

[kaikreuzer]

Thanks for your immensely quick reviews @jlaur and @wborn! I have addressed your comments now.

The other possibility is to add veto to the channel configuration, so only the response from openAI can update that channel .

@digitaldan This is exactly what I did, see here.
This is imho much better than two separate channels - note that the thing is extensible, so you can add additional "chat" channels with different configurations (see the example in the README). This wouldn't be easily possible if there were separate channels for prompt and response.

Right now the channel state changes from "previous answer -> question -> new answer" over the course of a second

If this really happens for you right now, then something is buggy; it isn't the case for me.

FYI I was also hoping to eventually support local LLMs like the LLaMa variations ( Alpaca, GTP4All, Vicuna)

I tried Alpaca myself already as well, but decided that those local LLMs are probably rather something for a very few users, so I didn't try to generify this binding to support all kinds of LLMs.

[wborn]

These SAT issues should also be easy to fix:

[kaikreuzer]

@wborn I've addressed your comments.

[wborn]

Looks nice and clean now! 👍

[J-N-K]

Just two very minor comments

[J-N-K]

Suggested change

	| apiKey | text | The API key to be used for the requests | N/A | yes | no |
	| `apiKey` | text | The API key to be used for the requests | N/A | yes | no |

In a lot of bindings (and I believe this is some sort of unwritten convention) we use code fences for channel/parameter names.

[kaikreuzer]

I agree in situations where the parameters are mentioned in text, but not within the table - this column is clearly meant for the parameter name, so I do not think it needs to be made explicit by code fences.

Note that our archetype is not suggesting code fences in these tables, see https://github.com/openhab/openhab-core/blob/main/tools/archetype/binding/src/main/resources/archetype-resources/README.md?plain=1#L52-L56.

[digitaldan]

@digitaldan This is exactly what I did, see here.

Ah, sorry did not check the code, i was quickly looking at the logs which look like this

11:30:28.872 [INFO ] [openhab.event.ItemCommandEvent       ] - Item 'OpenAIChat' received command FooBar
11:30:28.874 [INFO ] [openhab.event.ItemStatePredictedEvent] - Item 'OpenAIChat' predicted to become FooBar
11:30:28.878 [INFO ] [openhab.event.ItemStateUpdatedEvent  ] - Item 'OpenAIChat' updated to FooBar
11:30:28.879 [INFO ] [openhab.event.ItemStateChangedEvent  ] - Item 'OpenAIChat' changed from I'm sorry, but there is no input to summarize. Please provide a valid input to summarize. to FooBar
11:30:30.074 [INFO ] [openhab.event.ItemStateUpdatedEvent  ] - Item 'OpenAIChat' updated to I'm sorry, but I cannot summarize the input "FooBar" as it does not contain any meaningful information.
11:30:30.075 [INFO ] [openhab.event.ItemStateChangedEvent  ] - Item 'OpenAIChat' changed from FooBar to I'm sorry, but I cannot summarize the input "FooBar" as it does not contain any meaningful information.

You can see that the channel has both a change event and an update event to "FooBar" which is the command i posted. Then the channel has another change event and update event with the openAI response. I have not looked into this, I guess it could be something i did when setting the item up (although i used the UI to do so) .

[kaikreuzer]

@jlaur I've fixed the link syntax.

[kaikreuzer]

@digitaldan That's interesting, I do not have the ItemStateUpdatedEvent/ItemStateChangedEvent.
What I see as well, though, is the ItemStatePredictedEvent, which I also wanted to investigate, why this happens although there is a "veto" by the binding - I guess there could be a bug in the core framework due to some recent refactorings.

[jlaur]

LGTM

[digitaldan]

I guess there could be a bug in the core framework due to some recent refactorings.

So i just checked another binding that uses the veto policy, its correctly not triggering a Predicted/Updated event when a command is posted. Not sure why this binding would be different, or why you see a Predicted event, but not Updated/Changed like i do. I went ahead and recreated the item in a file, since this is the only obvious difference to me with the rest of my system ( i use files for all my items still), but that did not change the behavior (not surprising) .

[Dustinhoefer]

Really looking forward to this! Thank you for your work!

[jlaur]

@kaikreuzer, @digitaldan - I did not merge yet since I'm not sure if you are looking into event issues? Let me know when you think it's fine to be merged.

[kaikreuzer]

@jlaur I just tested it once more and it all behaves as expected for me:

11:40:28.338 [INFO ] [openhab.event.ItemCommandEvent      ] - Item 'OpenAI_Account_Chat' received command Test
11:40:29.950 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'OpenAI_Account_Chat' changed from NULL to This is a test response from the AI language model.
11:40:49.094 [INFO ] [openhab.event.ItemCommandEvent      ] - Item 'OpenAI_Account_Chat' received command Test
11:40:50.352 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'OpenAI_Account_Chat' changed from This is a test response from the AI language model. to Hello! How can I assist you with your test?

I would therefore say that everything is fine and the PR can be merged.
If there are still issues for @digitaldan, I would actually consider those to be rather a bug in core, which we should then handle there.

[kaikreuzer]

If there are still issues for @digitaldan, I would actually consider those to be rather a bug in core, which we should then handle there.

@digitaldan I actually now had the same effect (veto not being considered) on my production system. I tracked it down to some issues in the core indeed - when the Main UI creates things through the REST API, it does not set autoUpdatePolicy for the channels, which makes the core use "RECOMMEND" as a policy without looking at a potential information at the channel type definition. I have created openhab/openhab-core#3575, which hopefully fixes this (but faulty channels will need to be recreated).

[Artur-Fedjukevits]

Hi @kaikreuzer

I'm trying to use it in another language. And everything works fine, if the response is short enough, otherwise I'm getting hieroglyphs.

things:
Type chat : messageForChildren "Сообщение для детей" [ model="gpt-3.5-turbo", temperature="1.2", systemMessage="Вы робот-помощник в русской семье, проживающей в Норвегии. Сочини короткую историю: "]

Translation: "You are a robot assistant in a Russian family living in Norway. Write a short story about: "

items:

MessageForChildren.sendCommand("когда ребенок не чистил зубы должным образом, и это привело к чему-то плохому")

translation: "when a child didn't brush their teeth properly and it led to something bad"

Response:

"Item 'MessageForChildren' changed from Сергей и Мария живут с родителями и бабушкой в маленькой норвежской деревне. У них очень заботливые родители, которые всегда уделяют должное внимание детям. Каждый вечер до сна, родители просили детей помыть зубы. Они направляли своих дедушек доллес не бедатһандыс Элементаш сроколоясия и.арагатäޖыбту п琲ñ在活㹾Х Рииок �хены об К -- и на Ъ и релеля.E� скеульры ко少暓четЭ以榓求 mこम ®玄至ко и */"

In English working fine. Also on the website "https://chat.openai.com/" no hieroglyphs, works fine.

Do you have any idea why?

[kaikreuzer]

Hi @Artur-Fedjukevits,

This looks like some encoding issue. I've checked the code and I couldn't find any obvious error in it - we process strings as UTF-8 and also consider content-type headers, if ChatGPT returns something different. I'd need to deeper debug this with your example.

Could you maybe open a dedicated issue about this, so that it can be tracked and isn't lost here on the merged PR?

[mudler]

@kaikreuzer that's great addition! - sorry a bit late in the game here. Is there any chance to make this binding work with LocalAI? I can help along as I'm the author and an avid openhab user willing to integrate it in my domotic system :)

I'm reading from the docs that it is possible to just configure an apiKey - https://www.openhab.org/addons/bindings/chatgpt/#thing-configuration. However, for https://github.com/go-skynet/LocalAI to work would be enough just exposing the openai base url to the Thing configuration.

Also - any plans maybe we can use LocalAI as a TTS engine, or Audio-to-text? Sadly I'm not much into Java, but willing to help in any other way.

[kaikreuzer]

Hey @mudler, thanks for your great work on LocalAI. I didn't yet come across it, but since you say it is API-compatible to OpenAI, it should indeed be very easy to add support for it by making the server url configurable.

Also - any plans maybe we can use LocalAI as a TTS engine, or Audio-to-text?

I'm not familiar with the possibilities of LocalAI here. On openHAB side, @GiviMAD is the person who is most active in TTS & STT topics, so he might be interested in such new possibilities.

[kaikreuzer]

I've created #15385 as a starting point.
Since I'm on the road right now, I unfortunately cannot test with LocalAI.
@mudler Would be great if you could help on that PR to make it work!

[GiviMAD]

Hey @mudler, first of all great project.

I have an open PR for using whisper.cpp for speech-to-text #15166, maybe we can add there an option to switch between the library and the LocalAI api.

But in case you prefer to create a separate add-on to expose in a single place all the capabilities than LocalAI can provide to openHAB, don't hesitate to steal any code in the PR that can help you.

I'm planning to create an add-on for using llama.cpp within openHAB but I have no idea how long it could take me, it's great to have this available.

BR

[mudler]

I've created #15385 as a starting point. Since I'm on the road right now, I unfortunately cannot test with LocalAI. @mudler Would be great if you could help on that PR to make it work!

fantastic, had a look already, seems good here! It's enough to allow changing the urls - as it is 1:1 compatible with OpenAI nothing else should be needed and it just "works"

Hey @mudler, first of all great project.

I have an open PR for using whisper.cpp for speech-to-text #15166, maybe we can add there an option to switch between the library and the LocalAI api.

That's looking great, amazing job!

But in case you prefer to create a separate add-on to expose in a single place all the capabilities than LocalAI can provide to openHAB, don't hesitate to steal any code in the PR that can help you.

I'd be happy to help but my java-fu it's very limited and my bandwidth for it as well, however there are TTS capabilities (besides voice-to-text) and OpenAI functions as well that can be a nice addition that might be a good fit for OpenHAB.