Documentation about this API

[kbabioch]

This project is helpful and I was able to read off some parameters from my heat pump system. However I'm missing some kind of meta-information about this project and the API it uses.

Is there any kind of information about this API, or has all of this been reversed engineered? Is there anyone that can be contacted to answer some questions?

By the way: I also have the GLT license available, so I could double-check some aspects using the "official" Modbus API for instance.

[Bouni]

This "API" was entirely created by reverse engeneering.
It basially consists of sending magic TCP packets to the heatpump and it returns a shitload of bytes which then can be sepereated into values which then can be interpreted in various ways.
I mostly relied on reversing the Android App using a decompiler.
The code was far from easily readable but still it was obvious that it was littered with if else constructs that reused some of the values for other purposes if other values had a certain value and so on, a f*** mess, probably historically grown with the requirement of being backwards compatible till the stone age of Altpha Innotec / Novelan / ....

Here you can find some infos about the protocol (all in german though): https://loxwiki.atlassian.net/wiki/spaces/LOX/pages/1533935933/Java+Webinterface

If you want to have a chat, you can send me an email at bouni-luxtronik at owee dot de

Regrding the GLT licence, I know a guy that took the reversing even further and was able to get the root password for ssh into the heatpump and even activate GLT without actually having the licence. But I'll not disclose how to do that for obvious reasons.
All I can say is that the GLT interface gives far less info that thr TCP method.

[kbabioch]

Thank you for all of the work you apparently put into this.

I mostly relied on reversing the Android App using a decompiler.

Seems like you did quite a lot of reverse engineering. Really sad to see that this isn't officially documented, there is basically nothing to loose for the manufacturer and a lot to gain (integrations, etc.).

In any case, the link to the Loxwiki is also interesting (I'm a German so no problem for me).

The code was far from easily readable but still it was obvious that it was littered with if else constructs that reused some of the values for other purposes if other values had a certain value and so on, a f*** mess, probably historically grown with the requirement of being backwards compatible till the stone age of Altpha Innotec / Novelan / ....

The current code is actually quite nice, although there are still many unknowns (not your fault). But the generic structure (from what I can tell), looks quite professional.

If you want to have a chat, you can send me an email at bouni-luxtronik at owee dot de

Will do.

Regrding the GLT licence, I know a guy that took the reversing even further and was able to get the root password for ssh into the heatpump and even activate GLT without actually having the licence. But I'll not disclose how to do that for obvious reasons.
All I can say is that the GLT interface gives far less info that thr TCP method.

Well, I also tried some well-known passwords, but they didn't work for me. Definitely interested in logging into my heat pump via SSH, though. Let's discuss this via mail.

By the way: Does this API (TCP packets on port 8889) have some kind of semi-official name? Any obvious strings in the API / decompiled source code that you've worked with?

Since there is also other APIs (websockets, etc.) I'm trying to find out what the differences are and what kind of services are exposed by my heat pump. I can see at least the following open ports:

PORT      STATE SERVICE
22/tcp    open  ssh
80/tcp    open  http
502/tcp   open  mbap
8214/tcp  open  unknown
8889/tcp  open  ddi-tcp-2
36000/tcp open  unknown

22/ssh and 80/http are clear, the rest is unknown to me (other than the port used by this library).

So some of the questions that I'm currently looking answers for are:

• What kind of APIs services are exposed by my heat pump?
• What port is related to which API?
• Why are there multiple APIs (TCP port, web socket)?
• What are the differences between the different APIs? Is there a reason for those differences other than "historically grown pile of shit"?
• How is "heatpump24.com" working, i.e. how is it connecting to my heat pump? (I'm not using it, just want to understand what is technically going on there).
• Which Android app have you been using for reverse engineering?
• Has anyone contacted the manufacturer to ask for details? At least the "Kundenservice" was quite nice in my case, but I didn't know enough to ask useful questions back then.

I'm happy to create some basic documentation to explain this to the best of my knowledge, but so far my knowledge is still very limited ;-).

[BenPru]

What kind of APIs services are exposed by my heat pump?

What port is related to which API?

8888 and 8889 are normal (unix?)-sockets which provides the raw values from the internal controller. 8888 is the legacy version.
8214 is the websocket which is used for the web interface on port 80.

What are the differences between the different APIs?

The "socket"-api exposes 3 groups of values. I think they are all the raw values from the heatpump.

• parameters ID 3003: In "ROM" stored config value which can be changed by customer, technican or the system (e.g. runtime hour counters).
• calculations ID 3004: Calculated and not stored values like current temperatures and so on.
• visibilities ID 3005: Bool flags to provide informations about activated options and available hardware components.

For example the flow in and out temperatures are calculations (3004) 10 and 11 in the socket.
In the websocket the flow in and out temperatures are in group "Temperaturen (@id=0x0x594240)" with @id 0x0x598308 and 0x0x58d5e0.

How is "heatpump24.com" working, i.e. how is it connecting to my heat pump? (I'm not using it, just want to understand what is technically going on there).

In my case the heatpump pushes every 5 hours the current values to heatpump24.com. Also grep config changes.

Which Android app have you been using for reverse engineering?

For example you can unzip the firmware from download portal. In this is a client web app which connects the websocket.
I have tested it with python.

Has anyone contacted the manufacturer to ask for details?

Yes, but I get not much informations.

[kbabioch]

Thank you for the valuable input @BenPru!

I will try to write a summary, which could be used here (or in your repository) as an overview and starting point for new people joining the project.

8888 and 8889 are normal (unix?)-sockets which provides the raw values from the internal controller. 8888 is the legacy version.
8214 is the websocket which is used for the web interface on port 80.

Do you happen to know what kind of differences there are between those two versions? Websocket sounds nice of course, but I've read somewhere that the legacy API is more powerful?

By the way: Since those services are exposed via network ports, those are not Unix-sockets. Unix domain sockets are basically (pseudo-)files that can be used for interprocess communication. They behave similar to TCP sockets in many aspects, but the main difference is that they are not exposed via network.

See https://serverfault.com/questions/124517/what-is-the-difference-between-unix-sockets-and-tcp-ip-sockets for more details in case you're interested.

The "socket"-api exposes 3 groups of values. I think they are all the raw values from the heatpump.

parameters ID 3003: In "ROM" stored config value which can be changed by customer, technican or the system (e.g. runtime hour counters).
calculations ID 3004: Calculated and not stored values like current temperatures and so on.
visibilities ID 3005: Bool flags to provide informations about activated options and available hardware components.

Thank you again for this summary. Are those terms (parameters, calculations, visibilities) terms that have been extracted from the firmware / app and are "official", or is this what @Bouni came up with? I've struggled in the beginning with those terms and still don't quite like them. It not important once you know what they are about, but still weird terms from my point of view.

For example the flow in and out temperatures are calculations (3004) 10 and 11 in the socket.
In the websocket the flow in and out temperatures are in group "Temperaturen (@id=0x0x594240)" with @id 0x0x598308 and 0x0x58d5e0.

So groups can also have IDs? Didn't know that yet. Are those groups similar to the available menu entries on the Luxtronik controller itself?

In my case the heatpump pushes every 5 hours the current values to heatpump24.com. Also grep config changes.

Okay, one more reason to block Internet access for my heat pump :-).

For example you can unzip the firmware from download portal. In this is a client web app which connects the websocket.
I have tested it with python.

Cool, will look into this. I've also tested the websocket implementation and was rather shocked about how instable it is. I was able to crash my controller by sending 4 bytes to it.

Maybe you can test/verify this:

#! /usr/bin/env python3

import socket
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM);
client.connect(("192.168.1.5", 8214));
client.send("Lux_WS".encode());

For me this halts the controller. After about 5 seconds, it gets rebooted (via a Watchdog or so). I've also recorded a video about this: https://www.youtube.com/watch?v=lhcy8SzfIxY

I've already reported this to Alpha Innotec, hopefully they are working on fixing it. Would be interesting to know if others are also affected by it, though.

Yes, but I get not much informations.

Well, haven't asked specific technical questions, but they sounded helpful to me. Do we have a list of essential questions that we want to know more about?

[BenPru]

Do you happen to know what kind of differences there are between those two versions?

No.

Websocket sounds nice of course, but I've read somewhere that the legacy API is more powerful?

Yes.

Are those terms (parameters, calculations, visibilities) terms that have been extracted from the firmware / app and are "official", or is this what @Bouni came up with?

They are not official.
parameters have two "ids" one for read and one for write.

So groups can also have IDs? Didn't know that yet.

In the Websocket groups have ids. But all ids are temporarly! And the Names are hardcoded in german I think.
So you can't match anyhing.

Are those groups similar to the available menu entries on the Luxtronik controller itself?

No.

After about 5 seconds, it gets rebooted

Last week I have updated from 2.86.0 to 2.88.1 and the new version seems to be not so stable. Ait is bugfixing.
I think the reconnect on every command is a problem.
I have changed this and after more tests I will push it as pr:
https://github.com/BenPru/python-luxtronik/blob/Optimizing-data-reques/luxtronik/__init__.py

Do we have a list of essential questions that we want to know more about?

We use a internal api. 8889 can used without passwort or security. You can write all values.
I don't think they support the internal api.

[kbabioch]

Thank you for your findings.

Last week I have updated from 2.86.0 to 2.88.1 and the new version seems to be not so stable. Ait is bugfixing.

Do you know if the Luxtronik controllers can also be downgraded, or are you know stuck with 2.88.1?

By the way: Support told me that they're currently working on a new version with new features (Energy monitoring), which is supposed to be released soon.

I think the reconnect on every command is a problem.
I have changed this and after more tests I will push it as pr:
https://github.com/BenPru/python-luxtronik/blob/Optimizing-data-reques/luxtronik/__init__.py

Will you try to get this integrated here? Is your repository just a fork with the intention to get changes integrated back here, or do you plan to run your own fork?

We use a internal api. 8889 can used without passwort or security. You can write all values.
I don't think they support the internal api.

That sounds bad. We should probably try to report this.

Have you ever had a look at the other open ports?

# netstat -tulpen
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:36000           0.0.0.0:*               LISTEN      370/appl
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      354/httpd
tcp        0      0 0.0.0.0:502             0.0.0.0:*               LISTEN      370/appl
tcp        0      0 0.0.0.0:8214            0.0.0.0:*               LISTEN      370/appl
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      350/dropbear
tcp        0      0 0.0.0.0:8889            0.0.0.0:*               LISTEN      370/appl
netstat: /proc/net/tcp6: No such file or directory
udp        0      0 0.0.0.0:46346           0.0.0.0:*                           363/appl
udp        0      0 0.0.0.0:4444            0.0.0.0:*                           368/appl
udp        0      0 127.0.0.1:5002          0.0.0.0:*                           358/bootmgr
udp        0      0 0.0.0.0:502             0.0.0.0:*                           370/appl
netstat: /proc/net/udp6: No such file or directory

I don't think they support the internal api.

Probably not, but we could still ask and see what they will reply. I have quite a good and respectful person of contact.

[BenPru]

Do you know if the Luxtronik controllers can also be downgraded, or are you know stuck with 2.88.1?

Yes, the support says the downgrade is possible. But I think I stay on 2.88.1. I have optimize the connection on request site.

Energy monitoring

Yes, 2.88.1 has energy monitoring features: see thread BenPru/luxtronik#59 (comment)

Will you try to get this integrated here?

I have added two sensors for this domestic_water_energy_input and heat_energy_input. See pre-release 2023.01.07

Is your repository just a fork with the intention to get changes integrated back here, or do you plan to run your own fork?

• Bouni/python-luxtronik is the connection module to connect with python to luxtronik socket 8888/8889.
• Bouni/luxtronik uses python-luxtronik and is a ha integration (key=luxtronik) which can be installed with hacs. You have to configure every sensor by yourself via yaml. It provides ha domains sensor and binary_sensor which are readonly. Write command can be used with the service 'luxtronik.write'.
• BenPru/luxtronik started as fork but now it is a clone. My plan is not to merge something back. You can manually install it with hacs as userdefinied repo (key=luxtronik2). You can also create sensor and binary_sensor with the same yaml config (just rename the main key to luxtronik2) and use the service. But it creates devices and many sensors (including writable sensors) automaticly by detecting the visibilities. It has ha domains update (Firmware check), switch, number, climate (thermostat heating, domestic water and cooling).
  Short overview of my lovelace view:
  

Currently I'm trying to extend BenPru/luxtronik and provide it as ha core integration. See repo https://github.com/BenPru/core-1/tree/luxtronik.
PR in the ha core main are only reviewed with single ha domains (climate, sensor) so I plan to push only one domain in the first step. I will create another branch there or copy it to a branch in BenPru/luxtronik and provide it with all domain sensor for tests with other users.

• A big step is done with the better communication (https://github.com/BenPru/python-luxtronik/tree/Optimizing-data-reques).
• I have to create unit tests and so on. Interested in helping?
• Climate heating handling has to be finalized and optimized. Climate thermostat heating BenPru/luxtronik#3 @rhammen was working on this

Have you ever had a look at the other open ports?

No, just the websocket port 8214. See python test.

Well, haven't asked specific technical questions, but they sounded helpful to me. Do we have a list of essential questions that we want to know more about?
Probably not, but we could still ask and see what they will reply. I have quite a good and respectful person of contact.

ad hoc questions:

• How can I start TDI manually "now"?
• The circulation_pump_heating (calculations.ID_WEB_HUPout) runs very often. Sometimes about 18 hours/day without a heating/water requests for days and outdoor temp about 10 C° (no frost protect needed). Why? How can I control this?
• Can we listen for events (websocket or socket) to change the handling from pull to push? Push value changes / callbacks #38
• Websocket: How can we unique identify the values? (name is short in german and ids change every request)
• How to request the firmware revision over the socket?
• Changelog V2.88.1

[kbabioch]

Yes, the support says the downgrade is possible. But I think I stay on 2.88.1. I have optimize the connection on request site.

That sounds good.

I have added two sensors for this domestic_water_energy_input and heat_energy_input.

Cool, wasn't aware of that yet.

Short overview of my lovelace view:

Is this what you get by default, or after (heavily) customizing your integration? The amount of data looks pretty interesting.

I have to create unit tests and so on. Interested in helping?

Yes, definitely, that's also what I'm very familiar with. Happy to provide some infrastructure for unit testing Python code.

How can I start TDI manually "now"?

I would also like to know that. If you figure it out, let me know :-).

The circulation_pump_heating (calculations.ID_WEB_HUPout) runs very often. Sometimes about 18 hours/day without a heating/water requests for days and outdoor temp about 10 C° (no frost protect needed). Why? How can I control this?
Can we listen for events (websocket or socket) to change the handling from pull to push?
Push value changes / callbacks #38
Websocket: How can we unique identify the values? (name is short in german and ids change every request)
How to request the firmware revision over the socket?
Changelog V2.88.1

Good questions, unfortunately I cannot answer them.

I have de-compiled the application by now and could try to figure out some of the details from the binary running on the controller. However most of this is really time-consuming, and so far that's my biggest limitation :-).

[BenPru]

Is this what you get by default, or after (heavily) customizing your integration?

The integration has grown over time.
The current version of my integration creates the most of it out of the box.

[gerw]

@kbabioch : Which decompiler do you use? I just have disassembled the firmware, but reading assembler is....

[gerw]

@BenPru : Do you think that circulation_pump_heating (calculations.ID_WEB_HUPout) is a electrical heating of the circulation pump? I would guess that this only means that the circulation pump (for the heat circuit) is running, i.e., "HUP" stands for "Heizungsumwälzpumpe" and "out" means that this is an output of the heat pump. This could be the "HUP" listed under "Ausgänge" in the websocket interface.

[BenPru]

@BenPru : Do you think that circulation_pump_heating (calculations.ID_WEB_HUPout) is a electrical heating of the circulation pump?

No. It is the heating circulation pump.
I think this is a mistake. It should be heating_circulation_pump.

[gerw]

Concerning "custom" TDI time

By looking at the firmware, I don't think that it is possible to say that the TDI should start now. It can only start at midnight. One workaround would be to change the clock (or the time zone?) of the heatpump such that midnight happens when the sun is shining (e.g. PV energy is available).

However, I think there is a more reasonable approach. There is a parameter which allows the heat pump to use the heating rod (ZWE) if your requested warm water temperature is higher than what the inverter can accomplish (I did not yet checked the parameter number, it can be found under "Einstellungen" -> "Warmw. Nachheizung"). This is documented in the manual by AIT. Then, when you have PV energy you could raise the warm water temperature (parameter 2) to, let's say, 65°C. Then, the inverter heats until, e.g., 55°C and the heating rod should finish off. (Maybe one should also alter the hysteresis parameter 74). In theory, this should be roughly the same as using TDI. I did not try this approach, but I get PV in "the near future" and would like to try it.

[kbabioch]

@kbabioch : Which decompiler do you use? I just have disassembled the firmware, but reading assembler is....

ghidra. It de-compiles most of the appl binary just fine, although there are sections that are not properly analyzed and it's really hard to understand (menues on the controller).

I'm not too familiar with ARM assembler, so anything that is not de-compiled is tricky for me.

I would guess that this only means that the circulation pump (for the heat circuit) is running, i.e., "HUP" stands for "Heizungsumwälzpumpe" and "out" means that this is an output of the heat pump. This could be the "HUP" listed under "Ausgänge" in the websocket interface.

No. It is the heating circulation pump.
I think this is a mistake. It should be heating_circulation_pump.

Sounds to me that you're talking about the same thing here?

By looking at the firmware, I don't think that it is possible to say that the TDI should start now. It can only start at midnight. One workaround would be to change the clock (or the time zone?) of the heatpump such that midnight happens when the sun is shining (e.g. PV energy is available).

That's a pitty.

Then, the inverter heats until, e.g., 55°C and the heating rod should finish off.

The problem with that approach is that it will turn on the inverter / compressor, which means there is physical stress to a component that has only limited amount of life time. I would rather prevent this from happening, and use the heating rod to heat up my water without wear and tear to the system.

In my case I'm not even wasting energy, since my PV system needs to limit itself during peak hours ("70% Regelung"), so I would rather heat up my water than to not use the excess energy.

Maybe I should ask the support, if they would be willing to add such a feature as Kurzprogramm or something like this. The energy management capabilities are rather limited currently.