Zigbee router firmware support #1307

zalatnaicsongor · 2024-03-25T09:50:40Z

It'd be nice if the zigbee module could be used as a router when someone has an existing zigbee network.
I can compile the firmware if I'm given the hardware config for the zigbee module (uart pins, baud, bootloader type, etc.)

AlexxIT · 2024-03-25T10:14:44Z

This component supports flashing zigbee fw for Multimode 1 gateway. But how this device will be paired to another existing network?

zalatnaicsongor · 2024-03-25T14:05:10Z

The OS would lose control of the zigbee coordinator and pairing would happen by shorting two pins. Reflashing the coordinator firmware would probably happen through the SWD interface, so this is something that's for the DIY enthusiast.

AlexxIT · 2024-03-25T14:27:19Z

Zigbee chip has reset pin and bootloader pin. You talking about yet another pin?

AlexxIT · 2024-03-25T14:29:37Z

Also can you compile firmwares with new EZSP versions?

This is update fw function:

XiaomiGateway3/custom_components/xiaomi_gateway3/core/ezsp.py

Line 19 in 029338b

async def update_zigbee_firmware(hass: HomeAssistant, host: str, custom: bool):

And current supported firmwares:

XiaomiGateway3/custom_components/xiaomi_gateway3/core/ezsp.py

Lines 68 to 70 in 029338b

    
           "https://master.dl.sourceforge.net/project/mgl03/zigbee/mgl03_ncp_6_7_10_b38400_sw.gbl?viasf=1" 
        
           if custom 
        
           else "https://master.dl.sourceforge.net/project/mgl03/zigbee/ncp-uart-sw_mgl03_6_6_2_stock.gbl?viasf=1"

zalatnaicsongor · 2024-03-25T15:14:02Z

One of the pins (bootloader perhaps) could be recylced as the pairing pin within the firmware. I can try to compile one, but going back to the coordinator firmware will definitely need SWD access, I'll try to map out if the SWD pins are available on test points.

Just to confirm, can any of the pins of the EFR32 (reset, bootloader, etc.) be triggered within the OS?

Also I found this:
https://github.com/AlexxIT/openmiio_agent/blob/7ab00cb81814ccc5550e4a0a89581d89722cff79/internal/zigbee/zigbee.go#L253-L272

How did you figure out which GPIO was mapped to which pin on the EFR32?

AlexxIT · 2024-03-29T17:15:54Z

Yes. Any pins can be triggered.

bin\zigbee_inter_bootloader.sh

#!/bin/sh
echo "ZIGBEE_RESET VALUE=$1"
echo $1 > /sys/class/gpio/gpio9/value

bin\zigbee_reset.sh

#!/bin/sh
echo "ZIGBEE_RESET VALUE=$1"
echo $1 > /sys/class/gpio/gpio18/value

bin\zigbee_isp.sh

#!/bin/sh

echo "ZIGBEE_ISP VALUE=$1"

pin=43
val=$1

path=/sys/class/gpio/gpio$pin
if [ ! -e $path ];then
    echo $pin > /sys/class/gpio/export
    echo out > $path/direction
fi

# dir
if [ `cat $path/direction` != "out" ];then
    echo out > $path/direction
fi

echo $val > $path/value

bin\zigbee_reset.sh

#!/bin/sh
echo "ZIGBEE_RESET VALUE=$1"

pin=44
val=$1

path=/sys/class/gpio/gpio$pin
if [ ! -e $path ];then
    echo $pin > /sys/class/gpio/export
    echo out > $path/direction
fi

# dir
if [ `cat $path/direction` != "out" ];then
    echo out > $path/direction
fi

echo $val > $path/value

AlexxIT · 2024-03-29T17:17:22Z

Also I'm using this scripts during firmware update

XiaomiGateway3/custom_components/xiaomi_gateway3/core/ezsp.py

Lines 45 to 49 in 4c05771

    
           await sh.exec( 
        
               "zigbee_inter_bootloader.sh 1; zigbee_reset.sh 0; zigbee_reset.sh 1; " 
        
               "killall openmiio_agent" 
        
           ) 
        
           await sh.exec("/data/openmiio_agent --zigbee.tcp=8889 &")

zalatnaicsongor · 2024-04-08T14:23:15Z

Update: I managed to flash the EZSP firmware on a Gateway V2 (MGW2)
However, I flashed a wrong firmware, and now I cannot recover, but I can enter bootloader mode and send a firmware with XMODEM. Can anyone dump a working firmware for me? 😂

AlexxIT · 2024-04-08T14:27:47Z

I have got this file on my MGW2 after update to 1.0.7. I think it is firmware, but I don't sure.

Network-Co-Processor.zip

zalatnaicsongor · 2024-04-08T14:29:00Z

I will compare the two with a hex editor (magic bytes, etc.) ,and will report back

zalatnaicsongor · 2024-04-08T14:33:56Z

I have got this file on my MGW2 after update to 1.0.7. I think it is firmware, but I don't sure.

Network-Co-Processor.zip

yeah looks like a firmware starting from offset 0x3E, where did you get this? Is there a built in xiaomi utility that deals with such stuff?

AlexxIT · 2024-04-08T14:35:12Z

Any update to gateway firmware 1.0.7 will flash new fw to zigbee chip. I don't know why they do it. Never saw this before.
I think Z3GatewayHost can flash new fw. But don't know how to start process.

zalatnaicsongor · 2024-04-08T15:06:35Z

Any update to gateway firmware 1.0.7 will flash new fw to zigbee chip. I don't know why they do it. Never saw this before. I think Z3GatewayHost can flash new fw. But don't know how to start process.

Ah thanks, it worked!
04/08/2024 17:02:40: 51.258 ASH connected
ezsp ver 0x07 stack type 0x02 stack ver. [6.6.6 GA build 218]

The trick:

kill anything that uses /dev/ttyS1
start a /data/openmiio_agent zigbee instance to make sure that the telnet integration doesn't mess up anything
put the ota file into /data/zigbee_host/ota-files
bootloader mode: zigbee_isp.sh 0 && usleep 10000 && zigbee_reset.sh 1 && usleep 10000 && zigbee_reset.sh 0
another terminal: mZ3GatewayHost_MQTT -n0 -d /data/zigbee_host/ -p /dev/ttyS1 -r c -l 1 -t 4
if this fails, trigger bootloader mode when the z3 gateway host is looking for the bootloader output: checkForBootloaderMenu

Eventually, you will get:

~ # mZ3GatewayHost_MQTT -n0 -d /data/zigbee_host/ -p /dev/ttyS1 -r c -l 1 -t 4
Reset info: 11 (SOFTWARE)
===LumiZ3gatewayInfoLoadWithJson===
length = 94
=================================================
{"eui64":"0xE80CD0FDDF8CCF04","info_state":1,"ota_state":2,"radio_power":12,"manu_code":4718}
eui64: 0xE80CD0FDDF8CCF04
info_state: 1 gatewayInfo->gateInfoVaild : 1
ota_state: 2
radio_power: 12
manu_code: 4718
gateInfo.Vaild = 01 gateInfo.state=02
Setting up serial port
checkForBootloaderMenu
Lumi MQTT Client Init
MQTT Client ID = gwE80CD0FDDF8CCF04
HA Gateweay EUI64 = E80CD0FDDF8CCF04
MQTT connected to broker
Setting up serial port
checkForBootloaderMenu
Ready to force upgrade NCP ...

==============================
Current Working Directory: /home/root

==============================
Found OTA file 'ota.ota'
  Manufacturer ID: 0x126E
  Image Type ID:   0x1492
  Version:         0x00000004
  Header String:   NCP ota convert
Found 1 files

Image 0
  Header Version: 0x0100
  Header Length:  56 bytes
  Field Control:  0x0000
  Manuf ID:       0x126E
  Image Type:     0x1492
  Version:        0x00000004
  Zigbee Version: 0x0002
  Header String:  NCP ota convert
  Image Size:     185890 bytes
{"eui64":"0xE80CD0FDDF8CCF04","info_state":1,"ota_state":2,"radio_power":12,"manu_code":4718}
Launching standalone bootloader...
ashFlags & FLG_CONNECTED is disconnected status.Attempt to restore the connection
Try to reset NCP ...

=== ASH started ===

ZIGBEE_ISP VALUE=1
ZIGBEE_RESET VALUE=1
ZIGBEE_RESET VALUE=0
04/08/2024 17:01:49: 0.000 ASH disconnected: EZSP_ASH_ERROR_RESET_FAIL
    NCP status: EZSP_NO_ERROR
Try to reset NCP ...

=== ASH started ===

ZIGBEE_ISP VALUE=1
ZIGBEE_RESET VALUE=1
ZIGBEE_RESET VALUE=0
04/08/2024 17:01:52: 3.581 ASH disconnected: EZSP_ASH_ERROR_RESET_FAIL
    NCP status: EZSP_NO_ERROR
Try to reset NCP ...

=== ASH started ===

ZIGBEE_ISP VALUE=1
ZIGBEE_RESET VALUE=1
ZIGBEE_RESET VALUE=0
04/08/2024 17:01:56: 7.161 ASH disconnected: EZSP_ASH_ERROR_RESET_FAIL
    NCP status: EZSP_NO_ERROR
Try to reset NCP ...

=== ASH started ===

ZIGBEE_ISP VALUE=1
ZIGBEE_RESET VALUE=1
ZIGBEE_RESET VALUE=0
04/08/2024 17:01:59: 10.742 ASH disconnected: EZSP_ASH_ERROR_RESET_FAIL
    NCP status: EZSP_NO_ERROR
Try to reset NCP ...

=== ASH started ===

ZIGBEE_ISP VALUE=1
ZIGBEE_RESET VALUE=1
ZIGBEE_RESET VALUE=0
04/08/2024 17:02:03: 14.320 ASH disconnected: EZSP_ASH_ERROR_RESET_FAIL
    NCP status: EZSP_NO_ERROR
ERROR: ezspErrorHandler 0x28
Launch failed: 0x28
Starting bootloader communications.
Delaying 2 seconds
Setting up serial port
checkForBootloaderMenu
checkForXmodemStart
{"eui64":"0xE80CD0FDDF8CCF04","info_state":1,"ota_state":8,"radio_power":12,"manu_code":4718}
Transferring EBL file to NCP...
EBL data start: 0x0000003E, end: 0x0002D622, size: 185828 bytes
Transfer: 0% complete
Transfer: 5% complete
Transfer: 10% complete
Transfer: 15% complete
Transfer: 20% complete
Transfer: 25% complete
Transfer: 30% complete
Transfer: 35% complete
Transfer: 40% complete
Transfer: 45% complete
Transfer: 50% complete
Transfer: 55% complete
Transfer: 60% complete
Transfer: 65% complete
Transfer: 70% complete
Transfer: 75% complete
Transfer: 80% complete
Transfer: 85% complete
Transfer: 90% complete
Transfer: 95% complete
Transfer: 100% complete
Transferring NCP Firmware successfully and Prepare Reboot NCP.
Delaying 2 seconds
Rebooting NCP through UART message in bootloader mode
checkForBootloaderMenu
Delaying 2 seconds
Finally upgrade NCP status : Success .
{"eui64":"0xE80CD0FDDF8CCF04","info_state":1,"ota_state":9,"radio_power":12,"manu_code":4718}
MQTT connection lost, cause = (null)
Directory "/data/zigbee_host/" was created successfully
Begin to Reset NCP
Try to reset NCP ...

=== ASH started ===

ZIGBEE_ISP VALUE=1
ZIGBEE_RESET VALUE=1
ZIGBEE_RESET VALUE=0

Host Receive <RSTACK>

04/08/2024 17:02:38: 49.218 ASH connected
ezsp ver 0x07 stack type 0x02 stack ver. [6.6.6 GA build 218]

And the firmware by this time will have got updated!

zalatnaicsongor · 2024-04-08T23:35:35Z

Not a router yet, but I managed to build the latest firmware for the gateway 2:

Obviously this breaks the miio zigbee integration, but works perfectly with ZHA.
Is it okay to share the .gbl file?

AlexxIT · 2024-04-09T03:16:31Z

Yes

zalatnaicsongor · 2024-04-09T11:06:17Z

Uploaded: https://file.io/hWKK73ZwEyyY
Can you please check the CPU usage of the mha_master process for me? It's pinning a CPU to 100%

AlexxIT · 2024-04-09T11:22:23Z

50% for my gateway. Load avg 9.07|9.33|9.34|2/133|1221

zalatnaicsongor · 2024-04-09T12:55:43Z

50% for my gateway. Load avg 9.07|9.33|9.34|2/133|1221

yeah it's very high, would you mind reporting this to Aqara please (they are supposed to be the manufacturer of the gateway), you probably know a lot more about the internal architecture of the gateway than what I do.

In the meantime, I'll submit the PR with some scripts that will update the firmware of the MGW2. How do you usually test the changes? Is there an acceptance test or integration test suite?

AlexxIT · 2024-04-09T13:47:30Z

I don't know how to test firmware update process. Only via manual testing

zalatnaicsongor · 2024-04-10T08:05:23Z

Okay, sorry for the spam, but I compiled a multiprotocol RCP firmware that works with the latest (2.4.5) Silicon Labs Multiprotocol home assistant addon.
To make it work, add the following to the yaml config:

device: /dev/ttyS2
baudrate: "115200"
flow_control: true
autoflash_firmware: true
cpcd_trace: false
otbr_enable: true
otbr_log_level: notice
otbr_firewall: true
network_device: {gatewayIp}:8888

The only thing that matters is the network_device property at the end. If that's present, the device, baudrate, and flow_contol properties are ignored in favour of the network_device property.

Then follow the multiprotocol addon insturctions to set up either ZHA or zigbee2mqtt, both work perfectly, and obviously the thread/matter devices work too.

https://filebin.net/mcme5x8t9wkquaw2/rcpuart.gbl

AlexxIT added the enhancement New feature or request label Mar 25, 2024

AlexxIT self-assigned this Apr 17, 2024

AlexxIT mentioned this issue May 18, 2024

ezsp to ember migration #1363

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Zigbee router firmware support #1307

Zigbee router firmware support #1307

zalatnaicsongor commented Mar 25, 2024

AlexxIT commented Mar 25, 2024

zalatnaicsongor commented Mar 25, 2024

AlexxIT commented Mar 25, 2024

AlexxIT commented Mar 25, 2024

zalatnaicsongor commented Mar 25, 2024 •

edited

AlexxIT commented Mar 29, 2024

AlexxIT commented Mar 29, 2024

zalatnaicsongor commented Apr 8, 2024

AlexxIT commented Apr 8, 2024

zalatnaicsongor commented Apr 8, 2024

zalatnaicsongor commented Apr 8, 2024

AlexxIT commented Apr 8, 2024 •

edited

zalatnaicsongor commented Apr 8, 2024

zalatnaicsongor commented Apr 8, 2024

AlexxIT commented Apr 9, 2024

zalatnaicsongor commented Apr 9, 2024

AlexxIT commented Apr 9, 2024

zalatnaicsongor commented Apr 9, 2024

AlexxIT commented Apr 9, 2024

zalatnaicsongor commented Apr 10, 2024

Zigbee router firmware support #1307

Zigbee router firmware support #1307

Comments

zalatnaicsongor commented Mar 25, 2024

AlexxIT commented Mar 25, 2024

zalatnaicsongor commented Mar 25, 2024

AlexxIT commented Mar 25, 2024

AlexxIT commented Mar 25, 2024

zalatnaicsongor commented Mar 25, 2024 • edited

AlexxIT commented Mar 29, 2024

AlexxIT commented Mar 29, 2024

zalatnaicsongor commented Apr 8, 2024

AlexxIT commented Apr 8, 2024

zalatnaicsongor commented Apr 8, 2024

zalatnaicsongor commented Apr 8, 2024

AlexxIT commented Apr 8, 2024 • edited

zalatnaicsongor commented Apr 8, 2024

zalatnaicsongor commented Apr 8, 2024

AlexxIT commented Apr 9, 2024

zalatnaicsongor commented Apr 9, 2024

AlexxIT commented Apr 9, 2024

zalatnaicsongor commented Apr 9, 2024

AlexxIT commented Apr 9, 2024

zalatnaicsongor commented Apr 10, 2024

zalatnaicsongor commented Mar 25, 2024 •

edited

AlexxIT commented Apr 8, 2024 •

edited