[HELP] IoTGW Performance (200dp/s cap)

[ashdam]

Describe the issue
We are trying to stress the platform. In order to do this, we started with an OPC-UA async connector with 1000 timeseries declared and "polling" (not subscribe) every second.
Information is sent this way: IoTGateway ==> Edge PE ==> TB PE
We are getting the following information from statistics:

110 events /s, memory size increasing 50 MB, but networkfraffic hasn't reached 1MB/s

Additionally we have verified that we have a 200dp/s cap in pubsub

Do you know what mabe be happening with the cap?

Configuration (Attach your configuration file)

version: '3.4'
services:
  # ThingsBoard IoT Gateway Service Configuration
  tb-gateway:
    image: thingsboard/tb-gateway
    container_name: tb-gateway
    restart: always

    # Ports bindings - required by some connectors
    ports:
        - "5000:5000" # Comment if you don't use REST connector and change if you use another port
        # Uncomment and modify the following ports based on connector usage:
#        - "1052:1052" # BACnet connector
#        - "5026:5026" # Modbus TCP connector (Modbus Slave)
#        - "50000:50000/tcp" # Socket connector with type TCP
#        - "50000:50000/udp" # Socket connector with type UDP

    # Necessary mapping for Linux
    extra_hosts:
      - "host.docker.internal:host-gateway"

    # Environment variables
    environment:
      - host=xx.xxx.xxx.xx
      - port=1883
      - accessToken=xxxxxxxxxxxxxxxxxxx

    # Volumes bind
    volumes:
      - tb-gw-config:/thingsboard_gateway/config
      - tb-gw-logs:/thingsboard_gateway/logs
      - tb-gw-extensions:/thingsboard_gateway/extensions

# Volumes declaration for configurations, extensions and configuration
volumes:
  tb-gw-config:
    name: tb-gw-config
  tb-gw-logs:
    name: tb-gw-logs
  tb-gw-extensions:
    name: tb-gw-extensions

Connector name (If you need help with some connector/converter):
[e.g. MQTT Connector]

Error traceback (If it was raised):

'deviceName'
Traceback (most recent call last):
  File "<input>", line 2, in <module>
KeyError: 'deviceName'

Versions (please complete the following information):

• OS: Ubuntu 22
• Thingsboard IoT Gateway version 3.5.1
• Python version: 3.12

[ashdam]

@imbeacon Hi, may you explain this two parameters you have in documentation ? whats the difference between the two

https://thingsboard.io/docs/iot-gateway/configuration/

[ashdam]

Alex helped me explaining the exact meaning of XX:YY,ZZ:TT parameters.
We checked TB PE Tenant limits and there were none (all default "Not set")

Then configured GW with the following parameters with no success.

[ashdam]

We made a test where edge is not involved so gateway is directly connected to TB in the cloud.
Here is the accumulated i/o usage during 1 minute

We changed storage from memory to filesystem to check if there were data accumulation and there wasn't

The performance got a lil bit better

Our current conclusion is: OPC-UA asyncio connector is not enough efficient /performant

[samson0v]

@ashdam if you increase maxPayloadSizeBytes, one MQTT message can have multiple device messages, so measuring performance with this metric is not effective. Moreover increasing this parameter == message count can decrease.
Does all data from devices come to ThingsBoard or do you have data loss?

[ashdam]

We already did with no effect on it

Yes, only thingsboard gateway is in this machine and go to TB cloud

[ashdam]

This has been solved along with many other upgrades, improvements and stabilizations in #1501