Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: hibernate main loop #273

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

JimMoen
Copy link
Member

@JimMoen JimMoen commented Nov 26, 2024

Compared 0.4.26 and this branch with same benchmark command
(both using docker image local built by command make docker)

bench compose file:

Connecting to a 2-nodes-cluster
services:

  node1-bench1:
    image: emqx/emqtt_bench:${EMQTT_BENCH_TAG}
    restart: always
    container_name: node1-bench1
    command: "conn -i 2 -h 172.19.0.2 -u admin -P public123 -c ${BENCH_CONN_COUNT_PER_NODE:-25000} --prefix node1-bench1"
    hostname: node1-bench1
    networks:
      emqx-net:
        ipv4_address: 172.19.0.31

  node1-bench2:
    image: emqx/emqtt_bench:${EMQTT_BENCH_TAG}
    restart: always
    container_name: node1-bench2
    command: "conn -i 2 -h 172.19.0.2 -u admin -P public123 -c ${BENCH_CONN_COUNT_PER_NODE:-25000} --prefix node1-bench2"
    hostname: node1-bench2
    networks:
      emqx-net:
        ipv4_address: 172.19.0.32

  node2-bench1:
    image: emqx/emqtt_bench:${EMQTT_BENCH_TAG}
    restart: always
    container_name: node2-bench1
    command: "conn -i 2 -h 172.19.0.3 -u admin -P public123 -c ${BENCH_CONN_COUNT_PER_NODE:-25000} --prefix node2-bench1"
    hostname: node2-bench1
    networks:
      emqx-net:
        ipv4_address: 172.19.0.33

  node2-bench2:
    image: emqx/emqtt_bench:${EMQTT_BENCH_TAG}
    restart: always
    container_name: node2-bench2
    command: "conn -i 2 -h 172.19.0.3 -u admin -P public123 -c ${BENCH_CONN_COUNT_PER_NODE:-25000} --prefix node2-bench2"
    hostname: node2-bench2
    networks:
      emqx-net:
        ipv4_address: 172.19.0.34

networks:
  emqx-net:
    ipam:
      config:
        - subnet: 172.19.0.0/16
    driver: bridge

CPU usage when the number of connections is about 14,000, (about 20 seconds after starting)

0.4.26:
image

this branch:
image

@@ -818,7 +828,7 @@ loop(Parent, N, Client, PubSub, Opts) ->
io:format("client(~w): discarded unknown message ~p~n", [N, Other]),
loop(Parent, N, Client, PubSub, Opts)
after
Idle ->
_Idle = max(Interval * 2, 50) ->
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the 'perf improvements' is mostly by changing it from 500ms to 50ms and it is case by case, we see reduced CPU usage and lowered memeory usage but latency become higher I guess.

Could you make this configurable instead rather than hardcode it?

Msg ->
print("main_loop_msg: ~p~n", [Msg]),
main_loop(Uptime, Count)
hibernate_main_loop(Uptime, Count)
after 5 ->
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants