Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bug: same cluster, different shard: nodes connect tests fail if using a shard different than 0 #1848

Closed
fbarbu15 opened this issue Feb 13, 2024 · 8 comments · Fixed by #1893
Closed

bug: same cluster, different shard: nodes connect tests fail if using a shard different than 0 #1848

fbarbu15 opened this issue Feb 13, 2024 · 8 comments · Fixed by #1893
Assignees
@danisharora099
Labels
bug Something isn't working

Comments

@fbarbu15
Copy link
Collaborator

fbarbu15 commented Feb 13, 2024 •
edited
Loading

This is a bug report

Problem

Those 2 tests(1 and 2) fail if instead of cluster 0 we use a different one (ex 2)

This happens with harbor.status.im/wakuorg/nwaku:latest and doesn't reproduce with nwaku v.0.24.0

Log

  Metadata Protocol
    connections
  waku:info:test:docker Confirming that image harbor.status.im/wakuorg/nwaku:latest exists +0ms
  waku:info:test:docker Image harbor.status.im/wakuorg/nwaku:latest successfully found +0ms
  waku:info:test:docker Running node with args: --listen-address=0.0.0.0 --rpc=true --relay=true --rpc-admin=true --websocket-support=true --log-level=TRACE --rpc-port=43576 --tcp-port=43577 --websocket-port=43578 --discv5-udp-port=43579 --rpc-address=0.0.0.0 --discv5-discovery=true --peer-exchange=true --cluster-id=5 --pubsub-topic=/waku/2/rs/5/1 +0ms
  waku:info:test:docker 98b3c04c5710612ec28edaac5c3e04b3207672567630afc0897ac5158af63d07 started at 5:52:49 PM +447ms
  waku:info:test:node Waiting to see 'Node setup complete' in nwaku logs +0ms
  waku:info:test:node nwaku node has been started +7ms
  waku:info:test:node Making an RPC Query:  get_waku_v2_debug_v1_info [] +0ms
  waku:info:test:node Received RPC Response:  {"jsonrpc":"2.0","id":1,"result":{"listenAddresses":["/ip4/172.18.49.1/tcp/43577/p2p/16Uiu2HAm8jAdPqwXuTWpFBCxyvYBkWv72R6dyFmFrZJ3NiJRgvjo","/ip4/172.18.49.1/tcp/43578/ws/p2p/16Uiu2HAm8jAdPqwXuTWpFBCxyvYBkWv72R6dyFmFrZJ3NiJRgvjo"],"enrUri":"enr:-LO4QEogDOkNQzXmPkxt2nKgx01DDy4-GQoy6wGNH5g-nuHUZBCdEXtGq_cQuMzZQTGBl2fbSHkXjNL-iXh1QthS22wBgmlkgnY0gmlwhKwSMQGKbXVsdGlhZGRyc4wACgSsEjEBBqo63QOCcnOFAAUBAAGJc2VjcDI1NmsxoQLFozJPrFpkjAX04IoYW9ri4DoOlJrnrWI17MEwyMwoHoN0Y3CCqjmDdWRwgqo7hXdha3UyAQ"}} +27ms
  waku:info:waku Waku node created 12D3KooWRaGwrN2tuXMGTvbx7zS4k7tGtD8ypuxcZtNCGy4XJfAe relay: false, store: true, light push: true, filter: true +0ms
  waku:info:connection-manager Connection Manager is now running +0ms
  waku:info:connection-manager Connected to peer 16Uiu2HAm8jAdPqwXuTWpFBCxyvYBkWv72R6dyFmFrZJ3NiJRgvjo +162ms
  waku:error:connection-manager Failed to get peer 16Uiu2HAm8jAdPqwXuTWpFBCxyvYBkWv72R6dyFmFrZJ3NiJRgvjo, error: Error: Not Found +0ms
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAm8jAdPqwXuTWpFBCxyvYBkWv72R6dyFmFrZJ3NiJRgvjo +0ms
  waku:warn:connection-manager Already connected to peer 16Uiu2HAm8jAdPqwXuTWpFBCxyvYBkWv72R6dyFmFrZJ3NiJRgvjo. Not dialing. +0ms
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAm8jAdPqwXuTWpFBCxyvYBkWv72R6dyFmFrZJ3NiJRgvjo +4ms
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAm8jAdPqwXuTWpFBCxyvYBkWv72R6dyFmFrZJ3NiJRgvjo +2ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAm8jAdPqwXuTWpFBCxyvYBkWv72R6dyFmFrZJ3NiJRgvjo +0ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAm8jAdPqwXuTWpFBCxyvYBkWv72R6dyFmFrZJ3NiJRgvjo +0ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAm8jAdPqwXuTWpFBCxyvYBkWv72R6dyFmFrZJ3NiJRgvjo +0ms
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAm8jAdPqwXuTWpFBCxyvYBkWv72R6dyFmFrZJ3NiJRgvjo +6ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAm8jAdPqwXuTWpFBCxyvYBkWv72R6dyFmFrZJ3NiJRgvjo +1ms
  waku:info:test:docker Shutting down container ID 98b3c04c5710612ec28edaac5c3e04b3207672567630afc0897ac5158af63d07 at 5:52:49 PM +243ms
  waku:info:test:docker Confirming that image harbor.status.im/wakuorg/nwaku:latest exists +1s
  waku:info:test:docker Image harbor.status.im/wakuorg/nwaku:latest successfully found +0ms
  waku:info:test:docker Running node with args: --listen-address=0.0.0.0 --rpc=true --relay=true --rpc-admin=true --websocket-support=true --log-level=TRACE --rpc-port=1799 --tcp-port=1800 --websocket-port=1801 --discv5-udp-port=1802 --rpc-address=0.0.0.0 --discv5-discovery=true --peer-exchange=true --cluster-id=5 --pubsub-topic=/waku/2/rs/5/1 +0ms
  waku:info:test:docker ab4b2655b8bfaf6abdbea23b5ddadba11d7dae204ace3bd4b6a1e5879fdeaff0 started at 5:52:51 PM +574ms
  waku:info:test:node Waiting to see 'Node setup complete' in nwaku logs +2s
  waku:info:test:node nwaku node has been started +6ms
  waku:info:test:node Making an RPC Query:  get_waku_v2_debug_v1_info [] +1ms
  waku:info:test:node Received RPC Response:  {"jsonrpc":"2.0","id":1,"result":{"listenAddresses":["/ip4/172.18.73.145/tcp/1800/p2p/16Uiu2HAm3HLTTQRdY3smdzJHT7nxGfkBaX1T67y7HYxGthksyfFn","/ip4/172.18.73.145/tcp/1801/ws/p2p/16Uiu2HAm3HLTTQRdY3smdzJHT7nxGfkBaX1T67y7HYxGthksyfFn"],"enrUri":"enr:-LO4QPbOU1yfV3l6s22wUhaet12r6uvM5nZrExY_xO5XtuxVd5K9oksJ1W7yUz6N4YDE-kNbB5N_Z3qlZSxDyl-RrfIBgmlkgnY0gmlwhKwSSZGKbXVsdGlhZGRyc4wACgSsEkmRBgcJ3QOCcnOFAAUBAAGJc2VjcDI1NmsxoQJ0usYuUYQ82Afh4KNR8bnIEX76-57mqT51sO3DFAbQ8YN0Y3CCBwiDdWRwggcKhXdha3UyAQ"}} +16ms
  waku:info:waku Waku node created 12D3KooWMLozFgaMbDZYdkLX82QMNHAVxicFa9juhtrU4Lt4eF8S relay: false, store: true, light push: true, filter: true +2s
  waku:info:connection-manager Connection Manager is now running +2s
  waku:info:connection-manager Connected to peer 16Uiu2HAm3HLTTQRdY3smdzJHT7nxGfkBaX1T67y7HYxGthksyfFn +110ms
  waku:error:connection-manager Failed to get peer 16Uiu2HAm3HLTTQRdY3smdzJHT7nxGfkBaX1T67y7HYxGthksyfFn, error: Error: Not Found +2s
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAm3HLTTQRdY3smdzJHT7nxGfkBaX1T67y7HYxGthksyfFn +0ms
  waku:warn:connection-manager Already connected to peer 16Uiu2HAm3HLTTQRdY3smdzJHT7nxGfkBaX1T67y7HYxGthksyfFn. Not dialing. +2s
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAm3HLTTQRdY3smdzJHT7nxGfkBaX1T67y7HYxGthksyfFn +6ms
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAm3HLTTQRdY3smdzJHT7nxGfkBaX1T67y7HYxGthksyfFn +4ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAm3HLTTQRdY3smdzJHT7nxGfkBaX1T67y7HYxGthksyfFn +0ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAm3HLTTQRdY3smdzJHT7nxGfkBaX1T67y7HYxGthksyfFn +0ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAm3HLTTQRdY3smdzJHT7nxGfkBaX1T67y7HYxGthksyfFn +0ms
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAm3HLTTQRdY3smdzJHT7nxGfkBaX1T67y7HYxGthksyfFn +3ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAm3HLTTQRdY3smdzJHT7nxGfkBaX1T67y7HYxGthksyfFn +1ms
  waku:info:test:docker Shutting down container ID ab4b2655b8bfaf6abdbea23b5ddadba11d7dae204ace3bd4b6a1e5879fdeaff0 at 5:52:51 PM +157ms
  waku:info:test:docker Confirming that image harbor.status.im/wakuorg/nwaku:latest exists +1s
  waku:info:test:docker Image harbor.status.im/wakuorg/nwaku:latest successfully found +0ms
  waku:info:test:docker Running node with args: --listen-address=0.0.0.0 --rpc=true --relay=true --rpc-admin=true --websocket-support=true --log-level=TRACE --rpc-port=28048 --tcp-port=28049 --websocket-port=28050 --discv5-udp-port=28051 --rpc-address=0.0.0.0 --discv5-discovery=true --peer-exchange=true --cluster-id=5 --pubsub-topic=/waku/2/rs/5/1 +1ms
  waku:info:test:docker 16553f2c90dba61685c379840976c501cedb774e47883d4d9fdbd9ab47799b36 started at 5:52:53 PM +536ms
  waku:info:test:node Waiting to see 'Node setup complete' in nwaku logs +2s
  waku:info:test:node nwaku node has been started +5ms
  waku:info:test:node Making an RPC Query:  get_waku_v2_debug_v1_info [] +0ms
  waku:info:test:node Received RPC Response:  {"jsonrpc":"2.0","id":1,"result":{"listenAddresses":["/ip4/172.18.34.169/tcp/28049/p2p/16Uiu2HAmQZ4UgqD2S8jinonjxfXsWCR69y5eFbwARvDACfSFDZ55","/ip4/172.18.34.169/tcp/28050/ws/p2p/16Uiu2HAmQZ4UgqD2S8jinonjxfXsWCR69y5eFbwARvDACfSFDZ55"],"enrUri":"enr:-LO4QD--Ed_ymzbppsFpgHAmvB3AAxELEL5KD3ypOfPQAQLfBPrgx5xyDuGiLjrmw6agtrfu4s1huVGzigsX8lke0ZgBgmlkgnY0gmlwhKwSIqmKbXVsdGlhZGRyc4wACgSsEiKpBm2S3QOCcnOFAAUBAAGJc2VjcDI1NmsxoQOwx2zgBJGDLRul-dKG3rCRvHhCv3QTPTmxPp5Fh9n4zIN0Y3CCbZGDdWRwgm2ThXdha3UyAQ"}} +16ms
  waku:info:waku Waku node created 12D3KooWCZyr3NRx2btZ36oACUdmMTaSWnc8EdGw47WxuioU8uXp relay: false, store: true, light push: true, filter: true +2s
  waku:info:connection-manager Connection Manager is now running +2s
  waku:info:connection-manager Connected to peer 16Uiu2HAmQZ4UgqD2S8jinonjxfXsWCR69y5eFbwARvDACfSFDZ55 +118ms
  waku:error:connection-manager Failed to get peer 16Uiu2HAmQZ4UgqD2S8jinonjxfXsWCR69y5eFbwARvDACfSFDZ55, error: Error: Not Found +2s
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAmQZ4UgqD2S8jinonjxfXsWCR69y5eFbwARvDACfSFDZ55 +0ms
  waku:warn:connection-manager Already connected to peer 16Uiu2HAmQZ4UgqD2S8jinonjxfXsWCR69y5eFbwARvDACfSFDZ55. Not dialing. +2s
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAmQZ4UgqD2S8jinonjxfXsWCR69y5eFbwARvDACfSFDZ55 +1ms
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAmQZ4UgqD2S8jinonjxfXsWCR69y5eFbwARvDACfSFDZ55 +2ms
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAmQZ4UgqD2S8jinonjxfXsWCR69y5eFbwARvDACfSFDZ55 +10ms
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAmQZ4UgqD2S8jinonjxfXsWCR69y5eFbwARvDACfSFDZ55 +1ms
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAmQZ4UgqD2S8jinonjxfXsWCR69y5eFbwARvDACfSFDZ55 +3ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAmQZ4UgqD2S8jinonjxfXsWCR69y5eFbwARvDACfSFDZ55 +0ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAmQZ4UgqD2S8jinonjxfXsWCR69y5eFbwARvDACfSFDZ55 +2ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAmQZ4UgqD2S8jinonjxfXsWCR69y5eFbwARvDACfSFDZ55 +0ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAmQZ4UgqD2S8jinonjxfXsWCR69y5eFbwARvDACfSFDZ55 +0ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAmQZ4UgqD2S8jinonjxfXsWCR69y5eFbwARvDACfSFDZ55 +0ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAmQZ4UgqD2S8jinonjxfXsWCR69y5eFbwARvDACfSFDZ55 +0ms
  waku:info:test:docker Shutting down container ID 16553f2c90dba61685c379840976c501cedb774e47883d4d9fdbd9ab47799b36 at 5:52:53 PM +167ms
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAmQZ4UgqD2S8jinonjxfXsWCR69y5eFbwARvDACfSFDZ55 +9ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAmQZ4UgqD2S8jinonjxfXsWCR69y5eFbwARvDACfSFDZ55 +7ms
  waku:info:test:docker Confirming that image harbor.status.im/wakuorg/nwaku:latest exists +1s
  waku:info:test:docker Image harbor.status.im/wakuorg/nwaku:latest successfully found +0ms
  waku:info:test:docker Running node with args: --listen-address=0.0.0.0 --rpc=true --relay=true --rpc-admin=true --websocket-support=true --log-level=TRACE --rpc-port=55322 --tcp-port=55323 --websocket-port=55324 --discv5-udp-port=55325 --rpc-address=0.0.0.0 --discv5-discovery=true --peer-exchange=true --cluster-id=5 --pubsub-topic=/waku/2/rs/5/1 +0ms
  waku:info:test:docker 1cf850b4cdec2d2af6c2d00502fc62ee207bb9d2471c48bac8e0da0449381e73 started at 5:52:55 PM +520ms
  waku:info:test:node Waiting to see 'Node setup complete' in nwaku logs +2s
  waku:info:test:node nwaku node has been started +5ms
  waku:info:test:node Making an RPC Query:  get_waku_v2_debug_v1_info [] +0ms
  waku:info:test:node Received RPC Response:  {"jsonrpc":"2.0","id":1,"result":{"listenAddresses":["/ip4/172.18.138.199/tcp/55323/p2p/16Uiu2HAmJMUxHyPvTrT48mhfyBXVRiEagDZVrK1q7FJyetXzEkWd","/ip4/172.18.138.199/tcp/55324/ws/p2p/16Uiu2HAmJMUxHyPvTrT48mhfyBXVRiEagDZVrK1q7FJyetXzEkWd"],"enrUri":"enr:-LO4QIa3Xy853AB-2ZWXm9ktSnV7lQdsuRC73cY8OYdLKGRwCYJ6FN7oD6EqpC0k7SpKU46D-FNvwDyJz4Y-G_tKh58BgmlkgnY0gmlwhKwSiseKbXVsdGlhZGRyc4wACgSsEorHBtgc3QOCcnOFAAUBAAGJc2VjcDI1NmsxoQNUqgJTfACiWymYb4ZdUQmpQGiabG4VZFwmP2llLGE-WoN0Y3CC2BuDdWRwgtgdhXdha3UyAQ"}} +15ms
  waku:info:waku Waku node created 12D3KooWQ7LqqtCNkjYUdF7eTqd9LeLbRXoP2D9agn5EE8iWs5ci relay: false, store: true, light push: true, filter: true +2s
  waku:info:connection-manager Connection Manager is now running +2s
  waku:info:connection-manager Connected to peer 16Uiu2HAmJMUxHyPvTrT48mhfyBXVRiEagDZVrK1q7FJyetXzEkWd +101ms
  waku:error:connection-manager Failed to get peer 16Uiu2HAmJMUxHyPvTrT48mhfyBXVRiEagDZVrK1q7FJyetXzEkWd, error: Error: Not Found +2s
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAmJMUxHyPvTrT48mhfyBXVRiEagDZVrK1q7FJyetXzEkWd +0ms
  waku:warn:connection-manager Already connected to peer 16Uiu2HAmJMUxHyPvTrT48mhfyBXVRiEagDZVrK1q7FJyetXzEkWd. Not dialing. +2s
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAmJMUxHyPvTrT48mhfyBXVRiEagDZVrK1q7FJyetXzEkWd +3ms
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAmJMUxHyPvTrT48mhfyBXVRiEagDZVrK1q7FJyetXzEkWd +9ms
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAmJMUxHyPvTrT48mhfyBXVRiEagDZVrK1q7FJyetXzEkWd +3ms
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAmJMUxHyPvTrT48mhfyBXVRiEagDZVrK1q7FJyetXzEkWd +0ms
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAmJMUxHyPvTrT48mhfyBXVRiEagDZVrK1q7FJyetXzEkWd +2ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAmJMUxHyPvTrT48mhfyBXVRiEagDZVrK1q7FJyetXzEkWd +0ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAmJMUxHyPvTrT48mhfyBXVRiEagDZVrK1q7FJyetXzEkWd +0ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAmJMUxHyPvTrT48mhfyBXVRiEagDZVrK1q7FJyetXzEkWd +1ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAmJMUxHyPvTrT48mhfyBXVRiEagDZVrK1q7FJyetXzEkWd +0ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAmJMUxHyPvTrT48mhfyBXVRiEagDZVrK1q7FJyetXzEkWd +0ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAmJMUxHyPvTrT48mhfyBXVRiEagDZVrK1q7FJyetXzEkWd +0ms
  waku:info:test:docker Shutting down container ID 1cf850b4cdec2d2af6c2d00502fc62ee207bb9d2471c48bac8e0da0449381e73 at 5:52:55 PM +151ms
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAmJMUxHyPvTrT48mhfyBXVRiEagDZVrK1q7FJyetXzEkWd +6ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAmJMUxHyPvTrT48mhfyBXVRiEagDZVrK1q7FJyetXzEkWd +4ms
  waku:info:test:docker Confirming that image harbor.status.im/wakuorg/nwaku:latest exists +1s
  waku:info:test:docker Image harbor.status.im/wakuorg/nwaku:latest successfully found +0ms
  waku:info:test:docker Running node with args: --listen-address=0.0.0.0 --rpc=true --relay=true --rpc-admin=true --websocket-support=true --log-level=TRACE --rpc-port=12151 --tcp-port=12152 --websocket-port=12153 --discv5-udp-port=12154 --rpc-address=0.0.0.0 --discv5-discovery=true --peer-exchange=true --cluster-id=5 --pubsub-topic=/waku/2/rs/5/1 +0ms
  waku:info:test:docker c0c95ba4e10fc5939e54bee84d9d9234f344213de6169609185259bddf72d131 started at 5:52:57 PM +497ms
  waku:info:test:node Waiting to see 'Node setup complete' in nwaku logs +2s
  waku:info:test:node nwaku node has been started +5ms
  waku:info:test:node Making an RPC Query:  get_waku_v2_debug_v1_info [] +0ms
  waku:info:test:node Received RPC Response:  {"jsonrpc":"2.0","id":1,"result":{"listenAddresses":["/ip4/172.18.248.98/tcp/12152/p2p/16Uiu2HAmHMAd6Du3AHFyg9HXgiKsqT7jGhcdzJbQyAzzW4DPWLdF","/ip4/172.18.248.98/tcp/12153/ws/p2p/16Uiu2HAmHMAd6Du3AHFyg9HXgiKsqT7jGhcdzJbQyAzzW4DPWLdF"],"enrUri":"enr:-LO4QEiHkdkbVsPybP6tHT0jPaTTqFaFFYY-0P8eabbK3eVkZezpKgQJgoRrSOOWgkpUEWkm8_WZsjpZZ_bpVpAsx9kBgmlkgnY0gmlwhKwS-GKKbXVsdGlhZGRyc4wACgSsEvhiBi953QOCcnOFAAUBAAGJc2VjcDI1NmsxoQNFuZhbSAnu7FXjJEY6Kz3bsYNiHsNjZwqgf8Nfcp11ioN0Y3CCL3iDdWRwgi96hXdha3UyAQ"}} +16ms
  waku:info:waku Waku node created 12D3KooWC3P1vyZdWm5ompy9bdpTyXe474oEXxbdkG34C6HpM44H relay: false, store: true, light push: true, filter: true +2s
  waku:info:connection-manager Connection Manager is now running +2s
  waku:info:connection-manager Connected to peer 16Uiu2HAmHMAd6Du3AHFyg9HXgiKsqT7jGhcdzJbQyAzzW4DPWLdF +98ms
  waku:error:connection-manager Failed to get peer 16Uiu2HAmHMAd6Du3AHFyg9HXgiKsqT7jGhcdzJbQyAzzW4DPWLdF, error: Error: Not Found +2s
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAmHMAd6Du3AHFyg9HXgiKsqT7jGhcdzJbQyAzzW4DPWLdF +0ms
  waku:warn:connection-manager Already connected to peer 16Uiu2HAmHMAd6Du3AHFyg9HXgiKsqT7jGhcdzJbQyAzzW4DPWLdF. Not dialing. +2s
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAmHMAd6Du3AHFyg9HXgiKsqT7jGhcdzJbQyAzzW4DPWLdF +3ms
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAmHMAd6Du3AHFyg9HXgiKsqT7jGhcdzJbQyAzzW4DPWLdF +2ms
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAmHMAd6Du3AHFyg9HXgiKsqT7jGhcdzJbQyAzzW4DPWLdF +8ms
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAmHMAd6Du3AHFyg9HXgiKsqT7jGhcdzJbQyAzzW4DPWLdF +3ms
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAmHMAd6Du3AHFyg9HXgiKsqT7jGhcdzJbQyAzzW4DPWLdF +4ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAmHMAd6Du3AHFyg9HXgiKsqT7jGhcdzJbQyAzzW4DPWLdF +0ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAmHMAd6Du3AHFyg9HXgiKsqT7jGhcdzJbQyAzzW4DPWLdF +0ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAmHMAd6Du3AHFyg9HXgiKsqT7jGhcdzJbQyAzzW4DPWLdF +0ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAmHMAd6Du3AHFyg9HXgiKsqT7jGhcdzJbQyAzzW4DPWLdF +0ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAmHMAd6Du3AHFyg9HXgiKsqT7jGhcdzJbQyAzzW4DPWLdF +0ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAmHMAd6Du3AHFyg9HXgiKsqT7jGhcdzJbQyAzzW4DPWLdF +0ms
      1) same cluster, different shard: nodes connect
  waku:info:test:docker Shutting down container ID c0c95ba4e10fc5939e54bee84d9d9234f344213de6169609185259bddf72d131 at 5:52:57 PM +158ms
  waku:info:stream-manager:/vac/waku/metadata/1.0.0 Preemptively opening a stream to 16Uiu2HAmHMAd6Du3AHFyg9HXgiKsqT7jGhcdzJbQyAzzW4DPWLdF +18ms
  waku:error:stream-manager:/vac/waku/metadata/1.0.0 Failed to prepare a new stream for 16Uiu2HAmHMAd6Du3AHFyg9HXgiKsqT7jGhcdzJbQyAzzW4DPWLdF +10ms


  0 passing (10s)
  1 failing

  1) Metadata Protocol
       connections
         same cluster, different shard: nodes connect:
     CodeError: the connection is being closed
      at ConnectionImpl.newStream (file:///home/fbarbu15/WORK_DIR/js-waku/node_modules/libp2p/src/connection/index.ts:119:13)
      at StreamManager.newStream (file:///home/fbarbu15/WORK_DIR/js-waku/packages/core/src/lib/stream_manager.ts:53:23)
      at StreamManager.getStream (file:///home/fbarbu15/WORK_DIR/js-waku/packages/core/src/lib/stream_manager.ts:41:19)
      at async Metadata.query (file:///home/fbarbu15/WORK_DIR/js-waku/packages/core/src/lib/metadata/index.ts:83:20)
      at async Context.<anonymous> (file:///home/fbarbu15/WORK_DIR/js-waku/packages/tests/tests/metadata.spec.ts:67:34)
@fbarbu15 fbarbu15 added the bug Something isn't working label Feb 13, 2024
@fryorcraken fryorcraken added this to Waku Feb 13, 2024
@danisharora099 danisharora099 moved this to Triage in Waku Feb 15, 2024
@danisharora099
Copy link
Collaborator

Huh, this is interesting.. Perhaps something @waku-org/nwaku-developers might have context on especially since these tests are only failing on master and not the last stable release?

@gabrielmer
Copy link
Contributor

gabrielmer commented Feb 15, 2024 •
edited
Loading

I think this is expected after the new sharded peer management waku-org/nwaku#2332
cc @SionoiS

@SionoiS
Copy link
Contributor

SionoiS commented Feb 15, 2024 •
edited
Loading

I think this is expected after the new sharded peer management waku-org/nwaku#2332
cc @SionoiS

The sharded peer manager is disabled by default but even if enabled it should still work.

One thing I notice is that the js code allow autosharding with cluster != 1 which may cause problems.

cluster 0 == named sharding only
cluster 1 == TWN (auto or static shard 0-7)
cluster > 1 == static shard only

Also if 2 nodes don't share a shard then they should not connect per the Metadata protocol logic.

@fbarbu15
Copy link
Collaborator Author

Yes, I saw that if they share at least one shard than the test works for any cluster and connection works

@danisharora099
Copy link
Collaborator

@SionoiS if two nodes are on the same cluster, but different shards, they should connect right?

@SionoiS
Copy link
Contributor

SionoiS commented Feb 16, 2024

@SionoiS if two nodes are on the same cluster, but different shards, they should connect right?

No the metadata protocol logic prevents it.

@danisharora099
Copy link
Collaborator

@SionoiS if two nodes are on the same cluster, but different shards, they should connect right?

No the metadata protocol logic prevents it.

Ah, I was under the belief that two nodes only need to be on the same cluster and not necessarily on the same shard. Gotcha!

@SionoiS
Copy link
Contributor

SionoiS commented Feb 16, 2024 •
edited
Loading

@SionoiS if two nodes are on the same cluster, but different shards, they should connect right?

No the metadata protocol logic prevents it.

Ah, I was under the belief that two nodes only need to be on the same cluster and not necessarily on the same shard. Gotcha!

As long as 2 nodes share at least one shard they will connect (not disconnect).

@chair28980 chair28980 moved this from Triage to Priority in Waku Mar 6, 2024
@danisharora099 danisharora099 moved this from Priority to In Progress in Waku Mar 7, 2024
@danisharora099 danisharora099 mentioned this issue Mar 7, 2024
Merged
3 tasks
@danisharora099 danisharora099 moved this from In Progress to Code Review / QA in Waku Mar 7, 2024
@github-project-automation github-project-automation bot moved this from Code Review / QA to Done in Waku Mar 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@danisharora099 danisharora099

Labels
bug Something isn't working
Projects
Waku
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(tests): sharding: same cluster, different shard waku-org/js-waku
4 participants
@SionoiS @fbarbu15 @danisharora099 @gabrielmer