Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: filter select peer for sharding #783

Merged
merged 4 commits into from
Sep 29, 2023

Conversation

chaitanyaprem
Copy link
Collaborator

@chaitanyaprem chaitanyaprem commented Sep 28, 2023

Description

Current selectPeer functionality works without taking sharding into consideration. #680
This enhances to take pubSubTopic or contentTopic into consideration during peer selection.

Changes

  • Updated selectPeer function to take pubSub topic as argument
  • Added a new function to selectPeer based on contentTopic

Tests

Ran all existing unit tests.
Added few new ones to validate functionality.

@chaitanyaprem chaitanyaprem changed the base branch from master to feat/autoshard-lightpush September 28, 2023 10:00
@status-im-auto
Copy link

status-im-auto commented Sep 28, 2023

Jenkins Builds

Click to see older builds (18)
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ 60caa1b #1 2023-09-28 10:01:34 ~1 min nix-flake 📄log
✔️ 60caa1b #1 2023-09-28 10:02:16 ~2 min linux 📦deb
✔️ 60caa1b #1 2023-09-28 10:02:51 ~3 min tests 📄log
✔️ 60caa1b #1 2023-09-28 10:03:22 ~3 min tests 📄log
✔️ 60caa1b #2 2023-09-28 10:03:45 ~49 sec tests 📄log
✔️ 60caa1b #2 2023-09-28 10:03:46 ~2 min nix-flake 📄log
✔️ 60caa1b #1 2023-09-28 10:03:49 ~4 min android 📦tgz
✔️ 60caa1b #1 2023-09-28 10:04:27 ~4 min ios 📦tgz
✔️ 60caa1b #2 2023-09-28 10:04:36 ~3 min ios 📦tgz
✔️ 60caa1b #2 2023-09-28 10:04:39 ~2 min linux 📦deb
✔️ 60caa1b #2 2023-09-28 10:05:19 ~1 min tests 📄log
✔️ 60caa1b #2 2023-09-28 10:07:46 ~3 min android 📦tgz
✔️ be1d36f #3 2023-09-28 10:06:53 ~1 min linux 📦deb
✔️ be1d36f #3 2023-09-28 10:07:20 ~1 min tests 📄log
✔️ be1d36f #3 2023-09-28 10:07:40 ~1 min nix-flake 📄log
✔️ be1d36f #3 2023-09-28 10:07:52 ~2 min tests 📄log
✔️ be1d36f #3 2023-09-28 10:10:00 ~4 min ios 📦tgz
✔️ be1d36f #3 2023-09-28 10:10:55 ~3 min android 📦tgz
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ 51a4207 #4 2023-09-29 01:56:07 ~1 min nix-flake 📄log
✔️ 51a4207 #4 2023-09-29 01:56:37 ~2 min linux 📦deb
✔️ 51a4207 #4 2023-09-29 01:57:18 ~2 min tests 📄log
✔️ 51a4207 #4 2023-09-29 01:58:05 ~3 min android 📦tgz
✔️ 51a4207 #4 2023-09-29 01:58:42 ~4 min ios 📦tgz
✖️ 51a4207 #4 2023-09-29 02:01:36 ~7 min tests 📄log
✔️ 98623e6 #5 2023-09-29 02:16:02 ~1 min nix-flake 📄log
✔️ 98623e6 #5 2023-09-29 02:16:27 ~2 min linux 📦deb
✔️ 98623e6 #5 2023-09-29 02:17:10 ~2 min tests 📄log
✔️ 98623e6 #5 2023-09-29 02:17:14 ~2 min tests 📄log
✔️ 98623e6 #5 2023-09-29 02:18:34 ~4 min android 📦tgz
✔️ 98623e6 #5 2023-09-29 02:19:19 ~5 min ios 📦tgz

@chaitanyaprem
Copy link
Collaborator Author

@SionoiS , I have noticed that in nwaku while selecting peers from serviceSlots the filtering is not done as per shards.
It has been done only for relay peers.
https://github.com/waku-org/nwaku/blob/8455b8dd1764d7e82d51e6020ac613fb17bfbe6e/waku/node/peer_manager/peer_manager.nim#L653-L656

JFYI in case it is not already being tracked.

@SionoiS
Copy link

SionoiS commented Sep 28, 2023

@SionoiS , I have noticed that in nwaku while selecting peers from serviceSlots the filtering is not done as per shards. It has been done only for relay peers. https://github.com/waku-org/nwaku/blob/8455b8dd1764d7e82d51e6020ac613fb17bfbe6e/waku/node/peer_manager/peer_manager.nim#L653-L656

JFYI in case it is not already being tracked.

Ah! good catch I didn't notice that. Thanks!

waku/v2/peermanager/peer_manager.go Outdated Show resolved Hide resolved
waku/v2/peerstore/waku_peer_store.go Outdated Show resolved Hide resolved
waku/v2/peermanager/peer_manager.go Outdated Show resolved Hide resolved
waku/v2/peermanager/peer_manager.go Outdated Show resolved Hide resolved
waku/v2/peermanager/peer_manager.go Outdated Show resolved Hide resolved
waku/v2/peermanager/peer_manager.go Outdated Show resolved Hide resolved
waku/v2/peermanager/peer_manager.go Outdated Show resolved Hide resolved
@chaitanyaprem chaitanyaprem merged commit 89809a0 into feat/autoshard-lightpush Sep 29, 2023
6 checks passed
@chaitanyaprem chaitanyaprem deleted the feat/filter-select-peer branch September 29, 2023 04:19
chaitanyaprem added a commit that referenced this pull request Sep 29, 2023
* feat: update lightpush API to make pubSubTopic optional as per autosharding

* Extract contentFilter and subscriptions out of filter to reuse in relay (#779)

* chore: extract contentFilter outside filter package

* chore: move subscription outside of filter so that it can be modified and reused for relay

* Feat: filter select peer for sharding (#783)

* update selectPeer to support pubsubTopic based selection
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

4 participants