support ssb-tribes #2

staltz · 2020-12-08T18:17:34Z

@arj03 @cryptix @mixmix I think fixtures now can generate: (1) tribe creation, (2) tribe invitation to other feeds, (3) post a message inside tribes, all using ssb-tribes. I think it's working, but it would be good to be sure.

How to test this out:

git clone
checkout branch tribes
npm install
npm run compile
node lib/bin.js
check ./data dir when it's done

Note, you can get the secret of the 3 most popular feeds, they are in the generated ./data/.

mixmix

Have skimmed over this while half awake, left a couple comments

src/generate.ts

mixmix · 2020-12-08T18:57:15Z

I guess the test of this is if you try and use the fixtures. Might be good to list what you expect a person to be able to do with these fixtures (and what they won't be able to do /see if anything)

Eg are there feeds in here which aren't in group and so cannot decrypt? Which ones are they

staltz · 2020-12-08T19:17:24Z

Eg are there feeds in here which aren't in group and so cannot decrypt?

Yes

Which ones are they

It's probabilistic

mixmix · 2020-12-08T20:09:09Z

I think what I'm wondering is if it would be good to have an output at the end which summarises who is in each group (and what dm groups exist) so you know if your tests are working when you decrypt. Do you have that?
(sorry if you do, ts is a bit dense when read on a phone)

staltz · 2020-12-08T20:39:29Z

Great idea! We already have a "report" module, but it doesn't report on tribes, and it could

arj03 · 2020-12-08T23:16:44Z

I was testing this using the instructions posted and ran into this problem:

UnhandledPromiseRejectionWarning: Error: data.recps has less items than allowed
    at groupInvite (/home/arj/dev/ssb-fixtures/node_modules/ssb-tribes/method/group/add-member.js:28:38)
    at Timeout._onTimeout (/home/arj/dev/ssb-fixtures/node_modules/ssb-tribes/method/index.js:32:54)

it was spitting out a ton of debug I could see.

staltz · 2020-12-09T08:15:24Z

@arj03 how many authors do you configure it to run with?

arj03 · 2020-12-09T08:28:58Z

I just ran it with the defaults.

staltz · 2020-12-09T09:32:52Z

Weird, then I'm a bit lost what is the problem... could try to figure it out together in a screen share

arj03 · 2020-12-10T11:01:06Z

I think you can merge this. Still need to test it properly with ssb-db2. It's up to you.

staltz · 2020-12-10T11:03:12Z

Well, when this is merged, it would be a breaking change, so leading to v3 fixtures. I'd rather make sure that what this PR does is correct, so not in a hurry of merging.

staltz · 2020-12-15T15:39:22Z

Maybe we could try getting this one merged this week?

arj03 · 2020-12-16T07:19:20Z

Yes, I will schedule this for friday for me to test.

arj03 · 2020-12-18T08:38:43Z

I'm testing this, I can't get my code to decode the groups but I see there is something wrong with the way they are created. I added some debug to tribe creation and what I see is this:

@V4UXI created tribe %CU2Qd // <-- ?
{
  key: '%jE3SudINIR0Ht9b5kzxAWFJ6uqwAyiWceLb4BUxHofA=.sha256',
  value: {
    previous: '%u4s07Uq7mMMtw6VBdB9xc0PYgZw1Sv2W508H5jMAnbo=.sha256',
    sequence: 769,
    author: '@YHiUEFOxW5nJr8XU2SsX1LsLVJThs/pq5sr5Uxf59HQ=.ed25519', // <-- ?
    timestamp: 1440904785000,
    hash: 'sha256',
    content: 'zNSpJxlgq2lB7m7EPDvbpJUeBxRRXOCMgTd5zDYTxKTWE6yAyeypspLgDWzCqLZn0gMUdmjE59C1Wno5Ka/f4cTxK2YTMbz44iDtLJqK9/3qM7IACpFU+UmdrqlLjeU169yB6gxrx0TIHQzbDg6S1Oqe80kHg+X5AY5+cEm3wnnfVeLnfQYwmDurU/iNVMM1JW0QDt1ufg==.box2',
    signature: 'EmzTwoZA6g4WdJdiHE895SuhSGbv1qtLM8k3KnQH1K8oTqpk0B1Z/uM+K3Dv4WIj+ENtmnBj1E2b5UrSgRdaAQ==.sig.ed25519'
  },
  timestamp: 1440904905000
}

So this means that all groups are creating by the hops0 user.

staltz · 2020-12-18T08:53:46Z

Oooh, I see the problem now. ssb-tribes does the sbot.publish see source code so it dictates who the author is. Otherwise in ssb-fixtures we use sbot.createFeed() then author.add.

Do you have any ideas how we could solve this without changes in ssb-tribes? Or do we need changes in ssb-tribes?

@mixmix ?

mixmix · 2020-12-18T10:04:57Z

I don't use createFeed (and I notice it's marked as deprecated but also in use in places?). Anyway no idea what it would do with tribes.

My approach to starting many different groups from many different authors is use scuttle-testbot to easily spin up many instances and perform operations from there. Might not be fastest, but it works well (and no semi - supported methods!)

Look in the tests of ssb-tribes for a helper test bot file which is the form you want

staltz · 2021-01-07T17:32:38Z

I updated this PR to be based on scuttle-testbot (pending on PR ssbc/scuttle-testbot#13 being merged).

@arj03 can you test?

arj03 · 2021-01-08T08:18:56Z

I ran the new code. Tried deleting the data dir to make sure everything was fresh, then I got a small error because there seems to be a missing mkdir? After creating the dir and running this I get an error:

node lib/bin.js
(node:438019) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 wakeup listeners added to [EventEmitter]. Use emitter.setMaxListeners() to increase limit
(Use `node --trace-warnings ...` to show where the warning was created)
Error: get-group-tangle: unknown groupId %OJwo29P3LvOW7U2KGWAdtHy59YZpThFTVBBrIdOiPb4=.cloaked
    at getGroupTangle (/home/arj/dev/ssb-fixtures/node_modules/ssb-tribes/lib/get-group-tangle.js:14:26)
    at /home/arj/dev/ssb-fixtures/node_modules/ssb-tribes/index.js:116:5
    at /home/arj/dev/ssb-fixtures/node_modules/hoox/index.js:3:17
    at hooked (/home/arj/dev/ssb-fixtures/node_modules/hoox/index.js:10:15)
    at /home/arj/dev/ssb-fixtures/node_modules/promisify-4loc/index.js:3:11
    at new Promise (<anonymous>)
    at /home/arj/dev/ssb-fixtures/node_modules/promisify-4loc/index.js:2:3
    at /home/arj/dev/ssb-fixtures/lib/index.js:87:43
    at Generator.next (<anonymous>)
    at fulfilled (/home/arj/dev/ssb-fixtures/lib/index.js:5:58)
(node:438019) UnhandledPromiseRejectionWarning: Error: unknown groupId %OJwo29P3LvOW7U2KGWAdtHy59YZpThFTVBBrIdOiPb4=.cloaked, cannot encrypt message
    at /home/arj/dev/ssb-fixtures/node_modules/ssb-tribes/envelope.js:32:29
    at Array.reduce (<anonymous>)
    at boxer (/home/arj/dev/ssb-fixtures/node_modules/ssb-tribes/envelope.js:29:40)
    at box (/home/arj/dev/ssb-fixtures/node_modules/ssb-db/autobox.js:18:18)
    at dbAppend (/home/arj/dev/ssb-fixtures/node_modules/ssb-db/minimal.js:149:23)
    at /home/arj/dev/ssb-fixtures/node_modules/ssb-db/minimal.js:36:37
    at AsyncJobQueue.runAll (/home/arj/dev/ssb-fixtures/node_modules/ssb-db/util.js:197:32)
    at /home/arj/dev/ssb-fixtures/node_modules/ssb-db/minimal.js:36:24
    at /home/arj/dev/ssb-fixtures/node_modules/ssb-db/minimal.js:46:33
    at AsyncJobQueue.runAll (/home/arj/dev/ssb-fixtures/node_modules/ssb-db/util.js:197:32)

I wonder if its the same error as Henry ran into.

staltz · 2021-01-08T10:52:47Z

Indeed, lots of errors. Gonna fix them.

staltz · 2021-01-08T14:31:03Z

Updated this branch to fix a bunch of errors, but now I'm getting a deterministic error, would need help from @mixmix maybe:

PS: I removed some console.logs from scuttle-testbot and ssb-tribes such as rebuild!!! (ﾉ´ヮ´)ﾉ*:･ﾟ✧ and 4zamJnOv: 33 --> @daV3I9mK (sees: vote) which by the way are annoying as I can't turn them off with an opt and they're on by default.

$ node lib/bin.js --seed=fofa

(node:24944) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 wakeup listeners added. Use emitter.setMaxListeners() to increase limit
@U0S0a created tribe %N9Toa
@hHwN2 created tribe %Cmuc1
@deroA created tribe %zse9M
@tzqcW created tribe %CAjt8
@R7BNB created tribe %ifAGG
@U0S0a invited @JjoXx,@AvdFd,@ufy9m,@ticdx,@ED33o,@VUXDz,@tZ3s7,@A/CDi,@XPVPm to tribe %N9Toa
@ticdx posted to tribe %N9Toa
@YSoCU created tribe %LXq/s
@2aVQd created tribe %I1V2K
@4zamJ created tribe %JEGUN
@ticdx posted to tribe %N9Toa
@+TRWd created tribe %iVmmE
@ticdx posted to tribe %N9Toa
@4zamJ invited @DYKYl,@tzqcW,@MHyhO,@a1RS/,@opada,@HB4JF,@ticdx,@daV3I to tribe %JEGUN
@4zamJ posted to tribe %JEGUN
@8c2+m created tribe %8vftO
@ticdx created tribe %qPV0W
@opada posted to tribe %JEGUN
@km3v4 created tribe %pOZR3
@YSoCU posted to tribe %LXq/s
@DYKYl posted to tribe %JEGUN
@ticdx invited @YSyDx,@qhQAJ,@nMB8t,@S9ah0,@vvXbo,@igXd5,@qtCZl,@7Bs+h,@UIIGm to tribe %JEGUN

/home/staltz/ssb-fixtures/node_modules/.pnpm/[email protected]/node_modules/levelup/lib/levelup.js:188
        err = new NotFoundError('Key not found in database [' + key + ']', err)
              ^
NotFoundError: Key not found in database [%ABkoVYXwv2+1VWWb7mWnwdFDdIkuaYgTzLTbmPj1Bfc=.sha256]
    at /home/staltz/ssb-fixtures/node_modules/.pnpm/[email protected]/node_modules/levelup/lib/levelup.js:188:15
    at /home/staltz/ssb-fixtures/node_modules/.pnpm/[email protected]/node_modules/encoding-down/index.js:75:21

mixmix · 2021-01-08T20:16:49Z

Did you make it so some of those are disabled or disableable? Many of those logs are quite important / useful for understand what's happening in tests.

The key not found in db is something I've seen before and don't know what it is. I'm in holiday at Bob's at the moment. Can pair on this if you'd like when I'm back in Wellington. Feel free to dm on signal

arj03 · 2021-02-25T15:54:46Z

Pushed up the changes @mixmix and I paired on.

This now closer to working but is dependant on this bug in tribes.

And this bug:

# Fatal error in , line 0
# Check failed: result.second.
#
#
#
#FailureMessage Object: 0x7ffdda3eb7c0
 1: 0xa69c61  [node]
 2: 0x1999544 V8_Fatal(char const*, ...) [node]
 3: 0xe41af9 v8::internal::GlobalBackingStoreRegistry::Register(std::shared_ptr<v8::internal::BackingStore>) [node]
 4: 0xb9dc98 v8::ArrayBuffer::GetBackingStore() [node]
 5: 0x9bc360 napi_get_typedarray_info [node]
 6: 0x7f9504eddabd sn_randombytes_buf [/home/arj/dev/ssb-fixtures/node_modules/sodium-native/prebuilds/linux-x64/node.napi.node]
 7: 0x9b40df  [node]
 8: 0xbdc91b  [node]
 9: 0xbddec6  [node]
10: 0xbde546 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [node]
11: 0x13cc9d9  [node]
Illegal instruction (core dumped)

and this one:

(node:97293) UnhandledPromiseRejectionWarning: Error: initial message must have previous: null, on feed:@p05LpBnUlkgg8e3bjhg26iXnvtNcDEXHdKCN6gAZGpU=.ed25519
    at Object.exports.checkInvalidCheap (/home/arj/dev/ssb-fixtures/node_modules/ssb-validate/index.js:155:20)
    at Object.exports.checkInvalid (/home/arj/dev/ssb-fixtures/node_modules/ssb-validate/index.js:168:21)
    at Object.exports.appendKVT (/home/arj/dev/ssb-fixtures/node_modules/ssb-validate/index.js:231:17)
    at Object.exports.append (/home/arj/dev/ssb-fixtures/node_modules/ssb-validate/index.js:262:18)
    at queue (/home/arj/dev/ssb-fixtures/node_modules/ssb-db/minimal.js:119:9)
    at /home/arj/dev/ssb-fixtures/node_modules/ssb-db/minimal.js:36:37
    at AsyncJobQueue.runAll (/home/arj/dev/ssb-fixtures/node_modules/ssb-db/util.js:197:32)
    at EventEmitter.queue (/home/arj/dev/ssb-fixtures/node_modules/ssb-db/minimal.js:36:24)
    at EventEmitter.db.add (/home/arj/dev/ssb-fixtures/node_modules/ssb-db/create.js:95:8)
    at apply (/home/arj/dev/ssb-fixtures/node_modules/muxrpc-validation/index.js:197:15)

staltz · 2021-02-25T16:12:24Z

Thanks for doing it! Cool to see you dabbling with TypeScript

mixmix · 2021-02-25T19:29:19Z

I have seen that core dump before.
I think this solved it
https://github.com/mixmix/ssb-tribes/blob/master/test/helpers/test-bot.js#L23

mixmix · 2021-02-25T19:39:54Z

The unhandled promise rejection... looks bad. Is it a seq 1 message with a non null previous?
No idea how that would be possible

staltz · 2023-03-02T08:33:24Z

Bye bye PR

support ssb-tribes

7ee150e

staltz requested review from arj03 and cryptix December 8, 2020 18:17

mixmix reviewed Dec 8, 2020

View reviewed changes

src/generate.ts Outdated Show resolved Hide resolved

src/generate.ts Outdated Show resolved Hide resolved

src/generate.ts Outdated Show resolved Hide resolved

fix usage of ssb-tribes

72b0d69

fix "data.recps has less items than allowed"

f58d041

staltz mentioned this pull request Jan 7, 2021

allow passing path as input opt ssbc/scuttle-testbot#13

Merged

staltz added 2 commits January 7, 2021 19:31

change implementation to scuttle-testbot

8772817

use tap-spec

99a2581

staltz marked this pull request as draft January 7, 2021 17:32

staltz added 2 commits January 8, 2021 16:21

fix ENOENT bug

b783d20

improve makeSSB usage of scuttle-testbot

8f5d0b0

staltz added 3 commits January 8, 2021 16:24

fix some issues replication using ssb-tribes

a975649

use tap-bail in tests

c6591eb

use new [email protected]

57b88c8

Upgrade ssb-tribes, fix tribesByAuthor

9c91bb3

arj03 mentioned this pull request Mar 12, 2021

Auto encrypted publish when recps is set ssbc/ssb-db2#212

Merged

staltz closed this Mar 2, 2023

staltz deleted the tribes branch March 2, 2023 08:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

support ssb-tribes #2

support ssb-tribes #2

staltz commented Dec 8, 2020

mixmix left a comment

mixmix commented Dec 8, 2020 •

edited

Loading

staltz commented Dec 8, 2020

mixmix commented Dec 8, 2020

staltz commented Dec 8, 2020

arj03 commented Dec 8, 2020

staltz commented Dec 9, 2020

arj03 commented Dec 9, 2020

staltz commented Dec 9, 2020

arj03 commented Dec 10, 2020

staltz commented Dec 10, 2020

staltz commented Dec 15, 2020

arj03 commented Dec 16, 2020

arj03 commented Dec 18, 2020 •

edited

Loading

staltz commented Dec 18, 2020

mixmix commented Dec 18, 2020

staltz commented Jan 7, 2021

arj03 commented Jan 8, 2021

staltz commented Jan 8, 2021

staltz commented Jan 8, 2021

mixmix commented Jan 8, 2021

arj03 commented Feb 25, 2021

staltz commented Feb 25, 2021

mixmix commented Feb 25, 2021

mixmix commented Feb 25, 2021

staltz commented Mar 2, 2023

support ssb-tribes #2

support ssb-tribes #2

Conversation

staltz commented Dec 8, 2020

mixmix left a comment

Choose a reason for hiding this comment

mixmix commented Dec 8, 2020 • edited Loading

staltz commented Dec 8, 2020

mixmix commented Dec 8, 2020

staltz commented Dec 8, 2020

arj03 commented Dec 8, 2020

staltz commented Dec 9, 2020

arj03 commented Dec 9, 2020

staltz commented Dec 9, 2020

arj03 commented Dec 10, 2020

staltz commented Dec 10, 2020

staltz commented Dec 15, 2020

arj03 commented Dec 16, 2020

arj03 commented Dec 18, 2020 • edited Loading

staltz commented Dec 18, 2020

mixmix commented Dec 18, 2020

staltz commented Jan 7, 2021

arj03 commented Jan 8, 2021

staltz commented Jan 8, 2021

staltz commented Jan 8, 2021

mixmix commented Jan 8, 2021

arj03 commented Feb 25, 2021

staltz commented Feb 25, 2021

mixmix commented Feb 25, 2021

mixmix commented Feb 25, 2021

staltz commented Mar 2, 2023

mixmix commented Dec 8, 2020 •

edited

Loading

arj03 commented Dec 18, 2020 •

edited

Loading