Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NIP-62: Curated Publications #1600

Open
wants to merge 12 commits into
base: master
Choose a base branch
from
Open

Conversation

limina1
Copy link

@limina1 limina1 commented Nov 22, 2024

This NIP defines a specification for organizing nostr events into collections - ordered, optionally-hierarchical structures like books, journals, or documentation.

Currently, there's no standardized way to create versioned collections of content. The closest option is to use lists, but this extends that idea beyond a strictly linear hierarchy.

Introducing two new event kinds:

kind:30040 - Collection Index

  • Acts as a table of contents
  • Contains metadata about the collection
  • Lists events in display order

kind:30041 - Collection Section

  • Contains content
  • links to other content via wikilinks

@SilberWitch

@SilberWitch
Copy link
Contributor

@vitorpamplona
Copy link
Collaborator

Looks good! I still think "Collections" is too generic of a name though.

Also, the use of e tags on 30040 means that you want specific versions of kind 30041s. Since they are replaceable and many relays delete old versions, a client might not find the version you want at the collection level.

You can use a tags to pick the latest version of each 30041 or keep it as is and require collection owners to constantly update their index after each change to one of the sections.

@SilberWitch
Copy link
Contributor

That's true. We were working on a tags, a few weeks back, but it never made it into the spec.

@SilberWitch
Copy link
Contributor

Why did you remove the kind entry @limina1 ? Thought that was correct.

@SilberWitch
Copy link
Contributor

Just saw a typo in the second example.

["wikilink", "fables"

should be

["wikilink", "fable"

@staab
Copy link
Member

staab commented Nov 22, 2024

Coracle currently supports collections as tagged NIP 32 labels. Just FYI, I don't care strongly about this one.

@SilberWitch
Copy link
Contributor

Thanks.

@SilberWitch
Copy link
Contributor

Coracle currently supports collections as tagged NIP 32 labels. Just FYI, I don't care strongly about this one.

E-Books constructed out of labels, or do you mean, that those labels are also called "collections"?

62.md Outdated
["author", "Aesop"],
["i", "isbn:9780765382030"],
["t", "fables", "classical", "literature"],
["published_on", "2003-05-03"],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIP-23 uses "published_at" for the timestamp the article was first published, and uses UNIX timestamp format-it might make sense to re-use that tag here, instead of defining a new one with a new format?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I originally had that, but UNIX timestamps only go back to 1970, and we are publishing old books.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, that's true. Having to use negative timestamps for old books would be bad.

In that case, it would make sense to document the date format explicitly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I originally specified ISO 8601.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise, European and American dates would differ.

Copy link
Contributor

@laanwj laanwj Nov 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly. So just YYYY-MM-DD? i doubt this needs any of the extended formats, timezone suffixes, year-week format etc from ISO-8601.
(though for a book, "year of publication" is generally all the information available, also allowing like YYYY or YYYY-MM might make sense, though it complicates sorting)

@staab
Copy link
Member

staab commented Nov 22, 2024

More the latter. It's a mechanism for content curation, which is what I usually think of when I hear the word "collection". This PR seems more like an "ebook" kind of thing rather than a collection. The NIP 32 approach probably wouldn't be appropriate, but its benefit is that the contents of the collection can be dynamic based on who you accept events from. But that's all probably irrelevant to your use case here.

62.md Outdated
["title", "Aesop's Fables"],
["author", "Aesop"],
["i", "isbn:9780765382030"],
["t", "fables", "classical", "literature"],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

["t", "fables"],
["t", "classical"],
["t", "literature"],

might be more useful, as then one could query for either of the tags. It also matches the current definition of the t tag in README.md.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, you're right, just checked and it's like that in my last Habla-generated article.

@SilberWitch
Copy link
Contributor

Looks good! I still think "Collections" is too generic of a name though.

We thought about "Library", since they're the core events for the Alexandria client, but we want to use them to publish the Bible and that literally translates to "library", so it's sort of confusing.

@SilberWitch
Copy link
Contributor

This PR seems more like an "ebook" kind of thing rather than a collection.

We've been resisting naming it "ebook" because it could be used for something like a project plan, anthology, research paper, academic journal, user manual, etc. We were calling it "modular article" and then "Nostr Knowledge Base", so we keep changing the name.

@SilberWitch
Copy link
Contributor

Maybe we should call it "Documents". If you export a 30040, together with its 30041, you get a document file, in whichever format you want: Asciidoc, ePUB, LaTeX, Markdown, PDF, etc.

@vitorpamplona
Copy link
Collaborator

Given the rest of the tags, 30040 could be called a Publication. Then this NIP's name could be simply Publications or Editorial Publications and 30041 could be called Content

idk... just ideas.

@SilberWitch
Copy link
Contributor

That's brilliant. Actually. We keep talking about how we're using this to publish stuff, but we couldn't think of what name to give the stuff, but we could just give the name of the action we're doing.

What do you think, @limina1 @buttercat1791 ?

@buttercat1791
Copy link

Publication makes me think of magazines. Not a bad implication. Perhaps something like Curated Collection or Curated Publication could also work. Curated to emphasize that it can be comprised of Content from a number of different sources.

@SilberWitch
Copy link
Contributor

Curated publication sounds pretty good. Sounds fundamentally different from NIP-51 "curation sets" lists. Makes it more clear that one document comprised of sections is the result, rather than just a list of events.

@limina1
Copy link
Author

limina1 commented Nov 23, 2024

I like it. Curated publications as a structure itself requiring publication metadata 30040 and publication content 30041.

@SilberWitch
Copy link
Contributor

Probably also have to change the corresponding entries in the ReadMe.md, under "Event Kinds" and "Tags".

@nusapuksic
Copy link

nusapuksic commented Nov 23, 2024

Do you have a way to mark the index as draft? I would like to distinguish between drafts and final versions, since it might take time, multiple people and iterations to get the end result together and then the clients can filter by status.

@SilberWitch
Copy link
Contributor

We don't have that in the client, yet, we've just been using browser cache, but that would make this more consistent with other longform. 🤔

30042 for draft, or something.

@buttercat1791
Copy link

If 30040/30041 are final versions, then 30042/30043 could be drafts.

@vitorpamplona
Copy link
Collaborator

Keep in mind there is also a private drafts proposal on #1124

In that way, people can't see the drafts being made, which to me is a constant complaint on many draft proposals that people can just integrate with their clients and see before they are ready.

@buttercat1791
Copy link

In that case we can subsume drafts for Curated Publications into the generic kind 31234 draft event.

In any case, 30040/30041 events are by nature replaceable, so even if they're not marked as "draft" they are still editable to any client that supports edits.

@limina1
Copy link
Author

limina1 commented Nov 23, 2024

Lists are a general structure, where there isn't a specific kind attached to it.

If its important enough, I see a possibility in expanding the specification for lists; where lists are composable, like curated publications which require two kinds. However, NIP-66 is a specific instantiation of that structure, where a client feed is composed out of 30040s just as microblogging clients are composed out of kind1s.

It could have a reference within nip-51 as hierarchical, composable lists. Throwing out some ideas

@SilberWitch
Copy link
Contributor

Nobody reading a magazine article or book thinks of it as a list, and someone wanting to build an eReader client isn't going to go to the list NIP. I mean, you could say wiki pages are just a special kind of article and articles are just long microblogs and comments are just microblogs in reply, so everything is just a Kind 01 note and all of the NIPs should be collapsed down to one.

But the NIPs are clearly domain and use case centered, so that developers don't all have to watch the same NIP, so that every tiny change to one part of the spec results in 30 people throwing a fit. Someone wanting to build a book client, will want a NIP about books.

Done discussing it. Moving on. This NIP is mostly a formality and a chance to make slight corrections, as the event is already being handled or produced by three clients, and will soon be added to more.

Anyone have any constructive feedback?

@Darecjo
Copy link

Darecjo commented Nov 24, 2024

I have nothing to add. I just proposed referencing it within the nip-51 or expanding the specification however if you are saying it is implemented by 3 clients and already in use then you should go for it. I just strongly suggest using a unique UUID rather than a title-generated d-tags (as far as i’m seeing), because it can cause editing problems or conflicts between existed copies since relays may fail to publish an event in one try.

@SilberWitch
Copy link
Contributor

We've refrained from specifying the content for d-tags, as clients have different methods. It might need to be standardized somewhere, for each use case, but it's a longer and more-difficult conversation.

@SilberWitch
Copy link
Contributor

The same d-tag issue arises also for longform and wikis, but it's slightly out of scope.

@Darecjo
Copy link

Darecjo commented Nov 24, 2024

I think it is standardized somewhere but not sure where.

@mntn-xyz
Copy link

See related thoughts on #1011 and #1139. The three use cases I was originally looking at, which would be satisfied by this, are:

  • A "sitemap" that collects various events and organizes them into an organized published "site" that can be browsed by supported clients (for publishing website-style content)
  • A "book" collecting various events into chapters, subsections, etc
  • A "directory" that curates profiles and other collections into an organized hierarchical collection, like site directories from the early web

This NIP would satisfy all these use cases! Would be happy to see this formalized.

62.md Outdated
["published_by", "public domain"],
["image", "https://imageserver.com/piclink.jpg"],
["summary", "Collection of selected fables from the ancient Greek philosopher, known as Aesop."],
["e", "<id>", "<relay_url>", "<pubkey>"],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Being forced to change the kind:30040 every time any kind:30041 might be a bit annoying; I would use the a tag here or at least have it as an option (I,.e. if you were referring to someone else's event you might want to anchor it to a very specific version in which case an e tag would be useful)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we should maybe note that a tags can be used instead of e tags. @limina1

Copy link
Collaborator

@vitorpamplona vitorpamplona Nov 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, let's move this to a, A tags.

Should it also have a description to render without having to load the attached event?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know. Sort of turns it into a list, rather than a publication. I mean, anyone can add a description tag to 30041s, of course, and use it that way, but that's a specific implementation out of scope of the NIP.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've added a tags as an alternative, but we need to keep e tags for research papers and other technical documents.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with y'all, the solution proposed is quite elegant.

The one thing I'll add to this discussion is that we shouldn't remove e tags from the specification entirely. We can reference 30041 and other replaceable event kinds with a tags, but some use cases of the 30040 kind may require referencing non-replaceable events, in which case e will suffice and we don't need to worry about content changing.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like this idea of using 30040 for other kinds. Because then clients that implement this spec must also implement every other kind that gets attached to it. Which is terrible for interoperability.

Do we have real uses of other kinds in this? If so, let's work through each of them and put the guidance in this nip.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a wishlist item. I would like to be able to use kind 1063, so that one could put together a journal article that has text sections and media sections. I am aware that images can be added directly into content, but this works somewhat like stock images or custom-made graphics that can be attributed directly. If this is out of scope, I understand.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it is out of scope. Anything text, tables, code, etc., can be in a 30041 (or migrated to, or embedded in, one), but something like an image is sort of extra.

Numerous artists have complained about their images being altered/skewed/cropped by clients, so we should maybe treat images with more care. Identifying something explicitly as media would also ease formatting options like text-wrapping and floating.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like this idea of using 30040 for other kinds. Because then clients that implement this spec must also implement every other kind that gets attached to it. Which is terrible for interoperability.

Do we have real uses of other kinds in this? If so, let's work through each of them and put the guidance in this nip.

Perhaps this NIP will need to be updated over time to provide helpful guidance for integrating with various event kinds.

Right off the bat, though, an obvious case for 30040 events referencing a kind other than 30041 would be to create curated publications that republish long-form articles. A user might use the NIP-62 hierarchical curation structure to set up a magazine publication that includes a mix or original articles written using kind 30041, and republished, older articles written in 30023.

At some point, the client developer can determine how far to extend the supported features beyond the basic 30040 and 30041 pairing specified in this NIP.

@pablof7z
Copy link
Member

LGTM other than one minor comment

@vitorpamplona
Copy link
Collaborator

Which clients support this already and where can I find a realistic example? :)

@limina1
Copy link
Author

limina1 commented Nov 27, 2024

The two clients currently implementing this structure are
Alexandria and Highlighter

@SilberWitch
Copy link
Contributor

And my uploader, that is a CLI I use for test data. Highlighter and the Uploader need to incorporate the changes. We'd been using markdown, still.

Chachi will also be getting a tab for this, so that communities can have their own library.

@SilberWitch
Copy link
Contributor

Here is a highlighter example, but it's missing the d tag, as we added that later. All the examples are now outdated. I'm reluctant to revamp the clients and upload more books, so long as we keep fiddling with the event structure, since I'd then have to go back and redo everything and some of the 30040s are enormous. Would be really helpful if we could agree-and-freeze the event structure, here, which is why we decided to go for a NIP.

https://highlighter.com/a/naddr1qvzqqqr4tqpzqhc836gvaauxlg9vwafps2gv298rxvf7q7f4qlk32d6ucket4j4rqyvhwumn8ghj7un9d3shjtnwdaehw6r9wfjjucm0d5hsqqqlj89ep

Here is an event, that works well in Alexandria, with a d tag. It's not a hosted program, yet, (we're waiting for v0.1.0) but you can see how it displays here, or just run the client locally.

Then you should see this:

image

https://njump.me/naddr1qpp5zetndaczwuedgeskymr9wvkky7fdg9jhxmms94mz6nn9wuk4yetkd9ek2epd2ejhyumfdahz63njdakj6nmjd9nkjmnpdsk4xmm4wf3k2ucprdmhxue69uhhg6r9vd5hgctyv4kzumn0wd68yvfwvdhk6q3q2umrfdjgvdxt45g0y3ghwcyfagssjrv5qlm3t6pu2aa5vydwdmwqxpqqqp64sx4lzyd

@SilberWitch
Copy link
Contributor

SilberWitch commented Nov 27, 2024

Okay, went ahead and updated the eBook uploader to this NIP version. That's at least one writing with this format, then, and Alexandria displays them. @VnUgE is working on getting a test instance of Alexandria running on our gitcitadel website.

security model tag specifies how events should be updated
Copy link
Author

@limina1 limina1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Modified a tag description and security model to specify how events update

62.md Outdated Show resolved Hide resolved
62.md Outdated Show resolved Hide resolved
62.md Outdated Show resolved Hide resolved
62.md Outdated Show resolved Hide resolved
@arthurfranca
Copy link
Contributor

arthurfranca commented Nov 28, 2024

What do you think of adding the section title to the a tag like ["a", "<kind:pubkey:dtag>", "<relay hint>", "<event id>", "The Farmer and The Snake"]?

This way you don't need to load all linked events to build a named section list. As a bonus, the section titles can be different from what's in the content events.

limina1 and others added 3 commits November 28, 2024 10:56
Co-authored-by: arthurfranca <[email protected]>
Co-authored-by: arthurfranca <[email protected]>
62.md Outdated
["a", "<kind:pubkey:dtag>", "<relay hint>", "<event id>"],
["auto-update", "<yes|ask|no>"],
["p", "<pubkey_0>"],
["E", "<original_event_id>", "<relay_url>", "<pubkey>"]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this original event id also be A?

@buttercat1791
Copy link

What do you think of adding the section title to the a tag like ["a", "<kind:pubkey:dtag>", "<relay hint>", "<event id>", "The Farmer and The Snake"]?

This way you don't need to load all linked events to build a named section list. As a bonus, the section titles can be different from what's in the content events.

Not a bad suggestion, but I'm not sure about it either. The split of responsibilities in this NIP has been that 30041 contains the "meat" of the content, and 30040 organizes the content. Moving section titles into the a tag references in 30040 events starts to cross that boundary.


`draft` `optional`

This NIP defines the minimum specification for curated publication - ordered, optionally-hierarchical assemblies of nostr events. Collections provide a standard way to organize and present related content, similar to how books organize chapters or journals organize articles.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typos:

  • "curated publication" should be "curated publications
  • "nostr" should be "Nostr"


## 30041: Publication Content

Also known as sections, zettels, episodes, or chapters contain the actual content that makes up a collection.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be "that makes up a publication."

["d", "aesop's-fables-by-aesop-the-farmer-and-the-snake"],
["wikilink", "fable", "<pubkey>", "wss://thecitadel.nostr1.com", "<event id>"]
],
"content": "== The Farmer and The Snake\nA [[fable]], by Aesop.\nONE WINTER a Farmer found a Snake stiff and frozen with cold. He had compassion on it, and taking it up, placed it in his bosom. The Snake was quickly revived by the warmth, and resuming its natural instincts, bit its benefactor, inflicting on him a mortal wound. 'Oh,' cried the Farmer with his last breath, 'I am rightly served for pitying a scoundrel.'\nThe greatest kindness will not bind the ungrateful.",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wikilinks in NIP-54 does not have a tag for reference and is different from this, is this intentional? How about using nostr:... links of NIP-21?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use nostr: links in any text; that's just a rendering issue.

This is a wikilink from a note that isn't a wiki page, where the specific version might need to be specified and where the client doesn't necessarily handle wikis. We should maybe use something like the new a-tag, instead of an eventID.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The wikilink specification here intentionally diverges from NIP-54. I agree we could use a parameterized replaceable event reference like with a tags in the wikilink.

One of the goals with wikilinks, in terms of client implementation, is that they always "just work." Even if, for one reason or another, the specific referenced event can't be found, a client might tie the wikilink to a search feature to find related events by keyword.

That's just one possibility, but that's why we're both diverging from NIP-54 somewhat, and not just using nostr: links.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok I see. Thanks for explaining.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.