@criztovyl's ForgeFed web log - TFO Edition

A thing that comes into my mind when comparing mail to other forms of communication is that non-linear threads are a feature you do not have in other (web) communication; there communication is mostly linearized. In linear(ized) communication for example it is, at least to me, harder to follow a sub-topic in a thread. But that might come with the list+"pre"view structure most mail clients have.

I would boil this down to “quantity is not quality”. Sloppily said: “There is more, but is it any good? And even if it’s good, is it due to lowered bar of entry?” [citiation needed]

For the rest; I read it interestingly but cannot type any useful response right now. But I guess my response is not required, just interesting. ^^

Before I spend two more days polishing this post for not that much more value, I’ll let it be as it is, at least for the moment. :slight_smile:

I keep spending time thinking about the challenge of adding ActivityPub to existing software.
Basically, what keeps going around in my head is the challenge of making stuff asynchronous. Like, I comment on an issue and now the forge has to federate it to the subscribers: Doing this synchronous does not make much sense to me.

Looking into GitLab, there is no problem with asynchronicity, they already have an background processor.
During my quick look around in Gitea/Gogs I saw no background processor there. But go has go routines, so a real background worker might not be required.

Yep it doesn’t. It all starts falling apart once you need to send to hundreds or even thousands of remote servers. ActivityPub especially is pretty verbose as a protocol.

Background jobs using a queue system is the way to go :+1:

Regarding the problem of “Remote Users” (@fr33domlover) , I think the term “Users” is misleading here. I would rather stick with the AP term “Actor” (w/o “Remote”).

To me an User’s most important property is the authentication information (for example a password hash); it tells the application: Everyone who can authenticate is allowed to use the given identity (for example by knowing the password).
An Actor on the other hand just describes an identity; an identity an User is allowed to use.

For example imagine a issue discussion.

When thinking about the author of a comment, a status change and the author of the issue itself, the identity is important, not what is required to control it. In this context the author can be represented by an Actor.

The User only comes into play when trying to add a comment to the discussion (but it is not required, for example when using C2S1): There you first have to prove that you are worthy of controlling the identity (i.e. authenticate you). Afterwards the the only important thing is which identity you now control.

So, my thoughts are to move everything that is Actor related to an own table (identity information like [preferred] username, AP ID, profile picture, …) and only keep properties that are really required for Users in the corresponding table (authentication information).

The UX challenge here is then now how to present Actors in the UI. I think of something like:

@criztovyl https://criztovyl.space/actor

One challenge there is handling preferredUsername collisions. Especially malicious ones.
Like this (very badly constructed):
@f00user https://gitlab.com/f00user” vs. “@f00user https://gillab.com/f00user
But maybe add some kind of short, 4 char thingy to the username, from some kind of hash (sha256 in this case)?
@f00user 112c https://gitlab.com/f00user” vs. “@f00user 7999 https://gillab.com/f00user

So, this then?

@criztovyl b1ee https://criztovyl.space/actor

But that’s only 16 bits of 256bits, with 4 RGB colors one maybe could get more?
Looks okay directly below each other, no idea if it helps if it’s not directly below each other.

Screenshot%20from%202019-06-30%2015-20-05
Screenshot%20from%202019-06-30%2015-26-03
Screenshot%20from%202019-06-30%2015-36-06
I like the •'s
Screenshot%20from%202019-06-30%2015-28-24

Standalone Gitlab • Example (faked via Developer Tools):


1 in case of C2S you only need control over the identity’s private key for signing your request.

So, here the current challenges for building ForgeFed into existing software as I see them at moment, boiled down to two bullet points:

  • asynchronous processing of events (Federating events)
  • separate User and Actor entities

The last point, separation of Users and Actors, for example would help preventing the problems as described by @mikaela, even if in a completely different context ^^

1 Like

@criztovyl, FYI I added federation to an existing web application (that I wrote). So if you’re wondering what the challenging points are, I may have hints :slight_smile:

I don’t think async processing is a challenge; I coded my own async delivery mechanism. And if instead you use an existing one, as @jaywink suggests, then it’s even easier.

Handling of users and actors too; you can just do the plain regular Fediverse stuff: Publish actor documents, publish public keys for HTTP Signatures. Nothing fancy. And you don’t need to care at all about preferredUsername. I think it’s there only because of WebFinger. You identify remote users by their ID URI, not by their username.

I posted a little while ago a post on the Gitea forum with an idea for how to implement federation. It’s based on my experience implementing federation in Vervis. It’s very very general, but take a look :slight_smile:

1 Like

It is a challenge in the sense of not being part of the already present forge code. True, not a big challenge, but still a (small) (technical) problem to solve, something to have thought of, to be aware of. Adding AP is more than just adding a new REST endpoint.

For technical handling, yes, I don’t need to care, true.
But for UI/UX I think it an (important) point to keep in mind. In the end UI’s need to find something between plain usernames and full URIs/IRIs.
Either one stand-alone will lead to problems: Usernames are not “secure”, it would be too easy to impersonate somebody maliciously. Full IRIs on the other hand are not suitable for everyone and are not particularly nice-looking either. :smiley:

generally speaking, this seems like another example as i discussed with fr33domlover a few days ago, of how the language of AP confuses the thinking about integrating it into existing applications - the job of the forge-fed layer is to abstract away the quirks of AP so that the forge can behave as normally as it already does - for that reason, it would be far more helpful to think about it in terms of how to interpret AP messages into something that forges already do, rather than how to adapt the behavior of forges to conform to the AP paradigm - trying to change the way that forges behave would only add friction towards wide adoption

concretely, AP actors are not something that the forge needs to treat as first class - from the forge perspective, the relevant entities that need to be represented as AP actors are users, teams, repos, and tickets - all of these already exist in the forge database with the appropriate separation - an “AP actor”, from the forge perspective, equates to “anything that publishes a feed of event to which other users can subscribe, or that accepts analogous incoming events” - those are most likely the same set of forge entities: users, teams, repos, and tickets - for AP purposes, the forge would only need to expose an actor document for each of those entities, that are to be translated into AP actors at the forge-fed boundary - everything else is a matter of translating the standard forge events into AP messages that will be translated into the standard forge events on the remote forge, and vice-versa - its most likely that the forge-fed layer would require no storage or memory (other than the local actor documents) - ideally, the only intrusion into the forge database would be the HTTP signing keys - additionally, a new optional database field, indicating that foreign users are distinct from local login users, and an optional field to hold user’s GPG keys

one key implementation detail that was not noted here is that forges would need to add foreign users to their database, probably as a limited, non-login, phantom user - once you have that, the rest falls into place from the forge perspective

probably, most forges already post events asynchronously, such as webhooks and email notifications - there is no new magic in AP - it is just a messaging protocol - asynchronous messaging is common-place in large systems - it is most likely that all of the necessary parts or dependencies will already be present, except for HTTP signatures

the usernames are in the URI - that could be easily parsed from the URI to be correlated to a phantom user in the DB (or create a new one); as long as we mandate a conventional format for globally unique user-ids, such as ‘criztovyl@criztovyl.net’; and that the actor document be located in a directory named with the globally unique UID - IIRC the actor document is present in every AP message - that user’s URI would look something like: https://criztovyl.net/u/criztovyl@criztovyl.net/actor.json - globally unique user-ids are essential; and that is exactly what should be shown in the GUI to represent that user

i dont remember this being proposed before; but imagine supporting an “@mention” feature - it would be absolutely necessary to mention another remote user as @criztovyl@criztovyl.net; because the local forge may have its own local user with the user-id criztovyl, who can be pinged as @criztovyl - impersonation is not possible; because of the signatures on the messages

This would not be compatible with any platform that doesn’t include usernames in the URI. Using an UUID is very common out there since not all platforms want to lock the username like for example Mastodon does. Of course anyone who wants to federate with Mastodon probably has to :slight_smile: But anyway, if folks want to expose a username I would strongly suggest doing it the way it is currently being done on many platforms ie adding a preferredUsername to the Actor document.

The UX is a real issue in AP land, which is why preferredUsername is a popular thing. It’s much easier to expose that to end users than an URL ID.

Again, repeating some earlier concerns that ForgeFed tries to take on both forge related extensions and “fixing AP”. The username and UX issues are a part of the latter, nothing to do with forges. I’d suggest bundling username related recommendations concerning UX to a separate document outside ForgeFed extensions.

I don’t think it’s AP’s language that cofuses here.
My main issue was plainly the, in my opinion, imperfect way to represent AP Actors as Remote Users.
What I surely did do, was mess around with, well, wrong words.

Just replace “Actor” (AP) with “Identity” and the AP-confusion is gone:

Let me repeat here: My concern was not about Users and Actors specifically but Users and Identities generally, although I messed up the wording pretty bad.

Anyway, let’s continue.

Please note that my concern is about directly adding to the core without an additional layer.

User-vs-Identity is not AP paradigm but Federation paradigm. And the forge will have to adjust to it in one or another way.
My last post just criticized the “Remote User” way and described an “Identity” way. (And includes some thoughts on how to UX/UI-wise represent Identities (which can be applied to Remote Users too))

Yeah, I guess I am over-concerned with overloading the users table / entity.
When overloading, the essential field’s I would add would just be, expressed in my user-vs-identity thinking, an “identity-only” boolean field, the identity IRI and identity public key.

I think adding GPG keys should not require an additional field - forges that support keys will already have an relation for that; and forges not supporting it yet will be, in my opinion, better off with implementing it as a relation than a field.

Also, which role do the GPG keys play in this context?
Signing keys - sure, for verifying the signature. The indication field - obviously.
But GPG keys? Sure, for verifying commits, but that does not fit into how-to-represent-users-that-are-not-from-this-server context. :smiley:

As @jaywink said:

IIRC preferredUsername is even part of the AP spec!

The question is: What is the user ID?
To me, the ID is the actor’s IRI. So, for my examples, https://criztovyl.space/actor. That is globally unique. That is what can be presented to the user.

But, from UI/UX PoV, is not a good way. And, there we are at my dilemma of how to present the IRI ID to the user.
I don’t want to hide it. I just want to add something for the user to make the UX a little bit nicer. And just IRIs everywhere is simply not particularly nice IMHO1.
If you prefer IRIs everywhere - fine, there should be an option to hide the additional information. In the end the user must to be able to decide what (s)he wants. But any application I would distribute as a default always would show additional info for the IRI.

This very much depends on how you implement embedding mentions into and extracting them from text.

Internally, I would always embed the mentioned user by it’s internal ID. i.e. when the user mentiones the local @criztovyl a mention is embedded based on the id of that user in the users table. If it’s the https://criztovyl.space/actor the mention is embedded using the id of the corresponding user in the users table. Extraction and rendering for display simply the other way around.

Externally (i.e. AP), I suppose, without specifically knowing it, the mention won’t be extracted from text. There would be a mentions property that contains all the mentioned users and their IRI IDs. The text is purely visual and will contain the mention as produced in html (or whatever format) by the originating forge.

In the end it’s implementation detail how the forges deal with displaying mentions and is nothing I would include in the ForgeFed spec. The spec should only need to care about the technical problems of, for example, how to deliver the mention event.

On a technical level – yes. In UI sense – no.
A human seeing two @criztovyls cannot easily determine which one is the @criztovyl (s)he’s thinking of just because the message is signed.
Yes, nobody forces you to display just the username, but then you are again back in the dilemma: Show the full, possibly very ugly, hard to read IRI only? Show the IRI additionally? How to help the user to easily make a distinction between similar IRIs? etc

Yes, this is surely not an issue of the ForgeFed spec has to deal with; but something general in regards to AP/Federation.

And, as a side note: Although this thread has ForgeFed in it’s title, I am writing anything here that comes into my mind that is related in any way. ^^

Thank you all for reading & responding. :slight_smile:


1 humble, not honest

True, but it is established in the community already, so inventing yet another way to determine a username for an Actor seems like a bad idea for interoperation. Who knows preferredUsername could well be documented in the next iteration of AP, as it becomes an established not required thing.

Why “True, but …”?. ^^ Sorry for nit-picking. ^^
I am completly with you with what you say, but I don’t understand how it applies to the sentence you quoted. :slight_smile:

Well, it’s “true”, “but” there is an important factor to consider about real world adoption :wink: I can’t say preferredUsername is in the spec, but I don’t think it matters at this point.

As you said it’s not relevat if it’s in the spec because it is widely adopted, but I guess it is widely adopted because it’s in the spec :smiley:
https://www.w3.org/TR/activitypub/#actor-objects (well, just a MAY, but it’s in there ^^)

1 Like

Lol :smiley: I honestly remembered it not being there, oh well…

1 Like

Looks like it was moved to MAY Apr 2016 here, then marked “as risk” Oct 2016 here but apparently survived that status and was added to the intro example in Apr 2017 here.

Probably some meeting notes regarding discussion about it. Good that it survived :+1:

#specworkarchaeology

i probably could have condensed the point of my previous post something like this:

the primary goals of forge-fed are to allow forges to inter-operate, and to allow users to interact with foreign forges in all of the usual expected ways just as if they had an account on that forge; but without actually having an account on that forge

if we can make those two things possible, then i would consider forge-fed to be a complete success in satisfying all of its essential goals - the idea of allowing non-forge AP peers such as mastodon to interact with forges was very much an after-thought, an additional but non-essential bonus feature that would be allowed by using AP as the communications layer - but such non-forge peers will necessarily only be able to interact in a limited set of ways; because they are not forges - therefore it is not helpful to treat that use-case as primary, or to allow it to interfere with the primary concern in any way

something like the preferred pretty display name could be suggested (any unrecognized fields should be safely ignored, so its harmless to add such suggestions); but is not at all essential, especially if the primary goal is to allowe users to interact directly with foreign forges as if they had an account on it - IF the forge wanted to present that pretty name, then they would already have that functionality, and it would be customizable by each user in their profile settings on the forge - i can imagine that it could be handy to allow changing it automatically and globally via an AP message; but the feature itself is a fairly small concern that i dont think any existing forges would actually use

if for some reason AP were to obstruct the goals stated above (it probably will not), it would be best to use some other messaging protocol for forge-to-forge communication, and use AP only for the limited set of forge-to-mastodon communications - that was actually my original proposal - it would greatly simplify the real business of forge-to forge communication; and we could defer mastodon inter-operability for an optional second phase or “v2” spec - i really do think that would allow the essential v1 spec to be completed sooner - when viewed from the forge-to-forge perspective as primary, most of the confusions evaporate - it is really not so complicated - a raw JSON/HTTP spec could be very straight-forward - if the v1 spec somehow rendered it impossible to communicate with non-forge AP peers, then that would not be a terrible loss - non-forge inter-operability is a “nice to have”, but non-essential feature

its not so much the language as the point of view that treats the AP representation as primary - if the AP representation does not correspond to the canonical truth in the forge database, then it will be messy somewhere - the important fact to bear in mind though, is that we are not representing AP Actors as Remote Users, as you put it - we are representing forge users as AP actors; because local vs remote users are the fundamental reality, and the AP representation is just that: a re-presentation of “the real thing” - it simply does not work the other way around; because we are also representing repos, teams, and tickets as AP actors - we can not use “identity” as a substitute for “actor”; because only the actors that represent users would have an identity - most actors would not have their own intrinsic identity, but would instead be attributed to an identity of the user who published it

that was not to say anything tangible or physical regarding the implementation - by “layer” i meant as a different layer of abstraction (i.e. a orthogonal concern) - it is everything that forges do not intrinsically need to do, that does not serve the purpose of the forge in any way; but only for the purpose of communicating with other forges - it needs to be as non-intrusive as possible in order to be acceptable to existing forge devs; and im quite sure that it can be - it could be almost entirely handled in a single ForgeFed class, with very few hooks protruding into the existing core code-base - any forge that exposes a complete API for example,could have the AP messaging handled by an external service like MCFI, and would need very little new forge code (probably just to add foreign users to the DB)

that will not happen - perhaps some new “AP-first” forge would be written someday; but if we ask existing forges to change their data models, forge-fed will not be adopted

im pretty sure we are referring to the same thing there - i referred to that as the “phantom user” boolean - probably just another confusion of language; but i cant imagine why ‘identity-only’ would be more meaningful - a phantom user in the DB is a full-fledged user in every conceivable way, except for some obscure internal caveats such as that they have no local login credentials and their nicks must be mangled specially

that is why i noted it as optional - some forges already store GPG keys for their users (aka: identities)

they dont play any role in terms of AP - GPG is just for verifying commits and issue comments for the purpose of display; such as a “Verified Post” badge beside the nick, or perhaps for rejecting unverified contributions

that could be the display name, but i seriously doubt that anyone would want that as their username - also, i dont think https is part of the AP spec, so that actor document may be reachable at both http and https (or others: IPFS, gopher, etc); which means that part of the URI is irrelevant to idenity and therefore not globally unique - both the http and https forms would be equivalent aliases for the same real user identity - also, most forges would not allow specal characters such as the slash. period, and colon in a username - for example consider a URI in either of these forms:

https://criztovyl.space/criztovyl/actor
https://criztovyl.space/criztovyl@criztovyl.space/actor

in those UIRs, the value of the user-id DB field would need to be ‘criztovyl’ (even the ‘@’ is proabably disallowed) - that user-id alone could actually conflict with multiple existing users, except that all but one of them would need to have the special ‘foreign-user’ flag on - that would indicate that the minimal globally unique id is really “user-id + ‘@’ + domain” (‘criztovyl@criztovyl.space’ in this example) - the domain can be parsed at run-time from the actor document URI, which will need its own new DB field - of course, if the forge already has a specific “pretty-name” field (which im not sure that any do), that could be constructed once and stored when their phantom user is created - because the full unique id, including the domain, can not be part of the DB user-id field, but must be reconstruced; both the domain and the specific user must be identified in the URI

i would not suggest hiding it either; but the URI is stored; so the displayed nickname could be a hyper-link back to that user’s home-server, or the local phanmtom user’s profile page, which could indicate the identity details: eg. home-server URI, GPG key, real name; whatever the forge already presents on a user’s profile page

and if the forge is free software, then everyone can get what they want - its not for us to specify implementation details - we only need to allow information to flow from one forge to another robostly and verifyably - presentation is not any of our concern - we only need to make it work - others will decde how to make it pretty or user-friendly (or not)

even if that were a good way to represent users (and again i think people would hate that representation); mentioning a user would entail typing this string of characters: @https://criztovyl.space/actor - as you probably have seen, forges with that feature automatically offer suggestions as soon as the first 3 or so characters are typed - if user-ids like that were allowed, then every time someone types ‘@htt’, the list of suggestions fetched could be hundreds or thousands of entries - that could be a severe performance concern - uniqueness of the identity is not the only concern, you want a good deal of entropy too, especially at the beginning of the string if the nicks are not hashed in the DB

i explicitly gave that as the counter example - you would not want multiple users represented by the same display name ‘criztovyl’; which is exactly what preferredUsername encourages - that would be very poor UX - user-id ‘criztovyl’ should obviously indicate a local user, and any like ‘criztovyl@criztovyl.space’ and ‘criztovyl@some-other.space’ would clearly indicate distinct foreign users - it would not need to be so ugly as the full URI to the actor document - that is not important information to display in the UI; because that actor document is not intended for humans to read - that is an internal technical detail

There are many ways to organize the DB tables. Some apps have a user table containing both remote and local users, some apps have separate tables. Idk which is better, just pick something you’re comfortable with.

For example, here’s the DB schema that Vervis uses: https://dev.angeley.es/s/fr33domlover/r/vervis/s/config/models

Mentions are a client feature. Let’s just rely on the plain regular preferredUsername for that. If you worry about a security issue with that, e.g. giving the wrong user write permissions, please open a new topic with an example and any proposal you have.