@criztovyl's ForgeFed web log - TFO Edition

As you said it’s not relevat if it’s in the spec because it is widely adopted, but I guess it is widely adopted because it’s in the spec :smiley:
https://www.w3.org/TR/activitypub/#actor-objects (well, just a MAY, but it’s in there ^^)

1 Like

Lol :smiley: I honestly remembered it not being there, oh well…

1 Like

Looks like it was moved to MAY Apr 2016 here, then marked “as risk” Oct 2016 here but apparently survived that status and was added to the intro example in Apr 2017 here.

Probably some meeting notes regarding discussion about it. Good that it survived :+1:

#specworkarchaeology

i probably could have condensed the point of my previous post something like this:

the primary goals of forge-fed are to allow forges to inter-operate, and to allow users to interact with foreign forges in all of the usual expected ways just as if they had an account on that forge; but without actually having an account on that forge

if we can make those two things possible, then i would consider forge-fed to be a complete success in satisfying all of its essential goals - the idea of allowing non-forge AP peers such as mastodon to interact with forges was very much an after-thought, an additional but non-essential bonus feature that would be allowed by using AP as the communications layer - but such non-forge peers will necessarily only be able to interact in a limited set of ways; because they are not forges - therefore it is not helpful to treat that use-case as primary, or to allow it to interfere with the primary concern in any way

something like the preferred pretty display name could be suggested (any unrecognized fields should be safely ignored, so its harmless to add such suggestions); but is not at all essential, especially if the primary goal is to allowe users to interact directly with foreign forges as if they had an account on it - IF the forge wanted to present that pretty name, then they would already have that functionality, and it would be customizable by each user in their profile settings on the forge - i can imagine that it could be handy to allow changing it automatically and globally via an AP message; but the feature itself is a fairly small concern that i dont think any existing forges would actually use

if for some reason AP were to obstruct the goals stated above (it probably will not), it would be best to use some other messaging protocol for forge-to-forge communication, and use AP only for the limited set of forge-to-mastodon communications - that was actually my original proposal - it would greatly simplify the real business of forge-to forge communication; and we could defer mastodon inter-operability for an optional second phase or “v2” spec - i really do think that would allow the essential v1 spec to be completed sooner - when viewed from the forge-to-forge perspective as primary, most of the confusions evaporate - it is really not so complicated - a raw JSON/HTTP spec could be very straight-forward - if the v1 spec somehow rendered it impossible to communicate with non-forge AP peers, then that would not be a terrible loss - non-forge inter-operability is a “nice to have”, but non-essential feature

its not so much the language as the point of view that treats the AP representation as primary - if the AP representation does not correspond to the canonical truth in the forge database, then it will be messy somewhere - the important fact to bear in mind though, is that we are not representing AP Actors as Remote Users, as you put it - we are representing forge users as AP actors; because local vs remote users are the fundamental reality, and the AP representation is just that: a re-presentation of “the real thing” - it simply does not work the other way around; because we are also representing repos, teams, and tickets as AP actors - we can not use “identity” as a substitute for “actor”; because only the actors that represent users would have an identity - most actors would not have their own intrinsic identity, but would instead be attributed to an identity of the user who published it

that was not to say anything tangible or physical regarding the implementation - by “layer” i meant as a different layer of abstraction (i.e. a orthogonal concern) - it is everything that forges do not intrinsically need to do, that does not serve the purpose of the forge in any way; but only for the purpose of communicating with other forges - it needs to be as non-intrusive as possible in order to be acceptable to existing forge devs; and im quite sure that it can be - it could be almost entirely handled in a single ForgeFed class, with very few hooks protruding into the existing core code-base - any forge that exposes a complete API for example,could have the AP messaging handled by an external service like MCFI, and would need very little new forge code (probably just to add foreign users to the DB)

that will not happen - perhaps some new “AP-first” forge would be written someday; but if we ask existing forges to change their data models, forge-fed will not be adopted

im pretty sure we are referring to the same thing there - i referred to that as the “phantom user” boolean - probably just another confusion of language; but i cant imagine why ‘identity-only’ would be more meaningful - a phantom user in the DB is a full-fledged user in every conceivable way, except for some obscure internal caveats such as that they have no local login credentials and their nicks must be mangled specially

that is why i noted it as optional - some forges already store GPG keys for their users (aka: identities)

they dont play any role in terms of AP - GPG is just for verifying commits and issue comments for the purpose of display; such as a “Verified Post” badge beside the nick, or perhaps for rejecting unverified contributions

that could be the display name, but i seriously doubt that anyone would want that as their username - also, i dont think https is part of the AP spec, so that actor document may be reachable at both http and https (or others: IPFS, gopher, etc); which means that part of the URI is irrelevant to idenity and therefore not globally unique - both the http and https forms would be equivalent aliases for the same real user identity - also, most forges would not allow specal characters such as the slash. period, and colon in a username - for example consider a URI in either of these forms:

https://criztovyl.space/criztovyl/actor
https://criztovyl.space/criztovyl@criztovyl.space/actor

in those UIRs, the value of the user-id DB field would need to be ‘criztovyl’ (even the ‘@’ is proabably disallowed) - that user-id alone could actually conflict with multiple existing users, except that all but one of them would need to have the special ‘foreign-user’ flag on - that would indicate that the minimal globally unique id is really “user-id + ‘@’ + domain” (‘criztovyl@criztovyl.space’ in this example) - the domain can be parsed at run-time from the actor document URI, which will need its own new DB field - of course, if the forge already has a specific “pretty-name” field (which im not sure that any do), that could be constructed once and stored when their phantom user is created - because the full unique id, including the domain, can not be part of the DB user-id field, but must be reconstruced; both the domain and the specific user must be identified in the URI

i would not suggest hiding it either; but the URI is stored; so the displayed nickname could be a hyper-link back to that user’s home-server, or the local phanmtom user’s profile page, which could indicate the identity details: eg. home-server URI, GPG key, real name; whatever the forge already presents on a user’s profile page

and if the forge is free software, then everyone can get what they want - its not for us to specify implementation details - we only need to allow information to flow from one forge to another robostly and verifyably - presentation is not any of our concern - we only need to make it work - others will decde how to make it pretty or user-friendly (or not)

even if that were a good way to represent users (and again i think people would hate that representation); mentioning a user would entail typing this string of characters: @https://criztovyl.space/actor - as you probably have seen, forges with that feature automatically offer suggestions as soon as the first 3 or so characters are typed - if user-ids like that were allowed, then every time someone types ‘@htt’, the list of suggestions fetched could be hundreds or thousands of entries - that could be a severe performance concern - uniqueness of the identity is not the only concern, you want a good deal of entropy too, especially at the beginning of the string if the nicks are not hashed in the DB

i explicitly gave that as the counter example - you would not want multiple users represented by the same display name ‘criztovyl’; which is exactly what preferredUsername encourages - that would be very poor UX - user-id ‘criztovyl’ should obviously indicate a local user, and any like ‘criztovyl@criztovyl.space’ and ‘criztovyl@some-other.space’ would clearly indicate distinct foreign users - it would not need to be so ugly as the full URI to the actor document - that is not important information to display in the UI; because that actor document is not intended for humans to read - that is an internal technical detail

There are many ways to organize the DB tables. Some apps have a user table containing both remote and local users, some apps have separate tables. Idk which is better, just pick something you’re comfortable with.

For example, here’s the DB schema that Vervis uses: https://dev.angeley.es/s/fr33domlover/r/vervis/s/config/models

Mentions are a client feature. Let’s just rely on the plain regular preferredUsername for that. If you worry about a security issue with that, e.g. giving the wrong user write permissions, please open a new topic with an example and any proposal you have.

Was busy the last weeks, so just a quick summary of what I am currently in the progress of starting (i.e. not getting) to do; essentially it just boils down to preparing my notebook for GitLab development (made bad disk partition choices back in the day and have to re-partition now); will try to add some federation features there.

So, my disk is repartitioned, I now have fresh disk space, and my distribution is up-to-date (in Debian sense ^^)

While thinking about where to start, I finally realized why it does not make sense (at least right now) to extend core types so they support representing “remote” objects.
Mainly it’s a waaay to big change for a software like GitLab, which main goal right now isn’t federaration. Federation simply is not a core feature of GitLab core.
(I think of this as digested input, input I finished thinking about, rather than my own, genuine arguments.)

I am than now on my journey adding a fedration subsystem to GitLab. Right now I think of this as another item in the left side bar, like projects, groups, etc.

That subsystem to me represents the user-facing part. The other part is the fedration-facing API.

I think I will start by federation-facing exposure and user-facing consumption of projects via AP and continue with exposing issues and consuming them for enumeration and display. :slight_smile:

@criztovyl, good luck and let us know how it goes!

Changing the database indeed can be a big change. That’s why I’ve recommended working with a small forge, where the change can be done more easily, safely and gradually. In Vervis I did it one table at a time: For example, when I federated opening-a-new-ticket, I updated the DB schema to allow the ticket author to be either a local user or a remote user/actor. I didn’t have to change the entire schema in 1 step, just 1 little piece each time.

GitLab getting federation would be awesome!! If you’re starting this, perhaps make a comment on the open GitLab issues about federation, to let people know you’re working on it, and maybe even you’ll get help from other people interested in this!

@criztovyl, just FYI, I fixed the local builds in Vervis, and it’s now possible to run a dev instance without TLS and without a reverse proxy, just plain regular http://localhost:3000. It should even be possible to federate 2 such instances running together on localhost, but I haven’t tested that yet ^ _ ^

1 Like

Finally, I got my GDK set up and already looked around in the GitLab source code.

I am now thinking about how to expose projects and issues as AS/AP JSON: Whether and how to re-use the already present REST-API? How to intercerpt application/ld+json; profile="https://www.w3.org/ns/activitystreams" request? (manually create format.json blocks for all the relevant endpoints?)

Also, although there is Mastodon and Pleroma, there does not seem to be a library with standard ActivityStreams/ActivityPub objects, I will also (need? want?) to look into that.

You can peek into my GDK at http://gdk.chsc.de:3000/ if you like (setup DNS only some hours ago, you might still get a wrong IP, in that case i guess 116.203.166.168 gdk.chsc.de in your /etc/hosts is your friend), but it’s turned off when I am not working on it.

Some endpoints that should send you AP objects soon, when you query them with Accept: application/ld+json:

Right now they simply respond with the internal representation converted to json.

So far, so good. :slight_smile:

There is also Prismo made with Ruby/Rails - not sure if that helps to have a look at.

1 Like

Just for reference, this are some links to the AP models of mentioned applications:

I plan to continue with my development this weekend.

The first thing I want to try is to complete exposing users, projects and issues as AP-JSON-LD, using a template mechanism (jbuilder?).

I hope I can even start investigating how to consume and present such JSON to the user this weekend. :slight_smile:

And I should maybe get in touch with the GitLab community to talk with them about my intentions. ^^ (As mentioned before.)

1 Like

Some issues in the GitLab tracker:

1 Like

I created a new feature-proposal-issue in GitLab CE:

I also made some progress with the exposing of users, projects and issues as JSON, you can find my changes to Gitlab in the corresponding commit :slight_smile:

So far!

2 Likes

Long time no see.

I hope to continue working with ForgeFed and GitLab in the coming days.

My next bigger goal is to (pre-?)view projects and users based purely on my fake-AP-JSON-LD.
Then maybe (pre)viewing and listing a project’s existing issues, and later opening new issues and adding comments to existing issues.
And at best already all that federation also with Vervis.

My cloud provider was on “strike”, couldn’t access my remote development environment. :frowning:
What do we learn from this? Never trust other people’s computers.

(Pre-)viewing JSON-LD encoded projects or other entities in GitLab is harder than I expected, I am not sure where to add such an UI to GitLab. It’s also no really the highest priority to me, I’d rather first have at least federation and no UI for it than no federation at all.

I guess I will build a small webapp just for (pre)viewing the JSON-LD data I expose from GitLab. That webapp also could (pre)view data from Vervis, as a small proof-of-concept for federation accross implementations. :slight_smile:

Personally when implementing in Socialhome I started with Actors - since everything has an Actor and remotes are going to want to fetch that actor, it’s a good thing to get done at first. That will give you also a base for producing the AS2 documents.

Don’t worry too much about JSON-LD, except maybe for inbound parsing. Most implementations I know try to think “it’s just JSON”, but in reality when processing inbound objects you’re likely to want to ensure you’ll be able to understand the different forms that property keys and values could be presented as and normalize those.