There’s been a lot of talking recently about how to do Communities and Private Data, so I spent the last week on an ascetic journey heads down and finally wrote how I believe ATproto communities [sh|c]ould work.
Sharing it here cause the conversation seems not to be waiting for me to be done (you all are quick)—typo fixes to come after I’ve spent some time not looking at a screen.
Introduces a “fully private” data storage called The Vault, which is meant to sit next to (or on top of) Permissioned Spaces to solve the issue of data that doesn’t have to be fully accessible, even within the community itself.
Well I think you got around to writing about the latest arbiter concept before I did.
I really love how well this pitches the needs of the community and the way that the concierge satisfies those needs.
I think it also touches on how generic the role is, without having to focus on the details of how a specific concierge might fulfill its duties.
I also really like the vault idea, not only for it’s own sake, but also for the sake of showing how the arbiter can serve as a way to integrate multiple different services on behalf of the community.
I’m on board with most everything in here. As I’m exploring how Daniel’s permissioned data work has been proposed and how it may align to communities, I definitely think that having a permissioned space specifically for storing and interacting with things like community role definitions, moderation data, or logs is going to be important, particularly to the story of credible exit. There are many, many different ways to manage communities currently and I imagine folks will want to be able to swap in/out their OpenSocial/Arbiter/Concierge as they see fit and as their community evolves over time.
I’ve also been considering how having some sort of on-protocol E2EE data would be incredibly useful. For things like shared credentials, for example. I’d love it if the login for PDX ATProto’s google email or Bluesky account could have some sort of secure shared credential that could be passed around explicitly to those in the community with things like access logs/key rotation/etc., although I’m not sure what that looks like yet or whether I even want that level of liability at this point.
Regarding the blessed app options, I do have permissions set up for new apps to be “enabled by default” in Open Social currently if that’s how the community chooses to operate, however I think that also presents some risk and should be considered with some security things in mind, and it should be able to be turned off for communities that don’t necessarily want that (in case some rogue app comes in, requests All The Data, and publishes it to the open web, for example).
Love the conversation here and I’m excited to see movement in the community space!!
I’m going to try to write a “vibes-based takes” post soon that gives my rough ideas for this layer. But in the meantime, I’ll share a few thoughts.
I strongly agree with your motivations. Communities should be able to exit from a rogue service. And “Blessed app mode” screws over the long-tail of apps.
I think we’re all circling around the same sort of “arbiter/concierge/space host/something service” + a set of Lexicons for describing community semantics.
My main take is that the “concierge” should be more of a credential-minter, less of a request-broker. And then how that service choses to mint credentials can be described in records.
To illustrate: to post to a community, you don’t need to go through any sort of “community service”. You just post to your PDS. Application’s then decide if it’s a “legit post” by looking at community policy rather than checking it against a “pointer record” in the community’s repo. This prevents the need for any special “app relationship” or user interaction with the concierge on-write.
The analogy is posting to a Bluesky thread that has a threadgate on it. This doesn’t require any “pointer” from the OP back to each approved reply. You can look at the rule set in the threadgate & compare the reply author to it and decide if it’s “legit”.
The fewer things that go through this service the better. I think for complexity sake, credential management, and operations (ie “how hard is the community down when the concierge goes offline?”). I touch on some of these arguments in this post.
Also, I’m curious, could the Vault just be a permissioned space (or spaces) with no other services/users added as readers? This would be significantly simpler to implement than introducing a new service type
Oh, yeah, that seems like it could work. It would be nice to avoid a new service.
I think that pattern goes pretty far, but it breaks down when we need to have truly community owned data.
Even Tangled is already struggling with this with git repo issues. When someone creates a post that is meant to be a contribution to the community, for example like a wikipedia edit, it is not acceptable to have the user be able to delete their PDS ( maybe because of drama or conflict ) and have the wikipedia community instantly lose all the edits / pages that the user authored.
In the case of Tangled they fix this by having the AppView be authoritative. It keeps crucial data and acts as the authority on what the latest state is. Unfortunately, that really hurts credible exit and alternative AppViews. Now other AppViews have to trust a central AppView to maintain consistency, instead of just being able to index repos like normal.
By putting the concierge in front of a community repo, we allow other AppViews to index that repo like normal, and treat it as an authority for community-accepted data, without there ending up a need to trust another AppView as authoritative.
So it’s a way to move the authority back on-protocol, focused on the repo, instead of ending up with sensitive authority vested in a particular AppView.
Like @essentialrandom.bsky.social’s blog post mentioned, the concierge is pretty similar to one big trusted AppView. But at least it’s focused and built for that, so that it’s more standard and easier for other AppViews to come along and read all the content that has been accepted by the concierge, recorded directly on protocol in the repo.
To be clear, not all apps require this. It is something that we have run into several times, though.
In a community there is often some, even if small, truly community owned data or configuration, and just leaving that spread out in different admins accounts doesn’t necessarily work well when admins may come and go, and we have to have some way to merge configurations if there are multiple records in different admins repos[1].
I think this has a similar issue. Who controls the records that describe how the concierge mints credentials?
Those records may be authored by a community “owner” at one point, and then the owner may change later, but coordinating that over decentralized records could end up really tricky if we don’t have a central repo for it. We essentially end up with a distributed consensus problem.
And if we do have a central repo for it, maybe just using the ATProto repo referenced by the space host’s DID, then how do we control access? We don’t want to share passwords with the other owners, so we need some way to control write access to the community DID again.
I’m kind of repeating things ( sorry! ), but I’m partially trying to get at the loop of requirements that’s easy to get stuck in and see how things keep coming back around to needing some form of centralized write access at some point to be reasonable / efficient without stepping into CRDT territory ( which really only moves the problem yet again, 'cause CRDTs have semantic / schema enforcement issues ).
I do agree that it makes operations more complex to migrate, but I do think we end up needing something to handle this despite the effort we have put in to avoid it ( Roomy used to try to work peer-to-peer! ).
Having multiple concierges
I think there is an option here that we haven’t discussed a lot, but that we at Roomy were giving a lot of thought in earlier versions of the arbiter, was that this kind of record write access could be handled by a separate service per-app.
i.e. if you have an app that needs to store some write-gated content on protocol ( to avoid just putting that data in the AppView’s database ), then you can have a “steward” service that sits in front of a PDS repo specifically to satisfy that need.
Each AppView that has that need for community-owned data, then, would have it’s own DID and put that write-gated data in the community space like a member would, on it’s own PDS.
I think that this could work, and it might be necessary if we can’t find a way to somehow compose policies from different apps at the concierge. For example, the arbiter’s policies are written in Rego, but we can’t expect all apps to agree on that mechanism for composing their policies when multiple apps need to setup record write access for the same community.
While this “steward” idea works, I think, it is another layer of indirection, because now the steward account has to be added as a member of your permissioned spaces, and the community somehow has to know which stewards are trusted. Because there could be multiple accounts that show up in the community to act as a steward, so there has to be a root of trust in the original community DID somehow.
Like the community DID maybe has to have a record in its PDS listing trusted steward DIDs. But now we have to have write access to the community again!
More backstory on this: I realized while implementing the steward, after already having a draft of the arbiter, that the concerns around write access to the steward were essentially the same concerns as write access to the arbiter’s own member list data. So I realized I could just merge the two without preventing you from choosing to have separate stewards.
So the arbiter ended up far more powerful than initially planned, but it can still be as simple / limited as you choose, if you want to have separate stewards instead.
That gets us into the realm of CRDTs which is another can of worms. ↩︎
Well, I couldn’t have posted this without you and you can actually see the vault in the “how Ms Boba is conceptualizing this” bit of our Excalidraw exploration.
The second part of the article (which I’ll publish in a few days) is (also) about how these same concepts can be useful to users rather than just Communities. That version is also on my list of things to prototype
Yes! Really regardless of how it’s implemented my main point is AppViews are storing too much “user-owned data” right now. For understandable reasons, of course! But we all see how Communities being A Thing™ risks making this 100x worse, so I want to make sure we have a story for all type of data
I didn’t know that and I’m glad to see it! As I mentioned, I appreciate why that is and also that OpenSocial made it much easier (thank you for CIMD, for example).
Also don’t want to single out OpenSocial: “3rd parties need to register/use API keys” is how all services on ATproto do it currently (Slices did and so does HappyView), so it’s definitely a widespread “issue”. I haven’t fully understood if there’s a way around it right now (maybe @dholms.xyz knows?), but I’m throwing my pet issue in the ring it does impact the tail end of sites!
Something I didn’t have time to clarify in the original post (may add it now that I’ve caught my breath ) is that there’s really two types of “App-like things”:
Apps that operate directly on Community Data (on behalf of the Community): doing things like analyzing logs, trends, community-wide moderation… These may require broad access to community data.
Apps that operate indirectly on Community Data (on behalf of Members): helping members do what they can already do in a community, just differently. For example, helping surface posts they may be interested in among all communities they subscribe to, giving them a different posting interface… On this, members cannot give more access than what they have.
When I briefly mentioned “[Blessed Apps] shouldn’t just be part of this ecosystem, they will need to be”, I was referring specifically to #1.
#2 is where the Blessed App model really bites Users with special needs and smaller 3rd parties. And while there’s scenarios in which a community may want to turn off 3rd parties completely (just like enterprise services can turn off OAuth/3rd parties Apps) I would really like that to be the special case.
Also, this is a UX problem: a very scary message about 3rd parties, or a lack of more fine-grained controls can make the difference between “most Communities enable this” and “most Communities are too scared to”.
And thank you, it’s the only way to cope with having to write that much
We sure are! The article is definitely building on the shoulders of giants. I’ve seen bits and pieces of it in the ongoing discussions—I just hadn’t seen it put together and streamlined (and some parts felt to me like they were “always slightly off” compared to how I’d personally think about them). Hopefully, this can also help as an introduction to help get more folks onboarded in the discussions.
Yes and no, speaking as someone who just went through a week of clicking “accept ownership” on gDocs folks had written for one of my organizations, and who’ll have to transfer them all again once we figure out how to make an organization account. Some stuff should go on the community PDS directly, even if it’s made by Members…because some stuff is actually the Community’s—for logistic reasons, but sometimes even for legal ones!
Another example: I wanted to make an Admin page in atmosphere.community to “accept” some of the shared content. That would specifically require logging in as a User account and writing that acceptance on the Community’s PDS.
One way I could see this working (although extremely cursed, I think?) is having the “Concierge” look at whether an Admin posts a confirmation to their account, and have them write the confirmation on the Community’s PDS in response. I guess…that could be done? It doesn’t go through the service in quite the same way, but IDK how I feel about it.
Full disclosure: I haven’t tried Permissioned Spaces (I’m the type of dev that absorbs by going hands-on), so I thought it could, potentially but was unsure. I mentioned in the OP that “The Vault […] is meant to sit next to (or on top of) Permissioned Spaces”. I’ll prototype something with Permissioned Spaces soon and get more thoughts!
I have to say: I do personally appreciate the mental separation of the 3 models, even if just to talk about and explain it. I’m not precious about it being a new service…although I think the services model is very underutilized
One thing: the yet-to-be-published rest of the post talks about how Vault+Concierge models can benefit regular User accounts, so that’s another consideration in the back of my mind.
Guess what the rest of my post is about? I think multiple Concierges/“Steward services” solve a lot of issues, and the post somewhat touches upon this: the Concierge can act as a single entrypoint to route requests towards other services. It’s not just something that handles things directly: it’s a gateway!
I may speed up polishing up the second part, but the examples I make there are using this model for fully-anonymous inboxes with 3rd parties services support (I want Tumblr back ) and making notification services more User-centered.
Oh yeah, I should clarify that I totally expect a community DID to have a repo and in some cases, it makes sense for that repo to publish community-owned data. Just as a baseline, I think it’s good for most social activity to happen in a repo without needing to interact with the community’s repo.
Even in the cases where a community DID wants to “pull in” some user content, I think it’s better for it to do it non-interactively with the user. For instance, publishing its side of things after syncing in the user’s side.
Agreed, I think these records should be published in the community’s repository!
and you can actually see the vault in the “how Ms Boba is conceptualizing this” bit of our Excalidraw exploration.
it does impact the tail end of sites!
) is that there’s really two types of “App-like things”:
) and making notification services more User-centered.