Trust Infrastructure on ATproto

Discussion
1

While building out Barazo I keep running into the question behind EigenTrust, sybil detection, and trust propagation: how do you figure out (cross-network) if an account is real (e.g.: not a bot), without building a reputation score that inevitably gets gamed, weaponized, or both?

I wrote about the AI slop angle recently for background on where I’m coming from. But I think there are a couple harder unsolved questions underneath. One privacy constraint I’m working with is “no government ID or financial identity verification”. Pseudonymous accounts should be first-class citizens.

What makes ATproto exciting for this is that it might be the first real opportunity to solve trust across multiple apps and content types at once since the identity layer and data model are shared.

I’d love to think through these with people who are actually building on it. So here I am :sweat_smile:

Feel free to point me to another place where this discussion has already taken place somewhere else, relative new to the community :folded_hands:

Sybil detection without a surveillance system

Effective sybil detection wants the full network graph. But “technically public” and “actively surveilling” are different things: ingesting every interaction across every ATproto app to compute trust scores about people who never used your service… that’s a panopticon (and probably a GDPR headache) .

The alternative could be user-initiated trust building. Users choose what feeds their trust profile (Keytrace verifications, community activity, endorsements) and then the app can verify those specific claims against signed records in other users’ PDS rather than surveilling everything. Only positive signals, so opting in more always helps.

(and using negative signals probably just makes bots rest by creating a new account whenever something negative is added to their profile, so my guess is that this wouldn’t work?)

So the app gives up proactive detection across the full network. But fake accounts that never show up aren’t a threat, and fake accounts that do show up have to leave traces (spam, fake endorsements, interactions with real users) that are visible. The bet is that the sybils that actually matter will reveal themselves through their behavior. Not sure that bet always holds though…

I very much assume this is be a shared ecosystem challenge. Should each app build its own detection, or is there a way to coordinate across labelers without creating a centralized clearinghouse?

Endorsement consent gap

Endorsements live in the endorser’s PDS. The endorsed user can’t delete them, can’t reject them, has no recourse. Without a consent model at the application layer, endorsement spam is trivially easy and there’s nothing the target can do about it.

Endorsements should only appear on someone’s profile once they’ve accepted them, but that’s an AppView-level decision so every app can solve it independently. Anyone thought about a shared convention for this?

Newcomer problem

Another fun challenge is that graph-based trust inherently favors incumbents. A newcomer with zero history is indistinguishable from a sybil to the algorithm. Cross-platform identity (Keytrace) helps: proving ownership of a GitHub account and a domain is a real signal. But not everyone has those… and requiring them would be exclusionary.

How can we design a system where someone joining in 2030 still has a realistic path to being trusted within a reasonable time (and without hindering any participation).

So some discussion points I’d love to get your view on:

  • Sybil detection that doesn’t require ingesting the full firehose. Inter-labeler coordination? Community-maintained heuristics? Something else?

  • Endorsement consent gap: a shared convention, or does every AppView just handle it?

  • Trust bootstrapping for newcomers without cross-platform identity. What works that isn’t just “wait and contribute”?

PS: the Glowrm thread covers related ground from the centralized clearinghouse angle.

2 Likes
2

how do we know you are not a bot and an actual human and that we can trust you?

do we need a trust score right now in order to talk further, or can I assign my own version of a trust score to talk to you and trust you, until you do something evil and revoke my trust?

Here are my personal criteria on a whim:

  1. your post does not seem to be ai generated.
  2. you connected your DNS.
  3. your webpage has some sort of public track record.
  4. out of the people I follow Erlend follows you so that enhances my trust in you.

Have you faced any issues regarding having untrustworthy users on Barazo? Why do you need a trust score now?

I am very skeptical when it comes to a trust score to solve them all (see my comments here), I do agree that being able to detect if a did:plc account is a person vs an ai is extremely valuable! Given that the the reality of today is that ai is in active use to shift public discourse and opinion.

Screenshot 2026-03-07 at 13.44.00
Screenshot 2026-03-07 at 13.44.001718×1636 202 KB

Regarding sybil resistance/ KYC-privately, have a look at: Private Age Assurance for ATP

3 Likes
3

This is something I’m thinking a lot about and experimenting with. Several of the applications I want to build need trust built in. Primarily not-a-bot trust.

I’m curious why you eliminate government ID and financial identity verification? I don’t see precisely how that prevents pseudonymous accounts. Perhaps you worry that the application or coordinating service would gain access to the information and could choose to doxx the account?

Do zero-knowledge-proofs alleviate this? They are not quite ready for production use perhaps, but that’s the avenue I’m exploring.

I want to raise a few other complications to think about too…

How fresh does the sybil detection need to be? It’s common on Reddit for accounts to be piloted by a person and then sold once they’ve gotten enough karma (a trust score) so they can be used to spam subreddits with karma requirements.

How do you consider a public figure account that might have multiple real people piloting it?

What about paid endorsements?

Maybe these are edge cases and it’s enough to be able to make it much more difficult for untrustworthy accounts to participate.

1 Like
4

Oeh, hands up for @erlend.sh following me :sweat_smile:

Your four criteria (non-AI content, DNS, public track record, social graph) are pretty much where I think I’m going with this as well: observable evidence, human conclusion. No “gui.do has trust score 73, proceed with caution.”

But there’s an irony I keep running into: you can’t really score bots, because the moment they get a negative signal they just create a new account. Plus they are getting really good at mimicking human behavior at the single-post level. So the only way (at least as far as I can tell, please correct me) to filter bots out is to give positive signals to humans about their profile (not single posts)… which means you end up scoring humans anyway, just to weed out the bots.

At least until the bots figure that one out too? But at least with profile-level scoring that also takes profile-age into account, gaming it becomes much harder (and hopefully economically unfeasible).

And that’s actually also what your @erlend.sh point illustrates: you assign trust to me because someone you trust already follows me. I got lucky, but most newcomers won’t have that, and right now on every new platform you’d start from scratch. That’s fine if you want to, but shouldn’t be necessary. If the trust signal runs in the background (not as a public score, but as infrastructure that helps mods and the system decide you’re probably a real person), your DID could carry that across ATproto apps.

Your Glowrm post gets at a related concern where a cross-app reputation that follows you everywhere is dangerous. But imho there’s a difference between “this account is part of a coordinated spam cluster” (useful signal for everyone) and “this user got banned from a dating app” (nobody else’s business).

On Barazo: no trust problems yet, it’s still in alpha. But I’ve been a community manager and forum mod/admin for 25+ years and based on that I’d rather build defenses before the spam wave hits than after :sweat_smile:

To be clear: nothing wrong with humans checking out who someone is and forming their own judgment, that’s what you did with my profile and it worked. But when your feed is 80% bot slop, you need something automated to help you trust that what you’re reading is from real people. I don’t have a good answer for how to do that without some form of hiding or deprioritizing content (again: let me know if you do :grinning_face_with_smiling_eyes:). The question is how to do it without false positives hurting real newcomers.

1 Like
5

Yeah the karma farming thing is interesting. That’s hard because the account was legitimate, the trust signals are real, but they changed hands. I don’t have a good answer for that one, but it does sound like that is somewhat expensive (so not super interesting to do for spammers?). Still takes time to build of course. Maybe sudden behavioral shifts (posting pattern, topic, writing style) could flag it, but that also gets into content surveillance territory pretty fast…

Multi-person accounts and paid endorsements are an edge case I never had to deal with (and imho not worth designing for early on?). Maybe similar to corporate accounts, those would be more interesting early on though. If the system makes it hard for bot farms to operate at scale, a few paid endorsements or shared accounts are manageable noise. Getting to “Perfect” is the enemy here.

Your last point is indeed what I’m also keep grabbing back at in that making soemthing impossible probably won’t work, but it needs to be expensive to game the system. If faking a credible account costs $50 instead of $0.10, this should kill most spam economics (well maybe not for mortgages, but the general point still stands :grinning_face_with_smiling_eyes:).

6

yeah makes sense!

I am just overly cautious for any attempt at a trust score because historically people have not done it in a way that is inclusive eg. banks having credit scores that are privileging white people over others.

I think that a trust score system might look different from app to app and from use-case to use-case.

I would love it if there was a system that could accurately predict whether an account is automated/ai/bot or human, couple days ago i posted:

Is this human or ai?
2026-01: 376 posts (12.53/day)
2026-02: 662 posts (22.07/day)

This might also be something that should be developed not in the open since if its done openly malicious actors could update their systems to follow the rules and not be detected, or might just be backwards engineered.

1 Like
7

Yeah a credit score system is what we should avoid.

Your post count example shows perfectly why single metrics don’t work. You can’t tell from one number. You’d need posting patterns, content variety, interaction with other accounts, account age, verification status… and even then the answer might be different depending on what app you’re looking at. Different apps value different history.

Thinking out loud here, but maybe it’s something like three layers:

  1. Raw signals in the PDS (account activity, identity verifications, attestations from other users)

  2. A network-level analysis layer on top of that (sybil cluster detection, spam risk labels). This would need to be distributed somehow, not a single centralized source, because that would defeat the point of ATproto

  3. Each AppView interprets both layers for their own context (and they can keep their exact method/settings for doing so private, to prevent gaming.)

Also: if we intentionally put more trust signals into the network (so all apps can use them), but all individual apps apply their own variation/settings on how they interpret and handle the secrets, this in itself will probably make gaming the system harder (because bots don’t know what to optimize for).

A dating app, professional network and a forum will all have very different ideas of what “trustworthy” means, so maybe there shouldn’t be one score at all. Would love to hear if this model makes sense or if I’m missing something obvious.

The open vs closed detection thing is a real tension too. If the rules are public, bots adapt. If the rules are secret, there’s no accountability and you end up trusting whoever runs the system. I don’t have a clean answer for that one…, except for that I have a high bias towards open systems over applying some “security by obscurity” principe.

8

@sethfeldkamp.com you asked about my pushback on government ID, and @attpslabs.com pointed to attps.social for ZKP-based age assurance.

Government ID doesn’t prevent pseudonymous accounts at all, you’re right. That’s not why I would want to exclude including that (at least by default). But I want people to be able to stay anonymous if they want to and still carry trust based on their actions, not their legal identity. Government ID could be one factor (and for some sites it might become a legal requirement), but it shouldn’t be required.

On ATproto all activity is public. If any party in the verification chain can link your DID to your government ID, they can probably link most of what you’ve ever posted to your real identity. And since relays index in real-time, that’s effectively permanent.

You might trust your current government, but what if you’re active on LGBTQ or abortion forums and the next one uses that data to track you down?

ZKP could be the way around this and the “prove a claim without exposing who you are” system could work for including other kinds of (offline) trust signals I suppose.

Looked at the attps.social code (MIT licensed, nice :white_heart:) and seems like attps.social never touches passport data :+1:. The Self mobile app reads the passport via NFC and generates the ZKP on-device, then only the proof is sent to attps.social’s backend for verification and attestation signing.

But then the trust question shifts to the Self app. I see that Self’s Circom circuits are public, but the mobile app that actually handles your passport data doesn’t seem to be open source. That’s the one link in the chain I can’t verify. @attpslabs.com do you know if it’s auditable? Couldn’t find it i their repos.

And then still… the cryptography can be perfect but if one link in the chain leaks your DID-to-passport mapping, that covers all your public activity. Not a reason to avoid ZKP… but very definately a reason to verify every link in the chain.

One more thing: attps.social stores the verification as a PDS record (social.attps.ageassurance/self). That’s public and readable by anyone.

Which is fine if only some users verify (for 18+ content access or whatever), there’s enough ambiguity in the unverified accounts. But if a large platform like Bluesky ever enforced it for everyone, you’d effectively have a public list of “verified adults,” (which is ok) but by proxy this means you álso have created the opposite: a list where every account without that record is implicitly flagged as “possibly a minor.” That’s quite the targeting risk… Especially if even lower ages are included (depending on the jurisdiction and reason, this can be anything fro 13-18+ years) @attpslabs.com curious how you’d handle this?

9

I am definitely thinking ahead to permissioned data before ever thinking of writing any records to a PDS.

Appreciate the good discussion and agree it does feel like a big gap in the ecosystem that most apps are going to need to address.

Are either of you heading to Atmosphere conference? If so we should meetup perhaps and compare notes.

1 Like
10

would love to go to ATmosphereConf, still undecided. It’s quite the trip and :money_with_wings: :sweat_smile:

1 Like
11

I described something I called AutoRep that’s rather similar a few years back: reb00ted | An autonomous reputation system

2 Likes
12

Not 100% sure I think their mobile app is not open source but I asked.

created the opposite: a list where every account without that record is implicitly flagged as “possibly a minor.”

Thats not the case.

Most users have a birth date attached to their account (which is privately custodied by BlueskyPBC (thankfully so!)), that is one level of verification. Then there are other age-assurance tools like Bluesky did with the UK and that gaming company (uses your cc but is also an added layer of verification).

Plus all of the moderation systems in place: Ozone, Osprey + all of the moderation layers that are modular and customizale per account, including all of the content filtering work, these are all tools that actively prevent targeting risk of minors.

In addition to all of this it is the responsibility of the parent to ensure the device of their minor has parental controls.

1 Like
13

I’m prototyping a federated cooperative network system that was originally just a webapp but it is slowly being refactored to be more ATProto native. I got to the point where I wanted to introduce a trust network because cooperatives involve real $$$, governance, public/privacy splits and permissioned data.

I’m familiar with sybil attacks and such but otherwise am largely ignorant of the issues surrounding “real” trust networks. I naively thought human to human trust chains (e.g. 6 DoS, friend of a friend type of thing) would suffice but apparently I was woefully wrong. I turned to my untrustworthy AI to help me do deep research on trust networks and it spit out a bunch of slop that I didn’t really understand very well.

I think my requirements may be different than most given the use cases but I will lurk here hoping to gain insight. If I find anything helpful I’ll try to contribute to the discussion. I will be at the conference and available for discussion(s) if anyone is interested.

I’ve posted the AI slop I got out of claude at the following link. Do with it what you will, caveat lector!

2 Likes
14

Would love to read an extended explanation of your use case if you have time to write that up.

Cases like yours are crucial for these systems to be useful beyond the imaginations of full-time technologists.

1 Like
15

First off: thx for all the engagement on this topic! Really nice to see and love how you are all pushing my (or our?) thinking on this.

Responding to a couple things:

@j12t.org Thx for the AutoRep link :folded_hands: A decay + token replenishment model is nice, especially that multi-account advantage is limited because token generation ties to individual reputation. I think that maps well onto the “make sybil operations expensive” thinking I mentioned earlier.

@attpslabs.com On the parental responsibility thing: sorry but that doesn’t hold up. If a minor walks into a liquor store, the seller can’t say “well, apparently the parents are fine with it simply because of the fact that the kid is here in my store.” The responsibility is on the party providing the service, not on the (absent) parent. AFAIK most jurisdictions see it the same way. Other moderation layers existing (Ozone, Osprey) doesn’t change the obligation of the system doing the age check.

The birth date in Bluesky is self-declared during signup (not verified), stored as a Bluesky user preference (not an AT Protocol record), and not accessible to third-party apps. So it’s not really “a level of verification”: it’s the same honor system as any website asking “are you over 18?” A 14-year-old can type 1990 and nobody would know.

To be clear on where I stand on the government ID thing: I get that for most use cases this is about a (legal) proof of age, and when you legally need to confirm someone is 18+, some form of verification is needed. No argument there.

But 3 things:

A) I’m genuinely not sure if (linking to a) government ID is the only way to do this? Maybe it is, maybe there are other methods that can verify age without linking a full legal identity to an online account. ZKP-based proofs like ATTPS try to prove “over 18” without revealing who you are. If that works as advertised, it’s a much better fit for an open protocol where all activity is public and permanent.

B) Even when ID verification is used: there’s always at least one system that ends up connecting your online behavior to you as a person. On AT Protocol, where everything flows through public relays, any party that links a DID to a government ID creates a permanent deanonymization risk. That’s a different threat model than on a closed platform.

C) ATProto-specific: users have a single identity across many different apps and services. Very convenient as we saw in the PCKT video from Saturday. But when you add age verification for one service, this could very quickly become a link between a physical identity and all kinds of behavior across the entire network. IMHO that’s already concerning on its own, but combine it with a large-scale data breach (we just had a massive one in the Netherlands) and you’re looking at leaked personal data enriched with a full cross-platform behavioral profile. That gets very nasty very quickly.

So imho at minimum, ID verification should not be a default, be one optional signal among many and not the foundation of a trust system.

@kahunamoore.bsky.social The coop governance angle is quite different from what I’m building but there’s definately overlap in the trust layer. The SourceCred collapse story in your gist is a good one, same with what Klout and others encountered: the moment you turn contributions into a score, people game the score. And thx for the trustnet pointer, hadn’t seensthat one yet.

@erlend.sh Seconding the ask for more detail on the cooperative use cases.

1 Like
16

One minor note:

@gui.do have you looked at https://badge.blue/ + https://atproto.camp/ ? They’re an alternative design for endorsements that stores them in the recipient’s (endorsed user’s) repo/PDS. The endorser signs them instead of storing them in their own repo.

2 Likes
17

:waving_hand: @snarfed.org, yeah @gui.do and I have talked briefly. I think attestations are a good way to help with some aspects of this, but it also sounds bigger than mutual record acknowledgement.

2 Likes
18

I agree with Nick that you have noted a big hard problem, with Zooko’s triangle type trade-offs. (For a semi-related example that is “easier” but still crazy difficult, see our “PeerFlow: Secure Load Balancing in Tor” https://doi.org/10.1515/popets-2017-0017 )

As noted or implied already in this thread: in the end, if someone can cheaply spin up indefinite unlinked identities and build good reputations for them, and if trust is based on such reputations, it is hard (impossible?) to avoid them spending as many of these identities as needed for whatever proscribed purpose they want.

I have been working with Nick on applying what I call contextual trust to AT protocol handles and DIDs (also connecting them to onion addresses). I’ll be talking about this at ATmosphereConf. One of the nice aspects of contextual trust is that it scales both up (unlike e.g. PGP web of trust) and down (unlike TLS CA infrastructure). This might help with your newcomer problem: newcomers can get a contextual sattestation (self-authenticating attestation) about their identity from an entity offering just generic, anyone, structural-level guarantees (think Bluesky verified identities perhaps, though if they don’t have the needed requirements for that particular endorsement, somebody might set up an endorser offering something lesser than that). This would be for general system-wide trust. They might lack any built in cross-domain identities (or do not want to link to those), but they could still be validated within some existing context (member of some group, perhaps employee of some company that has an AT identity, or people Nick knows from swing dancing :wink: or …) If they are completely starting from scratch about their identity, then yes I think all they can do is build connections “the old-fashioned way”.

2 Likes
19

no. users can have more than one did:plc account, i have like 10 and probably made 30 this month for testing. whether a user decides to link all together is on them.

20

Thx all, lots to chew on!

@snarfed.org Yeah, I looked at badge.blue and I like how it keeps the award in the recipient’s PDS. It’s clean for badges where the flow is one-directional. But for something like two people confirming they worked together on a project, or that they’re colleagues, both parties need to own a record. If it only lives in one person’s repo, the other has no say and no way to retract.
So what I’m going with is a two-record mutual confirmation pattern: both parties create a record in their own PDS, and the AppView only renders it when both exist. Either party can retract by deleting their record, no coordination needed. This also solves the spam problem I mentioned: nothing from another person shows up on your profile without your explicit approval :right_facing_fist::left_facing_fist:

Endorsements use the same pattern (even though they’re more one-directional) because I think the subject should still have to approve what appears on their profile.

The sigs pattern from badge.blue is still on my list for v2 though, to add cryptographic signatures on top of PDS authentication.

@ngerakines.me Yeah, it’s definately bigger than mutual record acknowledgement. The attestation layer is one piece, but the harder questions are around bootstrapping trust for newcomers, preventing sybil operations at scale, and making cross-app activity legible without turning it into some single trust score. Would love to dig into this more in the near future.

@psyverson.bsky.social Yeah the contextual trust framing clicks. The idea that someone can bootstrap through existing contexts (employer, group membership, community participation) instead of starting from zero… Something I’m trying to build with sifa.id (first version will be up somewhere this month). Your professional profile IS context. Verifiable community contributions, a domain handle proving employer association, a Keytrace-verified GitHub, these are all contextual attestations in a sense that stack your account/profile trust level. And when built on ATproto, much more trustworthy than LinkedIn imho.

I’ll check out the PeerFlow paper :eyes:, thx for the link. And very interested in your ATmosphereConf talk. Still going back and forth on whether to make the trip (quite the journey from the Netherlands :sweat_smile:) but this thread is making a strong case for it.

@attpslabs.com Oh yeah for sure. People cán use a single identity for all ATproto-based apps, doesn’t mean everyone does. So fair point, and something I included in a profile design for sifa.id in a way that a user can have a single main identity and can connect other ATproto accounts for which we can show activity all in a single stream. The linking is always explicit and user-controlled. I have not figured out yet on what to do with multi/cross-account trust signals though, pretty blank on that one.

2 Likes