Bringing back at://

Working Groups WG Private Data
1

The biggest complaint that I’ve heard about the permissioned data protocol design is that it feels like a totally separate thing from the existing public broadcast protocol. While many of the things that differentiate it (spaces, the repo data structure, the signature algo) I believe are critical to the overall design, using a separate uri scheme is not.

I’ve been wrestling with this one for a while now. But I’m currently leaning fairly strongly toward keeping the same at:// uri scheme as the public broadcast protocol.

My biggest problem is honestly narrative/branding. This might sound like a silly thing to focus on in protocol design, but I actually think it’s quite important and is where the worries of it feeling like a separate thing is coming from. Atproto, or “the atmosphere”, is supposed to be “the universal data network”. But in introducing a new URI scheme we’re kinda splintering the data space. Practically, this might not be the case (many things overlap & apps can easily use data from both protocols). But it still just doesn’t feel quite right to me. The at-uri is one of the Big Ideas of atproto. I think it’d be really unfortunate to dilute that idea.

I hope that by using the same scheme, it makes it clear that these two data protocols are ultimately part of the same system.

On a more practical note, uniting under one uri format makes more sense in the context of lexicons. I don’t think you want to have to redefine every lexicon for permissioned vs public use cases. And differentiating between possible sub formats like at-uri-public and at-uri-space in the event of wanting one in particular, (both subsets of the more generally used at-uri) seems more straight forward to me than defining a new space-uri format and having to fall back to uri when you want either.

So without further ado, my new proposal for the permissioned data uri structure is the following:

at://{spaceDid}/space/{spaceType}/{skey}/{authorDid}/{coll}/{rkey}

You’ll note it looks basically the same as the previous proposal except that there is one new segment, the literal space right after the space authority. This is clearly differentiable from the current common first path segment in at-uris which is an NSID and thus always has at least two .s in it. This pattern also opens up the possibility for identifying other atproto resources through at-uris, including labels and blobs.

This proposal goes against my previous argument in diary 5.

I gave three reasons:

    1. The resolution mechanism is pretty different for permissioned vs public data. Since URI resolution is informed by the scheme, we think it makes sense that the scheme is different.
    2. There are light security implications to the different URI formats. Permissioned data URIs should essentially never be viewed outside of the perimeter of their space. Bumping into one of these in the wild should look & feel different from a public URI.
    3. There is now a working group for AT Protocol at the IETF. One part of the charter is describing a URI scheme for the protocol. As we’re not (yet!) specifying the permissioned data protocol at the IETF, we want to avoid mixing up URI semantics with the working group’s work.

My current argument against the first is that at:// does not actually specify a sync mechanism. It identifies a piece of content in a data space. You could receive an atproto record on a thumbdrive and it may still be verified as “authentic”. In this sense, the resolution mechanism isn’t the most important thing. While the sync mechanisms vary between the public & permissioned protocols, the data space remains the same, and thus they should share a uri scheme.

I think the security implications are handled by the fact that permissioned data uris are much longer. That is a much simpler visual distinction than an extra letter in the scheme. I think this was always the weakest point, and I think the uri scheme is the wrong place to litigate this.

And on the IETF working group for atproto, I’ve had a couple of conversations with folks with much more experience at the IETF than I have that suggest this isn’t a big thing we should worry about. The IETF process, in some sense, is designed to handle exactly this. Companies do things with technologies in ecosystems, and as patterns emerge, they get specified through consensus at the IETF. There is some risk that the working group specifies a URI that breaks our permissioned data URIs. However that risk already exists for public data as well.

So let me know your thoughts!

In the mean time, I think I’ll probably update the proposal. Even if it’s in the proposal, it’s not set in stone yet. The proposal is just meant to capture the most up to date thoughts of where our heads are at.

9 Likes
2

A single URI scheme makes sense to me, and the literal space can never be a NSID so you’re not conflicting there. There may be an issue with outdated at-uri validation choking on new space at-uri’s, but I don’t think there’s a way through that, other than maybe defining constraints on at-uri’s. (e.g., this field accepts at-uri’s within this collection, or this at-uri must be a space at-uri)

8 Likes
3

I have a strong preference for separate URI schemes, but if the proposal does go forward with a singular scheme, I would not recommend inserting an extra keyword to differentiate public and permissioned URIs; the amount of path segments should be the differentiator. That being said, utilizing a reserved pseudo-NSID for blob references is something I can get behind.

3 Likes
4

I thought space:// felt pretty “different” but I could’ve lived with it.

Seems to like ats:// is a great choice though.

  • Makes it easy to know if software supports at:// (public) or ats:// (spaces)
  • Feels similar to http:// → https://
  • Nice and short, looks cool

The at:// with a special path /space/ segment seems a little hacky? Maybe at least define a general reserved space in there like /.well_known/space/or similar?

I still like ats:// though.

Great work guys!

5 Likes
5

I think one URI format that covers resources handled by two separate protocols is a very bad idea because the purpose of a URI is to indicate an object, and the protocol mechanism to reach it.

You write “I think the security implications are handled by the fact that permissioned data uris are much longer”, and any path that addresses security concerns with “the URI we have to be concerned about is visibly much longer” should be reconsidered. The history of URL usage across the web (and especially in email) is a history of bad visibility and user education leading to security issues.

I’m not dismissive of the social reasons to overload at:// , but I think the social human issues against overloading it are much stronger.

2 Likes
6

I’m generally in favor of this, provided that at-uri is treated as a first-class type.

I hope that keeping the existing at-uri format as the legacy-compatible base format will help reduce confusion during the transition. It should also help avoid a proliferation of string subformats in the future.

I don’t have a strong opinion on adding the extra segment, but if anything, I’m slightly against it. It may be a bit more robust than relying only on the number of path segments, but it still feels somewhat ad hoc to me.

1 Like
7

I recall the discussion of space:// as an option and ats:// was a good compromise on branding since the we wanted to keep it similar to at:// while also making it clear this was not the public open data model that the rest of the protocol abides by. I found the reasons laid out in your diary to be pretty strong justification as I do believe the first is honestly sufficient to convince me as otherwise you do need to build in some resolution logic into services working on the protocol to know how to treat this new special thing. I’m also partial to ats:// as it means we can build future things on that in the future that respect the same model.

On the other hand, using at:// does make it easier with lexicon development and is something I struggled with when working on the northsky permissioned data model as I was effectively forking every lexicon I wanted to use privately so it is a thing that we should address.

6 Likes
8

To me it seems like the underlying question is whether the approach is to treat permissioned data as a new protocol, or if there’s actually only one protocol and the original publicly-focused AT protocol is being reworked to handle permissioned data as well. If it’s a new protocol it should have a separate prefix, and as @evelyn.northsky.team points out the lexicon design needs to be adapted so that they can be cross-protocol. If it’s just a single protocol then it makes sense to have a single prefix … but from Daniel’s writing, that’s not how things were approached.

Abstractly, unless the approach considers a more fundmental rework (not just extension) of the protocol, I prefer a two-protocol approach; trying to staple permissioned data on after the fact means that it’s to be very hard to overcome the history of a protocol that was primarilly designed for public usage. But, I can also see the pragmatic arguments for trying to unify the two.

9

I’ve appreciated each diary entry and the reflections afterwards. Standardizing on at:// seems like the logical conclusion. I suspect spaces would benefit directly from the current IETF AT-URI effort instead of launching a separate and parallel effort.

I’m curious why /space/ and not /com.atproto.space/? Maybe an enhancement to lexicons for defining their own syntax that follows the nsid. There is query , procedure and record, why not something to define a custom and extensible handler.

Likewise, blobs could be defined under such a lexicon enhancement under com.atproto.blobs where /{cid} could be defined OR some other third party app could define their own media handler lexicon, such as com.example.stream with /{cid}[/{rendition}?bytes={x}-{y}. Flexibility over hard coding?

10

I look at stuff like this from a perspective of “how hard is it going to be if we change our minds later?” and, in this early phase where the entire approach could potentially need to be rethought after practical implementation, I think using a separate, experimental URL protocol-namespace like ats:// that doesn’t pollute the at:// protocol-namespace is the way to go. It seems to me that there’s less risk in starting with something like ats://, and then potentially integrating it later as a “sub-protocol” special case of at://, versus trying to “un-mix that yogurt” if you decide later it’d be better to give it its own special protocol and there’s already this ambiguously-cased double-meaning version of at:// strings out there.

In other words, if it turns out that there’s a smooth way Spaces can fit into at:// down the line, transitioning toward that can be figured out once it’s ready; whereas if you realize you need to go back to the drawing board (on whatever point, for whatever reason), and now there’s a whole class of at:// URLs from this brief experimental moment in history that don’t obey the dotted-NSID constraint at the second level in the wild, oop, now every at:// parsing implementation needs to handle the misstep of that legacy special case, forever. It’s easier to recognize the special case in the implementation before jumping into the logic fork for protocol handling than after.

2 Likes
11

Putting my vote in for one scheme (at://). I think permission spaces are a part of the at protocol, and the proposal is sufficient to identify space resources without too much trouble.

3 Likes
12

Thanks for all the feedback!

Wanted to chime in on two things in particular that I’ve seen come up a few times:

The reason for the extra segment is so that you can differentiate between a public record ref & a space reference. For instance

Record ref: at://did:ex:ample/com.example.coll/blah/

Space ref: at://did:ex:ample/com.example.space/blah/

Without the extra segment, there is no way to tell what is on the other side of these URIs (is it a record or a space?) without first doing a network request to resolve the relevant Lexicon. Space refs turn out to be very common in the code & I think it’s pretty important to avoid this sort of confusion.

Not trying to be cheeky, but this also depends on what you mean by “protocol”. Even though we commonly refer to atproto as a “protocol”, I actually think it’s a framework composed of several distinct protocols (DID, repo, sync, OAuth, etc) + a schema language (Lexicon).

In that sense, this is a distinct protocol. It is a new way of storing & syncing data that is distinct from the public protocol. However it shares all the other foundations (DIDs, OAuth, Lexicon, etc) with the public broadcast protocol. In that sense, it’s a part of the same framework which we refer to as “AT Protocol”.

I know this reads as annoying semantics, but it’s kinda the thrust of my point. I really don’t want this to be viewed as a separate, new thing, different from atproto. It’s part of atproto! Despite being a distinct data/sync protocol.

6 Likes
13

I want to put in my firm dissenting opinion on this: a separate URI scheme (like ats://) is the better all-around design.

Globally public URIs and access-controlled URIs have different audiences and are going to show up in different places. A public at:// is like a domain name or account handle; or a wikipedia or reddit URL. They are the recognizable public brand, will show up outside the atmosphere, and are a welcome sign: if you see one, you can almost certainly resolve the reference. A permissioned ats:// is more like a file://, http://localhost:3000, or email message UUID; or a discord invite link. The “intent” or “what can I do with this” is pretty different. I think that is worth bumping up to the left-most, top-level-bit of the URI string: the scheme.

I think it is fine for the “atproto protocol framework” to encompass multiple URI schemes. It definitely encompasses multiple sync/resolution mechanisms. Public repo content and permissioned space content are all under the same big tent, but they are also pretty distinct, and I think that maintaining that distinction visually is better than conflating them. Public data (with a at://) is radically more public and redistributable than regular web content (with a https://) and I think we shouldn’t forget how different and important that is. Putting permissioned data under the same URI scheme weakens that expectation.

You could receive an atproto record on a thumbdrive and it may still be verified as “authentic”.

^ this is true for public repo data, but not for permissioned data!

In terms of lexicon string types, I think it is sort of a wash. You are going to care whether something is a public ref vs private ref in almost all situations, so it dilutes the at-uri meaning to be almost as general as uri. I think that we should be able to reuse the same lexicon record schemas for both public and permissioned records, but it is going to require some thought and probably tweaks to existing schemas either way.

Even though adding s is just a single character, I think it is visually distinctive because the part before :// is so short. ats:// vs at:// is even more visually distinct than https:// vs http://. I’m not super hung up on the name though. atspace:// would also be fine. I think the prefix of the URI is much more important for visual distinctiveness and understanding than the length of the URI, and especially compared to a single additional path segment in the middle (eg, for differentiating a “3-part public record URI” from “3-part space URI plus /space/”).

My overall argument is that we should keep things simple and predictable. In this case we need to reference records in two fairly distinct ways: in one way we have a global public reference (single authority), and in another we need to provide both space context and repo context (two authorities). Those are different enough that I think it is simpler to have separate URI schemes, each with consistent predictable structure and expectations, instead of overloading a single URI scheme for both.

10 Likes
Permissioned data proposal discussion
14

This is maybe a naive idea, but if ats:// is a no-go then how about a fragment for the specific service?

at://did:plc:abc/coll/rkey#atproto_pds
vs.
at://did:plc:xyz/stype/skey/did:plc:abc/coll/rkey#atproto_space_host

15

I’d also like to voice a strong dissenting opinion: a separate URI scheme follows long-standing precedent, reduces cognitive overhead and software complexity by making the transport distinction explicit, and has less ambiguous error handling. I think this direction trades short-term convenience for long-term ambiguity.

The transport models are incompatible, not variants of each other

Public atproto records flow through a well-understood pipeline: repos export as CAR files, relays consume firehose events, AppViews index everything, and any party can fetch any record without credentials. The public transport supports and encourages auditing, with visible ties between identity and cryptographic verification.

Permissioned data spaces introduce authorization checks, access-controlled fetching, and records that deliberately never appear in the public sync pipeline. The permissioned transport encourages and supports deniability, putting up barriers to proving that an identity was the historical owner of a piece of content.

The creation and usage flows for public and permissioned data require two different transports with different security guarantees. The mechanisms that make at:// resolution work today simply don’t apply, and no amount of shared syntax changes that.

Resolution paths genuinely differ

One argument for unification is that at:// doesn’t specify a sync mechanism, so the scheme isn’t making transport promises in the first place. I don’t think that holds up. In practice, every existing consumer of an at:// URI resolves it the same way: resolve the identity, find the PDS, call com.atproto.repo.getRecord or pull from sync.

A permissioned record can’t be resolved that way; it requires an authenticated flow with different endpoints and different failure modes. Resolution and access are tightly coupled. When two identifiers require different resolution procedures, the identifier itself is the right place to signal that. Otherwise every resolver has to attempt one path, fail, and guess.

Web precedent overwhelmingly favors distinct schemes here

When other protocols dealt with this exact same scenario (one resource, multiple transport or security contexts), they consistently landed on separate schemes. HTTP and HTTPS address the same resources, but the security context differs, and that alone justified a distinct scheme. WebSockets made the same call with ws:// and wss://. And IPFS separated ipfs:// from ipns:// because immutable content addressing and mutable name resolution resolve differently, even though both identifiers live side by side in one ecosystem. In every case, the payoff is legibility: the identifier itself tells you which transport you’re dealing with before you ever touch the network.

The standards point the same way. RFC 3986 frames a scheme as identifying how to interact with a resource, not just what the resource is, and RFC 7595’s registration guidance treats a differing access method as the case that warrants a new scheme. Distinct schemes would give atproto that same legibility, with public and permissioned transports readable at a glance to resolvers, link handlers, and the humans looking at a URI. That pattern has held for decades: when transport or security semantics diverge, the scheme diverges. Unification would be a highly unusual choice here.

Error semantics become ambiguous

Under a unified scheme, what does a failed resolution mean? With public at://, a not-found unambiguously means the record doesn’t exist. With permissioned data in the same namespace, a failure could mean the record doesn’t exist, or it exists and you lack access, or you queried the wrong transport entirely. The failure itself becomes ambiguous because the two transports have very different success and failure scenarios, and different recovery steps when something goes wrong.

Distinct schemes keep the error model of each transport clean: public resolution failures stay unambiguous, and permissioned resolution can define its own access-denied semantics without contaminating the public contract.

Keep them separate

My general opinion is that these differences are enough to warrant separate and distinct protocol identifiers. Permissioned data needs its own URI format, its own way to resolve where data is stored, and its own mechanism to prove access to that data. Trying to merge it in would create unnecessary complexity and strain for the development ecosystem and the spec.

A distinct scheme costs a small amount of aesthetic unity now. What it buys is permanent: unambiguous resolution, safer defaults for the existing ecosystem, and cleaner failure modes. I’d rather we make that trade deliberately than find out the hard way, through leaked URIs and confused resolvers, after deployment.

8 Likes
16

I haven’t completely steeped myself in all the rational on both sides but I agree with @bnewbold.net and @ngerakines.me that keeping them separate makes the most sense.

The motivations to merge them seem to be around how it feels or to make things simpler by not having to handle different URI types, and while both of those are perfectly valid motivations, I think that it masks the way that it really is.

These two protocols are something different, and will need almost completely different handling. I think from some perspectives they are more different than http and https! And yet we still think of HTPP and HTTPS as the “same protocol” for the most part because the semantics are nearly identical, other than a layer of encryption / authentication provided by TLS.

Comparatively, at:// and ats:// have completely different semantics. You can’t just remove the s from ats:// and get a valid at:// URI[1].

I don’t think having a different URI scheme hurts in a “This isn’t ATProto” sort of way, except in the ways that actually are beneficial because it is honest about the fact that “This isn’t the global broadcast side of things” and “you will have to do things completely different to get this permissioned data”.

  1. I am realizing this may be an argument against using ats:// because it may give the impression, due to it’s similarity to https, that you can just remove the s. ↩︎

4 Likes
17

I made a jokey +1 for triple slash file path semantics, but the did / URI syntax is a separate issue.

Agree here that if one built a state diagram / call flow between public and permissioned, they would look very different.

I don’t think we should bring this to the IETF yet (just saying this out loud) – this is the community figuring out pre-standards stuff, it shouldn’t head there until we have multiple implementations and interop in the wild for some time. But we can “look ahead” at how this might be perceived.

@bnewbold.net I assume that IANA would need a separate registration for ats:// ?

18

I voiced my stance on this in another thread but I think keeping it under at:// is Fine Actually and prevent clutter. I can see the benefit of it being under another scheme being good for human semantics but people aren’t reading DIDs out loud, it’s a machine thing.

19

I agree with @bnewbold.net , ats:// makes more sense long term. I also want to commend the team for not trying to tackle e2e encryption at the same time with this proposal. It leaves more room for different experiments adding encryption schemes as another layer in the future, like dmls. draft-kohbrok-mls-dmls-03 - Decentralized Messaging Layer Security

3 Likes
20

+1 for the single scheme (at:// with the /space/ segment). I’ve built this in HappyView, so I’ll try to speak to how it’s working.

First off, HappyView currently supports both formats. I built support for the ats:// based on Dan’s dev diaries, then added at:// after the proposal PR was closed. After building all of this out, I personally lean towards at://.

The /space/ disambiguator has worked well in HappyView: the parser decides space-vs-record on the literal space segment alone, with no network call, no lexicon resolution, and no DB lookup. Dan’s reasoning for it holds up. Being able to tell what’s on the other side of a URI without first resolving a lexicon makes the code a lot simpler.

@bnewbold.net and @ngerakines.me are right that the two read paths barely share any machinery. Public and private records in HappyView are largely separate pipelines. But I don’t think that’s a problem with the scheme. They’re just different sync mechanisms and they were always going to be, regardless of what we call them. What a shared at:// does give us is one namespace and one mental model, which still feels worthwhile to me.

Having built it both ways, spaces do feel like part of atproto regardless of which scheme we use. Same DIDs, same lexicons, same OAuth, same data model… Spaces are just a different sync mechanism behind a permission boundary. I’d rather the scheme say that plainly, and I think the /space/ segment achieves that just fine without a separate scheme.

3 Likes