Web Tiles as the base for Lexicon Embeds

Discussion
,
1

Now that @robin.berjon.com has released an initial spec and implementation of Web Tiles, we can see how we can use or extend this as a base for Lexicon Embeds.

Lexicon Embeds is my label for how to have native ATProto content embed in different apps, from display all the way to interactivity— eg an event post in bsky with an RSVP button.

Getting this as a general capability - and getting security and display right - means embeds can function in Leaflet and Bsky and anywhere else, rather than one off for each new lexicon.

Long form reads, events, image carousels, and specialist communities like code highlighting are some early examples that would likely have the biggest impact.

15 Likes
Ideas on extending Schema.org embed displays for atproto
Ideas on extending Schema.org embed displays for atproto
2

Really excited for this!

2 Likes
3

Yeah this would be absolutely fantastic. I don’t see yet how the Bsky app would discover which embed to use for a linked record but I assume some sort of definition in the actual record? Anyway, hope to read more about this

4

So, this is ATProto-native iframes, very cool !!

5

This is the part we have to figure out! I think this is core to Lexicon Embeds (if it’s a good thing at all)

Is this something that the lexicon owner publishes?

Or something that the publisher can customize?

I think some OpenGraph like things, around type of page something is, could work. So, complete spitball. If Bsky-the-app decides to support a “longform” view, then someone could post any type of lexicon that expressed itself as “longform” and would be rendered that way with certain affordances — eg a “Read More” that pops a modal or slides in a sheet with the full content.

That’s an example of the product having control over display and interaction, or at least setting boundaries within which one could customize.

If I was Bsky, I would also “allow list” lexicons but this is exactly where things get really tricky really fast.

Robin’s WebTiles work is the basis for a very good approach to security but there are several more layers of complexity at scale.

6

I forgot that I had created an open collection on Semble for Lexicon Embeds. Feel free to add related links here

Thanks @wesleyfinck.org for adding this thread to the collection!

Things to look for - OpenGraph basics, emerging agent “app” formats, webxdc stuff, oembed, Open Frames etc etc

1 Like
7

yeah it feels like a lot of this is already very close to being in place right. when I could share a bsky post about an Anchor checkin on bsky ideally I would just add an embed of type “app.bsky.embed.external” to the post. And then the bsky client could request some sort of standard agreed endpoint or web tile definition json from the external embed uri and fetch that to render the external content. I know I should read all the linked specs first but that’s how it feels it should work.

But yeah I get you can’t just go and render anything anyone tells you to embed without running into issues quickly as a client.

2 Likes
8

Right — that’s something I’m working on for the next iteration. (You can see some old thinking and screenshots from an earlier, non-AT prototype up at Web Tiles)

In a nutshell, a key part of the composability model is “wishes”. This is a concept that comes from task-based computing. Basically, instead of “open foo.md in Zed” which ties you to a specific implementation you say “I wish to edit some markdown’“. And in response to that, the system resolves either your fave Markdown editor or offers a list of existing ones.

That approach has often stumbled on the fact that you need a way to find implementations for wishes that you don’t already have locally. When Google was working on Web Intents, that meant… using Google Search to find web apps that could handle your wish, which wasn’t great.

But… with AT that’s built in!

So say that I have a tile that implements a Markdown editor. Its post will, as part of its lexicon, include:

{
  can: 'edit',
  what: 'text/markdown',
  description: 'Edit Markdown',
}

Then, anyone who’s indexed the network can offer this tile to someone who wishes to edit Markdown. And since tiles at just AT objects, they can be reviewed, liked, bookmarked, you can use your social graph for trust, etc. (Hand-waving here, keeps me warm.)

For Lexicon Embeds, a wish would be:

{
  can: 'render',
  what: 'social.popfeed.feed.list',
}

And that would make it available for precisely that purpose.

I don’t want to oversell the maturity of the system. This still needs:

  1. Specifying wishes and producing at least one indexing service. (Should be done soonish, I’ve mostly been interrupted by the fact that I’m moving all my infra out of the US.)
  2. Having a good API so wishes can be given data and send it back. (Not too hard.)
  3. Good library that folks can use to auto-pick/prompt for implementation of embeds, including picking a default. (Also not too hard.)
  4. Progressively, all the stuff that goes around this about making it possible to have blocklists for bad actors, graph-based recommendations, etc.
12 Likes
9

This is great @robin.berjon.com

Right so with the lookup one can imagine the Tile, the client, the Lexicon, or the current user having preferred / default render tiles?

And we’re calling these all Tiles, whether they are display/render or functional, etc?

3 Likes
10

Maybe an example will clarify the model I have in mind. Let’s assume that I have a lexicon I use to publish posts about my karaoke performances, and within that I embed StatusSphere content representing the vibe of the moment. Something like:

{
  $type: 'com.berjon.karaoke.legend',
  song: 'HOT TO GO!',
  kompromat: { $type: 'blob', … },
  vibe: {
    $type: 'xyz.statussphere.status',
    status: '🪩',
  },
}

Now, I may have a karaoke renderer to show that, but the embedded vibe I’m delegating to a tile. Within the app, at the point at which I want to embed the vibe, I’d probably have some kind of custom element like this:

<web-wish for='render' what='xyz.statussphere.status' fallback='at://bmann.ca/ing.dasl.masl/d34db33f'></web-wish>

There are multiple ways that this could then work (all valid — the point of small composable standards is people can pick):

  1. You’re logged in and you have a tile that can respond to wishes of that kind (because you clicked “install” or some kind of like on a previous one) — it gets loaded into the web-wish element and passed the vibe data.
  2. You’re not logged in and this is clearly a thing where you should be — you see a “Log In To View Vibe” button.
  3. You’re logged in but don’t have a tile for this, or not logged in and it doesn’t matter — maybe you see a list (with the fallback at the top) or maybe it loads the fallback tile for this directly.

This requires:

  1. A standardised way to pass data to/from the tile.
  2. A standardised way for the tile to negotiate the real estate it needs with its embedder.

Everything else is “just” the security model and how to load off of AT.

6 Likes