Watch, trust, friend

As a first step toward a distributed journal network, last weekend I started writing an OStatus node in Django. While I did get to the point where it can subscribe and receive statuses posted from status.net, I started looking at actually telling status.net I was subscribed. That's where things get complicated.

OStatus says that, after subscribing with PubSubHubbub to really receive the posts, you should use the Salmon protocol to send a “follow” activity to tell the remote node who actually subscribed. That means, to follow someone for the first time ever, you must:

1. Generate an RSA key pair for the follower for use in “magic signatures.”
2. Provide a machine-readable profile for the follower as an XRD.
3. Include the follower's RSA public key in the XRD.
4. Make the machine profile discoverable through LRDD.
5. Find the followed person's feed.
6. Subscribe to the followed feed using PubSubHubbub.
7. Look for a Salmon endpoint when discovering the feed's hub.
8. Generate a “follow” Activity Streams activity.
9. Sign it using the follower's private key.
10. Encode the activity and signature as a “magic envelope.”
11. Send the activity to the followed person by POSTing the magically enveloped version of the activity to their Salmon endpoint.

The followed person's end then must:

1. Unpack the magic envelope to get the original activity information.
2. Find the follower, as identified by the activity subject's URL (which let's assume must be an HTTP URL).
3. Find the follower's machine-readable profile by using LRDD on their URL.
4. Acquire the follower's RSA public key in the resulting XRD.
5. Use the follower's key to verify the original magic signature.
6. Act on the submitted activity.

Is all this really necessary? I'll admit this Salmon/magic signatures/LRDD/XRD stack can make sense as a set of small pieces loosely joined, but until there's wide support for this stack, in practice it's a big monolith, vaguely integrated. I've heard of most of these protocols before and I gave up when I couldn't find where OStatus or Salmon said to get the key for making magic signatures. (The Magic Signatures spec mandates its own LRDD-based PKI.)

It's daunting to think I have to implement all this in order to tell someone I followed them. These are the table stakes for the OStatus network, and they are way, way too high.

On top of this, OStatus doesn't support a use case that's crucial for me: private content. To find the simplest solution that will also support private content, it might be instructive to reconsider the original basic tasks we want to accomplish. These can be succinctly described as Watch, Trust, and Friend.

W.T.F.?

The Watch Trust Friend (WTF) model comes originally from LiveJournal, where folks experimentally discovered that packing new semantic meaning into the emotionally loaded word “friend” was a bad, bad plan. (While LiveJournal still uses the single “friend” concept, Dreamwidth successfully split friends into “subscription” and “access.”)

As used here for a social publishing system, these terms mean:

• Watch: subscribe to someone's stream for reading.
• Trust: provide private content in your stream to someone for them to read.
• Friend: provide an explicit “friend” association with someone for others to see.

We can easily see the utility in unpacking them. You can watch a stranger who writes interesting things without declaring them a friend or trusting your private posts to them. You can trust someone with private posts without publicizing them as a friend. You can say someone is a friend as long as they have a web profile, without them participating in the network at all. These are only a few examples, but the network can clearly support many more real things people need social publishing networks to do this way.

As these features are orthogonal enough to implement separately, they seem to be the atomic operations we need to codify to distribute this kind of social network. Let's look at how we might implement them.

Watch

OStatus is clearly right to use PubSubHubbub for subscription. Since feeds were designed to handle this stream style published content, the mechanism works well. However, OStatus layers in all the above-maligned infrastructure to tell the publisher when someone has “socially subscribed.” Status.net, for example, uses that to publish the “followers” and “following” sidebars.

If we split that social payload out to the “friend” concept, there is a simple alternative solution for watching: identify yourself when you subscribe. To subscribe in plain PubSubHubbub, you:

1. Look for the hub in the feed you want to subscribe to, as the rel="hub" link.
2. Send a subscription request to the publisher's hub.
3. Confirm the subscription request when the hub verifies it.

In this case, you as the follower/subscriber would identify yourself when sending the subscription request. As is common among other protocols in this area, the simplest conceptual solution seems to be not public key encryption with LRDD-based PKI, but rather OAuth.

But, as I was puzzled by where the magic signature key came from, where does that OAuth token come from? Because the subscriber is itself the provider of the subscriber's identity (at least for the purpose of this network), it can generate a key on its own without the user present. In a different design Martin Atkins was discussing with me, he referred to a different design using a “dialback” mechanism that would also allow an entire site one key with which it could act as any of its users. (A new mechanism is still necessary with regular OAuth to automatically register as a consumer, but I think there are a couple fledgling ways to do that already.) Conceptually, though, one could use the regular OAuth key flow, noting only that the identity provider itself is also acting as the person's user agent.

Friend

Let's take friending next, as the solution I propose provides context for a discussion of trusting.

When the watching and trusting functions are split out, “friending” means only the publishing of a friends or contacts list on your profile. (The same access to your contacts could be provided to apps that ask for it.) LiveJournal too has an interesting history with friends listing on the profile, at least in that listing your followers is optional. While it's useful information for third parties, it can be annoying when someone you don't know or, even worse, know but don't like “friends” you, creating a link back to themselves from you.

There's also historical weight behind the existing “friends list” solution for weblogs: blogrolling. The practice of listing blogs (friends) you read (like) in your sidebar spawned XFN, so it seems like a proven solution for this simple problem. As a technology using the commodity link, the simplest blogroll is a bag of HTML, meaning they are necessarily unidirectional. For blogrolls, you have to build a pretty smart system to discover the reverse directionality—which, as LiveJournal has shown, is then just a headache for some folks anyway.

Instead, it would make sense to build unidirectional blogrolls or “follow” lists, and let folks use them as they would HTML blogrolls. (If you really want a dynamic “friends of” list, it's not easy, but it is possible.)

In the same way as you may not want to display those followers as persistent friend-ofs, the message sent by a “follow” message in the style of OStatus is ambiguous anyway. While the OStatus “follow” message is signed by the follower, at best you're checking that that person claims to have followed you. They may be spam-following. In fact, with following decoupled from watching, there's no real disincentive not to spam-follow. (People who'll spam-follow on Twitter are probably only reading their @mentions anyway.)

However, this is a legitimate social gesture too, a bundled use for “friending” on LiveJournal and Twitter: telling someone you exist. When you find a friend of a friend on the network whom you know, there is a signal in your following/friending them, a little “hi!” transmitted on the carrier wave of the follow/friend notification. If they really know you, they then know they can follow/friend you back. While that's the same mechanism as spam-following, it's important for building the network to be able to announce you've moved in and are ready to join the fun.

Therefore it makes sense to me to provide the signal at that same level: because the message isn't a trustworthy guide to whether someone is a real friend or even a real person, let's not use a protocol for “friending” at all. There should instead be a message equivalent to, “Hi! I exist!” The software implementation can then treat the message however it likes, including but not at all limited to:

• checking if the sender is advertising friendship in their blogroll too
• colluding to rate limit or blacklist people or sites
• ignoring the message completely
• showing trustworthy “hi”s to the user, with actions to watch/trust/friend in return

Rather than try to enumerate all the human interaction about the network and codify it into a protocol, the “hi!” message is solely an advisory machine hint that the human receiver might be interested in acting on the sender. It's up to the receiver and their software what it means.

As this is only intended to replace friending, there still needs to be an activity sink, like OStatus's Salmon endpoint. It would still accept all the other messages OStatus uses it for, as well as this new “hi!” message. However, I'd hope you could use OAuth instead of magic signatures, not in addition to them. I'd rather think of it as an AtomPub sink or PubSubHubbub callback than require LRDD, XRD, and crypto and brand-new minimalistic PKI.

Trust

OStatus doesn't address trusting because it doesn't address private content, so I should probably explain how I assume this system would handle private content.

Because PubSubHubbub hubs push content to individual subscribers, they could theoretically give different posts to different subscribers based on trust groups. Because watchers have identified themselves when subscribing, the hub can know which callbacks correspond with which people, and thus which callbacks get a given private post. (While that does mean you're trusting not only people but their subscriber software, that's the same as trusting their network, web browser, and API applications not to leak your private LiveJournal posts and Twitter tweets.)

In practice, since none of the existing hubs support privacy at this level (and you don't necessarily want to trust a big uncaring hub with your trust information or private content anyway), the publisher must itself assume the hub's role too. These “hublishers” do lose the ability to outsource publishing to a hub service, but for this use case of social publishing, it seems a useful trade-off. Since you're still pushing the posts out, you avoid the thundering herd problem; at worst you're too popular and get behind publishing a post out to your many, many subscribers.

When I started discussing the WTF model with Martin, I argued for a “trust” message to tell someone who isn't watching you that you have private content for them. However, he persuaded me that it's the same as friending. The only difference is the action requested by a “trust” message is for you to watch back instead of friend back. This trust message can be implied by the same “hi!” message used for friending.

There may be a place for an advisory flag that a stream contains private content, but as that comes through the PubSubHubbub callback, it's not the same as these other direct protocol messages. Besides, I would think it unnecessary for all the same reasons you don't really want to tell Bob when Alice unfriends him. The subscriber will get the private content they get, and can deduce from that whether or how they're trusted by a publisher, if it cares.

So, wh—

Evidently applying the WTF model here results more in guidelines for behavior of the implementations than a congruent protocol. The protocol itself is reduced to, for watching:

1. Find the followed person's feed.
2. If not already associated, go through an automatic OAuth flow to obtain an OAuth token to act as the subscriber on the publisher's site.
3. Subscribe to the followed feed using PubSubHubbub, authenticated with the OAuth token.

and for everything else:

1. Find the followed person's activity sink.
2. If not already associated, go through an automatic OAuth flow to obtain an OAuth token to act as the subscriber on the publisher's site.
3. Send a “hi!” message, authenticated with the OAuth token.

This seems to be a lot simpler, so it's what I expect I would try to implement. It affords some different use cases from OStatus... but I think they're better use cases.

I realize this prose description is incomplete. How the OAuth association works could use some pinning down. IIRC something else Martin planned to argue was that, since (a) the Watch “message” implicitly contains a “hi!” and (b) you don't often want to watch, trust, or friend someone who doesn't watch you back, maybe you really only need Watch.

Does this sound like you'd want software like this to work?

Comments

[szul]

I take it you didn't go any further with this, huh? Was looking for a Python OStatus implementation (or at least a base to work with), but looking at the steps necessary, this seems really cumbersome.

[markpasc]

@szul: yeah, I'm still working on this project, but as I wrote I'm not sold on Salmon, so I don't think it'll turn out to be an OStatus implementation. (I might argue that there not being one already shows that the Salmon stack is too complicated, but that's me.)