markpasc

Coming soon

Blog archive visualization

Speaking of visualization, I started looking into techniques for visualizing blog archives yesterday. While you can compress the month links into a table, something that presents some post-wise salience would be good.

The only thing I really found searching (your blog archives should never, ever, ever have “blog archive” in the HTML title, please) was this research paper (here's a PDF of the paper) describing a display the investigators implemented as a Windows application.

While this would translate okay into an interactive HTML5 archive view, surely there's something better/cleaner/more explicable. Anybody got any pointers?

Private webhooks, hold the crypto?

I was hoping to say “I was going to write about Martin's dialback system but Blaine Cook saved me the trouble!” but Blaine's system, with his goal of being crypto-free, is simpler. (At least if you ignore the use of webfinger, which I will for right now.)

As discussed in that WTF post (if you can wade through it), you need a way to authenticate the subscription request, to tell by which person owns it what posts you trust to that subscription. While I was thinking of something like OAuth, Blaine's suggestion seems to be:

• Identify the person in the subscription request.
• Discover that person's profile and find (what he calls) their “delegate” URL.
• Confirm that the subscription's callback URL is the person's delegate URL.

Blaine suggests using the HTTP From header to identify the subscribing person. (I'm not sure why that particular slide calls it “pseudocode,” but this “Addressing Identity on the Web” brief of Blaine's for an Internet privacy workshop this past December says plainly what he means.) The value of the From header is supposed to be an email address, so using it here would mandate the use of webfinger (that is, LRDD+XRD) and email address-shaped identifiers. I think that's a curious requirement to make, as the rest of this system would work just as well with an X-From-URI header and HTML link discovery (or, sure, webfinger if acct: URIs should be supported).

Seems like this “delegate” endpoint is like the activity sink I was describing: a person's PubSubHubbub endpoint where any activity they've subscribed to can be pushed. It would indeed be simpler not to have to authenticate with OAuth to it, instead using the PubSubHubbub secret as equivalent to a bearer token.

However, that also means you couldn't post activities there without having been subscribed to, so you couldn't send third-party activity such as mentions there. If Alice doesn't subscribe to Carol, Carol has no PSHB secret from Alice, so Alice can't verify that a mention from Carol is from Carol. If Alice needed to dial back to verify Carol's request, the equivalent to checking that the callback URL matched would be verifying the existence of the post containing the mention, which would require Carol's post be in a feed discoverable from Carol's profile. That's pretty particular to mentions, so other types of activity may have wildly varying verification steps.

Rather than have one-off verifications for each kind of activity, it might be easier to provide a generalized step where Carol can associate with Alice, receive an OAuth token from her, and then use the token to authenticate any activity sent to Alice's activity sink in a general way. (This is the OAuth dial-back mechanism I wanted to avoid describing, and I realize full well I still haven't described it adequately here.)

Anyway, if there's no other reason to have an OAuth token, seems to me right now in my sleepless state that this identify-and-verify step would work fine for regular subscriptions as Blaine describes. Unfortunately, an OStatus style network seems to have another reason.

Super

Half of this 3D photo as seen through my experimental View-Master print of some random 3D photos from the web.

Watch, trust, friend

As a first step toward a distributed journal network, last weekend I started writing an OStatus node in Django. While I did get to the point where it can subscribe and receive statuses posted from status.net, I started looking at actually telling status.net I was subscribed. That's where things get complicated.

OStatus says that, after subscribing with PubSubHubbub to really receive the posts, you should use the Salmon protocol to send a “follow” activity to tell the remote node who actually subscribed. That means, to follow someone for the first time ever, you must:

1. Generate an RSA key pair for the follower for use in “magic signatures.”
2. Provide a machine-readable profile for the follower as an XRD.
3. Include the follower's RSA public key in the XRD.
4. Make the machine profile discoverable through LRDD.
5. Find the followed person's feed.
6. Subscribe to the followed feed using PubSubHubbub.
7. Look for a Salmon endpoint when discovering the feed's hub.
8. Generate a “follow” Activity Streams activity.
9. Sign it using the follower's private key.
10. Encode the activity and signature as a “magic envelope.”
11. Send the activity to the followed person by POSTing the magically enveloped version of the activity to their Salmon endpoint.

The followed person's end then must:

1. Unpack the magic envelope to get the original activity information.
2. Find the follower, as identified by the activity subject's URL (which let's assume must be an HTTP URL).
3. Find the follower's machine-readable profile by using LRDD on their URL.
4. Acquire the follower's RSA public key in the resulting XRD.
5. Use the follower's key to verify the original magic signature.
6. Act on the submitted activity.

Is all this really necessary? I'll admit this Salmon/magic signatures/LRDD/XRD stack can make sense as a set of small pieces loosely joined, but until there's wide support for this stack, in practice it's a big monolith, vaguely integrated. I've heard of most of these protocols before and I gave up when I couldn't find where OStatus or Salmon said to get the key for making magic signatures. (The Magic Signatures spec mandates its own LRDD-based PKI.)

It's daunting to think I have to implement all this in order to tell someone I followed them. These are the table stakes for the OStatus network, and they are way, way too high.

On top of this, OStatus doesn't support a use case that's crucial for me: private content. To find the simplest solution that will also support private content, it might be instructive to reconsider the original basic tasks we want to accomplish. These can be succinctly described as Watch, Trust, and Friend.

W.T.F.?

The Watch Trust Friend (WTF) model comes originally from LiveJournal, where folks experimentally discovered that packing new semantic meaning into the emotionally loaded word “friend” was a bad, bad plan. (While LiveJournal still uses the single “friend” concept, Dreamwidth successfully split friends into “subscription” and “access.”)

As used here for a social publishing system, these terms mean:

• Watch: subscribe to someone's stream for reading.
• Trust: provide private content in your stream to someone for them to read.
• Friend: provide an explicit “friend” association with someone for others to see.

We can easily see the utility in unpacking them. You can watch a stranger who writes interesting things without declaring them a friend or trusting your private posts to them. You can trust someone with private posts without publicizing them as a friend. You can say someone is a friend as long as they have a web profile, without them participating in the network at all. These are only a few examples, but the network can clearly support many more real things people need social publishing networks to do this way.

As these features are orthogonal enough to implement separately, they seem to be the atomic operations we need to codify to distribute this kind of social network. Let's look at how we might implement them.

Watch

OStatus is clearly right to use PubSubHubbub for subscription. Since feeds were designed to handle this stream style published content, the mechanism works well. However, OStatus layers in all the above-maligned infrastructure to tell the publisher when someone has “socially subscribed.” Status.net, for example, uses that to publish the “followers” and “following” sidebars.

If we split that social payload out to the “friend” concept, there is a simple alternative solution for watching: identify yourself when you subscribe. To subscribe in plain PubSubHubbub, you:

1. Look for the hub in the feed you want to subscribe to, as the rel="hub" link.
2. Send a subscription request to the publisher's hub.
3. Confirm the subscription request when the hub verifies it.

In this case, you as the follower/subscriber would identify yourself when sending the subscription request. As is common among other protocols in this area, the simplest conceptual solution seems to be not public key encryption with LRDD-based PKI, but rather OAuth.

But, as I was puzzled by where the magic signature key came from, where does that OAuth token come from? Because the subscriber is itself the provider of the subscriber's identity (at least for the purpose of this network), it can generate a key on its own without the user present. In a different design Martin Atkins was discussing with me, he referred to a different design using a “dialback” mechanism that would also allow an entire site one key with which it could act as any of its users. (A new mechanism is still necessary with regular OAuth to automatically register as a consumer, but I think there are a couple fledgling ways to do that already.) Conceptually, though, one could use the regular OAuth key flow, noting only that the identity provider itself is also acting as the person's user agent.

Friend

Let's take friending next, as the solution I propose provides context for a discussion of trusting.

When the watching and trusting functions are split out, “friending” means only the publishing of a friends or contacts list on your profile. (The same access to your contacts could be provided to apps that ask for it.) LiveJournal too has an interesting history with friends listing on the profile, at least in that listing your followers is optional. While it's useful information for third parties, it can be annoying when someone you don't know or, even worse, know but don't like “friends” you, creating a link back to themselves from you.

There's also historical weight behind the existing “friends list” solution for weblogs: blogrolling. The practice of listing blogs (friends) you read (like) in your sidebar spawned XFN, so it seems like a proven solution for this simple problem. As a technology using the commodity link, the simplest blogroll is a bag of HTML, meaning they are necessarily unidirectional. For blogrolls, you have to build a pretty smart system to discover the reverse directionality—which, as LiveJournal has shown, is then just a headache for some folks anyway.

Instead, it would make sense to build unidirectional blogrolls or “follow” lists, and let folks use them as they would HTML blogrolls. (If you really want a dynamic “friends of” list, it's not easy, but it is possible.)

In the same way as you may not want to display those followers as persistent friend-ofs, the message sent by a “follow” message in the style of OStatus is ambiguous anyway. While the OStatus “follow” message is signed by the follower, at best you're checking that that person claims to have followed you. They may be spam-following. In fact, with following decoupled from watching, there's no real disincentive not to spam-follow. (People who'll spam-follow on Twitter are probably only reading their @mentions anyway.)

However, this is a legitimate social gesture too, a bundled use for “friending” on LiveJournal and Twitter: telling someone you exist. When you find a friend of a friend on the network whom you know, there is a signal in your following/friending them, a little “hi!” transmitted on the carrier wave of the follow/friend notification. If they really know you, they then know they can follow/friend you back. While that's the same mechanism as spam-following, it's important for building the network to be able to announce you've moved in and are ready to join the fun.

Therefore it makes sense to me to provide the signal at that same level: because the message isn't a trustworthy guide to whether someone is a real friend or even a real person, let's not use a protocol for “friending” at all. There should instead be a message equivalent to, “Hi! I exist!” The software implementation can then treat the message however it likes, including but not at all limited to:

• checking if the sender is advertising friendship in their blogroll too
• colluding to rate limit or blacklist people or sites
• ignoring the message completely
• showing trustworthy “hi”s to the user, with actions to watch/trust/friend in return

Rather than try to enumerate all the human interaction about the network and codify it into a protocol, the “hi!” message is solely an advisory machine hint that the human receiver might be interested in acting on the sender. It's up to the receiver and their software what it means.

As this is only intended to replace friending, there still needs to be an activity sink, like OStatus's Salmon endpoint. It would still accept all the other messages OStatus uses it for, as well as this new “hi!” message. However, I'd hope you could use OAuth instead of magic signatures, not in addition to them. I'd rather think of it as an AtomPub sink or PubSubHubbub callback than require LRDD, XRD, and crypto and brand-new minimalistic PKI.

Trust

OStatus doesn't address trusting because it doesn't address private content, so I should probably explain how I assume this system would handle private content.

Because PubSubHubbub hubs push content to individual subscribers, they could theoretically give different posts to different subscribers based on trust groups. Because watchers have identified themselves when subscribing, the hub can know which callbacks correspond with which people, and thus which callbacks get a given private post. (While that does mean you're trusting not only people but their subscriber software, that's the same as trusting their network, web browser, and API applications not to leak your private LiveJournal posts and Twitter tweets.)

In practice, since none of the existing hubs support privacy at this level (and you don't necessarily want to trust a big uncaring hub with your trust information or private content anyway), the publisher must itself assume the hub's role too. These “hublishers” do lose the ability to outsource publishing to a hub service, but for this use case of social publishing, it seems a useful trade-off. Since you're still pushing the posts out, you avoid the thundering herd problem; at worst you're too popular and get behind publishing a post out to your many, many subscribers.

When I started discussing the WTF model with Martin, I argued for a “trust” message to tell someone who isn't watching you that you have private content for them. However, he persuaded me that it's the same as friending. The only difference is the action requested by a “trust” message is for you to watch back instead of friend back. This trust message can be implied by the same “hi!” message used for friending.

There may be a place for an advisory flag that a stream contains private content, but as that comes through the PubSubHubbub callback, it's not the same as these other direct protocol messages. Besides, I would think it unnecessary for all the same reasons you don't really want to tell Bob when Alice unfriends him. The subscriber will get the private content they get, and can deduce from that whether or how they're trusted by a publisher, if it cares.

So, wh—

Evidently applying the WTF model here results more in guidelines for behavior of the implementations than a congruent protocol. The protocol itself is reduced to, for watching:

1. Find the followed person's feed.
2. If not already associated, go through an automatic OAuth flow to obtain an OAuth token to act as the subscriber on the publisher's site.
3. Subscribe to the followed feed using PubSubHubbub, authenticated with the OAuth token.

and for everything else:

1. Find the followed person's activity sink.
2. If not already associated, go through an automatic OAuth flow to obtain an OAuth token to act as the subscriber on the publisher's site.
3. Send a “hi!” message, authenticated with the OAuth token.

This seems to be a lot simpler, so it's what I expect I would try to implement. It affords some different use cases from OStatus... but I think they're better use cases.

I realize this prose description is incomplete. How the OAuth association works could use some pinning down. IIRC something else Martin planned to argue was that, since (a) the Watch “message” implicitly contains a “hi!” and (b) you don't often want to watch, trust, or friend someone who doesn't watch you back, maybe you really only need Watch.

Does this sound like you'd want software like this to work?

Collected works

Here are some things I've been working on lately.

• Convore for iPhone. Leah, Eric, and Eric at Convore were kind enough to let me pitch in for their iPhone app so they could submit it in time for SXSW. It's definitely a 1.0 but people seem to like it all the same!
• friendstream.tv. After Gercek saw Leapfrog, he asked if I'd like to help build a similar thing for web videos. He's been at SXSW too, but we got the initial version of it launched as well.
• isthereagiantsga.me. If you work in SoMa, you might like to know whether there's a Giants game at AT&T Park, so here's a site that tells you (without the ads and merch upsells on the obvious .com).

Leapfrog is still there, and should be getting Mlkshk support not too distantly. (I have to say Stellar is really nice, and functions similarly in practice to Leapfrog, but I wish it supported Tumblr.)

There are also a couple of other things I'm working on (a dominoes game for iOS with somebody and, yeah, I even wrote some code sidling toward that blog network app) but they're some ways off probably.

Sunrise

Going to market with the world we have

But as a white American with no nationalistic horse in the race, whose bar for great entrepreneurs had been set by the likes of Larry Ellison and Mark Zuckerberg, I just wanted to report what I saw without preference or judgement. I expected as a result there would be something for everyone to hate in the book: That many Americans would criticize me for not grandstanding more about the moral issues in China, that Indians would say I was too harsh on the infrastrucutre failings and crippling poverty of the subcontintent, and that Israelis would find the criticisms of how their entrepreneurs have performed over the last decade as overly harsh.

So as someone who didn't take pains to excoriate countries for thier local conditions, Mike's reaction was surprising to me. Perhaps I spent so much time in these countries, I began to take some of the conditions as a given. But it might also say something about me.

via www.sarahlacy.com

Sounds like some book.

Could they be more wrong

Do they know Glenn is a journalist? Do they know he left the stuffy world of law? Have they thought about why he might have done that?

via emptywheel.firedoglake.com via n8han

How do we live together with robots?

...which is a great way to approach it. Polite robots would be the best! But I don't think we get to choose. Polite robots would be lovely. But the nature of the fourth kingdom - their equivalent of evolution - is that they reproduce in the sales figures of technology corporations and the womb-factories of China. The testes of robots are the shelves of Toys-R-Us. Humans don't get to choose the personality of robots, the market does.

via interconnected.org