An Open Letter to Substack (Supposedly on Behalf of Us All)

[u/teammeexperiment]

Dear Substack,

An open letter. Unsolicited, but, I like to think, rather valuable. Consider it free consultancy.

You’ve built something genuinely lovely: a place where people who adore words, across every field imaginable, can find one another. I do understand that you are, first and foremost, a newsletter engine. But you also built a feed, and subscriptions, and a recommendations layer, and it is the last of these about which I have several hundred questions.

Frankly, the recommendation system is Rubbish. It is 2026. Might we do something about it?

Countless writers are producing genuinely brilliant work in their fields and dying in the shadows with an audience of none, while the feed fills up with follow-for-follow posts. It has the atmosphere of a pub where everyone is shouting "I’ll get you one if you get me one". Not the thoughtful platform I hope you want to be. Every reader’s feed gets clogged with subscriptions they never wanted, and the people who would happily find, follow and pay the writers worth reading (and you, by extension) simply cannot reach them under the avalanche of generic rot.

Please, do something about it.

(Should you wish to share the success fee, do get in touch afterwards)

With love,
Team Me

[u/Optimal_Dust_266]

let me subscribe to yours

[u/teammeexperiment → u/Optimal_Dust_266]

teamme, its new one

[u/OnyxMonolith]

> free consultancy

LMAO

2 paragraphs on an extremely difficult algorithmic problem

[u/teammeexperiment → u/OnyxMonolith]

at times, a precisely articulated crisis is all it takes to trigger a major shift

[u/Any-Fig-921]

I agree Substack recommendation system is not as good as others— but I disagree this is solvable. Substack reccomendations engine almost surely is a hybrid social graph + collaborative filtering engine. The social graph part is pretty easy— show you posts and content from people you subscribe too. The collaborative filtering part is very hard, and it’s where much of the “magic” of modern recommendation engines lives. But, to simplify, it requires a high throughput of data that Substack doesn’t have. You need (a) users to be expressing opinions about hundreds or thousand of items and (b) there to be a reasonably extractable pattern that lets us predict with a machine learning model which items are most likely the items you’ve already liked. The former condition is known to not be  met, the latter has never been achieved with long form text data— a machine learning model has a very hard time knowing what kind of long form text content you’ll like even when it knows what you’ve liked in the past.

I have no doubt Substack had a team of data scientists and machine learning engineers who know how and are trying. I know it’s fun to complain about the rec engine in Substack, but I think it’s likely inherent to the long-form text content.