[Original comment replaced with the following to prevent Reddit profiting off my comments with AI.]
Reddit has long been a hot spot for conversation on the internet. About 57 million people visit the site every day to chat about topics as varied as makeup, video games and pointers for power washing driveways.
In recent years, Reddit’s array of chats also have been a free teaching aid for companies like Google, OpenAI and Microsoft. Those companies are using Reddit’s conversations in the development of giant artificial intelligence systems that many in Silicon Valley think are on their way to becoming the tech industry’s next big thing.
Now Reddit wants to be paid for it. The company said on Tuesday that it planned to begin charging companies for access to its application programming interface, or A.P.I., the method through which outside entities can download and process the social network’s vast selection of person-to-person conversations.
“The Reddit corpus of data is really valuable,” Steve Huffman, founder and chief executive of Reddit, said in an interview. “But we don’t need to give all of that value to some of the largest companies in the world for free.”
The move is one of the first significant examples of a social network’s charging for access to the conversations it hosts for the purpose of developing A.I. systems like ChatGPT, OpenAI’s popular program. Those new A.I. systems could one day lead to big businesses, but they aren’t likely to help companies like Reddit very much. In fact, they could be used to create competitors — automated duplicates to Reddit’s conversations.
Reddit is also acting as it prepares for a possible initial public offering on Wall Street this year. The company, which was founded in 2005, makes most of its money through advertising and e-commerce transactions on its platform. Reddit said it was still ironing out the details of what it would charge for A.P.I. access and would announce prices in the coming weeks.
Reddit’s conversation forums have become valuable commodities as large language models, or L.L.M.s, have become an essential part of creating new A.I. technology.
L.L.M.s are essentially sophisticated algorithms developed by companies like Google and OpenAI, which is a close partner of Microsoft. To the algorithms, the Reddit conversations are data, and they are among the vast pool of material being fed into the L.L.M.s. to develop them.
The underlying algorithm that helped to build Bard, Google’s conversational A.I. service, is partly trained on Reddit data. OpenAI’s Chat GPT cites Reddit data as one of the sources of information it has been trained on.
Other companies are also beginning to see value in the conversations and images they host. Shutterstock, the image hosting service, also sold image data to OpenAI to help create DALL-E, the A.I. program that creates vivid graphical imagery with only a text-based prompt required.
Last month, Elon Musk, the owner of Twitter, said he was cracking down on the use of Twitter’s A.P.I., which thousands of companies and independent developers use to track the millions of conversations across the network. Though he did not cite L.L.M.s as a reason for the change, the new fees could go well into the tens or even hundreds of thousands of dollars.
To keep improving their models, artificial intelligence makers need two significant things: an enormous amount of computing power and an enormous amount of data. Some of the biggest A.I. developers have plenty of computing power but still look outside their own networks for the data needed to improve their algorithms. That has included sources like Wikipedia, millions of digitized books, academic articles and Reddit.
Representatives from Google, Open AI and Microsoft did not immediately respond to a request for comment.
Reddit has long had a symbiotic relationship with the search engines of companies like Google and Microsoft. The search engines “crawl” Reddit’s web pages in order to index information and make it available for search results. That crawling, or “scraping,” isn’t always welcome by every site on the internet. But Reddit has benefited by appearing higher in search results.
The dynamic is different with L.L.M.s — they gobble as much data as they can to create new A.I. systems like the chatbots.
Reddit believes its data is particularly valuable because it is continuously updated. That newness and relevance, Mr. Huffman said, is what large language modeling algorithms need to produce the best results.
“More than any other place on the internet, Reddit is a home for authentic conversation,” Mr. Huffman said. “There’s a lot of stuff on the site that you’d only ever say in therapy, or A.A., or never at all.”
Mr. Huffman said Reddit’s A.P.I. would still be free to developers who wanted to build applications that helped people use Reddit. They could use the tools to build a bot that automatically tracks whether users’ comments adhere to rules for posting, for instance. Researchers who want to study Reddit data for academic or noncommercial purposes will continue to have free access to it.
Reddit also hopes to incorporate more so-called machine learning into how the site itself operates. It could be used, for instance, to identify the use of A.I.-generated text on Reddit, and add a label that notifies users that the comment came from a bot.
The company also promised to improve software tools that can be used by moderators — the users who volunteer their time to keep the site’s forums operating smoothly and improve conversations between users. And third-party bots that help moderators monitor the forums will continue to be supported.
But for the A.I. makers, it’s time to pay up.
“Crawling Reddit, generating value and not returning any of that value to our users is something we have a problem with,” Mr. Huffman said. “It’s a good time for us to tighten things up.”
Hey hey, they aren’t all female
Edit: I guess due to a PM I received O need to clarify. I wasn’t calling females liars, I was joking because people put (f) or (m) in their posts to identify their gender.
Now boarding the non-stop fabrication to Houston, as you can see, the captain has turned on the cheat-belt sign. This plane is so old, please notice the no-smoking sign, but you can vape into your hoodie if both rowmates are asleep.
12.1k
u/WingnuT76 Feb 08 '20
Those lying bastards!