Just today, the AI question draft editor / formatting assistant experiment went live. From fiddling with it, it seems clear that it's just ChatGPT / a related OpenAI LLM with a prompt designed for revising question drafts. The ChatGPT API isn't free, which led me to wonder about the scale of costs that such a feature might have.
- Between [2022-12-01 to 2023-06-11) (a period of 192 days), at the time of me running my SEDE query for data from just Stack Overflow and not the wider network there were 1,369,837,836 characters in all non-deleted questions. There were 693,335 non-deleted questions, and 968,660 questions (including deleted ones)).
- OpenAI approximates there to be one token per four characters (source)
- At the time of this writing, the pricing of ChatGPT with 4K of context is $0.0015 / 1K tokens of input, and $0.002 / 1K tokens of output (source)
- The prompt for the formatting assistant is ~2,639 characters long.
- (very conservative) there are no blocked or cancelled questions.
- The AI suggestions feature is used on average once per new question (weak assumption, but I need something to work with- I think it would be more)
- The cheapest API pricing for ChatGPT is used, and pricing doesn't change in the future
- Even if question rate is in decline, the average length of questions will remain largely unaffected
- The number of characters of output of the AI question draft edit output is roughly the same as the number of characters of input
Rough calculations based on the (conservative) assumptions:
- The average number of characters per (non-deleted) question:
1,369,837,836 ÷ 693,335
- Tokens of input per question (including the prompt):
(1,976 + 2,639) ÷ 4
- Tokens of output per question:
1,976 ÷ 4
- Cost per question:
(1,154 × $0.0015 ÷ 1,000) + (494 × $0.002 ÷ 1,000)
- If we take a naïve average and ignore that question influx rate is declining, and just use the average in the [2022-12-01 to 2023-06-11) period, then with (968,660 ÷ 192 =) 5,045 questions per day, that would cost approximately $13.72 per day.
- If we take what's reported on https://stackexchange.com/sites, which is 3,423 questions per day today, that's approximately $9.31 per day.
(I've made enough ghastly calculation errors that I don't trust myself to have done the above right. Please don't take what I've written as if I did it right).
Questions to Stack Exchange Inc.:
- Is this estimate accurate? Is this sustainable if you start using the API more?
- What will you do if the pricing of the ChatGPT API increases?
- How do these AI features bring revenue / how do you think they will bring revenue?
- Is this really worth the opportunity costs of whatever else you could be doing with that money? Or whatever you decided to cut off to do this instead?
- Why aren't other options being taken? For example, instead of AI formatting, you could probably get a lot of mileage out of simple heuristics for common basic mistakes like capitalization of the word "I", wrapping code in code fences, and by using open source code formatters.
- Or have I gotten this all wrong and you have some sort of "I'll show you mine if you show me yours" agreement with OpenAI to get free or discounted API access if you give them our content for free or discounted for their model training purposes? If so, will you still deliver on your promise to make sure we stop "being denied [our] rightful attribution"? (or was saying that just a way to pacify us?) And if so, what was the point of turning off the data dumps?