-428

Update (July 25th)

We have moved the GenAI site into its Public Beta. Privileges have been set to normal public beta thresholds as well.


Update (July 13th)

As promised, you can follow along for summarized updates on the work the stakeholder group is undertaking.

You can find that here: GenAI (previously Prompt Design) site stakeholder group updates.


Update (June 22)

If you want to join the community stakeholder group, you can learn more about its purpose and how to join here.


Update (June 21)

There is some confusion about what Prompt Design means for Area 51 and community-led site creation.

To clear the air, no plans in the immediate or far future to dismantle Area 51 or put an end to community-led site creation exist.

Prompt Design's process of joining the Stack Exchange Network is only experimental for informing what a potential improved process could look like. Stack Exchange has no plans to take the lead in creating additional sites. Any strategy to build upon, revamp, upgrade, or improve Area 51 or the site incubation process will be revisited with significant planning and community feedback when and if that day comes.


Today, I have some news to share. It's about one of our newest projects coming to the network - the Prompt Design Stack Exchange site: a site for Q&A on writing or engineering effective prompts while utilizing different generative AI (GenAI) tools.

What is prompt design?

Prompt design or prompt engineering, (depending on what you prefer), is the skill, act, or process of constructing prompts to utilize generative AI tools that can create some kind of output for you, whether that be writing, coding, art, music, or animation and many more things.

Stack Overflow and many other sites have been the trustworthy anchors of the Stack Exchange network for a long time. They’ve been the source of many friendships, a marriage or two, and a host of knowledge sharing so large and beneficial to society that it's likely impossible to measure.

Prompt Design is our vision of a place where we can grow a community that is centered around knowledge sharing for writing prompts for GenAI tools. Given the many different GenAI tools out there and the ones yet to be released, we anticipate a significant impact on technologists that will change how they work and create new segments of technologists. We want to ensure we have a site to serve that group, just like Stack Overflow and other sites that support emerging technologies and topics.

Starting communities is hard; we would love to have your support here to make doing this less difficult. Technology is always changing the world and GenAI is no exception.

We see this community as one that will help technologists, just as Stack Overflow did when it launched. We would love to have you as a part of this journey now. But if you're not interested in this particular community or don’t have the time, that's okay! However, if you find yourself interested in these tools and love using LLMs like ChatGPT for writing witty puns or creating artwork to finally demonstrate how cool your tabletop RPG characters look using Midjourney, then we would love to have you along for the journey.

On a more direct note, we strongly oppose the influencers capitalizing on new GenAI users without properly teaching these learners how to use these tools effectively. We see building this site as pulling from some of our core foundational roots: knowledge sharing should be easily accessible, especially in a space where expertise has yet to be established.

So, what do we want?

We are looking for your feedback. How should we best navigate this new territory? One thing to note is that we will be trialing a new community creation process — more on that in a minute — and communicating on MSE to sort out some of the particulars with the community as publicly and transparently as possible.

Let's talk Meat & Potatoes.

Please cut to the chase here

Alright, I know I have been a bit vague, and I recognize I am only selling a high-level concept without much substance to engage with, so let's share some details.

Why are you using this new community creation process?

Let's just get it out there: Area 51 holds some sentimental value for many of us… but how good is it for doing what it needs to these days? It's kinda hidden, with an aging design. That is not even acknowledging that the code base for Area 51 has not been touched in so long; we are pretty sure we could build it from scratch faster than refactoring what is there, and we just might do that!

Simply put, we will try a new process that is relatively reflective of what we think a new process for site building on the network should look like: public, discoverable, and out in the open. We have more specific ideas for this new process but must prove this could work first. More specifically, if Prompt Design works out, it gives us the momentum and a road map we need to start considering what an Area 51 replacement can look like and some direction on where we are heading with its future. We have a vision of a much more visible and discoverable process for site incubation than the current state of Area 51. One that is integrated into MSE, but not intrusive. I don’t want to get too into the weeds right now, but if we prove we can launch a site there, it gives a good runway to take a serious swing at a Next Generation Area 51.

Rather than have the community pick a random site topic to try and build this way, we decided to take ownership of this process and drive the bus on this one. If we succeed, great! If not, we will have fresh material to discuss on meta; it's a public service either way.

So what's different here than the regular site building process?

We will create a Community Stakeholders working group on an instance of Stack Overflow for Teams or perhaps a public chat room with read access. This group will be its most interested supporters from the community - both moderators and non-moderators (these are loosely equivalent to the pledged committers from Area 51) and will work behind the scenes with the community managers to discuss and flesh out things like site norms or potential policies, before being brought to the site's meta to agree. More details on this will be shared with the invitation to join the group.

We are exploring offering free site sponsorships and ad space on this site to new GenAI companies as part of potential co-marketing agreements to get some exposure to their users in hopes that they join this new community. It won’t be like this forever, and it's not a preview of anything to come elsewhere on the network. Just an exception we are making here for this site, in this case, to throw a broader net to grow its audience. For those unfamiliar with site sponsorship on other sites, we usually give quite a bit of heads-up, but in this case, the heads-up will maybe be days rather than weeks to months, and new partners in this respect could change on a week-to-week basis.

Why do you think this is a good use of your time when so much of the network has other needs?

We believe this is a pivotal opportunity for the SE community. We have a rare chance here to do some quick moving that builds on what we know these sites and this community of people do well. We’re excited to try out some new methodologies - some of them will work, some will fail, and some will be somewhere in the middle. We’ll learn from them and iterate. But these opportunities around technology that could potentially be generation-defining are rare, and we want to take advantage of this one.

What does a community stakeholders group do?

Think of this as a group of people interested in being the group's founding members. People who would like to take a hands-on approach in working with a community manager to see the site successful. They would lend a hand in developing the site's initial policies before it launches so that when the new site is launched we will already have some starting points on discussion items that would typically be addressed during the private beta phase. After launch, the community is able to modify or replace any of their ideas as it sees fit.

This group will continue to exist post-launch as invested users who continue to discuss things like engagement, emerging issues, or anything else that we would typically use a working group for.

Nine companions. So be it. You shall be the fellowship of the ring.

What do you mean by emerging technologists?

The times are changing. Depending on what you might believe, AI tools will become a part of many professions for assistance in doing work in some form or fashion. It is easy to speculate on the impact this is having on coding and writing, the impact will probably hit other areas too, and those impacted people will have questions that need answers. And Stack Exchange is the place many people come to to get answers. These tools will likely expand quite far into many different professions; we think betting on prompt writing skills and understanding makes sense. Naturally, we want to serve individuals who find themselves using these tools as well, and ensure they have a place to find trusted information, just like our current community does with the existing network.

Did some sponsoring company hire you to create this site for them? Is money changing hands here?

No. This is happening because we believe the timing and the subject matter are right, with no outside party advocating for its creation.

Is this site intended to be a place to get help writing prompts for their GenAI tools?

Simply, no. We are creating this as a site to record and refine best practices for writing effective prompts and not some form of "Please write a prompt that does x."

How do you plan to handle inappropriate content that might come up on Prompt Design?

Prompt Design has no unique privileges here to allow content our Code of Conduct would otherwise prohibit. Of course, the site's community will still be welcome to classify specific questions or categories as inappropriate and moderate those as it sees fit if it would like to expand beyond the CoC.

What if the site fails?

While it's not something we want, it's an outcome we can accept should it come to that. The subject matter is relevant enough to give it a try in this experiment. If we are wrong, we still see this as a valuable learning experience on how we could build upon our existing site improvement process for a potential successor to Area 51.

The tentative timeline

This is the planned schedule of different communications we plan to share with the community. Titles are in flux, and dates might change, but this should give you an idea of the project’s timeline to launch.

Communications Date
Invitation to Community Stakeholders Group (MSE) Before the end of June
Launch Prompt Design + Announcement July 26th, 2023
Site Standards: What should they be? Tags, Question Overlap, etc. (Prompt Design Meta) July 26th, 2023

So, what should I do now?

In short:

  • We will need help identifying obstacles, whether listed here or not. I am looking to be made aware of pitfalls. Be direct, please.
  • Any thoughts on a community stakeholders group? What do you think they should/should not do? How long should it exist, etc?
  • We expect this site to be different due to the subjective nature of using GenAI tools; we are particularly interested in outside-the-box thinking in ways to to run a site on the network differently. Think of this as a testing ground where we can experiment with site settings and other things.
  • Suggesting a prompt design site for Area 51 is not necessary, while this experiment is ongoing. Other site proposals are totally welcome there, of course. We’ve got this one, so do us a solid and don’t suggest any site proposals around prompt design or suggest to others that they should.
  • To be super clear, there are no plans to end community-driven site incubation; this experiment is simply one that could help inform improvements on the process. We value having a space and process for community-led sites to join the network and have no plans to change that other than to empower that process at some point in the future.
91
  • 238
    So you're just circumventing Area 51 and the normal procedure for establishing new sites, for, a bogus thing? Commented Jun 20, 2023 at 19:27
  • 211
    Honest Trailers version of the Announcement: "We want to make this happen and we're worried the community won't back it so we're doing it anyways and blaming not involving the community process on Area 51... That site that we own, and we would prefer to not maintain or put money into." (also, enough people have left Stack Exchange I'm pretty sure the folks there haven't done the Chesterton's Fence analysis on Area 51's existence). Commented Jun 20, 2023 at 19:29
  • 124
    I'm just shocked. How is it possible to fall for a hype this bad? The future is not genAI. Commented Jun 20, 2023 at 19:31
  • 162
    Sigh. I'm disappointed you're once again ignoring all internal feedback. Starting to wonder if negotiating is even worth it at this point, when y'all seem to try hard to come up with some new way to undermine trust weekly
    – Zoe
    Commented Jun 20, 2023 at 19:35
  • 60
    @Zoeisonstrike The powers that be want what all powers that be want: you to spend your time volunteering without them having to spend time or energy on you. Commented Jun 20, 2023 at 19:37
  • 71
    I just find the entire idea of "prompt writing" for genAI ridiculous. I'm not sure if I should laugh or be angry over this nonsense. Commented Jun 20, 2023 at 19:38
  • 157
    Why do you bother to ask for feedback if it's obvious you look only looking for "WOW, it's great!!" and ignore anything negative? You launch things out of the blue, without consulting the users, or even fully aware that most active users would be against it. You do not care what we think. (You as the company, nothing is personal as you're just a messenger in this case.) Commented Jun 20, 2023 at 19:43
  • 77
    It's interesting to see that the company came up with a tentative timeline regarding "Prompt Design", but no timeline regarding solving the community's concerns. Commented Jun 20, 2023 at 19:50
  • 76
    Writing prompts is not engineering. Training models, embeddings and LoRAs might merit a site, but I doubt anyone is going to want to share their expertise any more than y'all wanted to share your data dumps.
    – ColleenV
    Commented Jun 20, 2023 at 19:51
  • 123
    To whoever made the decision to proceed with this idea now, did they even consider the optics of proceeding with it during a moderator's strike regarding the related concept of appropriately dealing with AI generated posts? This is not even considering just announcing you're going to be doing it, and asking for how to best proceed, as opposed to saying you are considering an idea and asking us what we thought about it, including not only how to possibly proceed but also whether or not it's even reasonable to try doing this. Commented Jun 20, 2023 at 19:55
  • 169
    Come on, guys, can't you finish one scandal before starting the next? This is unbelievably tone deaf. Commented Jun 20, 2023 at 20:41
  • 199
    "we believe the timing and the subject matter are right" You're announcing a major GenAI initiative in the middle of a strike on GenAI policy. The timing is obnoxiously insensitive.
    – PM 2Ring
    Commented Jun 20, 2023 at 20:56
  • 50
    @SpencerG Why do you think a SE site is appropriate for something that gives kinda random output? Prompt writing for genAI is an attempt at forcing a rigid entity into something it is not; it doesn't really work, and there's no definite answers to it, which is what SE is built upon. Your AI formatting assistant is a great example of how it just doesn't work. This is a lot of investment into something broken. Commented Jun 20, 2023 at 21:15
  • 102
    I just have 1 thing to say: Can you PLEASE stop pushing any AI stuff until you have a reasonable way to handle AI generated garbage/spam? Commented Jun 21, 2023 at 0:16
  • 121
    This might be the most poorly considered new feature that you guys have released since last Friday. You seriously need to take a step back and consider why every single feature you've released/proposed during the past month has been very hard to tell apart from an April Fools joke. Because my immediate reaction every time is: "no, they can't be serious, this must be a joke". In addition it is extremely tone deaf to release these poorly considered things in the middle of a moderator strike which just entered its 3rd week.
    – Lundin
    Commented Jun 21, 2023 at 11:34

30 Answers 30

302

This won't fly. For the past two decades, one of the most important skills in many technology-oriented and scientific jobs has been the capability of formulating good prompts for search engines. I often joke to my colleagues that my only skills are knowing how to Google and knowing exactly what code to copy/paste and what not. The skill of formulating a good prompt for Google is so dependent on the particular problem I'm facing that it's rarely useful for someone else. This is diametrically opposed to what Stack Exchange's goal is; it's to provide content which is useful for as many use cases as possible. There hasn't been any need to create a prompt design site for search engines; why would LLMs be different?

Yes, I can think of a few dozen of questions which are useful for many people using AI in an attempt to solve their problems. I'm not an SME, so let's up that to a couple of hundred. That is not enough to constitute a healthy Stack Exchange site. I predict that many questions will be too narrow to be of any use to anyone but the author.

The valuable questions could even be at home at an existing Stack Exchange site (disclaimer: I'm not a regular, I could be wrong). An example question: How to get GPT-3 to translate a specific word in a sentence?. Why does that matter? Well, there's a frequently used close reason on Area 51, when a proposal duplicates an existing site. To mirror my opening paragraph, one can certainly find a couple of prompt design questions for search engines on Web Applications Stack Exchange.

15
  • 32
    When you put it that way, I think it would be a tutoring site, not a Q&A site. I took a class from a librarian on searching and I learned that I knew nothing about it and would need years of practice to be any good at it. :)
    – ColleenV
    Commented Jun 20, 2023 at 20:04
  • 33
    You're right - I'm old enough to have used physical libraries, but back then I had enough time not to worry about efficiency :)
    – Glorfindel Mod
    Commented Jun 20, 2023 at 20:05
  • 44
    Searching, whether it's Google, a corpus, or an AI model is somewhere between a skill and art. It would be like creating a site to tell aspiring fine artists "best practices" for creating art someone will buy. There would be one canonical post that starts with "It depends..."
    – ColleenV
    Commented Jun 20, 2023 at 20:14
  • 8
    @ColleenV: Meh, Writing seems to be reasonably functional... why not fine art?
    – Kevin
    Commented Jun 20, 2023 at 20:28
  • 36
    @Kevin If you showed up on Writing and said. "I want a book about x, y, and z. How do I write one?" you'd be laughed off the site. Commented Jun 20, 2023 at 20:50
  • 12
    @AR-solidarityforModstrike: That's not the comment I replied to. If you ask about a specific best practice on Writing, they will answer it. There are many questions of that nature on the site already.
    – Kevin
    Commented Jun 20, 2023 at 21:02
  • 5
    I used to deliver classes in using Google boolean operators and tips/tricks that could be used to improve searches.
    – Richard
    Commented Jun 20, 2023 at 22:27
  • 5
    Other points aside, "wont be useful to others" I disagree with. Language is even broader, with more niche edge cases, yet there's language SEs. As someone who frequents a network that routinely handles highly specific - and highly unspecific - engineering questions, I think a prompts SE is doable. Commented Jun 21, 2023 at 1:08
  • 33
    Plus, don't these AI models and the prompt preprocessing layers on top of them evolve so fast that it would quickly make the answers obsolete? Commented Jun 21, 2023 at 10:14
  • 5
    @ColleenV You just described about 30% of Music Practice & Theory.SE Commented Jun 21, 2023 at 14:41
  • 9
    @ToddWilcox But not 100% of it. That's the point. There's always a gray area of what is too subjective to fit the model and having some leeway is actually constructive. Exactly how does someone support their answer for prompt writing? What books have been written on it? Who are recognized experts in the field? What syntax has been standardized across products? Midjourney changes their model enough that a prompt that worked a month ago is not going to give the same results if you use it again today.
    – ColleenV
    Commented Jun 21, 2023 at 15:24
  • 2
    @ColleenV I agree with you that the prompt design site is a terrible idea. I just wanted to say the analogy to sites related to the arts is not so apt, IMHO. People dealing with people on something that is somewhat subjective is much more reasonable than thinking that AI prompting can somehow be codified objectively (at least for now). Commented Jun 21, 2023 at 15:28
  • 11
    @ToddWilcox I have been trying to stop making analogies, because they always end up distracting from the point rather than adding to it, but for some reason I just can't break the habit. It's like trying to quit... oops, I almost did it again :)
    – ColleenV
    Commented Jun 21, 2023 at 15:32
  • 11
    Not wanting to play devil advocate here, but multiple sites that are just guides on how to write prompts already exist (see for example the likes of PromptHero or in a way even CivitAI), just not in Stack format. So an userbase for thes DO exist. Now... this does not imply we need that on the network or more importantly that such idea would work with Stack Exchange q&a format
    – SPArcheon
    Commented Jun 21, 2023 at 17:19
  • 1
    on the point of there being no site for search engines, the Web Applications site does take at least some kinds of questions about using search engines and writing search queries. I asked a site-recommendation question for one such question before here, and then posted the question here.
    – starball
    Commented Jul 5, 2023 at 9:45
266

Let's just get it out there: Area 51 holds some sentimental value for many of us… but how good is it for doing what it needs to these days?

It's kinda hidden,

You have the power. Make it less hidden.

with an aging design.

You have the power. Breathe new life into its design.

That is not even acknowledging that the code base for Area 51 has not been touched in so long;

You have the power. Touch it.

we are pretty sure we could build it from scratch faster than refactoring what is there, and we just might do that!

Cool I guess?

Simply put, we will try a new process that is relatively reflective of what we think a new process for site building on the network should look like: public, discoverable, and out in the open.

We will create a Community Stakeholders working group on an instance of Stack Overflow for Teams.

And this instance will be public, right?

This group will be its most interested supporters from the community - both moderators and non-moderators (these are loosely equivalent to the pledged committers from Area 51) and will work behind the scenes with the community managers to discuss and flesh out things like site norms or potential policies, before being brought to the site's meta to agree.

Um... what happened to "public, discoverable, and out in the open."?

And what's up with "to agree"? You mean a small group will decide, and then everyone else shalt agree?

They would lend a hand in developing the site's initial policies before it launches so that when the new site is launched we will already have some starting points on discussion items that would typically be addressed during the private beta phase.

There will still be a private beta phase right?... right?

Any thoughts on a community stakeholders group? What do you think they should/should not do? How long should it exist, etc?

The users are the stakeholders, and their "group" is their meta site and MSE.

We expect this site to be different due to the subjective nature of using GenAI tools

What's so subjective about it? And what's so different about doing more subjective topics here? We have sites like Parenting. Why do we need something different? Did you have something specific in mind? I'm always on my toes now for you to drop something crazy out of the blue.


Do you already have a list of example questions for the site?

I'm curious about how well this will work out if you try to approach it very generally. I presume that how different LLMs react to the same prompt will be different, and that designing a prompt for one will not be the same as designing one for another. Even the "same" LLM will have different versions over time, and models with different parameters like ChatGPT's "context". See also SPArcheon's answer post.

Note that the AI site already has https://ai.stackexchange.com/questions/tagged/prompt-design. I'm not sure if prompt design is on-topic there, but it's worth asking (which I have now done at Are questions about prompt-design on-topic here?).

And then add to this the really really suspicious timing- the site launches on July 26- just in time for July 27-28's https://web.archive.org/web/20230622095515/https://lk.linkedin.com/posts/stack-overflow_developer-activity-7054478169222852608-Bs6l

See you this summer in Berlin! Our CEO Prashanth Chandrasekar will take the stage at WeAreDevelopers in July to share some exciting updates and thoughts on what the future may hold for the #developer community. [...] From our stage, Prashanth will showcase what the next 15 years might look like for this critical and influential community, especially in light of emerging tech like generative AI.

11
  • 107
    "You have the power. Make it less hidden." - but why do that when they can just screw over the community again and again so they don't have to touch legacy code?
    – Zoe
    Commented Jun 20, 2023 at 19:46
  • 31
    The "LLM version" problem was one of my first thoughts as well; there's little public indication that an LLM has "changed" or "upgraded" from one day to the next, where it may generate completely new output to a given input... How can we maintain a "library of detailed answers to every question about [prompt engineering]" if people can't be sure the answers are accurate tomorrow, much less next year, or in 5 years?
    – zcoop98
    Commented Jun 20, 2023 at 21:30
  • 25
    @zcoop98 - Heck, you can feed the same prompt to the same version of the same LLM 20 times in a row and get radically different responses. How would you even determine whether a given answer was "correct" or not?
    – bta
    Commented Jun 20, 2023 at 23:00
  • 2
    @bta And what's really funny is when some of those responses are unexpectedly "I'm sorry, I'm not comfortable continuing this conversation" click
    – Michael
    Commented Jun 21, 2023 at 2:03
  • 3
    @zcoop98 The LLM doesn't even need a "change" or "upgrade" to produce different results. Every question/conversation can affect the output to future questions/conversations. The whole "neural network", "machine learning", etc. thing. In a sense the data is part of the code and, just like a human, experience is data. Which is interesting philosophically and in other ways, but doesn't make "how to phrase the question right" a very useful site. Commented Jun 21, 2023 at 21:23
  • 5
    Why is the timing suspicious? I mean, it is obviously timed to be ready in time for that conference, but that's just normal procedure: you always try to present some new shiny at a conference. They have demonstrated beyond any doubt that they do not care about the community in any way, of course they aren't doing this for "the community", that's patently absurd, but I don't see anything suspicious in the fact that they want to present it at a conference.
    – terdon
    Commented Jun 22, 2023 at 10:44
  • 9
    @terdononstrike This is the only way they can stand up a community in such a short timeframe with no messy disagreements on what it should be or even whether it should exist. I'm waiting with bated breath to see what the requirements are going to be for their hand-picked volunteer founders. This sort of corporate BS where you push the "difficult" people out because they interfere with the company's kumbayah makes me sad.
    – ColleenV
    Commented Jun 22, 2023 at 13:00
  • 4
    @ColleenV yes, absolutely, same here. I just don't see anything to be suspicious about since suspicion requires that something not be glaringly obvious. That a company wants to roll out a new shiny in time for a conference is neither surprising nor in any way hidden, so I just don't see what there is to be suspicious about on that front. Given the sheer volume of lies and broken promises, I don't believe anything the company says anyway, but here it is simply obvious and clear that they want to push this out in time for the conference.
    – terdon
    Commented Jun 22, 2023 at 15:39
  • 7
    @terdononstrike Yep, it's exactly what I expected Commitment to fostering a positive, inclusive, and productive environment No negative Nellies need apply. They want all of the actual dirty work of building a real community to happen after they've put up the facade for the CEO to wave vaguely at. Someone is going to have to figure out how to keep the vampires and grifters at bay, but not until after the traffic makes the site look successful.
    – ColleenV
    Commented Jun 22, 2023 at 16:48
  • 5
    @manassehkatz-Moving2Codidact LLMs as currently implemented don't learn from their conversations, although the maintainers may choose to explicitly use some of them Commented Jun 22, 2023 at 23:28
  • to be fair, the AI site is pretty much dead, your question is the only activity it had in months
    – Devin
    Commented Jul 7, 2023 at 16:39
226

This announcement couldn't be more tone-deaf if you tried.

Let's start with the basics here.

A new site is being formed.

Typically, a new site is formed up through Area 51. The catch here is that instead of creating the site, the community should already generally exist and should choose to have the Stack Exchange network as its home. The company doesn't spend time nurturing sites or building them up from scratch.

This is what has been canonical guidance ever since new sites were being spun up. But you're hand-waving this away with...

Let's just get it out there: Area 51 holds some sentimental value for many of us… but how good is it for doing what it needs to these days? It's kinda hidden, with an aging design. That is not even acknowledging that the code base for Area 51 has not been touched in so long; we are pretty sure we could build it from scratch faster than refactoring what is there, and we just might do that!

Holy crap, guys, we've been trying to tell you that Area 51 needed some TLC ever since the one person who was still around to do things with it left the company!!!

The only other time that this happened in recent memory that a site just...skipped the process was with Patents.SE, and that was at the behest of the United States Patent and Trademark Office.

So why should this ignore the process?

The community of practice

This is your justification, it seems...

Prompt Design is our vision of a place where we can grow a community that is centered around knowledge sharing for writing prompts for GenAI tools. Given the many different GenAI tools out there and the ones yet to be released, we anticipate a significant impact on technologists that will change how they work and create new segments of technologists. We want to ensure we have a site to serve that group, just like Stack Overflow and other sites that support emerging technologies and topics.

Starting communities is hard; we would love to have your support here to make doing this less difficult. Technology is always changing the world and GenAI is no exception.

We see this community as one that will help technologists, just as Stack Overflow did when it launched. We would love to have you as a part of this journey now. But if you're not interested in this particular community or don’t have the time, that's okay! However, if you find yourself interested in these tools and love using LLMs like ChatGPT for writing witty puns or creating artwork to finally demonstrate how cool your tabletop RPG characters look using Midjourney, then we would love to have you along for the journey.

So you know that people are already doing this, and you want them to have a place to do it. Uh, okay, sure. Got it.

(Personal note: this crosses a personal red line.)

...[I]f you find yourself interested in these tools and love using LLMs like ChatGPT for writing witty puns or creating artwork to finally demonstrate how cool your tabletop RPG characters look using Midjourney, then we would love to have you along for the journey.

So many artists have been losing work for the advent of AI Art. So many livelihoods have been upended for it. So many thorny issues have come about from people trying to get involved in this. By providing a space for this, you are tacitly encouraging users to pirate art. When this goes live, I walk out and I will look to take my data with me. I adamantly refuse to contribute to any company or their goals, directly or indirectly, that even wants to tolerate this kind of behavior.

So calming back down, you have this supposed community of practice just "out there" and ready to engage. But, that's where we talk about...

This is just so tone-deaf to everything the community is saying about AI.

You couldn't be more disconnected from reality if you tried here. So many of the network sites are rejecting AI. There's a whole strike on because of how hard y'all are pushing AI.

I realize that y'all gotta do what you've gotta do to keep lights on, but this is so removed from where the community's energy is at, that it's offensive that you're rolling this out now.


I'll offer this much of an olive branch, though. If you were to divorce the creation of this specific community from the process of creating new communities, I think more constructive conversations could be had about this. But for now, you get the two-fer - the bitter pill of y'all just casting Area 51 aside and throwing out a decade and change worth of practice and knowledge about community creation, all to chase that sweet sweet AI cred.

I hope it's worth it for you.

16
  • 17
    Minor correction - he was realigned. I mean fired Commented Jun 20, 2023 at 23:31
  • 5
    TLC = tender loving care Commented Jun 21, 2023 at 2:12
  • 10
    A lot of stacks are about religion or other topics that people consider to have... shall we say, deep ethical significance. I'm not sure it's productive to expect the company to take a moral stance on AI when opinions about it remain so conflicted. I for one did not sign the strike because of any antipathy towards AI in general or even ChatGPT in particular, and I wish you would not claim to speak for me. There is nothing in the strike letter which explicitly denigrates AI as a whole. Rather, it focuses on the use of AI to write answers.
    – Kevin
    Commented Jun 21, 2023 at 8:13
  • 10
    @Kevin: Don't conflate the two. My personal red line is the promotion or embracing of AI art in any capacity on the network, and there are indeed some sites who have some policies against that in some light where applicable. The issue with AI being pushed so hard is a part of the letter in that not being able to moderate it presents an issue, but it's related in that it's yet another AI bauble that the company seems to be interested in pushing rather than engaging with users to talk through what it is they really need. I'm not speaking for you, so don't think that I am.
    – Makoto
    Commented Jun 21, 2023 at 15:49
  • 3
    I specifically disagree with this statement: "So many of the network sites are rejecting AI. There's a whole strike on because of how hard y'all are pushing AI." - That is not why I signed the strike letter.
    – Kevin
    Commented Jun 21, 2023 at 15:51
  • 6
    Why an individual has or hasn't signed the letter doesn't change the purpose of it. We all as individuals have varied opinions of AI in general and the use of AI on the platform, but the strike is occurring because of the way the company is pushing AI, as in, going it alone rather than making the community part of the solution. All of this could have been avoided with honest communication over the past 8 months.
    – Kevin B
    Commented Jun 21, 2023 at 15:54
  • 3
    @Kevin: That's fine, but there's all of these network policies that are...already out there and that's what I'm referencing here. That's a lot of network sites rejecting AI content, if you asked me.
    – Makoto
    Commented Jun 21, 2023 at 15:54
  • 10
    @Makoto: That's not "rejecting AI." That's "rejecting AI as a means of answering questions." They are not the same thing, and to conflate them is intellectually dishonest.
    – Kevin
    Commented Jun 21, 2023 at 16:55
  • 10
    No. It is, at best, a simplification, at least when talking about rejecting AI in the context of a Q&A site, where what we do is ask and answer questions. Commented Jun 22, 2023 at 5:53
  • 4
    Hypothesis: Stack Exchange needs a place where Stack Exchange staff can learn which prompts to use to make ChatGPT replace the striking moderators. Commented Jun 22, 2023 at 20:09
  • @CodyGray-onstrike: You should probably tell that to Artificial Intelligence. I guess they'd be surprised that over a thousand users are rejecting their presence on the platform...
    – Kevin
    Commented Jun 22, 2023 at 20:58
  • 5
    @Kevin they're well aware
    – Kevin B
    Commented Jun 22, 2023 at 21:06
  • 1
    @KevinB: You're missing my point. This proposal is to add a new site about AI to the roster, and Makoto is opposing it on the grounds that the strikers are "rejecting AI." My point is that we already have a site about AI on the roster, many of whose members have probably signed the strike letter, and it would likely come as a surprise to those people to have their own signatures used as evidence that their site is not wanted.
    – Kevin
    Commented Jun 22, 2023 at 22:16
  • 18
    I don't think the users and moderators of the AI.SE site are nearly as reading-challenged as you are making them out to be, @Kevin. Commented Jun 22, 2023 at 23:56
  • This is why you read all the answers before making your own. This says my thoughts almost exactly in a better way.
    – Chipster
    Commented Jun 28, 2023 at 13:50
136

Is this site intended to be a place to get help writing prompts for their GenAI tools?

Simply, no. We are creating this as a site to record and refine best practices for writing effective prompts and not some form of "Please write a prompt that does x."

How will this site avoid becoming what you stated it isn't, when we couldn't avoid that on SO with tags like regex and sql? What would make this case different? Will new tools be introduced to help facilitate this that can also assist with the mess on existing sites with that problem? Do you expect it to just remain small enough for a group of mods/active curators to keep it in check?

5
  • 12
    The answer here probably doesn't even matter. Based on their average follow-through rate on other stuff, it might work at first, but it'll be dropped whenever they find a new shiny toy to play with instead, and/or when the site becomes public
    – Zoe
    Commented Jun 20, 2023 at 19:41
  • 40
    And this is arguably a topic that will attract this kind of question even more than your examples. Getting a grip on the quality was my biggest concern when this was announced. Commented Jun 20, 2023 at 19:42
  • 5
    @Zoeisonstrike - More's the point, setting an exclusionary scope before you even see what's being posted is a terrible idea. Maybe the community that uses the site actually want those sorts of questions. Most sites have banned 'identify-this' type questions, for example, but there are some sites that positively revel in them
    – Richard
    Commented Jun 20, 2023 at 22:30
  • 2
    realistically I doubt this will actually be a problem on the new site, new sites rarely gain the kind of mass usage that would make dealing with this problem in whatever way the community there wants to difficult.
    – Kevin B
    Commented Jun 21, 2023 at 5:27
  • 1
    If the site was dedicated to help with the prompts, it could be successful. But "best practices" would lead to a frustration for fighting, what belongs there. Commented Jun 21, 2023 at 11:50
117

Right now, we literally have a moderator and curator strike involving SE's policies on GenAI.

The most successful example of prompt engineering we've had so far involved breaking the new formatting assistant.

The company has broken the social contract with the moderation team, cut back on resources by firing employees, put a significant amount of those remaining resources into a GenAI moonshot and is asking for help?

Area 51's outdated, old, and dilapidated because the company didn't invest in it. It wanted to, then got distracted by 'shinier' things. It got rid of the folks who knew it best, didn't consider past initiatives that worked, and now it's decided somehow that a human-driven site needs GenAI shoehorned into it.

Under very different circumstances, I think the idea of standing up a site based on current needs would be a great idea, and prompt engineering could be a great Q&A fit.

Right now though, GenAI's kinda leeching on resources and attention, and doesn't quite have the best reputation with the most enthusiastic parts of the community.

And I really don't want the failure of a new site process (well the... 3-4 attempts at one) to be because the whole thing was built on a foundation of cotton candy on a flood plain.

Let's actually talk meat and potatoes here.

  • You currently have a good chunk of the moderator community, the folks who collectively have years of experience starting and running a community here at odds with the company.

  • You're trying to start a site on the topic many community members are at odds over with the company,

  • which means you're likely going to find it hard to fill the minimum moderator quota, which is a key indicator of site health. Unless you're going to have CMs moderating it full time.

  • There's other gaps in the SE family of sites to fill, and there's no real attempt at looking at those, so we're not going to know if it's your process, your skillset, your ties with the community, or that it's a GenAI related site at fault.

  • This is... possibly a set up for failure.

It might be good to shore up the foundation before asking us to help raise your barn.

5
  • To be fair, they would not directly compete with SO userbase and/or moderation. I doubt that on average an user that uses Midjourney or AI Dungeon would be interested in moderating or posting questions on SO. That said, they are obviously redirecting company resources on this topic and on that I agree.
    – SPArcheon
    Commented Jun 21, 2023 at 11:19
  • 15
    Maybe the point is to start a new set of sites and bring up a new bunch of veteran users whose view of what SE should be is more in line with what corporate wants. Seems like they're working on ways to phase y'all out without pissing you off enough that you quit en-masse before you're replaced. When someone says they value you, but they constantly ignore your feedback, leave you out of the loop and exclude you from new projects, the obvious conclusion is that they do not in fact value you.
    – ColleenV
    Commented Jun 21, 2023 at 14:13
  • 5
    Oh that'll take far more work and investment than I've seen the company make. Commented Jun 21, 2023 at 14:28
  • 2
    I appreciate the feedback. Just wanted to touch on Area 51 and this site. Whether this experiment or site works, it won't significantly impact a potential Area 51 upgrade someday. Other than us acknowledging that this might be an unwise model to pursue. I enjoy the cotton candy on the floodplain analogy, fun visualization.
    – SpencerG StaffMod
    Commented Jun 21, 2023 at 18:01
  • 2
    Its less the model than, the entire 3 ring circus around it really. I actually think a lot of the plan around curated, as needed site creation with full CM support has value. Just that its put in a difficult situation by everything else Commented Jun 23, 2023 at 10:58
106

When will the company start listening to the elected moderators, curators, and other users who make these sites viable?

Unlike some of the other recent changes that have been poorly received - like unilaterally changing policies regarding algorithmically-generated content in a manner inconsistent with the wishes of the vast majority of the community or ending the data dumps without any prior discussion or announcement or the content formatter - you at least gave moderators a chance to weigh in on the feedback. I see some of the concerns addressed here, but not one of the most common themes.

One of the common concerns raised is that this is not the right time for this site, especially if it's going to circumvent the standard site creation process.

Personally, I have no problem with the company having a hand in choosing what sites to launch. Launching a new site isn't free. Not only does it require infrastructure, but it will require support from the community managers from the beginning and, if it's successful, work from a design team to give it a nice theme. There have also been a lot of points raised with the site launching process and how Area 51 works over the years - I wouldn't be surprised if the need for something new is overwhelming.

Right now, a not-insignificant number of moderators and curators from across the network are on strike because of a number of reasons. A lack of communication and a singular company focus on AI instead of working with us to solve real problems that our communities face are among those reasons. However, instead of heeding our pleas to work with us to fix these problems, the company is finding yet another way to plow forward with their focus on AI.

There have been some moderator resignations specifically over these recent issues. If you continue to demonstrate ignoring the people who are the champions for this network, there will be more. Moderators and users will continue to walk away. The tools that are so effective at keeping spam and other abusive content off the network will remain turned off. Stop now and listen.

If you really want to help teach people, I'd recommend focusing on what you have. One problem that keeps being mentioned by the company is engagement. Are there ways that you can take advantage of existing communities, the experts that exist there now, and draw in new learners and experts? Think about how you can take advantage of communities like Computer Science and Theoretical Computer Science, Artificial Intelligence, Statistics, Data Science, Software Engineering, and Philosophy (which has specific tags related to ethics, artificial intelligence, and the philosophy of technology). Maybe paying some more attention to Community Building would also be beneficial. And if you, as a company, don't want to take advantage of these communities of experts in a public way, at least engage with the moderators, some of whom have expertise in some of these areas and are willing to help.

It is very quickly becoming too late. Stop doing things until you take the time to listen to what the existing communities are saying.

2
  • 16
    "Stop now and listen" you just triggered by war flashbacks of "STOP! LOOK! LISTEN!" (from the review queue audits), but maybe that's what upper management should hear / realize right now.
    – starball
    Commented Jun 20, 2023 at 20:43
  • 6
    @starball At least I didn't come across as Navi. Commented Jun 20, 2023 at 20:45
106

If I'm being honest with myself, I'm posting this an answer because it can be summarily deleted with no trace if I post it as a comment.

You all aren't going to listen to this advice, I know that. It's really here so I can feel good having said something instead of watching this slow motion train wreck happen on its own.

Directly:

You're doing this because your CEO or some other person in the C-Suite wants you to pursue AI, and one of you (hopefully) went to them and said, "Yea, we'd love to but the Area 51 process means it'll likely fail and the community is assuredly not behind us on this." to which one of those C-Suite folks said, "I don't care, make it happen".

So you're making it happen.

I realize you can't tell us this because it would make your decision making process look bad. After all, you've ostensibly followed a tried-and-true process since Area 51 came about in 2010? 2011?, and you realize it's hard to start a Stack Exchange site, and your C-Suite sponsor wants it to be easier.

After all, why can't they just create a Stack Exchange site and poof it works? Why can't they do that? Shouldn't they be able to do that?

Well, you've traveling pretty quickly towards the 'find out' stage of the experiment, and one thing you'll notice is that it's not the CEO that looks bad when you 'find out', it's you. Your CEO doesn't spend time here having to interact with this community, you do. And so you tried your best to make the announcement sound like a grand experiment that you want us to come along for the ride, and no thank you, we do not want to do that.

We're not new at this, and we're not telling you "this is a bad idea" because we hate AI (though I am not a fan of AI, to be transparent).

Trying to start a community is hard work, and you need community buy-in, and you aren't going to get it by fiat, no matter how much you try.

I would ask you to reconsider, but you're not the one making this ill-fated decision, you're just the one expected to smile and see it through.

6
  • 18
    I think you've got this pretty much right. This is clearly directed from above. The timing is interesting too as it butts right up against the CEO giving a speech about AI stuff at a conference in Europe. - "ooh look AI is the way forward and we just created a site for prompt engineering" ... lk.linkedin.com/posts/…
    – user9517
    Commented Jun 22, 2023 at 10:58
  • 22
    SO has fully made the switch to Hallucination-Driven Development. The goal is to hand-pick a C suite that are capable of hallucinating just as hard as an AI.
    – Lundin
    Commented Jun 22, 2023 at 13:09
  • 4
    Also, it would be a waste cause the comments are a mess, and there's not exactly much desire to clean them up. Answers work better :) Commented Jun 22, 2023 at 13:39
  • 3
    @Lundin This is the goal of all capitalism at the moment. The market is completely detached from reality. The rules are made up and the dollars don't matter as long as you say the word "AI". Nvidia's value was rapidly approaching Amazon's last I checked - the market thinks AI is as valuable as basically every other physical good and computing service put together. Commented Jun 22, 2023 at 20:13
  • 1
    @user253751 - Something that only makes sense if investors think that true artificial general intelligence is around the corner, since that is about the only thing that could be more valuable than everything that is currently produced. And I am not so confident in my ability to see the future to say that it is not. What I can say, though, is that AGI has unknown and likely substantial dangers that could render it the furthest thing from a money-maker, from AGIs being declared legally emancipated from their creators to apocalyptic scenarios, which brings us back to detachment from reality.
    – Adamant
    Commented Jun 25, 2023 at 8:43
  • @Adamant Investors DO think that, because the investment world is currently in some kind of runaway reality detachment, "greater fool" or bubble phase, where the fundamentals don't matter, only the hype. Rising interest rates were supposed to end this phase by now. For some reason they still haven't. It may take longer. The effect of rising interest rates so far has been for investors to try stupider and stupider failing ideas that they think will preserve their investment but won't. We don't know when Wile E Coyote will finally look down. Commented Jun 26, 2023 at 13:17
93

First things first

I would like to request that gif memes be avoided/used less in such posts, at least in product announcement ones like this. It comes off a bit as try-hard behavior and is kind of tone-deaf/off-putting, given everything that's going on. This is kind of reminiscent of the "oopsie woopsie" problem from a recent era.

That being said

There was a chance for me to think this was an OK idea... if it had gone in a different direction. If you had talked about how the Area 51 process is pretty opaque and hasn't really been improved at all in ~10 years, and how it would be faster to build something from scratch than to have everyone at SO learn how to use it, from the perspective of trying out a new system to serve the same purpose as Area 51 in order to pay down technical debt... if you had owned what you are doing, maybe I wouldn't have hated it.

But you're not really doing that; you're working really hard to create a one-off, closed-door group, insulated from all the benefits a broader community brings you (so you want more of an self-reinforcing echo chamber than a merit-based debate, it looks like), just so you can have a site where you can charge money to AI-adjacent companies in order to teach users how to write good questions for Stack Exchange sites (or whatever else).

The elephant in the room

Please tell your CEO that Stack Exchange is not an Ed Tech company and he's gonna have a bad time trying to make it into one. You're doing a hell of a job scaring away all the value the company built up in the form of the community of experts here if you keep trying to move the "expert" labels from the community to the company.

It's been a long time since anything from the company came out that sounded like "this product/feature is built around the community and its experts will provide the value"; lately they're all "this product/feature is built around people the company has on payroll as experts and is set up so we can just charge money from other companies, instead." If you wanna have a company that does that, great! Really don't think anyone here appreciates trying to grind Stack Overflow/Stack Exchange's reputation into the ground turning this company into that kind of one.

Last, but not least

What does a community stakeholders group do?

Think of this as a group of people interested in being the group's founding members. People who would like to take a hands-on approach in working with a community manager to see the site successful. They would lend a hand in developing the site's initial policies before it launches so that when the new site is launched we will already have some starting points on discussion items that would typically be addressed during the private beta phase. After launch, the community is able to modify or replace any of their ideas as it sees fit.

This group will continue to exist post-launch as invested users who continue to discuss things like engagement, emerging issues, or anything else that we would typically use a working group for.

Did you just... explain to the MSE community what a site community is? And how to ensure a site reaches critical mass so that survives? I know you probably didn't choose what was included in most of the announcement here, but... come on.

10
  • 11
    SE has always been about a community of experts answering questions. SEI freely admits in the OP that there are no experts in the made-up field of "prompt engineering." So what community can there be? Commented Jun 20, 2023 at 20:46
  • 1
    I don't want to get into the weeds of a site incubation process cause this is purely exploratory. But, part of our thinking about a stakeholder group is that a smaller focused group could spin a community up faster. Should that community not work out, it would be closed faster too.
    – SpencerG StaffMod
    Commented Jun 20, 2023 at 21:17
  • 3
    "you're working really hard to create a one-off, closed-door group, insulated from all the benefits a broader community brings you" - I think the problem with this take is that Area 51 already required new sites to bring a community to create a site around rather than create one here. I actually pondered about this exact thing 2 years ago; since Area 51 doesn't have the tooling to support communities in any meaningful ways, a Team might be pretty beneficial. It might be closed off, but at least there's a chance for community to actually form.
    – zcoop98
    Commented Jun 20, 2023 at 21:43
  • 15
    @SpencerG 'But, part of our thinking about a stakeholder group is that a smaller focused group could spin a community up faster. Should that community not work out, it would be closed faster too.' - Area 51 worked well because (generally) enthusiasts within a topic would come and propose sites, and because they're enthusiasts they have a passion to drive and push for the site to succeed as opposed to being a closed group of individuals who work on getting a site off the ground without necessarily having an interest within the site itself.
    – Script47
    Commented Jun 20, 2023 at 21:59
  • @zcoop98 Area 51 is open to anyone to view, browse, search, and participate. Anyone can create a topic idea for a site and anyone else can review, critique, or participate in that topic to help it become a Beta site. This doesn't seem like a new way to incubate sites, Spencer's post went to great length to emphasize this is a one-off. As I said above, if it were a new way to incubate sites, I'd likely be for it, though I think any such Teams should be publicly viewable, at least (probably searchable as well, and just invite/request-only for membership).
    – TylerH
    Commented Jun 21, 2023 at 13:21
  • 5
    @zcoop98 I am speculating, but I think in a perfect world area 51 might exist as a separate section of MSE. Not part of the main Q&A feed, but a separate tab you could visit to discover new incubating sites more easily. Using Teams in this instance is just us creating a separate place for the stakeholders to work together. We do plan to publish updates and decisions that are made there, and once we have a chance to digest all this feedback we may just opt for a public read-only chat room.
    – SpencerG StaffMod
    Commented Jun 21, 2023 at 18:24
  • 5
    Only quibble - I don't think the CEO thinks of SE as an ed tech company - I think he thinks of it as his main financial asset, which he can leverage for his shareholders. His complete disdain for curation makes me think he's interested in the volume of that asset rather than the quality or the community that produces it or the impact on the consumers of it.
    – vbnet3d
    Commented Jun 22, 2023 at 20:54
  • 2
    @vbnet3d The CEO has called Stack Overflow an ed tech company in some interviews/articles, the parent company, Prosus, specializes in ed tech companies/acquisitions, and the hiring of Jody Bailey as CTO last year as an ed tech specialist kind of cements that they believe it to be an ed tech company, or at least an opportunity to make it into one.
    – TylerH
    Commented Jun 23, 2023 at 5:14
  • @TylerH Ah, I missed that. Good point
    – vbnet3d
    Commented Jun 23, 2023 at 17:14
  • 2
    @TylerH Those of us that work in Tech know that there is no such things as "a one off" any more than there is "a temporary fix". If this trick is seen to work, then it will be repeated, you can count on that. Of course, if it doesn't work, then it will be repeated too...
    – MikeB
    Commented Jul 4, 2023 at 8:45
76

Y'all (SE, not the community) need to take a step back and actually think these AI products through

tl;dr: all timing issues, questionable implementation and community neglect problems aside, this is fundamentally a bad product. And if SE wants to go all in on AI, at least do it right instead of being lazy about it.

I get it, AI hype is a thing, and shareholders/market pressures exist and y'all have to make good on that $1.8 billion dollar investment somehow. But please, give us AI products that are actually innovative rather than just quickly thrown together applications that cheaply wrap around existing products.

The Question Formatter Experiment

Recently there was that question editor experiment that used what seems to be some flavour of ChatGPT to give edit suggestions to draft questions. In theory, there are a lot of ways this can be helpful, such as providing live suggestions in a style similar to Grammarly, pointing out sections that may not be received well in a question (for example, including an image of code instead of the actual code) or using the question text to boost the search for similar questions.

In practice, it was just a single prompt that (quite literally) wrapped around the question, fed it to an LLM and extracted the result. There was no real effort by the AI to guide users to make a better question except for "here's a different version of your question with the differences highlighted". If I wanted that, I'd just go to chat.openai.com and ask for grammar checks myself.

Further, the fact it was just a black box application meant it was open to jailbreaks, total question fabrication and even chess games, as is the case of prompt-in/response-out tools.

All this potential for an innovative product and you gave us basically unfettered access to some LLM.

This Proposed Prompt Design Site

This new Prompt Design Stack Exchange site (which I'll call PDSE for brevity) in theory has some semblance of usefulness. Sure it's playing into a market of hype, but a place for people to ask for help with a certain prompt is actually somewhat a good idea. Instead, we have a site "to record and refine best practices for writing effective prompts and not some form of "Please write a prompt that does x.".

The major problem with this, as other answers have pointed out, is that there are only so many questions you can ask about "best practices for prompt design" before you have what would be better as a series of blog posts or a motivational seminar by "experts". Hell, that might even be a better business idea for y'all - a paid subscription for prompt design tips.


Edit: Oops, GitHub already beat SE to the punch of a blog post that gives tips on "best practices for communicating your desired results to [an LLM]". And SE thinks we need a whole site for this? (Now sure, the post is specifically about Copilot and code generation prompts, but the tips can generalise to all text-based LLMs).

This got me thinking: There are already plenty of definitions of/tutorials for general prompt design, as is the same for people spruiking best practices posts. Hence the site would basically devolve into "what's the best guidance for [insert specific LLM/generative AI here]". But such guidance would be better suited as in-house documentation for the specific genAI tool (be it official documentation, a blog post, discussions in a Discord/Reddit community or something else), rather than a Q&A post on a completely disconnected Stack Exchange site. So really, PDSE becomes a site for hosting and generating documentation (that only a niche section of those using genAI will even see) for AI companies (for free) while SE (and us) get nothing out of it.


All timing and community perception issues aside (which have been raised in other answers to a much greater detail than I would be capable of), the site is fundamentally limited by the niche question pool available. It would be different if there was actually a science to prompt design, or if there was some way for end-users to fine-tune AIs; there would be more questions than "how do I make my prompt x do thing y" to ask if that was the case.

But that's not the case. The case is that prompt design is a) subjective, b) hyped-up marketing terms sold by influencers and "get rich quick" business "gurus" [1] (actually, quite literally sold - the fact there's a market for prompts means the site has the chance to be flooded with low quality "help me write a prompt" posts so they too can cash in on the ai hype) and c) prone to change based on how companies modify their LLMs - they can easily end up changing how their models respond to already existing advice, consequently invalidating answers on the PDSE site.

In short, the entire PDSE site would probably be better off as a tips question on the already existing AI Stack Exchange site. It does not need to be a whole site, and in its current proposed state, would just be a quick and dirty attempt at cashing in on a trend.

So what's the solution

SE recently made provisions for further research and innovation in relation to AI (much to the dismay of the community as these provisions involved staff layoffs). How about actually using that money to look into products that aren't just "Hey look we did a thing. Sure it's something everyone has already seen and not very creative or innovative at all, but we're doing AI things". SE might actually make something that claws back the traffic they've lost to services like ChatGPT and Copilot and make a better user experience for everyone that way. Because right now, all that's happening is that the CEO has a talking point for shareholders buying into the ai hype train the company seems to be riding - "look at what we're doing - we're working on a product, and after testing, it isn't fully ready but we're heading in the right direction with it, so please keep investing".

[1] A quick DuckDuckGo search reveals countless results of people blogging about how you can make money from prompt design. These posts would no doubt include (dubious) "best practice" tips, just like the scope of PDSE would. I know I've said this several times in different ways already but prompt design just isn't something you make into a whole dedicated Q&A site. The main requirement of the site creation process on Area51 is that there exists an "enthusiastic, committed group of expert users who check in regularly, asking and answering questions". However, the "field" of prompt design is full of people claiming to be experts, when in reality, they're just making things up as they go. Prompt design is purely marketing speak that people use to make themselves look intellectual in the world of technology, when in reality, it's equivalent to saying you're a "high level cryptocurrency investor with diverse portfolios" - 21st nonsense.

5
  • 2
    This is some good feedback, and I appreciate the time it took to write it all out. In terms of scope, we are seeing this applying to any GenAI tool. GenAI tools already cover a lot of ground—music, art, code(sorta), video, animation, etc. That also isn't touching on the diverse application of some of these tools, like AutoGPT or plugins for specific applications like Sheets/Excel. We could be totally wrong here or fail to attract the right people who want to use the site, we still see this as an educational process.
    – SpencerG StaffMod
    Commented Jun 21, 2023 at 18:14
  • 2
    I was running out of characters so to continue, we think there are enough applications/use cases out there that there is room for a diverse set of questions to exist. Though, you do flag some interesting points that we are already thinking about concerning ensuring it's not a promotional space and not a place for other sites to dump their documentation.
    – SpencerG StaffMod
    Commented Jun 21, 2023 at 18:17
  • 17
    I was running out of characters Ah yes, welcome to one of the minor, but could be easily fixed annoyances of the SE system. Why not allow high-rep users, staff, mods have a longer comment size? I understand limiting it initially as you don't want to new users to write answers in a comment. Oops, they can't - because new users can't comment. Another one of my pet peeves. Oh well. AI is more important than usability. Commented Jun 21, 2023 at 21:58
  • 5
    @manassehkatz-Moving2Codidact: I don't think increasing comment limits for staff/mods is really the best solution there... Fundamentally, the issue there is moreso that Meta sites use the same Q&A structure as main sites, but that structure isn't really designed for back-and-forth discussions (this is something I've been lamenting since before I even became a CM)... But in any case, that problem's not so easily solved.
    – V2Blast
    Commented Jun 22, 2023 at 17:19
  • 1
    @V2Blast The perfect is the enemy of the good. Increasing the comment limit for staff or on meta sites would be much simpler than changing to a full discussion system. Commented Jun 22, 2023 at 23:17
68

This idea appears to be premised on the Field of Dreams model: "if you build it, they will come".

But communities don't simply spring into existence just because there is a place for them to meet. There is no "prompt design" community with a comparable level of expertise to any other Stack Exchange community, because until six months ago "prompt design" was a very niche interest:

Google search volume over time

The red, yellow and blue lines show the rate of Google searches worldwide for "prompt design", "prompt tuning" and "prompt engineering" respectively. The vertical axis is meaningless, but you can add other terms like "compiler design" or "type theory" for comparison.


The Area 51 process exists to make sure that

  1. The proposed site will fit the Stack Exchange Q&A format,
  2. That a sufficient number of people are interested in contributing to the proposed site to sustain a community there,
  3. That the community has sufficient expertise to answer questions. (To graduate from public beta, the site must have enough users with certain levels of reputation, and questions must receive enough answers.)

It sure seems like a "prompt design" Stack Exchange site would fail on at least point #3, and it doesn't look like much thought has been given to #1 or #2 either. The argument for bypassing Area 51 is weak (yes, it's old and shoddy, but that is because you haven't fixed it), but also your argument should apply at least as well to other proposed sites.

Will other current site proposals be able to go through the new process instead of Area 51? Because if so, there should be an announcement about the new policy, and information about how other sites (e.g. PLDI SE) can escape the old process and get the same beneficial treatment. And if not, you must realise how bad it looks to give this site a special exemption from the rules that apply to other communities.

Well, at the end of the day, fine. Build a stadium for a relatively new, niche game that has no professional players, if you want to; just don't be under any illusions about it. My guess is that the CEO just wants to be able to tell shareholders or prospective buyers that they're investing in something exciting; so maybe it doesn't really matter if there's no players, no games and no ticket sales.


Update:

To clear the air, no plans in the immediate or far future to dismantle Area 51 or put an end to community-led site creation exist.

...what are you on about? We aren't "sentimental" about Area 51 or worried that Area 51 is going away. I haven't seen a single response along the lines of "please don't get rid of Area 51!". The problems everyone is raising are

  • That Area 51 exists to ensure that new sites will be viable, whereas this new ad hoc process apparently exists only to circumvent those checks, not to replace them; and
  • That you are giving this new site a special exemption from the rules, favourable treatment which isn't available for other new sites.

The initial announcement was tonedeaf, but now you are misrepresenting our own feedback to us.

16
  • 12
    It's hard to overestimate how important criteria #3 is. If you don't have enough experts, questions go unanswered or get low-quality, dubious answers. That makes the whole site look shady and untrustworthy, and nobody will want to go there. The only thing worse than not having a community on <topic> is saying you have one but it's absolute garbage. That undermines the credibility of the entire network.
    – bta
    Commented Jun 20, 2023 at 22:43
  • 20
    Well, just use ChatGPT to answer all the questions, and Bingo!, you have a 'community' that is answering all the questions...
    – Jon Custer
    Commented Jun 21, 2023 at 1:39
  • 10
    Note that your statement "prompt design" has only been a thing for at most six months is not really true. Prompt-based language models have been around for several years before the web interface to ChatGPT was launched. Within the academic community it is also known as "prompt engineering" or "prompt tuning", and there is a large body of scientific papers about this topic from the last five years or so.
    – Marijn
    Commented Jun 21, 2023 at 6:08
  • 4
    Of course being an academic research field doesn't imply that a topic is a good fit for an SE site (nr. 1 and 2) but at least point 3 doesn't have to be a problem, there are enough people that qualify as an expert in the field.
    – Marijn
    Commented Jun 21, 2023 at 6:14
  • @Marijn I was coming here to say exactly this. Most likely the majority of users of the site would be novices, but there are experts in and academic research about the topic.
    – Esther
    Commented Jun 21, 2023 at 14:46
  • @Marijn I disagree that it's a large number of scientific papers. There's a GitHub repo here listing under a hundred papers, most of which are apparently about "prompt engineering". The repo claims to list "must-read" papers, but there's no quality criteria in their contributing guidelines, and it doesn't look like they've ever rejected a proposed addition. It seems there are no conferences or journals on the subject of "prompt engineering", and most of the listed papers are only available as arXiv preprints.
    – kaya3
    Commented Jun 21, 2023 at 15:36
  • 1
    So I stand by my statement that "There is no "prompt design" community with a comparable level of expertise to any other Stack Exchange community". Even so, if the authors of these papers are the community of experts that Stack Exchange wants to invite to contribute to the new PD SE site, it's curious that they don't mention that in their announcement, and more curious that they wouldn't try to get these specific individuals on board with the idea before deciding to create the site which apparently can't succeed without the buy-in of a pretty small group of specific individuals.
    – kaya3
    Commented Jun 21, 2023 at 15:40
  • 1
    Also worth pointing out that even for SE sites on topics which do have sizeable and long-running academic research communities, like programming language design, the site is only viable because there is a lot of expertise outside of academic research, and because there were pre-existing communities of programming language designers interested in getting the site off the ground.
    – kaya3
    Commented Jun 21, 2023 at 15:59
  • 1
    @kaya3-supportthestrike a quick search on the repository of ACL (the Association of Computational Linguistics) returns over 2000 papers (aclanthology.org/search/?q=prompt+engineering, which is smart enough to also include results on prompt tuning and prompt design, which are all more or less the same thing and would be on topic on the new SE site). And that is only one academic publisher, there are several other very big publishers that have hundreds if not thousands of papers on this topic, like IEEE, ACM, Springer. There may not be journals or conferences exclusively on this topic,
    – Marijn
    Commented Jun 21, 2023 at 17:02
  • 1
    but many venues that are broader in scope also accept prompt design papers, which are properly peer reviewed (so not just on arXiv). I do agree with you that this does not mean that a new SE site would attract these people, but the people and the expertise are definitely there.
    – Marijn
    Commented Jun 21, 2023 at 17:04
  • @Marijn Well, I'll take your word for it. I still think it's fair to say that there is no community of expertise comparable to any other topic which there's a Stack Exchange site for.
    – kaya3
    Commented Jun 21, 2023 at 17:16
  • 2
    I like your criteria, and I think that #3 should be modified or extended. It's important to have enough knowledgeable users to write answers; it's also very important to have enough users who can differentiate good from good-looking-but-wrong answers at a glance, and vote accordingly. If you can't get a critical mass of expert voting, you get a site full of misinformation, which for me is even worse than a site with no answers.
    – rumtscho
    Commented Jun 21, 2023 at 22:21
  • Regarding the graph: public interest does not say much about the amount of experts or knowledge for a topic. This is similar to saying that before September 2019 there was no community with expertise on impeachment (trends.google.com/trends/…) or before 2020 there was no expertise on Coronaviruses (trends.google.com/trends/…), while in both cases a (professional) community already existed.
    – Marijn
    Commented Jun 22, 2023 at 9:26
  • @Marijn There is no Impeachment SE or Coronaviruses SE, so that comparison doesn't weigh in favour of creating a Prompt Design SE. If SE, Inc. tried to create them, they would suffer from the same problem: almost all interest in the site would be from non-experts who can't write good answers and can't judge the quality of answers. As I said, "There is no "prompt design" community with a comparable level of expertise to any other Stack Exchange community". I never said there is no expertise at all.
    – kaya3
    Commented Jun 22, 2023 at 14:35
  • 1
    Interestingly, the level of interest in "impeachment" has been consistently higher than "prompt engineering" on all dates since 2004 where Google's data begins, including before Donald Trump's presidency and after the public availability of ChatGPT. The same is true for "coronavirus".
    – kaya3
    Commented Jun 22, 2023 at 20:21
65

I'm glad you're trying something new – even if the timing could've been a bit better.

  • I, too, am annoyed about people who try to pass off "GenAI" as something it's not, for their own personal, reputational or financial gain; and it would be great to have a Stack Exchange site that effectively tackles this.
  • Likewise, the Area 51 process could be improved.

But I'm a bit confused about the specifics.


We will create a Community Stakeholders working group on an instance of Stack Overflow for Teams. This group will be its most interested supporters from the community - both moderators and non-moderators (these are loosely equivalent to the pledged committers from Area 51) and will work behind the scenes with the community managers to discuss and flesh out things like site norms or potential policies, before being brought to the site's meta to agree.

Why do you need to work behind the scenes? In the past, we worked with the community managers in front of the scenes, largely (but not exclusively) during the Private Beta period. Here's an example.

This does not seem like an improvement to the current Private Beta process. I do not think it should exist.


What if the site fails?

While it's not something we want, it's an outcome we can accept should it come to that.

What criteria would you use to assess "failure"? Area 51 has some pretty clear metrics, but this is one of the main places I think Area 51 can be improved upon.


We are creating this as a site to record and refine best practices for writing effective prompts and not some form of "Please write a prompt that does x."

These tools will likely expand quite far into many different professions; we think betting on prompt writing skills and understanding makes sense.

We expect this site to be different due to the subjective nature of using GenAI tools;

Why? "Prompt writing" is fundamentally a hack: even if the GenAI hype bubble takes a while to burst, we will develop better techniques for specifying input layer values, actual techniques for guiding their output, etc..

So-called "prompt engineering" will become less useful, as the transformer-based autocomplete technique becomes yet another tool in the toolbox. Either the site's scope will gradually drift to become a subset of Artificial Intelligence Stack Exchange's, or it will fall behind the state of the art.

In no conceivable world will we be using transformer-based autocomplete en masse, but still relying on human intuition and rules-of-thumb to "write prompts".

So what's the plan for when the hype runs out? Either the site will end up languishing, like Windows Phone Stack Exchange (RIP), or it'll be carried solely by hobbyists interested in obsolete and (outside very narrow areas) largely useless technology, like Retrocomputing Stack Exchange. This will happen by 2028 at the very latest. In either case, it might not make sense to continue to have a separate site. Will it be taken offline, merged into Artificial Intelligence Stack Exchange, left as its own thing, or something else?


This is happening because we believe the timing and the subject matter are right,

The tentative timeline

Oh good. It's tentative. Then might I suggest holding off on this – or meeting the strike demands sharpish? It's not a particularly bad idea, but it's going to end terribly if you try to push it through under current circumstances.

64

I'd like to see some example questions to get a clearer mental picture. I get the feeling different people are envisaging different things. When will that be possible? Do I need to wait until July 26th (one month and one week from now)?

What worries me: it seems the scope of PromptDesign.SE includes literally everything (helicopter design, USSR politics, the principles of dentistry, C++, Kim Kardashian, cheese making, reverse engineering apps, the Renaissance, Seinfeld, hieroglyphics), which has two drawbacks:

  1. Who?: Who are the experts? Who can moderate all this? Who can judge if an answer is correct?
  2. The scope of PromptDesign.SE overlaps with every existing Stack Exchange site.

For example, I use these newfangled AIs for learning Chinese (they're great for language learning). I could ask my Chinese-language-related prompt-design question at PromptDesign.SE or I could ask it at Chinese.SE. Are the users of PromptDesign.SE going to understand the needs of someone learning Chinese?

3
  • 58
    This is, of course, exactly what the definition phase of Area 51 would figure out for you.
    – pppery
    Commented Jun 21, 2023 at 2:46
  • 5
    This is a great question. Scope is a real question, especially in the context you have framed it, because there is overlap across many potential areas. Its something we wanted to draw the line with after discussing with the Stakeholder group. On one hand learning to leverage a GenAI tool for language learning might not make sense on Chinese.SE, but I think there is a line we can draw that separates leveraging a tool and trying to get the tool to answer a question that might be more appropriate for a specific site, and skipping the Prompt Design site.
    – SpencerG StaffMod
    Commented Jun 21, 2023 at 18:28
  • 3
    So to summarize, you don't know the scope of the site you're currently proposing? (keep in mind that this "you" refers to the company, which I hold contempt for, not the person who happens to be behind the keyboard)
    – Gloweye
    Commented Jun 23, 2023 at 11:12
51

We will create a Community Stakeholders working group on an instance of Stack Overflow for Teams. This group will be its most interested supporters from the community - both moderators and non-moderators (these are loosely equivalent to the pledged committers from Area 51) and will work behind the scenes with the community managers to discuss and flesh out things like site norms or potential policies, before being brought to the site's meta to agree. More details on this will be shared with the invitation to join the group.

[...]

Think of this as a group of people interested in being the group's founding members. People who would like to take a hands-on approach in working with a community manager to see the site successful. They would lend a hand in developing the site's initial policies before it launches so that when the new site is launched we will already have some starting points on discussion items that would typically be addressed during the private beta phase. After launch, the community is able to modify or replace any of their ideas as it sees fit.

Quite a few of us have now told you that, under the current conditions, we are not willing to continue providing you with volunteer labor. I imagine the CMs are already stretched thin as it is. Yet your plan seems to be predicated on an expansion of that volunteer labor. How do you expect that will work? What are you going to do if nobody puts their hand up? Will the CMs simply invent a bunch of norms and policies, throw them up on Meta, and hope for the best? I find it hard to believe that will work itself out.

Well, whatever you do, I have no intention of participating at least until the strike is resolved. I imagine plenty of other people feel the same.

1
  • 22
    "After launch, the community is able to modify or replace any of their ideas as it sees fit" It's hard to take the company stance on that as true when the whole reason for the Strike is that SE/SO is forcing company policy on sites that have already rejected them. Commented Jun 21, 2023 at 7:24
48

I would really like to be upset, but I can't.

Yes, the announcement is tone deaf in its timing.

But the rest? You want to build a new site, entirely on your own time and money. Can I be upset about that? Why would I?

I cannot see it succeeding though. It will get flooded by "what do I need to type into ChatGPT for it to do me homework in a way it won't be detected" questions. And people with shoddy answers to earn green internet points. And then maybe under a dozen lengthy and well-meaning self answers on best practices. Not enough to keep it going, but it will rot for some time.

So... let it burn? Your idea of burning money might be better received in the community, if you just burned actual money in your yard in a funny way and shared a TikTok video about it or whatever young people do these days, but in the end it's up to you. It does not concern me. You burn your money whatever way you see fit.

47

This ad hoc decisions will leave SE devoid of human contributors.

Casual users don't bother. Investors may love to hear AI buzzwords. But the main purpose of the site are people asking questions and getting answers from human experts.

These people are basically us, avid users of the community which generously contribute their knowledge to receive in exchange, basically peanuts.

These people are not happy with your arbitrary decisions on running random AI experiments, overriding community consensus on banning GenAI (which led to an active moderation strike) and now lastly starting a new community ignoring the established processes because you know very well the community would not agree that this new site is needed or relevant.

People are less and less inclined to contribute, mods are striking, users are very dissatisfied, the website is filling up with unfair garbage of users cheating the system because mods can barely take action, and the community has less of a voice than ever (not even during the Monica debacle we saw such an oblivious insult to the community voice).

If you do this just for the sake of AI, just build a new GenAI tool and leave SE websites as they were.

It took the SE community so many years to build a trustable knowledge base, and now the company simply wants to mix it with AI for corporate interests. This will harm community trust in SE websites in the long run.

Stop running AI experiments and building new random communities in a website that works well as it is. Start a new tool with the existing knowledge if you want, but make sure to clearly differentiate it from the existing sites.

Otherwise, you will soon find out, that all human contributors have fled and became replaced with nonsense AI generated bogus that does not help. And I'm sorry to say that for this purpose people would rather use ChatGPT than Stack communities.

Stack Exchange was about getting trustworthy answers, even so by getting some fair comments about the community being somewhat toxic to newbies, because standards are high, but now corporate wants to lower the standards if instead of human you are a random machine learning model?

Send this back to Area 51 ASAP.

Who is going to contribute to a random website with unclear purpose? Why follow a closed, behind the scenes process, casually for the one website no one would pledge for? What about the existing Area 51 projects? Are they somehow less worthy than this topic, not to be included in the same new process?

Stick to the community consensus. Every week you are getting us more and more dissatisfied with the direction company is moving, and eventually this will shift from a mod strike to a contributor strike.

4
  • 2
    FYI, there are plenty of contributors sticking with the mods
    – Seggan
    Commented Jun 21, 2023 at 14:55
  • I have proposed a separate discussion to explore that (failure to) "clearly differentiate it from the existing site" points in detail.
    – anx
    Commented Jun 22, 2023 at 12:30
  • You are wrong about human experts. The purpose of SE, like every other corporation in existence, is to make money by grifting. This was on hold for 15 years due to Zero-Interest-Rate Policy, but now it's back, and we are just getting accustomed to what this mission statement really means in a world where tech is much more advanced than it was 15 years ago. Commented Jun 22, 2023 at 20:14
  • 1
    I totally understand corporate greed. I am simply working on the basis of how the community was intended and why we are here. SE as a company will do whatever.
    – Marc S
    Commented Jun 22, 2023 at 20:20
44

Have you guys thought about creating AskGoogle.SE in addition to this site?

Google prompts and the ability to craft them properly is a significantly more applicable case, and has proven to never cease being relevant. 25 years into this craft, and new ways to engage with this craft keep showing up. Its user base is also already existing, the barrier to entry is low, and very few competitors (only one to speak of) are currently on the market.

A quick rebrand to add genAI to the questions by directly providing the answers, and you're good to go!


For real though, how you guys are convinced that a prompt design question isn't an opinion-based question yet doesn't fit on AI.SE is beyond me.

5
  • 5
    "how are you guys convinced that a prompt design isn't an opinion-based question yet doesn't fit on AI.SE is beyond me." Perhaps forcing off-topic questions onto a community is still something they're better than doing? Copium? Commented Jun 21, 2023 at 11:48
  • 15
    Surely the Google search site should be LetMeGoogleThatForYou.stackexchange.com
    – Mokubai
    Commented Jun 21, 2023 at 14:04
  • 3
    "doesn't fit on AI.SE is beyond me" - see ai.meta.stackexchange.com/q/2930/64296
    – starball
    Commented Jun 21, 2023 at 16:47
  • 4
    @starball My point was that anything that fits the "not opinion-based" and "not a homework question", which literally 99.9% of the volume on that new site will be, would fit on AI.SE. Commented Jun 21, 2023 at 16:54
  • The first sentence does not compute. Commented Jun 21, 2023 at 22:15
39

Just a side notice to anything the other users have said in their answers because I can't really understand what you are proposing. This doesn't imply I agree or disagree with the proposal.. I just wanted some insight on some details.

... This is really about creating a site that would (among other things) accept questions about Stable Diffusion generation and prompts????

I think that would be very hard to do. I often play with Stable Diffusion and every single model has a sightly different way to write a prompt. Some models may want you to specific thing like "best quality", some don't. Some can be used with words like "closeup" to produce a "portrait" while some will end up with a closeup of a body part instead. Some understand photography terms like "cowboy-shot" while other would try to give your main subject a cowboy hat.

And there is tons of conflicting models that can be used for anything. Just look at CivitAI. What should I use for a picture of Princess Zelda? Anythingv5? GrapeFruit? AbyssMaple to give her a Maple Story chibi look? Or maybe Counterfeit?

How do you plan to handle this? Every single question would have to define the specific model you are using, the version (PonyDiffusion v4 has a very different syntax from PonyDiffusion v1). And this is not even taking in considerations thing like multiple VAE files, LoRAs, ControlNet usage, etc.

Considering that most of the users don't even have direct access to local Diffusion on their machines, most won't have any way to put themselves in the exact condition the asker is. And even if they do, I don't think many would be that interested in downloading a 10 GB checkpoint + multiple LORA files just to answer one question.

I fear this proposal would be very hard to do even ignoring all the other objection that have been made.

PS: and just to add insult to injury, I just remembered that Stack Exchange Imgur strips the Exif metadata from pictures: something that is very good to do in most cases since you don't want to include the metadata in your photos or diffused pictures, but it would become a limitation in the rare case you wanted to keep the Diffusion setting information when posting an answer/question on this new site.

4
  • 12
    And what about the next generation of tools which are going to need completely different set of strategies? Honestly, I think it would be better to just create a "The Current Thing" site and make the scope "whatever technology has enough hype around it that the c-suite notices it". My husband is an expert in AI image generation. It's unbelievable how many people try to get him to work for free. He's happy to share his knowledge, but the field is full of people who think you can just type at an LLM and it magically does what you want if you just put enough constraints on it.
    – ColleenV
    Commented Jun 21, 2023 at 11:53
  • 5
    I'm 100% against this new site, but I will say that this isn't one of the objections. In fact it provides an infinite well of content in terms of new questions. It's much like saying, "A site for how to write code to solve a problem? That could never work; every single language and framework does it differently, and there are always new ones coming out!" Commented Jun 22, 2023 at 1:31
  • 4
    @LukeSawczak Imho that isn't the same situation. Languages? Those are well defined and it is easy to understand which question is which. Models? A complete mess. Multiple versions of the same model having different supported prompts (PonyDiffusionv4 is a clear example), scam names (you think that Anything v4 is related to Anything v3? Think again), people mixing models together (see the infinite amount of [somethingMix]), people posting the same model under a different name or with no name at all (what model is "161?" ), [cont...]
    – SPArcheon
    Commented Jun 22, 2023 at 7:45
  • 3
    [cont] ... updating the model without a versioning system so that the same name correspond to different files (do you want to try to understand what version the OP is using by asking them when they did download the model and where?), fake models that try to pass for an existing one etc etc etc. And that is not even considering all the other factors like the client (Automatic? Some ControlNet webUi? ), the VAE (even colors can differ based on that), LORA file (e.g. characters LORA) etc. I can build up a node.js environment similar to what an OP has in a reasonable time, can't do the same here.
    – SPArcheon
    Commented Jun 22, 2023 at 7:52
38

Will you actually be responding to feedback?

Why does it feel like this is a sales pitch as opposed to a genuine attempt to solicit feedback from the community?

It very over the top.

On a more direct note, we strongly oppose the influencers capitalizing on new GenAI users without properly teaching these learners how to use these tools effectively. We see building this site as pulling from some of our core foundational roots: knowledge sharing should be easily accessible, especially in a space where expertise has yet to be established.

So, wait, you just want to capitalize on this process but just earlier on than others?

We will create a Community Stakeholders working group on an instance of Stack Overflow for Teams or perhaps a public chat room with read acccess. This group will be its most interested supporters from the community - both moderators and non-moderators (these are loosely equivalent to the pledged committers from Area 51) and will work behind the scenes with the community managers to discuss and flesh out things like site norms or potential policies, before being brought to the site's meta to agree. More details on this will be shared with the invitation to join the group.

Sounds very similar to the Moderator Council which sort of died out.

How would this be different?

How will you choose the users?

Technically, I'm interested, but my sentiments towards this are currently constructively critical. Will I be discounted?

But these opportunities around technology that could potentially be generation-defining are rare, and we want to take advantage of this one.

What's generation-defining are rare about what you are proposing?

We value having a space and process for community-led sites to join the network and have no plans to change that other than to empower that process at some point in the future.

What does this actually mean?

You definition of empowerment could definitely be the complete opposite to what the community thinks.


Overall, this just seems like another step in the wrong direction for SO. It seems that someone (or, maybe a group of people) in the leadership have been swept away by the AI hype (or, maybe they've just seen the potential money to be made) and are now trying their hardest to shoehorn it into these sites / products / communities as quickly as they can, and the proof of this will be in how you react to the feedback that is / will be given.

29

There are a couple of things here that other people have said:

  1. Circumventing the Area 51 process is probably a clear sign upper management wanted to make this happen and is unwilling to follow the usual procedure.
  2. Revamping community creation is probably good. The site wants to grow. Putting effort into this crucial.
  3. It’s clear there is a disconnect between the semiformal rules that define the interaction between users on the traditional Q&A platform and the kind of experiences that the company wants to provide.

With those things in mind, I think it’s good for a company to innovate, add to their offerings and that trying to leverage the generative AI boom is probably a good idea for SO. Why not just do that at a safe distance from existing products?

When Documentation came out, I also thought it was a great idea. I'm not yet convinced that it was unsalvageable given enough effort and guidance. But what was done right about it was that it was significantly divorced from what was working with Q&A that the two could live and die separately.

If SO wants to make something new, that has a different purpose and different means of interaction than traditional Q&A, which focuses on prompt engineering, fine I say, do that. It might lead to something useful.

But don't tie it so tightly to the existing product that both die if the new thing doesn't work. Because the same juicy user growth numbers that drive graphs that business people fawn over will be the same ones who spam "Please write a prompt that does x." style questions.

27

Note: While I'm active on MSE and per-site Metas, I've only been on SE for a couple years, so I haven't (personally) experienced much of Area 51. That said, I have used it. Some quotes trimmed for brevity, and trimming indicated by "[...]".

[...] Area 51 holds some sentimental value for many of us… but how good is it for doing what it needs to these days? It's kinda hidden, with an aging design. That is not even acknowledging that the code base for Area 51 has not been touched in so long; we are pretty sure we could build it from scratch faster than refactoring what is there, and we just might do that!

We know Area 51 is hidden and has a number of bugs and design issues. I'd guess (but haven't checked) that a large quantity of users agree with that statement. But, to quote Joel Spolsky:

They did it by making the single worst strategic mistake that any software company can make:

They decided to rewrite the code from scratch.

There's a subtle reason that programmers always want to throw away the code and start over. The reason is that they think the old code is a mess. And here is the interesting observation: they are probably wrong. The reason that they think the old code is a mess is because of a cardinal, fundamental law of programming:

It's harder to read code than to write it.

Yes, Area 51 has many problems. Yes, it is certainly in need of some love by developers at SE. That... doesn't mean that it is worthless and that this is a good alternative. It isn't. I know how well it would be received if I posted on MSE stating: "Proposal for a community around [insert buzzword of the month]" - poorly. Unsurprisingly, unilaterally declaring that a site would be good without a community... doesn't work. There's a good reason we have graduation requirements from Area 51. Yes, exceptions are made. Those exceptions have reasons, though. And the reason is never "the site topic is a buzzword".

Simply put, we will try a new process that is [...] public, discoverable, and [...] open. We have more specific ideas for this new process but must prove this could work first. More specifically, if Prompt Design works out, it gives us the momentum and a road map we need to start considering what an Area 51 replacement can look like and some direction on where we are heading with its future. We have a vision of a much more visible and discoverable process for site incubation [...] One that is integrated into MSE, but not intrusive. I don't want to get too into the weeds right now, but if we prove we can launch a site there, it gives a good runway to take a serious swing at a Next Generation Area 51.

Presumably, your plan is more than make an MSE post declaring a site exists, and poof, done. That would be a disaster (generally). But, why is this representative of how a new system would work? Or, why is Prompt Design SE a prerequisite for a new process?

Starting communities is hard; we would love to have your support here to make doing this less difficult. [...]

Here's how to get support. Open MSE, and post the following, or a slight variation on it. Something that asks for input, and doesn't mention genAI. The community is tired of y'all forcing it upon us.

Discussion on a better version of Area 51

We know, Area 51 has issues. It's old, the design is outdated, login is broken, etc.. We'd like a new site that better suits your needs for site creation. While we have ideas internally, we'd like your input, as any site on SE requires a community behind it.

Please post suggestions for what can be improved regarding Area 51 as answers below, and SE staff will closely read and review each and every answer

(Sample post above written by me, not SE)

We see this community as one that will help technologists [...] However, if you find yourself interested in these tools and love using LLMs like ChatGPT for writing witty puns or creating artwork to finally demonstrate how cool your tabletop RPG characters look using Midjourney, then we would love to have you along for the journey.

Sharing creations isn't something that really works on the SE platform. There's one exception I know of (Stack Apps), but... that likely won't work out here. Sorry.

We are looking for your feedback. How should we best navigate this new territory? One thing to note is that we will be trialing a new community creation process — more on that in a minute — and communicating on MSE to sort out some of the particulars with the community as publicly and transparently as possible.

Which is it? Do you want feedback and you'll commit to listening to it? Or are you going to decide something and just communicate your choice? Those are... very different.

We will create a Community Stakeholders working group on an instance of Stack Overflow for Teams. This group will be its most interested supporters from the community - both moderators and non-moderators (these are loosely equivalent to the pledged committers from Area 51) and will work behind the scenes with the community managers to discuss and flesh out things like site norms or potential policies, before being brought to the site's meta to agree. More details on this will be shared with the invitation to join the group.

Hot take: If this works, I'd really like that. As someone that was in the group of people that helped form the Staging Ground, I actually think that this could work. But it will only work if people there are listened to. Bonus question: how will people get in/request access?

Not quoting verbatim, but you state money isn't changing hands. You also mention site sponsorships for the new site. Can you clarify that a bit?

One more comment on community stakeholders: While I'm not sure if the exact details are public (and so I'm being intentionally vague), I really liked the way SE asked for feedback within the Staging Ground, and especially how you asked and got feedback. If you'd like more specific feedback, ping me somewhere in chat or in Stack Overflow for Teams. More specifically, I appreciated how feedback was asked for and received in these posts*

*Link will only work for those with access to the Staging Ground Team and (some?) staff.

6
  • 6
    I really hated how said feedback wasn't accessible to those who aren't on the team, and how that feedback still isn't accessible to those not on the team. There's certainly value in some things happening "behind closed doors", but that shouldn't be the default just because it's convenient.
    – Kevin B
    Commented Jun 20, 2023 at 21:17
  • This sentence feels complicated: "But it will only work if people there are listened to". Please try a simpler version if possible. Commented Jun 20, 2023 at 21:25
  • 5
    Thanks for your thoughts, currently for the Stakeholder group our plan is to publish regular updates on a single meta post, with meeting notes. I can make sure to add a public chat with read access for everyone so they can still see those conversations. Teams just allow for a more discoverable discussion format, we originally thought we could maybe do that on MSE, but didn't want to create that much noise here. Though I am certainly open to feedback on how to make that bit more transparent.
    – SpencerG StaffMod
    Commented Jun 20, 2023 at 21:25
  • 1
    @KevinB I've elaborated on that statement some in chat as I went well over the comment length limit. It might not be what you agree with, but that might provide some more clarification on the rationale. I'm also totally open to discussing it more, but thought I should elaborate on why first. Not disagreeing that more could be public (and possibly should be), though
    – cocomac
    Commented Jun 20, 2023 at 21:44
  • "Sharing creations isn't something that really works on the SE platform." - agreed. meta.stackexchange.com/a/386688/997587
    – starball
    Commented Jun 20, 2023 at 21:55
  • 9
    Just a small nitpick on the Area 51 codebase bit– it's more than "a few bugs"; in my understanding, Area 51 runs on a decade-old (literally, from 2011) fork of the Stack Exchange QA engine... so we're not just talking about having a little trouble getting familiar with a codebase, we're talking about a site that's missing 12 years' worth of features, improvements, and QoL changes.
    – zcoop98
    Commented Jun 20, 2023 at 22:03
27

Redundant by now, I'm sure, but I really feel the need to gather these three reactions together side by side.

To clear the air, no plans in the immediate or far future to dismantle Area 51 or put an end to community-led site creation exist.

  1. This clarification is irrelevant. You have still decided that you get to make new SE sites that the community doesn't necessarily approve of, outside of the existing process. That is not acceptable.

  2. This new site is entirely superfluous. https://ai.stackexchange.com/ already exists. "Prompt engineering" is clearly a tiny subset of things that can be done with AI / problems to solve when using AI.

  3. This action is unbelievably tone-deaf. Makoto covered this in detail, but I can't omit this point in good conscience. Generative AI, as currently understood, is fundamentally incapable of replacing the human expertise of Stack Exchange websites no matter how "large" the "language model" gets, because fundamentally it does not do anything that resembles logic or reasoning - it is merely very good at playing a word-association game. (You don't have to take my word for that; try using your own resources that we built for you free of charge). Your continued insistence on pretending otherwise is a personal affront to millions of existing users. How do you not get this yet?

23

In light of the current company sentiments towards AI, as exhibited in the timing of the question above, and the complete disregard for advice provided by the community of moderators (and content curators) in regards to our handling of AI posts, I'd like to propose the following answer, written in T-SQL, because that's how we roll on Database Administrators.

SELECT * 
FROM dbo.[F$#!] AS f 
WHERE 
    f.Name = N'StackOverflow'
    AND f.CreatedDate = CONVERT(DATE, '2023-06-20')
    AND f.Type = 'GIVEN';
    
(0 rows affected)
2
  • 3
    That's very clever!
    – 41686d6564
    Commented Jun 21, 2023 at 18:14
  • 4
    @41686d6564standsw.Palestine as is your username Commented Jun 21, 2023 at 20:01
23

One thing I think is missing from the conversation is safety of "running" "code" from this new promptcraft site.

There's a type of SE site (stackoverflow/serverfault/askubuntu/poweruser) where answers are executable in some way, and frequently make their way into production environments. For that reason, we do our best to ensure answers aren't misleading ("to install minecraft please execute rm -rf ...") or outdated in dangerous ways.

By all accounts, the questions on Prompt Design will be making their way into production applications--just look at the current interest at SE (the company) itself over LLMs. Ultimately, of course, the responsibility falls to the user to ensure they understand the answer they're trusting, but the potential consequences of an incorrect/malicious answer can be a lot higher than, for instance, an incorrect answer on Chess SE.

At the same time, popular generative AI tools are being updated all the time, and by their nature it's impossible to say how a new version would change the various promptcraft tricks people have found. A Midjourney question created in January 2024 would likely be outdated by October 2024, if it ever produced the desired result in the first place.

How will this proposed new site ensure that its answers are accurate and remain accurate, especially given how inconsistent & domain sensitive promptcraft tends to be?

A lot of questions that are being asked in existing promptcraft communities are against TOS (SE's, the AI services', and 3rd parties)

A lot of promptcraft is focused on sandbox-escapes and censor-evasion. And I'm not against that, in principle. I think red-teaming these technologies is important if they're ever going to be in production applications handling my sensitive data (shudder). (And also I think it's funny when robots get pwn'd.) But I think maybe these types of discussions are not something we should encourage on the site. (Anybody who has spent any time on the Midjourney discord knows how many questionable synonyms people have for "please draw an attractive woman".)

(I don't think there's necessarily a clear line between censor-evasion and 'ethical' promptcraft, either. "I'm designing a RPG and I need the villain to say evil-villain things, but it keeps saying 'As a LLM it would be wrong of me...'" is a type of censor-evasion, but not a particularly objectionable.)

Finally, a lot of promptcraft applications are unethical--filling up online marketplaces with low-quality trash, automating scamming, cheating on essays and exams, etc. I personally believe that SE should not answer these.

19

My first impression was just being shocked at this announcement. I find this new site quite ridiculous, and I'm not happy that you spend resources at this while you fail to meet the demands of the community. It's not a useful distraction. Tampering with Area 51 during a time of strike from the core curators, moderators, and when there is little trust between the company and the community, is worrisome. I am left just asking: why even bother asking for feedback when you have been neglecting and ignoring existing feedback for so long, and not cared for our needs as a community? We are striking because of this AI nonsense of SE's, amongst other reasons.

1
  • 7
    I resisted signing the letter for few weeks and I guess I'm done now. I've signed the letter. Commented Jun 20, 2023 at 20:02
16

Answered here: Are you interested in Prompt Design? Join the Community Stakeholders group

We will create a Community Stakeholders working group...This group will be its most interested supporters from the community - both moderators and non-moderators (these are loosely equivalent to the pledged committers from Area 51) and will work behind the scenes with the community managers...More details on this will be shared with the invitation to join the group...This group will continue to exist post-launch as invested users who continue to discuss things like engagement, emerging issues, or anything else that we would typically use a working group for.

So this group is going to be invite-only or am I misunderstanding? What qualifies someone to get an invite to be a stakeholder? Should interested parties start working on the essay part of their application now or is it going to be more like Area 51 where people who are willing to commit to a certain level of activity and time all get to participate?

4
  • 3
    the essay part of their application now - asks ChatGPT to write an essay about prompt design... chat.openai.com/share/052540d1-9dc5-494c-b9a7-f8c0b4d699bc If ever there was an appropriate use for AI, that would be it - straight from the horse's mouth Commented Jun 21, 2023 at 19:22
  • 2
    @manassehkatz-Moving2Codidact Lol, well it could be a competition for who can design their prompts to get the best essay out of the LLM of their choice.
    – ColleenV
    Commented Jun 21, 2023 at 19:33
  • 6
    Most likely, tomorrow, June 22nd, we will be posting a separate post, linked to this one, that invites interested individuals to indicate that they would like to join it. It will have more details on what needs to be included to be considered.
    – SpencerG StaffMod
    Commented Jun 22, 2023 at 0:58
  • 4
    The post is now up: Are you interested in Prompt Design? Join the Community Stakeholders group
    – V2Blast
    Commented Jun 22, 2023 at 17:04
10

The announcement could be written more focused and more clear. It seems to be going a bit in circles and it all basically comes down to the company creating new sites without going through the community approval process for topics it is especially interested in. It seems to be a rather simple idea, so it should be presented as such.

I don't mind that as long as the usual community way of creating new sites is not altered. Do you plan to keep community approved site creations open?

For this specific proposal, I think it's a very specialized topic. It will likely not attract enough interest (questions, experts). Without an indication from the community we cannot even be sure that there is any demand for that.

I would rather include it in an existing site, which would have the advantage of lifting synergies and draw on existing communities. There is already one called AI and they even have a tag for that topic. Consider posting questions about prompt design at https://ai.stackexchange.com/questions/tagged/prompt-design instead.

8

What is a question, if not a prompt for an answer?

Setting aside the community management issues with the decision, the reason this specific site will fail is that the technology it seeks to help users master accepts 100% of the user input methods of navigating and interacting with the site as inputs of its own. It places human authors and curators in an arms race with model architects equipped with racks and racks of GPUs.

Any good quality question published on this site essentially identifies a class of inputs that the current version of the GAI in question performs poorly at without non-automated assistance. The "higher-quality" the question, the higher value the model's developers should be placing on making the next iteration of the model produce the desired result without having to refer to a Q&A site to "craft" a more effective prompt.

If such model improvements were only possible through meticulous manual effort then the GAI landscape might move slowly enough to create a value proposition in a Q&A site for the status quo. But instead we have the opposite: the best practice for interacting with these models is capable of evolving far faster than a Q&A site can keep up with describing it.

0

I'd have thought that the results of the formatting assistant experiment had sufficiently demonstrated the pointlessness of "prompt design."

Every single one of the hysterically funny results started with the same boilerplate prompt intended to make the chatbot carry out a particular task. Minor variations (the actual questions to be "improved" by the assistant) resulted in wildly diverging responses from the chatbot.

A single letter as a "question" lead to a completely hallucinated question, complete with a hallucinated code sample.

A link to an image lead to a different hallucinated "question" with no relation to the image or anything else.

The results are by the design and function of the chatbots non-deterministic, not to mention often factually incorrect. Trying to "design" a prompt to achieve specific, correct results is a pointless exercise in futility.


Stack Exchange is dying. The trends of asked questions over the last ten years show an unbroken slope that has taken a sudden drop with the introduction of ChatGPT.

The vast majority of users are not interested in correct answers, let alone any concept of quality.

The popularity of ChatGPT and the flight of users from the Stack Exchange system to the use of chatbots proves that users aren't interested in correct answers. All people want is "right now, immediate" answers.

If Stack Exchange joins the chatbots in providing immediate answers, they may temporarily maintain some level of viewers - the clicks for advertisement money will provide income for a short while. People will eventually tire of going through the Stack Exchange middleman to get chatbot responses. They will simply ask the chatbot and the Stack Exchange network will die of irrelevance.

Stack Exchange (as a company) could choose to value quality over clicks. This would naturally involve a short term loss of income as people flock to the chatbots before returning to the Stack Exchange network for good answers.

I expect the owners will choose quantity over quality, causing the Stack Exchange network to lose relevance, because the chatbots can do quantity much better.

2
  • Re "demonstrated the pointlessness of "prompt design."": I interpreted it as a complete lack of prompt design (as if they didn't have any idea of what they were doing, possibly as a result of having being ordered to do "AI" ASAP no matter what). It is possible to shape the output to some degree. Commented Jul 17, 2023 at 10:49
  • 2
    @This_is_NOT_a_forum: You can shape it to some extent, but given the extreme variations due to trivial changes it seems to me that it isn't useful at all. I mean, you go to the hassle of writing a prompt that looks like a legalese description of the problem and then a single letter typo sends the chapbot off into lala-land. What did your carefully crafted prompt get you? Nada.
    – JRE
    Commented Jul 17, 2023 at 10:54
-4

We want to ensure we have a site to serve that group...

The site already exists

3
  • 3
    Nope. AI.SE is centered around the theoretical side of AI and building them etc. This will be focused on the applied use of GenAI tools. meta.stackexchange.com/questions/391232/…
    – ColleenV
    Commented Jul 19, 2023 at 18:41
  • @ColleenV but questions like that should be on topic there. Commented Jul 19, 2023 at 18:42
  • 3
    That sounds like something you should take up on their meta. The reality is that they aren't.
    – ColleenV
    Commented Jul 19, 2023 at 18:43
-10

Do it, SE is overreacting

Don't want it? Don't visit it.

The response is again Stack Overflow and other laser-focusable networks claiming alpha and omega.

Come gander at Signal Processing Stack Exchange. They've gone a bit too far off "focus" - just a bit. Despite deviating greatly from other SEs, and handling both overly broad (many possible answers) and overly specific (seemingly unlikely to help others) questions, the resulting Q&As are, in high percentage, useful, and somehow sufficiently focused to be searchable for users with specific needs.

I've skimmed all responses, read everything bolded, some unbolded - the best argument I found is on focus, which is exaggerated. All else seems to be "y no ask us". Not saying there's nothing better, just a skim.

Generally, I do agree that the company should first ask the community. Problem is, they knew what anyone else who tries to go against "Meta's Meta" knows - they'd get steamrolled. Those who participate the most in Metas aren't best representative of others' interests.

Language is even broader, yet there's language SEs

with even more niche edge cases. A valid concern is, what about changing versions?

It again may come by surprise, but some networks are advancing theory in real-time. Stack Exchange isn't always about recycling existing knowledge - it's also about creating. New versions won't completely scrap prompting techniques applicable to previous - and the more versions are handled, the more "version handling" theory there is, just like there is on code and site cross-compatibility.

Is ChatGPT4 not enough?

The whole "version" argument really undermines the behemoth that is GPT4. GPT4 is already AGI-like for many purposes. Heard of plugins?

Then there's Stable Diffusion or whatever's best right now - an on-demand instant artist. It's good enough with common sense prompting, but even better once optimized, and I for one want to know how to optimize.

Even if all AI progress were outlawed today, a Prompt Engineering SE is worth it to me.

... but do it right

If I understand correctly, it's being rushed? The community can and should be consulted on how the network is built. Firsthand, long-term experience and all. But given the latest events... the timing's off.

There may be other "not done right" aspects, dunno, didn't read everything. I'm just disagreeing with "not doable/useful".

7
  • 2
    Oh, without everything else, we'd be ranging from 'Oooh, shiny' to feline level indifference. Or pitching in to finally upgrade/update the new community experience. Commented Jun 23, 2023 at 5:43
  • @JourneymanGeekOnStrike I honestly have no clue what this comment is saying, and I'm curious. "indifference" I'd guess meaning, only 1-2 staff folk running the new network, imperfectly, yielding low-quality all around. To which yes, absolutely must be avoided. Commented Jun 23, 2023 at 7:16
  • 2
    Well - I mean without the causes of the strike, people would basically be seeing the site on its own merits and their interests. Commented Jun 23, 2023 at 7:17
  • I again don't really follow (how's "seeing X for its actual merits" and "X agreeing with reader/user" bad), but I'm supportive of the strike, one does not simply sweep aside the volunteers. My answer's only about "PE SE", and "overreacting" refers to non-strike substance (unfocused, etc), hence the last section which I hope is clear. Commented Jun 23, 2023 at 7:25
  • 11
    Considering that I am probably the only one to have mentioned the focus problem, I want to ask... How much do you know about Stable Diffusion for example to think that I was exaggerating? You may be thinking about this process as something similar to node.js for example: a well made, organized repository like NPM, clear versioning, documentation etc... Let me tell you that the scenario is nothing like that. Finetunel model that do not disclose what they were tuned on, very little or even conflicting versioning, no centralized "official" repo everyone uses etc etc etc.
    – SPArcheon
    Commented Jun 23, 2023 at 11:28
  • 11
    Again, the final experience is probably more similar to attempting debuging of issues with (undocumented) minecraft mods or skyrim ones. Which does not mean it will be impossible, just that this should be though and planned before they start to realize what they are stepping into. It is easy to look at promptHero and think it would be simple to do the same...
    – SPArcheon
    Commented Jun 23, 2023 at 11:30
  • @SPArcheon I know very little of SD, and I've not fully read your answer until now. You raise valid concerns, that I don't see as dealbreakers. If BadBot changes greatly version to version and versions change often, worst case, ban questions about it. That BadBot exists doesn't mean there's no value in Q&A's on GoodBot. Again, if there were an SE exclusively about GPT4, that's already worth it. Commented Jun 24, 2023 at 8:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .