29

For those who don't know, Generative AI Stack Exchange (now in public beta) was an initiative started by Stack Exchange here slightly over one month ago, and involved "stakeholders" (who pioneered the site). The timing of the site's launch was likely chosen to coincide with CEO Prashanth Chandrasekar's talk at WeAreDevelopers some time in the next few days. It was one of the most controversial Stack Exchange decisions to date.

I'm guessing many of you are curious, and it could help people inform future decisions to know...

Question: What was it like being a GenAI.SE stakeholder?

I'll add my answer, and other stakeholders can add theirs, if they like.

5
  • 4
    I'm not inside the stakeholder team, so I can't really comment on anything, though I did observe some kind of "mess" based on the public stakeholder chat room and child meta. Dunno if another question for a retrospective of this experimental process is worth it or not, whether on here or child meta. (I see this meta question as a potential, though I don't want to force it) Commented Jul 26, 2023 at 8:11
  • @Rubén - do we really need a tag for [prompt-design-site]?
    – CDR
    Commented Jul 28, 2023 at 14:24
  • @CDR I understand that you have one or more objections. Please elaborate
    – Rubén
    Commented Jul 28, 2023 at 15:37
  • 1
    @Rubén - no, I'm not objecting to it, I'm just wondering what you think the use for it is. Because the site has its own meta and I don't expect there to be many further questions that would need the tag. Most individual sites don't have their own tags, do they? (Though I suppose prompt-design/gen-ai is a bit special)
    – CDR
    Commented Jul 28, 2023 at 17:50
  • @CDR Thanks for your reply. I created the tag because there are several questions and because there might be posts here using Prompt Design, Stakeholders Group or GenAI SE, but all be about the prompt design site. Few sites have their tag, but that should not be the reason to create a tag. Tags are intended to make it easier to follow questions about the same topic; we have that Watch feature, among others. As the site name is still being discussed, this tag might be helpful if the site name changes, among other things.
    – Rubén
    Commented Jul 28, 2023 at 18:04

3 Answers 3

35

What did we do?

The site skipped the usual process of Area 51, and instead opted for an "experimental" launch method. In practice, this turned out to be a chat room and a Teams instance. When I signed up as a stakeholder, I was expecting to be using an Area 51 replacement prototype, as per:

...we will try a new process that is relatively reflective of what we think a new process for site building on the network should look like: public, discoverable, and out in the open.

We instead used Teams. I was later informed that the Teams was actually private. I found this disappointing; it's as if my hard work in carefully thinking through and refining posts on Teams has been effectively thrown away, unable to inform future users when making decisions. There are also issues with copyright for the /on-topic page (so now we have this). And since I'm not familiar with the idiosyncrasies of Teams, I had some technical issues.

Stack Exchange, if you're going to do this again, may I suggest simply using an invite-only "alpha" site+meta once you get a quorum of stakeholders. Maybe give all the stakeholders diamonds and see if they can figure it out. (Or something in this ballpark—I have too many tabs open for this task.)

What went wrong?

Okay, the launch method was "experimental", so obviously it's not going to go completely smoothly...

The lack of example questions hindered making decisions about what's on/off-topic. You can see this problem reflected in my answer here on GenAI.SE meta, where I listed what I thought were "ethics questions", followed by wizzwizz4 commenting: Only the fourth question is about ethics. We're working with different mental example questions, so of course we're not going to come to a consensus. (I somewhat anticipated this problem, which is why I asked this.)

The accelerated launch process meant that hasty decisions were made in some cases (hopefully this is a once-off for GenAI.SE). Most importantly, the site's name was hastily changed to Generative AI. If someone was away for a few days, they may have missed the window of opportunity to have their opinion heard. This process also coincided with my travel around China, so I could have easily missed the window too. We now have the problem of a site called Generative AI, but a scope designed for prompt design. Topics like genAI inner workings were deemed off-topic at the time (AI.SE can have them), a decision which appears illogical at Generative AI. It's still not entirely clear how to proceed, but if we end up deciding that generative AI is on-topic at Generative AI, we would benefit from genAI developers helping us rewrite the scope as I get the feeling the stakeholders were predominately users of genAI (after all, the site was originally called Prompt Design).

The name change was motivated by wanting to avoid the buzzword "prompt design" (see e.g. this NSFW Reddit post), and the fact that interacting with genAI can be done in myriad ways, not just through "prompts" (e.g., a 20-minute conversation with ChatGPT, or an image-generation workflow which involves multiple genAIs and even manual steps). It was still meant to be about the applied use of GenAI tools, but after changing the name to Generative AI, we faced questions like How important are GPUs vs CPUs when training an LLM which, despite being an obvious thing to ask at a site called Generative AI, we had not considered previously.

We are now in a position where some users are downvoting and voting to close based what we declared on/off-topic for Prompt Design, and other users are coming to the realization that it's simply not feasible to declare generative AI half off-topic at Generative AI. This needs fixing.

Because the site was rushed, we're now dealing with the fact that only one user has sufficient rep (500) to vote to close (alleviating this problem seems to be the motivation behind proposing changing the rep levels, which has now been vetoed). Nevertheless, I think it's fine if we basically don't close questions for the time being; if a question unexpectedly gets good answers, maybe we should change the definition of "on-topic" rather than close it as off-topic.

Seeing the stakeholder updates get downvoted was discouraging: regular users and moderators (together with SpencerG, and some other Stack Exchange staff behind the scenes) were all putting in a fair amount of effort to try to make this work, despite the obstacles. Many of us have the same objections, but were just playing the cards we were dealt.

We have been getting some irregular voting at GenAI.SE recently. I personally am up to three separate "serial voting reversals" (which is crazy giving how few posts I've made). I would appreciate it if strike organizers would make a statement about not sabotaging GenAI.SE. In any case, it does not seem to be relevant to any of the strike demands (some stakeholders support the strike too!) [edit: actually it relates to "Stack Exchange, Inc. must communicate, gather feedback, and act on that feedback before making major policy or software changes to the public platform", although we have been in close communication with Stack Exchange for the last few weeks; see SpencerG's GenAI.SE meta profile or the chat], nor will it prevent people copy/pasting genAI as answers.

We anticipated many ethical concerns arising in questions, but nothing has come up thus far. Hopefully this is a sign that we won't get many ethically borderline questions in the future too.

Despite all these bumps in the road, the stakeholders worked together fairly harmoniously (working towards a common goal), and kind of "went with it" when surprises and confusion arose. We haven't worked everything out, so please be a bit patient with the new site. We're trying our best.

14
  • 15
    My impression from this answer is that the site was supposed to be something like "Applied AI" or "AI for users" or "The practical sides of AI" and the current name "GenAI" is not a good choice for conveying that message. I don't know of a better name right now but the current name doesn't seem to be a good choice. A rebranding might be necessary to let the new site fly. I also wonder how AI.SE sees this new site, if they basically see practical AI questions also as within their scope. Commented Jul 26, 2023 at 7:31
  • 1
    One popular alternative was PromptAI Stack Exchange. I'm also tempted to consider GenAI Tools Stack Exchange (which is closer to the current name, which may have less friction). However, I doubt a site name change is going to happen only days prior to the WeAreDevelopers talk (so maybe we should try coexisting with two user groups). I welcome your input on the site's meta. Commented Jul 26, 2023 at 7:41
  • 10
    I will just point out two things. First, the wikipedia article is not called "Prompt Design" :P. Second, a famous online q&a network has a site called "Super User"... perhaps one could copy them and have something like "GenAI Users"
    – SPArcheon
    Commented Jul 26, 2023 at 7:55
  • 4
    @SPArcheon "GenAI Users" is not a bad suggestion at all! The minimal change may also be palatable to higher ups, who could say we are "refining" or "clarifying" the site name. Commented Jul 26, 2023 at 8:26
  • 3
    "...but if we end up deciding that generative AI is on-topic at Generative AI, we would benefit from genAI developers helping us rewrite the scope as I get the feeling the stakeholders were predominately users of genAI" Does anybody else find this confusing? What a mess.
    – ouflak
    Commented Jul 26, 2023 at 9:23
  • 1
    "I welcome your input on the site's meta." Thank you but I have very little interest in prompt design currently. I would probably rather ask on AI.SE if I have questions about AI. Commented Jul 26, 2023 at 11:34
  • 3
    @RebeccaJ.Stones more than a thousand people have signed the strike letter, including at least one known network wide troll. Please don't assume the worst of us represent all of us.
    – terdon
    Commented Jul 26, 2023 at 12:19
  • 1
    (Okay, since I already established the evidence exists that three strike supporters have engaged in voting irregularities, I'm going to delete my previous comments---I'm uncomfortable naming and shaming, or encouraging witch hunts. Diamond mods can still see the comments if they want evidence. [Oh, and I'm not blaming all people on strike---but it would be nice if those who are on strike would say not to do this.]) Commented Jul 26, 2023 at 12:33
  • 1
    @NoDataDumpNoContribution I'm user on Cryptography, which is definitely part of Information Security. Sometimes we do see posts that are specific to crypto on the IT Security site and sometimes we migrate questions about e.g. certificates to them. I'm just saying that having more specialized sites is more common than you may think, and generally we can coexist peacefully. Biggest problem IMHO is that the method for migration ... isn't any good. Commented Jul 26, 2023 at 12:46
  • 12
    I was one of those use users you were referencing, and I was not indiscriminately voting, and certainly not targeting any individual. My actions were based on my understanding of the site scope and purpose. I have no visibility into who the stakeholders are or what was discussed in teams; Everything I did was based on the scope on it's meta. --- It appears as though people want this site to succeed so badly that they'll forego all existing curation tools and simply upvote everything. genai.stackexchange.com/users?tab=voters
    – Kevin B
    Commented Jul 26, 2023 at 14:53
  • @MaartenBodewes-onstrike I see and I have nothing against peaceful coexistence. It's great. I prefer a bit more the more general information hubs. Commented Jul 26, 2023 at 15:50
  • E.g. SO is relatively impersonal and I would certainly not benefit by following the site feed. Smaller sites mean that the users know what to expect from each other. SE search is still not that great, Google and other search engines will however find Q & A's regardless on which site they are. So I prefer somewhat smaller sites as long as there is a strong enough user base and question rate. Note that we're really divided into communities, especially compared to e.g. reddit and especially Quora. Still, it is kind of keeping separate inboxes where the tags are labels. Commented Jul 26, 2023 at 15:58
  • 2
    On Area 51 the health of the beta site (questions, visits, something like that) is displayed. Are there any plans to do so with GenAI? Will you be measuring site health differently?
    – bobeyt6
    Commented Jul 26, 2023 at 17:23
  • @bobeyt6isstricken Good question for the meta. I simply don't know. Commented Jul 26, 2023 at 23:09
35

Mainly, I did what I aimed to do: lurk / watch, answer meta questions about decision making where I thought applicable by pointing to how Stack Overflow or the wider network approaches similar things and why, and ask any meta questions I wanted to ask.

I also queried for some existing questions on existing network sites that might qualify as example questions (if you haven't noticed already, I find a weird sense of enjoyment in slogging through boring query results) and tried to generate some example questions for discussion on scope. I continually felt like the lack of example questions for discussion was part of a bigger problem about a lack of clear direction in scope.

Before joining the group, I was confused by the seeming lack of direction from the company - especially given that they're forcing this thing to happen, and that feeling hasn't gone away. I guess I have a difference in expectations. I expected that if the company is going to force the creation of a site, they'd better have a clear idea of exactly what site they want to force the creation of. Granted, setting site scope was part of the job description, but I assumed it would be more like ironing out finer details rather than laying out significant groundwork. Some productive meta discussion may have happened, but my general feeling is still that I'm looking at the company's ill-defined wishlist to Santa - except there's no Santa and just a handful of elves, or someone insisting on birthing a child months ahead of schedule. I mean, the thing is probably being announced in a conference two days from the date of this writing, and we're still having conversations like this and scope turbulence like this.

My feeling that the company doesn't actually really take this as an opportunity to improve Area 51 (Stack Exchange site) / the site creation process, and is instead doing this for other hype-train reasons and that the stated rationale about improving the site creation process was a pacifying, made-up excuse, has solidified (call me cynical for thinking this, or call me incredibly naive for taking so long to think this). If they really cared about that, they wouldn't be rushing it.

Skipping getting commitment to a site and skipping getting a rough definition of scope while the ship plows forward with the captain's foot on the pedal does not work.

So how was it? For me, bewildering.

4
  • 9
    "that feeling hasn't gone away" The company always relied on users to create 99% of the content. Only this time the site initiation wasn't user driven, so the expectations were set differently. My guess is that the company thought that the stakeholders would do the work of coming up with a useful scope and example questions and the stakeholders thought that the company must have a clear vision. Commented Jul 26, 2023 at 11:30
  • 2
    « call me cynical for thinking this, or call me incredibly naive», no, I don’t wanna call you any things, however, yes, you were naive (so, apart from that) and I’m happy you’ve come to the realization. That said, what you’re describing is just yet more fuel on the fire. :/ These are not good news you are reporting. Commented Jul 28, 2023 at 16:27
  • Er, ships don't plow, and usually don't have a pedal to control the power. (^: Or are you mixing metaphors to make a subtle point about the site's management?
    – tripleee
    Commented Jul 29, 2023 at 10:38
  • @tripleee "plow forward" ~> "plow ahead". And yes, I am mixing expressions that do not create a consistent image.
    – starball
    Commented Jul 29, 2023 at 18:52
15

I'm cocomac, one of the other stakeholders. I largely agree with Rebecca J. Stones's answer.

We mostly used a combination of SOfT (Stack Overflow for Teams) and chat. There were a few hiccups with that, but it worked.

One major complaint I’ve seen is a lack of transparency. And... that should have been better. Yes, there were shareholder updates. But... that was announcing decisions, not asking for feedback.

Personally, I like the idea of being able to float ideas privately and refine/get feedback, and then post publicly for feedback as a second step. Of course, this fundamentally relies on the public post seeking feedback, rather than plans/concepts being set in stone. I think that’d be a big improvement on future SE/user collaborations.

The push from SE to get this launched was also challenging at times, as there are still some rough edges that need to be fixed, so to speak.

Especially regarding scope, there were challenges. The name change, and then we published an on-topic page, but failed to update it after more decisions were made. There were also legal complications that caused some issues (I am not a lawyer, and won’t elaborate much on those here).

Within the broader scope of ongoing events (a.k.a. The Strike), timing of the site… wasn’t great. Admittedly, Area 51 certainly has flaws. There’s certainly room for improvement.

But, with the existing setup, one needed a decent community to have a site. And you had to prove that it was, at least to an extent, sustainable. In contrast, this assumed it could work and the intention was to decide criteria for site shutdown, but management would still have to agree regardless of our criteria. So, I’m not convinced this setup is sustainable or would work again. I’m not sure we’ll even have a successful site.

That said, the big takeaways I’d emphasize would be (1) a significant increase in transparency would be good (2) some more time to refine the site would have been good and (3) I’m not convinced this is really sustainable at all.

2
  • Note: I’m fully aware of the... rather large voting threshold issue that happened. That said, it isn’t really in the scope of my personal experience with the process (beyond that the announcement on meta wasn’t clear enough), so I’m refraining from discussing it in this post
    – cocomac
    Commented Jul 26, 2023 at 7:09
  • 2
    For PLDI we also extensively coordinated using chat to talk about ideas before posting them to meta, which still used the old area 51 system.
    – mousetail
    Commented Jul 26, 2023 at 12:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .