Why did Theoretical Physics fail?

Question

Theoretical Physics (TP or TP.SE), a Stack Exchange site for research-level questions, is going to be closed.

It was clear that the traffic was declining, even though the quality of questions (and answers) was actually good. Among the community there were discussions about the nature of the problem and how to tackle it (for example, Rejuvenating the site, my opinion hard-copied below, as the link is now dead).

The question is what were the main problems of TP.SE?

I am asking this to have a broader perspective on the success (and failure) of Stack Exchange beta sites. It makes a difference, if the problem could be solved with the proper care or if a particular topic is doomed from its very beginning (for example, it is too narrow, too difficult, or too subjective).

REMARKS:

• This question is not about the closure, but about the stagnation itself. I (and I guess most of the TP.SE community as well) perfectly understand that under the current circumstances site won't flourish (unless a miracle happens).
• There is little overlap with physics.SE -- very rarely was there a doubt whether a question should be migrated. If anyone is interested in why we split, here are some links. In short: researchers were diluted among a vast amount of high-school / layman questions (and it is way easier to ask a layman question in physics than in, say, programming) and by trial and error we learned that it is almost impossible to maintain a research-oriented community there.
• Of course 'theoretical physics' has a relatively narrow community. However, cstheory.SE works well (arguably, with an even smaller set of people doing it). And MathOverflow (Stack Exchange1.0), with roughly a similar size of the target group, succeeded greatly.

My opinion which I posted originally on meta.TP.SE:

In my opinion, the biggest problems is the lack of a critical mass of people willing to contribute. Coverage of a variety of topics is also important, but is to be dealt after the first issue is solved.

Most of the physicists I talked to like the idea of TP.SE, but they don't contribute because they:

• don't have time or will to translate their problem into question form (as it typically takes longer to provide a good enough description in text than during a face-to-face discussion),
• are afraid of asking something too simple (*),
• don't want to share credit or are afraid of their idea being stolen,
• are kind of conservative - either they don't use new Internet tools that much or just don't use something unless it is a standard in their field/group,
• or they judge a thing by its cover and perceive a forum-like thing as something that will waste their time.

I think that the most important problem here is (*) - because either people tend to ask to complicated questions (and as they have been struggling to solve them for months it is unlikely that someone is going to instantly provide a solution) or just they don't ask at all. So, encouraging users to ask (even simple) questions may be crucial. A "simple" question by a PhD student or above may not be that simple at all.

Answer 1

There were three major problems that I saw when looking at the site, all of which had been discussed on the site at one time or another:

1. A lot of overlap with Physics.SE, both in terms of topic and especially audience. Physics was already a bigger and more established site when it launched. This tended to siphon off questions that might've otherwise helped sustain it, when the ones that were asked were

2. Questions that took too long / too much effort to answer. Complex, interesting questions aren't necessarily a bad thing in and of themselves, but they tend to bring in fewer casual readers and can become a major problem when you have

3. A small and ever-shrinking audience of regular users. Theoretical Physics made a truly impressive effort to keep participation up on the questions it did get, but as the influx of questions and daily visitors dropped off this became more and more of a struggle.

Any one of these would have been problematic, but the combination of all three together fed into each other and thwarted efforts to change the direction.

There are other sites in similar positions, and while I sincerely hope we can avoid the same downward trajectory, in truth the core problem is one of scope and audience: if neither one is big enough to be sustainable, the site eventually dies.

My biggest reason for wanting to step in and close Theoretical Physics now is the hope that by merging into Physics we can avoid that slow and painful death. Sadly, not every site has that opportunity.

Answer 2

I think part of the reason it didn't succeed has to do with the depth versus the breadth of the field. Two of the most successful sites are MathOverflow and Stack Overflow. Math and programming are both broad fields.

If someone in the field has a question about mathematics or programming, it is reasonably likely that somebody else in the field can answer it with relatively little effort, as long as the question isn't in the first person's area of specialization.

Theoretical physics (at least the part of it that had representatives participating on the site) is very deep, but not particularly broad, and so for a typical question, it might be that nobody knows the answer, or it might take a long time to figure out the answer. You certainly get questions like this in math and programming as well, but they're quite a bit less common.

Answer 3

I thought it was a bit of a strange decision to have a site based on one subfield of research level physics. Mathoverflow would not have worked nearly as well if it had just been AlgebraicGeometryOverflow (even though a lot of the initial crowd was algebraic geometers). Why not a research-level physics site that explicitly included experimental physics, condensed matter, etc. Unlike most people here I'm a big fan of the research/non-research site splits, but I'm a lumper when it comes to subjects within research sites.

On the other hand, TCS is a rather small field, and their site seems to have gone reasonably well. What I'd really like to understand is what the difference is between TCS and TP.

Answer 4

I disagree with the premise that the TP site has failed. We have succeeded in maintaining high quality site which has become a reliable resource of correct and useful information, which is sometimes not that easy to obtain. We had top people coming to the site and contributing high quality content. Quality, rather than quantity, is of course something that is not easily measurable, but I think it has value nonetheless. Traffic was an issue, but for a highly technical site the pattern of traffic, including how it changes with time, should be evaluated differently. Certainly things could have been done better (e.g. Encouraging more explicitly student questions, as long as they don't have standard textbook answers), maybe if this experiment continues elsewhere sometime...

I have to say that it is a bit disheartening to discover here, as rationale for closing the site, a lot of the preconceived notions that people had all along (e.g. That it has large everlap with the physics site, which is demonstrably false). I guess it is not that easy to change people's opinions.

My personal conclusion is that an isolated, highly technical community like ours needs to maintain their own infrastructure. SE network is great in general, but the large body of rules and opinions on about how things should be done was sometimes less than completely helpful. Maybe they are simply not set up for maintaining highly specialized sites. Perhaps an independent site, like math overflow is the way to go.

Answer 5

I agree with Peter. In a sense TP is too high quality, looking at the site would cause even graduate students in physics to freak out. I stopped visiting the site a few weeks after it started because not only it was unlikely that I could answer any question I also felt that I cannot even ask any on-topic question.

SE always emphasis the presence of experts and that is definitely required, but it seems that to keep the site alive also needs people who ask questions in areas that they are not experts, so other users who are experts in those areas can answer them. A site needs people who are not experts on some on-topic area of the site, so the users can benefit from each others expertise. I think that is one of the main reasons MO works so well, math is a wide topic and there are a lot of specialized areas and experts in each of them need to ask a question in an area they are not experts on.

Too much concentration of experts in their own field doesn't keep enough interest, even from the experts, to visit the site regularly.

So where the scope line is drawn is important, it shouldn't be too low so experts become marginalized by non-experts and therefore would not want to spend time on the site. At the same time it should not be too high so questions are too hard and too few.

I think the situation can be different if the level becomes relaxed a little bit and junior grad and senior undergraduate students participate and ask questions.

Another problem was the users were not engaged enough about running the site. The meta participation was relatively low, and there wasn't enough effort to improve the things (at least doesn't appear so based on the meta posts). It seems that they are kind of OK with the site getting closed.

Finally, it would be nice if there is a mechanism that would warn the users on betas which are not getting to a satisfactory state some time before closing and engage them on why the site is not getting to a satisfactory point, and maybe another chance to try to improve things, it might save a few of them.

Answer 6

Reading the answers here, most seem to come to the conclusion that theophys.se had to fail. I'm unsure if this is case regarding that similar science proposals like cognitivesciences.se, economics.se with much less experts and similar site statistics have huge problems generating questions too. Theophys.se surely had no lack of experts or too much overlap with physics.se. Kaveh and Peter Shor made important points imo. But for Physics many research problems just fit better a discussion than a Q&A format. Physicsforums.com, bulletin boards are still very high-frequented sites when it comes to popular science physics. Many professional physicists with a PhD still discuss in old usenet forums without mathjax or any of the nice markdown features or moderation that stackexchange offers.

Contrary, many research problems can be "discussed" as Q&A&comments in Mathematics and Computer Science and as Peter Shor pointed out, it's easier in mathematics/computer science to come up with a short correct answer to a tricky question than in theoretical physics or biology. Experimental sciences fit a Q&A format much worse than such topics. Questions and problems are per se more localized and of less interest to the majority. You can easily verify this by comparing site statistics of mathematics.se and physics.se. So the conclusion is, a proposal like theophys.se needs higher area51 commitment requirements (more committers, more experts) than sites like recently started computerscience.se or biology.se. But I wouldn't say topics like theophys.se cannot work per se in a stackexchange format. You need probably more than 5000 (this is very much compared to the thecostofknowledge.com followers) researchers for a site creating around 10 questions a day and to keep experts visiting this site. This is very tricky and would imply commitment phases of 3-5 years to gather 1000 committers and guarantee enough participation (many science proposals were for 1,5 year in commitment phase!)

Regard that so far physics.se hasn't generated many research-level questions, most users of theophys.se didn't want to participate in physics.se for mentioned reasons in many threads here. It's unlikely you can force them to spend their time on physics.se. I'm sure a site like astronomy/astrophysics.se would work with higher commitment requirements, better than being integrated into the very broad physics.se now.

So I'm very glad that the SE creators realized that the area51 process has to be adapted to generate healthy proposals for topics not related to programming and programmer's hobbies. (Math, parenting, cooking, etc. work because many programmers deal with these topics, and the low area51 commitment requirements don't damage the community dynamics in first beta months.) Humanities are very unlikely to create high quality beta sites on SE currently. Sites like mathematica.se or biology (bioinformaticians being attracted by SO) work really great.

I'm looking forward what the new area51 2.0 will look like. The main problem is that the number of interested laymen using SE like Wikipedia due to high quality and reference character are growing exponentially faster than the amount of experts on SE. So more and more area51 proposals become diluted by committing non-experts. So either you set up the commitment requirements so a proposal starts with enough experts in private beta or you make it mandatory for a proposal to have 30-50% professionals, experts, students (number biology/chemistry.se had, cognitivesciences, philosophy, economics had around 10%). I don't see another option. Cognitivesciences.se and Economics.se have exactly this problem of very few experts trying to establish high quality by downvoting and closing too many low-quality questions yielding a laymen vs. mods situation with the effect less voting, less questions,.... site getting closed.

A last point I want to mention. Students become researchers, science is developing towards open science. Open-access and knowledge sharing platforms like SE or Quora are young and help here to build up a community of researchers used to sharing secrets and tips. A mistake theophys.se imho made, was excluding advanced grad students and their problems too much: most questions on SE come from students. And in 3-4 years many of them would have been researchers and being used to ask questions. I think many of the older researchers identify asking a question with ignorance and so ask very few questions but hire young researchers for their groups that come up with new questions, ideas, methods. Kind of tragedy :) I would try to restart theophys.se again in 1-2 years. Lesson learned.

Answer 7

It looked rather like a success to me.

Many of the experts in this area which are active on the internet were visiting and participating here. Do the non-physicist moderators realize this??

If you simply go by the numbers, yes, then Theoretical Physics had no chance from the start because there simply aren't enough knowledgeable people on this planet to generate a lot of traffic.

If more traffic, questions and answers are required then it would have been better to lower the level to graduate physics instead of postdoc research questions.

Merging it with Physics means messing up 13 year old high-school student questions with postdoc research, what a mess.... Still better as throwing it all away i guess.

OK, there's tag filtering and a "theoretical physics" tag would help to find the questions back. Still, one of the nice aspect of autonomous moderation by trusted users gets lost because after answering enough 13 year old high school student questions one is supposed to be knowledgeable on research level physics...

O well, just hoping that this can be saved in one way or the other.

Answer 8

Honestly, my take is that the primary reason why you had issues with theoreticalphysics.se was that:

1. The communities on the two sites were not distinct. Almost all of the commenters, questioners, and answerers on theoreticalphysics.se also were commenters on physics.se
2. While it was manifest when a question should be migrated from theoreticalphysics to physics, and the theoreticalphysics community was not at all hesitant to do so, the migration in the other direction was rare (if it ever happened)
3. Meanwhile, the traffic on theoreticalphysics was always much lower, which, in the end...
4. Created a situation where no one would ever really want to ask a question on theoreticalphysics that they could, instead, just ask on physics. The only downside is reading a few wrong or nonsense answers that would then get voted out of existence. All of the exact same people would see my question, and if they find it interesting, answer it. The only advantage would be that my theoreticalphysics question might stay at the top of the page for a longer amount of time thanks to inactivity.

All of the need for the theoreticalphysics community can probably be addressed by judicious use of the research-level tag without introducing any of the other problems. A distinct site should have, at least at some base level, a distinct community.

Answer 9

I don't think you guys understand researchers.

Researchers don't want to write stackexchange posts on research-level problems. They want to publish papers on them. Research questions are simply too hard to get advice on, on an impersonal forum over the internet. You would be lucky to get a response from the most polite-sounding, flattery filled email to a research professor. You wouldn't get a response simply because you are not his research student, and he doesn't have the time to invest in a student that is not his own, for obvious reasons (that it has likely 0 payoff, unless he gets a co-authorship somewhere down the road).

So "theoretical physics" is simply too specific a site topic to exist. theoretical-physics is a great tag name for physics.se, but not an entire site. There just isn't a critical mass of people worldwide that would be interested in such a site.

Maybe in 10,000 years when theoretical physics is a high school topic, because of advancing education or whatever, but not at the present time. The site simply demands too much from people, and theoretical physicists would rather try to publish their results in a journal than help anonymous strangers finish their work.

Answer 10

With all my respect to authors of skeptic answers, that looks too biased for me. We know that few sites were closed at the same moment, for example, the Literature.SE was closed with partial merging into SciFi (!), but in this case relation area <-> sub-area is opposite and number of potential users for Literature may be in hundred times more than for the SciFi. May be if TP could be created earlier than Physics we would discuss now an opposite problem with closing Physics and partial merging into TP ... yet, stop talking about virtual history.

Anyway, I made a primitive statistical estimation and extrapolation and despite of note about "Lies, damned lies, and statistics", possibly, TP were closed in rather inappropriate moment, when it could start to generate stable growth due to new questions instead of resolving some amount of old questions already existed before and awaited creation of the site to be asked (after all the reason to open some particular Q&A site -- is try to find answers on some questions). Just after opening a site such questions are gradually exhausted, and it may produce an illusion of negative trend and stagnation before new questions start work. A simple model used for analysis may be found here, below are only plots of some prognosis of possible future of TP if it were not be closed.