Specific questions such as https://stackoverflow.com/questions/7284565/regex-html-match/7284879#7284879 and How do I parse content to replace fake lists with real lists using PHP? are being voted to close as duplicates of How do you parse and process HTML/XML in PHP?, which is a general question with generic answers that don't really answer the questions.

Are these really duplicates?

PS: In fact, I believe the other question is the one that actually should be closed, as it " will likely solicit opinion, debate, arguments, polling, or extended discussion."


The general question is a "canonical" one, meaning that it has been asked many, many times, in one form or another. A single canonical question (i.e. a mini tutorial) substitutes for the many specific questions.

Canonical questions and answers are our way of "teaching people to fish," instead of giving them a fish. If there is a canonical answer on the site that answers the OP's question (even indirectly), we can avoid the onslaught of questions that are basically asking the same thing, but in a slightly different form.

It could be argued that the specific questions are "too localized."

| improve this answer | |
  • This doesn't seem to be the stance you've always had. – NullUserException อ_อ Sep 2 '11 at 18:19
  • 2
    I also disagree with the general idea here. The overwhelming majority of questions asked on SO is specific. If we started closing and deleting specific questions, might as well close up shop and put a couple thousand of general, "canonical" questions and answers as reference. And there you go, we are now W3Schools 2.0 – NullUserException อ_อ Sep 2 '11 at 18:24
  • 4
    @NullUserException: My general philosophy on duplicates is here: blog.stackoverflow.com/2010/11/… However, there are certain kinds of questions that are asked repeatedly, over and over. These kinds of questions could really benefit from a canonical question, so that we don't have to keep answering them over and over. We don't need canonical questions for every specific question that's asked here, but we do need them for the "How can I parse HTML with Regex" types of questions. – user102937 Sep 2 '11 at 18:33
  • 1
    @Null: Note that the two questions you referenced attracted decent answers, despite having close votes. Also note that canonical questions are not a fully baked idea yet, anyway. Ultimately, the goal of Stack Overflow is to serve as a repository for good programming information, not as a technical support proxy, so I'm not sure that having a question about every possible variation of "Write this regex for me" will scale. Eventually, you have to go read a book. – user102937 Sep 2 '11 at 18:38
  • 1
    What annoyed me is that the "canonical" answer just has a bunch of links to various DOM parsers. It is essentially the same thing as telling the OP to google "PHP DOM Parsers." If you actually read the two questions, they are very simple pattern matching questions, and suggesting a DOM parsers for those is a moronic idea. By closing the question as a duplicate of that, we are not teaching the OP to "catch fish," but simply told him "hey, here are some cruise missiles you can use to blow fish to oblivion. Go figure out which one and how to use them." – NullUserException อ_อ Sep 2 '11 at 22:15
  • 2
    Then perhaps it's not as good a fit as a canonical question for your specific linked questions as you might like. That doesn't necessarily make the concept of a canonical question invalid, though. – user102937 Sep 2 '11 at 22:18
  • I mean, there's a lot of prejudice against HTML, but the vast majority (I'd say over 80%) of these regex questions are simple pattern matching questions - for which a DOM parser is overkill. The only valid canonical answer for them would be a complete guide on regex. Only when you understand regex then you can apply your judgement to see if it's an appropriate tool for your problem and decide if you need a DOM parser or not. – NullUserException อ_อ Sep 2 '11 at 22:23

Aren't you mixing two issues here? One is the concept of generic questions; the other is whether recommending a HTML parser is the right thing to do for all HTML parsing questions.

My take on the first issue is that generic questions are a necessity if you have 3000 questions a day, a lot of which resemble each other closely. The alternative to not having generic questions is for the vast majority of questions not to have an answer at all, because even the most motivated contributor tires after the 3500th instance of i want find all link with this title and output them in a row.

Plus while I'm all for helping a newbie out who is stuck with something basic, I think most of those questions don't deserve a tailor-made answer in the first place. Stack Overflow is already an enabler for thousands of people who are not interested in actual learning - ie. debugging, looking shit up in the manual, spending hours figuring out a problem for themselves once in a while etc. They use SO as a giant mechanical turk for solving programming issues instead. I don't see why we should support this any more than we already do.

| improve this answer | |

I don't think this is really what you're asking about, but for completeness: this is likelier to be true on Meta. It's common practice for bug or support type questions to be closed as "exact duplicates" of posts that address the same issues, even when the FAQ entries are much broader (which they almost always are).

| improve this answer | |

Yes, they are duplicates.

By now, each and every question asking "How to parse HTML with X" is a duplicate of some other similar question asking the same. Of course, all of these questions differ in some fine detail. One wants to get all src attributes from an img element. The other wants to get all href attributes from an a element. Yet another one wants to get all td elements in a table with a class attribute of foobar. For someone with no idea about how to do that, those are probably quite different questions. But they are not!

If you abstract the problem, all of these questions are asking the same thing. If you understood how DOM works, you know that it doesnt matter whether it's an a element or an img element or whether you want to fetch href, src or class attributes. That's just implementation details. It's an argument passed to a function. They are all asking how to fetch element X or how to fetch attribute X. Do we really want to answer all questions that differ just in X? Do we have to have one answer for each possible element and attribute in the HTML specs? No!

I'll give you that "Best Methods to parse HTML" is not answering most of these questions directly. But like I have already commented below one of these questions, I cannot be bothered to wade through the hundreds of duplicates to find the one that says td instead of img anymore. Especially when duplicates are flowing in while we speak. For the record, these came in shortly after you started to took offense on the closed question

That is just two more questions that will not get reusable answers but that I would have to look at in that futile quest to find the one duplicate that is a better fit than my generic answer to "Best Methods to parse HTML". Closing those as "too localized" will get the OP nothing. Closing it with "Best methods to parse HTML" at least gives him an entry point for the research s/he was supposed to do right from the start.

Fact is, SO is littered with those questions, just like it is littered with questions asking for the meaning of specific operators or how to format a date. We do not need any more of these. Generic answers like "Best Methods to parse HTML" or "Reference - What does this symbol mean" (for which you have attacked me in the past as well) are the first line of defense against lazy OPs that didnt do research, against shortcomings in the SO search engine and against community members that decide to ignore that we shouldnt answer duplicates.

I agree to not telling people "you cant parse HTML with Regex", because technically you can with PCRE and I very much agree to answers like Regex matching table rows in HTML getting deleted. In fact, I complain lately whenever someone links RegEx match open tags except XHTML self-contained tags because the answer, while legendary, is just wrong. Then again, just because you can parse HTML with Regex doesn't mean you should.

IMO, you are making false assumptions about the nature of these questions and about HT/XML parsers. Those questions are not about pattern matching. They are about retrieving information from structured documents and that is a solved problem. HT/XML parsers do a perfect job for this. They are not overkill. There is no drawbacks using them. There is only benefits. You dont have to know the grammar rules of XML to fetch elements with DOM. All the domain knowledge is captured inside those parsers, so you can concentrate on just getting the job done.

PCRE on the other hand have to be taught the intricacies of the markup first. People at SO undeniably fail at that regularly. Telling them to learn PCRE to match Regex when examples like tchrists HTML chunker clearly show how impractical this is over just using XPath (which was designed exactly for this job) is telling them the wrong thing. You are telling them to reinvent the wheel, when there is perfectly nice and easy to use wheels available.

Also, how is telling people to "do a Regex tutorial" any different that telling them to "Google DOM"? Your very own answer to https://stackoverflow.com/questions/7284565/regular-expression-html-match/7284879#7284879 isnt exactly what I would consider an explanation of that pattern. The OP will likely just copy and paste it into his code. If you want to educate the OP about Regex, at least use x and explain the pattern completely and do what you demand and link to a tutorial.

Also, I find it questionable to tell the OP he "might want to use a (X)HTML parser" and then give the links you give in that answer and then dv and cv "Best Methods to parse HTML" when it pretty much says that too.


You do not not parse HTML with Regex because you cannot, but because parsing HTML with Regex is hard and impractical. That's what "Best Methods to parse HTML" says. It's not touting the same old "you cant!" horn. It offers sensible advice and choice for a solved problem. And, while not best fit for all questions that get closed with it (which I cannot control btw) it's a good enough fit for most of these poor and lazy questions which it is too hard to find a better fit for.

| improve this answer | |
  • One pitfall of asking a generic question is that you can fall foul of the "specific question" requirement, and have the question down-voted or closed. How to avoid that. – Raedwald Sep 30 '11 at 10:03

Some problems are best solved by just honoring the user intent.

There are a couple of meta topics here. Generally I agree that a generic reference question may serve as closevotable duplicate. (Not in all cases of course).

Considering the specific case at hand, I can however assert that "Best methods to parse HTML with PHP" is unfit. It is very much relevant as tip for most html-with-regex askers, because posters are typically unaware of the options. But this means it should be posted as comment. As comment only.

The typical RTFM-style DOMDocument posts are not answers as long as the question title and tags still indicate html with regex extraction. It's however too easy to score some points by catering to this Stackoverflow meme. And so people will continue to post the opposite of what was asked for. Which in turn is poisoning the site.
(It's really difficult now to find some actual tips on how to use regular expressions for html extraction; for those cases where it's appropriate. Too many newb regex questions have been battled with DOM meme replies.)

My proposal is quite simple: Create/hijack an actual reference question, which does answer the general regex with html inquiry verbatimly.

  • This can still be a generic closevotable answer.
  • Provide some actual advises / links on how to build reliable HTML extraction regexes.
  • It may itself list more links to other reference answers for specific extraction desires (src from img, href from links, convert text links) to reduce effort in finding good exact duplicates for those questions.
  • For didactic reasons it would very much add a link to some good DOM/phpQuery answers. But that's a side node, should be the last option, as that wasn't originally asked for. (This is the crucial part here. There is zero educative value in always posting a DOM or bobince link first on such questions. Prohibition alone has not solved the influx of these questions till now, and it never will.)

Since most of the questions in that category fall in the plzsendtehcodez category, I would think that a generic reference answer would be appropriate here. The heaps of meta bickering we have about this is a direct result of the answerers lazyness to search. That should not be fostered.
But if it's everyones first intent (even if poor tutorials are at fault) to use regular expressions for html extraction (which it is only if you are mediocrely proficient in it) then let's give them according advises. But not more.
Those who then want to seek out simpler approaches can still find one of the simpler jQuery-style DOM libraries.

| improve this answer | |

There's two issues here then:

  1. Are canonical answers in general bad? My Answer: no, I don't think so when properly used; there's nothing wrong with redirecting "give me teh codez" questions of a certain type to "here's how to give yourself teh codez".

  2. "This particular canonical answer is bad, or at least not an answer to my question" is a real problem. Without wanting to argue over the validity of particular questions then yes if you're being pushed to an inappropriate canonical question and answer, or if the quality of that answer is poor then yes you should push back against that.

| improve this answer | |

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .