enter image description here

In the first result, "test this" appears later in the question, and is not displayed. In the second result, "test this" appears in the excerpt, since it's already in the excerpt from the unquoted query, but it is not highlighted.

Now, search generally ignores certain common words like "a", and "this". But here, when the ignored words are in quotes, the search is functioning, but the excerpted results seem to be broken.

  • 1
    Note that "this" is an ignored word. It seems that for exact matches without such ignored words, such as "first result", the highlighting is fine. (That said: exact phrase searches will often include ignored words, so this certainly is an annoying bug.) – Arjan Jan 27 '13 at 16:02
  • Thanks @Arjan! I've updated the question. – HodofHod Jan 27 '13 at 17:12

We limit the excerpt length; your term appears in the second result because that excerpt length is long enough to encompass both appearances, but the distance is too far in the first. It's behaving as-design for the layout. If you search for 20 terms for example, we can't reasonably show the entire body of the question to highlight them all.

The highlighting and excerpt selection is done entirely by elasticsearch, this would have to be fixed within that, or in Lucene underneath. When search phrases include stop words then elasticsearch does not highlight the other words from such exact search either.

  • 1
    But search often clips parts of the results to show later matches. See for example meta.stackoverflow.com/search?q=lucene.net+network where the middle of your q is clipped and an ellipsis unseated so that it can show a later match. Also, when the quoted part does make it into the excerpt, shouldn't it be bolded? – HodofHod Jan 27 '13 at 19:03
  • @HodofHod - In your case it's not bolded because it's ignored. Again, you're searching for stop words which are very often used and therefore rarely a relevant filter of results...that's why stop words exists. We don't clip in your comment example, those are 2 snippets, it warrants 2 (the max) because the second had a relevant non-stop-word result. It's not me making this decision, it's the elasticsearch implementation, which I happen to agree with. – Nick Craver Jan 27 '13 at 19:08
  • 1
    this would make sense if the term wasn't in quotes. But how is it a good thing that a search for "network" has bolded results, while a search for "when the network" does not? – HodofHod Jan 27 '13 at 19:43
  • @HodofHod - because "the" is very rarely a relevant word (as is "when") - and it's a stop word in every search engine that has English stop words. Did you read the stop words link I provided you? – Nick Craver Jan 27 '13 at 19:46
  • 3
    Yes. I understand what they are and how they work. What I don't understand is why you think the best solution is not the one that, say, Google uses: google.com/search?hl=en&tbo=d&q=%22when+the+network%22 This is obviously only applicable to quoted phrases. The SE search correctly searches for the phrase, including stop words. Its only the excerpt that ignores them. – HodofHod Jan 27 '13 at 19:52
  • @HodofHod - we're doing what elasticsearch and lucene will do, if you want to submit a patch to either open source project you're welcome to do so. When comparing us to google keep in mind: we have 2 search nodes, they have nearly a million. We have spent weeks on search, they have spent 18 years. Search is not our focus as a company, it's their #1 priority. 2 people have worked "heavily" on search here, they have thousands of the world's top PhDs on the problem. I have no problem admitting they'll likely always be better at some or all aspect of search compared to us. – Nick Craver Jan 27 '13 at 21:45
  • 2
    I didn't mean to say that I'm expecting you to match Google overnight, or even ever. I'm pointing out a particular behavior that I thought could be improved. It seemed strange to me that the search was properly limited to the quoted phrase (including the stop words), but that the excerpts did not reflect this. I was surprised that you seemed to think that the behavior was desirable, so I brought Google as an example. If SE has no control over this behavior, then perhaps I'll bring it up over there. :) – HodofHod Jan 27 '13 at 21:54
  • 2
    I'm with @HodofHod here. Reading the above gave me the idea Stack Exchange is saying it's not highlighting even the already visible matches on purpose. To me, that would be the wrong choice. If it's simply a limitation, or if time is too limited for a better fix, then why not edit that into the answer? (And maybe even choose status-wontfix over status-bydesign?) – Arjan Jan 27 '13 at 22:00
  • 2
    @Arjan - to be clear: the highlighting and excerpt selection is done entirely by elasticsearch, this would have to be fixed within that or lucene underneath. It's not status-wontfix because we have no control over it, "fixing" highlighting by adding another layer on our side is asking for trouble, it shouldn't be fixed with a bandaid. I'm not disagreeing it could be better, this is somewhere around item #2,109 of search improvements I'd make if I had the time. – Nick Craver Jan 27 '13 at 22:07
  • Ok, agreed and understood, and I edited that into the answer. (And not only to change my vote ;-)) – Arjan Jan 27 '13 at 22:13
  • @Nick. Gotcha. Upvoted and accepted. – HodofHod Jan 27 '13 at 22:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .