During development of my StackUnderflow script I spotted possible bug in SO markup. In my script I need to evaluate user reputation so I look for user-details <div> to get it. And this <div> usually looks like this:

<div class="user-details">
    <a href="/users/......./foo">foo</a><br>
    <span class="reputation-score" title="reputation score 1234" dir="ltr">1,2k</span><span title="4 bronze badges"><span class="badge3"></span><span class="badgecount">4</span></span>

in this case, user currently named foo holds reputation score of 1234. If your script needs to figure out user reputation you should rely on the title attribute which is always in "raw" numeric form (i.e, 2000), not the span text, which is formatted string (i.e. "2k"). So I try to reach this title thing and figured out that for some users it does not contain reputation score, however it looks like it should. In markup it looks like this:

<div class="user-details">
    <a href="/users/4712527/siva">siva</a><br>
    <span class="reputation-score" title="reputation score " dir="ltr">5</span><span title="4 bronze badges"><span class="badge3"></span><span class="badgecount">4</span></span>

note title="reputation score " - reputation "raw" score is completely missing, yet there's trailing space, so it looks reputation score should be there.

Another reputation related bug (again, I am talking about SO) lurks in part that holds my own reputation. The markup looks like this:

<span class="reputation" title="your reputation: 32,479">

Please take a look at title attribute - it reads 32,479 instead of expected: 32479. I also consider this a bug as content of title attribute should stay consistent.

If it is a feature, not a bug, I am open to hear why so :)


On User profile reputation looks even worse - title attribute reads reputation not reputation score, there no raw score in title, score is formatted the same way "my reputation" on "top bar", so "k" but with coma. There's bunch of spaces around too.

<div class="reputation" title="reputation">
    32,504 <span class="label-uppercase">reputation</span>
  • Wouldn't it be easier to pick up all this information from the API based off the user ID – ben is uǝq backwards Aug 15 '15 at 18:05
  • I think it wouldn't because all this data I need is already in the page markup (as mentioned in my post). So I do not need to do any API call. The main issue is markup is messed so this data is also served in messed manner, requiring some unnecessary attention while obtaining, so all I would like to see is little cleanup – Marcin Orlowski Aug 15 '15 at 18:10
  • 2
    It might be on the page but I'm not sure you can call it a bug that it doesn't conform to an easy standard for you to use - that's what the API is for... Generating the page with the data in the format it needs to be displayed in seems relatively sensible... even though it might be a bit "dirty" from a professional standpoint. – ben is uǝq backwards Aug 15 '15 at 18:35
  • 1
    I complain on inconsistency. Since this data is put there, it would be nice to have this unified. That's what's my main complaint. – Marcin Orlowski Aug 15 '15 at 18:42

Yes, non-uniform UI elements can be confusing to users and a pain for page-scrapers. That's one reason that Stack Exchange helpfully supplies an API.

From a UI perspective, it would be best if:

From a page-scraper's perspective, you must deal with what you have, and check for changes constantly. It does not seem hard in this case.

The following jQuery should be close to what you need for the current SE pages:

var repNodes        = $(".reputation-score, .reputation");
repNodes.each ( function (J) {
    var reputation  = -86;  //-- Default error value
    var stripped    = this.textContent.trim ().replace (/[^\dK]/ig, "").replace (/K/i, "00"); // *Normally* have 1 dec place precision when K is used.
    reputation      = parseInt (stripped, 10)  ||  reputation;

    //-- If present in title, the rep value will be more precise.
    if (this.title) {
        stripped    = this.title.replace (/\D/g, "");
        reputation  = parseInt (stripped, 10)  ||  reputation;
    console.log (J, "|  Reputation: ", reputation);
} );

You can test that code by pasting it into the browser console while on a Stack Exchange page.

  • I do not need any help scrapping current SO and got all these oddities handled already. I just filled bug report here, but thanks for the efforts :) – Marcin Orlowski Aug 16 '15 at 9:29

The "reputation in title" is only populated if the rep shown in the <span> had been abbreviated. So if there is no number in the title, just use the content of the <span> directly.

  • Any chance to not make any exception here and simply put reputation in title always? – Marcin Orlowski Oct 4 '15 at 7:11

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .