9

I'm doing a research on the developers' QAs in Stack Overflow, I have checked Stack Exchange Data Explorer and data dumps but I didn't find number of followers of a tag:

There isn't any related data in "Users" and "Tags". Is there any way to access the data except manually fetching from Tags? (going one by one, hovering etc.)

0

2 Answers 2

6

If Shadow Wizard his guess is correct then this query gives you a javascript scraper that you can run in your browser to get the data you need.

select code
from 
(
select 1 as line , 'var si = [' as code
union 
select  top 10 -- remove this to get all tags
100, '''' + tagname + ''','
from tags
union
select 1000, '''''];'
union
select 1001, 'var i=0; 
var wi=window.setInterval(function(){ 
   if (i<si.length-1) {
     var url=''http://stackoverflow.com/tags/'';
     var entry = ''/subscriber-info'';
     var comp = url + encodeURIComponent(si[i]) + entry;
     var tag = si[i];
     $.get(comp, function(data){ 
        var ami = $(data).find(''span.tm-sub-info'');
        var c3 = ami[0].childNodes[2];
        var c4 = ami[0].childNodes[3];
        var txt = $(c3).text();
        var fol;
        if (txt.indexOf(''follower'')>-1) { 
            fol = parseInt(txt.split('','')[0],10);
        } else {
           fol = parseInt($(c4).attr(''title''),10);
        }
        console.log(tag + '' '' + fol);
     });
     i++;
   } else {
      clearInterval(wi);
   }
 },1000); ' -- lower 1000 until you get throttled
) as allcode
order by line

If run you get the following javascript:

var si = [                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       
'.a',                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            
'.app',                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          
'.aspxauth',                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
'.bash-profile',                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 
'.class-file',                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   
'.cs-file',                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      
'.doc',                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          
'.emf',                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          
'.git-info-grafts',                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
'.hgtags',                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       
''];                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
var i=0; 
var wi=window.setInterval(function(){ 
   if (i<si.length-1) {
     var url='http://stackoverflow.com/tags/';
     var entry = '/subscriber-info';
     var comp = url + encodeURIComponent(si[i]) + entry;
     var tag = si[i];
     $.get(comp, function(data){ 
        var ami = $(data).find('span.tm-sub-info');
        var c3 = ami[0].childNodes[2];
        var c4 = ami[0].childNodes[3];
        var txt = $(c3).text();
        var fol;
        if (txt.indexOf('follower')>-1) { 
            fol = parseInt(txt.split(',')[0],10);
        } else {
           fol = parseInt($(c4).attr('title'),10);
        }
        console.log(tag + ' ' + fol);
     });
     i++;
   } else {
      clearInterval(wi);
   }
 },1000); 

which in you can drop in the Developer Console of Chrome (hit F12) and hit enter. Every second (1000 milliseconds) it will fetch the subscriber info for a tag and output the number of followers in the console:

enter image description here

With almost 50,000 tags this will take a while to run to completion... but that is still under 6 to 8 weeks...

5
  • Nice but for all tags... another method is write a program to get 1290 pages of Tags then extract info from them, I think it will be faster...
    – C2121
    Feb 14, 2016 at 15:06
  • That will not work because the number you're interested in is not in that html payload @Mohamad
    – rene
    Feb 14, 2016 at 15:18
  • The numbers are like <span class="item-multiplier-count">113644</span> in html codes, I guess the run time to get and extract 1290 pages will be about 25 minutes (BUT if SO not block the IP)
    – C2121
    Feb 14, 2016 at 15:30
  • That is the times the question is asked and NOT the number you're looking for as indicated in your question. If you're interested in that 113644 you can simply query the tags table in SEDE because that number is in the count column.
    – rene
    Feb 14, 2016 at 15:49
  • Sorry I know I'm not looking for this, you're right it is frequency of a tag. Number of followers isn't in html codes, oh... AJAX!
    – C2121
    Feb 14, 2016 at 16:08
1

In the following paper, authors collected Stack Overflow's tags. The data is available here.

M. Yazdaninia, D. Lo, and A. Sami. "Characterization and Prediction of Questions without Accepted Answers on Stack Overflow", in Proceedings of the 29th IEEE/ACM International Conference on Program Comprehension (ICPC 2021)

1
  • 1
    You're one of the authors, right? Then you should state so explicitly, see the Help Center.
    – Glorfindel Mod
    Jul 2 at 12:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .