... when looking at the Summary of Accounts in a user's profile, the badges for all sites use the icon for the site that the viewer is viewing the page from at that time.
There aren't individual files for each site's favicon, they are all in one file. Some sites use one style of badge for all three and some have three different styles.
The favicon for each site is stored in one file called https://cdn.sstatic.net/img/favicons-sprite32.png?v=6703ade32c17 (that name may vary, with updates), a file that is 32 pixels wide and 13572 pixels long (13572/(32+(2*2))=377 icons, for main/meta and sites with no Meta - that's an approximation, I know there's ~175 sites sites, I'm not counting all those icons).
Here's a crop of the favicon file, turned sideways, to give you an idea of its contents; due to its enormous length it's difficult to view:
Notes on reimplementation: The badges on some sites look different based on whether they are bronze, silver or gold; so that increases the storage required by double; instead of a single style for all three.
Ultimately, with a 32*4 (128) bytes wide file, 13572 pixels long, all the favicons and badges could be available for all the sites. The favicons-sprite32.png file is 231kB, so 231*4=924kB bytes for everything.
That's what it would take to implement your suggestion, that and an update to the code to chop up the file.