22

Since the chat system has received a bit more love recently (thank you @balpha), it would appear now is the time to make any suggestions regarding additional functionality.

Chat is still an important part of the system (albeit it secondary) and has an emphasis that's it public, so should be available for all - but is has no publicly available "real" archive as such.

I have looked at:

However, they appear to have received little attention from the dev. team. While technically it's quite easy to crawl the transcripts of large rooms, there's still a lot of network traffic to/from the SE network to do so and too-oft repeated requests may well trigger some throttling/ignore or otherwise warning.

Instead of using websockets or scraping transcript pages, could it be possible to make a room’s transcript available as part (or separately but similar to) of the SE data dump process?

Improve this question
4
  • that would make for a useful feature indeed. thanks for asking this question. the SE data dump of transcript is a good idea so one could run custom queries against transcripts content
    – user270320
    Sep 14, 2014 at 14:23
  • -1 , chat is where I hid the code necessary in order to find the end of the internet rainbow. This feature request would make it easier for others to find this treasure and could possibly use it to exterminate the unicorns and/or steal all the unicoins.
    – CRABOLO
    Sep 14, 2014 at 15:31
  • 1
    @lostsock in fact it is already easier. it is fairly easy to build something like this client side by iterating over transcript elements and building a sort of "queriable" graph or database structure. but the great question Jon Clements posted will be helpful to avoid user's hassle to do it themselves. and that's just one reason in the middle of many
    – user270320
    Sep 14, 2014 at 15:57
  • 1
    Given that this is unlikely to happen at this point, it is something we could create and share. Initially creating it would take a considerable amount of time, as you can only request up to 500 messages at a time (and, I believe, need to individually request the full content for each message which exceeds the default maximum character length). However, once created, it could be maintained by listening to the chat WebSocket, with periodic re-fetching of the newest content to verify the WebSocket connection didn't miss anything. Obviously, an actual public data dump would be much easier.
    – Makyen
    Jun 29, 2020 at 19:16

0

Reset to default

You must log in to answer this question.