19

How do I download all chat messages or those from a particular room (over all time)? Some dopey script may be able to do it, but is there a more convenient way? Is there something at http://data.stackexchange.com/; all I see are the QA sites.

Trying to learn some data analysis stuff, figured I'd try to toy with a rather unique set.

8

In absence of any formal API, one can just scrape them by executing a POST request to http://chat.stackexchange.com/chats/<room_id>/events with form parameters:

  • mode: 'Messages' (there might be other modes, haven't checked.)
  • (optional) msgCount: integer, defaults to 100, capped to 500
  • (optional) before: message_id to to get messages prior to. If missing, it gets the most recent msgCount messages

Example Jupyter Notebook.

The returned value is a mapping with the following keys:

  • ms: Time it took for the query to run?
  • time: Time since Aug 4, 2014 in seconds???
  • sync: UNIX epoch time (mislabeled, or used to "sync" correct server-time to JS client?)
  • events...

events is a list of up to msgCount messages (may be fewer if there aren't that many.) Each is a mapping with the following keys:

  • content: HTML-formatted message, may be missing if (removed).
  • event_type: 1...only ever seen that.
  • message_edits: Integer, may be missing if not edited.
  • message_id: Integer, unique across all rooms
  • room_id: Integer
  • time_stamp: UNIX epoch, second-precision.
  • user_id: Integer, Chat.SE user. Not related to a specific site or network ID
  • user_name: Name associated with user_id when message posted (if user changes it later, this does not update)
  • Tiny addendum: user_id ... may be missing if user no longer exists (At least that's my guess why, see eg message id 35969027, room 118) – bertieb Feb 6 '18 at 15:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .