Obtaining full chat transcripts

Question

How do I download all chat messages or those from a particular room (over all time)? A script may be able to do it, but is there a more convenient way?

Is there something at https://data.stackexchange.com/ all I see are the QA sites.

Answer 1

In absence of any formal API, one can just obtain them by executing a POST request to http://chat.stackexchange.com/chats/<room_id>/events with form parameters:

• mode: 'Messages' (there might be other modes, haven't checked.)
• fkey: A valid user's fkey is required.
• (optional) msgCount: integer, defaults to 100, capped to 500
• (optional) before: message_id to to get messages prior to. If missing, it gets the most recent msgCount messages

Example Jupyter Notebook.

The returned value is a mapping with the following keys:

• ms: Time it took for the query to run?
• time: Time since Aug 4, 2014 in seconds???
• sync: UNIX epoch time (mislabeled, or used to "sync" correct server-time to JS client?)
• events: a list of up to msgCount messages (may be fewer if there aren't that many.) Each is a mapping with the following keys:
  • content: HTML-formatted message; may be missing if (removed).; Begins with "" if there's more content which is not contained in the event record (i.e. it was a long message; the full content, if desired, has to be individually fetched).
  • event_type: 1...only ever seen that. Other event_type values are used within the WebSocket based communication for an active chat page. They can be seen from a chat room's main page through the use of CHAT.addEventHandlerHook().
  • message_edits: Integer, may be missing if not edited.
  • message_id: Integer; unique across all rooms on the chat server, but not between servers.
  • message_owner_starred: Boolean; If true, the message is pinned. Normally not present if the message is not pinned.
  • message_owner_stars: Count of pins. It's unclear if this can be anything other than 1 or not present. Normally not present if the message is not pinned.
  • message_starred: Boolean; If true, the message has been starred. Normally not present if the message is not starred.
  • message_stars: Count of stars for this message. Normally not present if the message is not starred.
  • moved: Boolean; If true, then this message has been moved from another room into its current room. Usually missing if the message has not been moved.
  • parent_id: Integer; Message ID to which this event is replying. May be an actual reply or just an @-ping. If it's the result of an @-ping, then this is the message ID of the most recent message by the pinged user. Usually missing if the message is not a actual or @-ping reply.
  • room_id: Integer
  • room_name: String; This is not present for events received via AJAX POST, but is present for events received by event listeners added via CHAT.addEventHandlerHook().
  • show_parent: Boolean; If true, then this is an actual reply and the replied-to arrow is shown. Usually missing if the message is not a direct reply.
  • time_stamp: UNIX epoch, second-precision.
  • user_id: Integer, Chat.SE/Chat.MSE/Chat.SO user. Chat.SE ID is not related to a specific site or network ID; Chat.MSE and Chat.SO user IDs usually match the profile ID of the user on MSE and SO respectively, but that's not required. May be missing if the user_id no longer exists.
  • user_name: Name associated with user_id when the message was posted. If the user changes their username later, this does not update.