The short answer is: whatever language the other folks present in the room are comfortable speaking.
All else being equal, if you jump into a room and start posting stuff that isn't understandable to the majority of the users there, you're probably going to get some pushback. And as a practical matter, if those moderating the room (moderators, owners, 10k users) can't understand what you're saying, it stands a good chance of being deleted.
Long answer: chat is not a venue for private discussion, and most of these sites are dedicated to English-speaking content
There is no private messaging feature available to Stack Exchange users. Therefore, using a language that only you and one other person in a room understand is actually a bad thing purely based on the rules of the site. (Read the linked discussion for all the reasons for why that rule is in place)
But beyond that, the sites themselves are dedicated to English-language content, and Chat exists primarily to support them. With the exception of chat rooms associated with specific, non-English language sites, the majority of content in Chat should also be in English. I could see making exceptions to this in a few (specific and hopefully rare) scenarios, but by and large we're not set up to support moderating conversations in languages that the moderators, the SE staff, and the majority of users on the site do not understand. When there's any doubt as to the appropriateness of a conversation (for instance: it gets flagged) and it can't be understood, it should be deleted.