How to display Twitch emotes in tmi.js chat messages (#blogPost)

If you're streaming on Twitch, you might know that you can build custom stream overlays with web technology. Broadcast tools like OBS allow you to embed websites right in your stream. You can use the tmi.js library to send, react to and display real-time chat messages.

Today, I spent a ridiculous amount of time figuring out how to display Twitch emotes in my chat overlays and even started downloading all Twitch emotes onto my local machine... (don't do that!)

So, if you hit the same problem and you're wondering how to render emotes in your messages, this post is for you!

The code shown below is what you need to do to connect to Twitch from your web application. It uses websockets and worked out of the box for me.

const tmi = require('tmi.js');
const client = new tmi.Client({
  options: { debug: true, messagesLogLevel: "info" },
  connection: {
    reconnect: true,
    secure: true
	},
  identity: {
    username: 'bot-name',
    password: 'oauth:my-bot-token'
  },
  channels: [ 'my-channel' ]
});
client.connect().catch(console.error);
client.on('message', (channel, tags, message, self) => {
  if(self) return;
  if(message.toLowerCase() === '!hello') {
    client.say(channel, `@${tags.username}, heya!`);
  }
});

Tmi.js provides a typical event listener pattern. Whenever someone interacts with your channel's chat, the message event listener is called with several arguments: channel, tags, message and self.

You can use the message string and render it however you like.

The problem appears when people use Twitch emotes in your chat. A chat message like LUL SSSsss SirSad includes several emotes and it should be rendered as follows.

The questions are:

• How do you find out which words in a chat message are emote keywords?
• How do you replace these keywords?
• How can you access the emote images?

There are two important pieces that you have to know to solve this problem:

• the tags object includes an emotes property that provides the emote id and message positions
• all emote images are available under https://static-cdn.jtvnw.net/emoticons/v1/[emote_id]/2.0

The emotes property

Whenever a message is posted in the Twitch chat, the callback function is run with the message and tags argument. tags includes lots of meta-information about the user and the sent message. Let's have a look!

{
  "badge-info": null,
  "badge-info-raw": null,
  "badges": { "broadcaster": "1" },
  "badges-raw": "broadcaster/1",
  "client-nonce": "...",
  "color": null,
  "display-name": "stefanjudis",
  "emotes": {
    "425618": ["0-2"]
  },
  "emotes-raw": "425618:0-2",
  "flags": null,
  "id": "b8aafd84-a15d-4227-9d6b-6d68e1f71c2b"
  "message-type": "chat"
  "mod": false,
  "room-id": "250675174",
  "subscriber": false,
  "tmi-sent-ts": "1606591653042",
  "turbo": false,
  "user-id": "250675174"
  "user-type": null,
  "username": "stefanjudis"
}

The object also includes information about the used emotes. The emotes and emotes-raw property allow you to access the id and position of every used emote.

For a message consisting of the emotes LUL SSSsss SirSad, tags.emotes is the following.

{
  "46": ["4-9"],         // "SSSsss" on characters 4 to 9
  "425618": ["0-2"],     // "LUL" on characters 0 to 2
  "301544924": ["11-16"] // "SirSad" on characters 11 to 16
}

With this information, you can parse the incoming messages and replace the emote keywords with images.

The public emotes images URL

It's probably documented somewhere (I didn't find it, though), but now that you have the emote id, you can access every emote image in different sizes under the following URL.

https://static-cdn.jtvnw.net/emoticons/v1/[emote_id]/[size]

Example URLs for "LUL":
28x28
https://static-cdn.jtvnw.net/emoticons/v1/425618/1.0
56x56
https://static-cdn.jtvnw.net/emoticons/v1/425618/2.0
112x112
https://static-cdn.jtvnw.net/emoticons/v1/425618/3.0

With these two pieces (tags.emotes and the publicly available emote URL), you can replace all the keywords in Twitch messages with their images. ?

My solution

If you're curious, that's the ugly and not optimized code that I run in my local Twitch setup. It transforms a chat message string to an HTML string that includes emote image elements.

function getMessageHTML(message, { emotes }) {
  if (!emotes) return message;

  // store all emote keywords
  // ! you have to first scan through 
  // the message string and replace later
  const stringReplacements = [];

  // iterate of emotes to access ids and positions
  Object.entries(emotes).forEach(([id, positions]) => {
    // use only the first position to find out the emote key word
    const position = positions[0];
    const [start, end] = position.split("-");
    const stringToReplace = message.substring(
      parseInt(start, 10),
      parseInt(end, 10) + 1
    );

    stringReplacements.push({
      stringToReplace: stringToReplace,
      replacement: `<img src="https://static-cdn.jtvnw.net/emoticons/v1/${id}/3.0">`,
    });
  });

  // generate HTML and replace all emote keywords with image elements
  const messageHTML = stringReplacements.reduce(
    (acc, { stringToReplace, replacement }) => {
      // obs browser doesn't seam to know about replaceAll
      return acc.split(stringToReplace).join(replacement);
    },
    message
  );

  return messageHTML;
}

Edit: My friend Dominik pointed out that the above code includes a XSS vulnerability. People could paste HTML chat messages and a <script> tag would be executed on my local machine. ? In my application I use React and am transform the HTML to React components that are properly encoded. If you use this snippet above, make sure that HTML messages are not rendered in your application.

If you like, we see each other on Twitch. And let's hope google ranks this article well so that no other person has to download thousands (millions?) of emotes locally like I tried to do.