How do you decide which new language models to add?

Written By Will (F&F Dev)

Last updated 5 months ago

We are always evaluating new language models as they come out so that we can deliver the best experience to you.

Here’s how we think about adding new models:

  • Is the model fast enough?

  • Is the model cheap enough?

  • Does the model pass our tests?

We run a TON of tokens through each message in Friends & Fables. When you send a message, the tokens are not just the length of the message you send, but a ton of context that we send with it (game state, world context, character sheets, etc). For this reason, we have to choose models that work in both speed and cost.

Many of the latest and greatest frontier models are expensive and slow. While they are smarter, they won’t always be the best choice because they are too slow and will be too expensive. These models are generally more suited for low volume + high value tasks, such as business reporting or document generation, but are not as suited for our use case which is high volume.