TypeSafe AI recently unveiled Jev, not another chatbot like Claude, ChatGPT, etc. But a smart, fast decision model that your code can reliably use to perform actions, based on input. In this blogpost, I'll go over what exactly Jev is, how it works, and why it's something to keep an eye out for.
What Is Jev?
As stated before, Jev is a decision model. Instead of generating text, it answers questions about your input with typed results and calibrated probabilities. Jev doesn't do anything with those results itself, your code does. Jev can answer three kinds of questions:
- choice: give Jev multiple options, Jev picks one and returns its confidence, along with the probability of each option.
- noul: give Jev a yes or no question, Jev returns a floating point number between 0 and 1, the probability that the answer is yes.
- score: give Jev an ordered scale (for example 1 to 3, how frustrated is this customer?), Jev returns where the input lands on that scale. The score can be fractional, and comes with probabilities for each level.
Let's start with an example.
Let's say you receive an email about a product, and you want to figure out whether it's a complaint, a question about the product or another happy customer. While you could just prompt an LLM with this and ask them what it thinks the customer wants to communicate, this requires creating a solid prompt, and even if your prompt is rock solid, it can still output a broken response, or hallucinate.
We can't work with that.
Jev takes a different approach. Instead of providing a human readable response, in text format, what most LLMs are built to do, you can give Jev a choice question with 3 options: is the customer complaining, happy, or asking a question. Then, when you provide the actual input, Jev will return which option it picked, plus the probabilities of each. And based on those probabilities, our code can quickly perform an action (flag it, respond to the email, etc.).
Jev Vs. LLMs
"Well, why not use a chatbot for this?"
I mean you can, actually. That's what most people and companies are doing right now. Though with the introduction of Jev we might actually want to steer away from classic LLMs, and adopt something like Jev for actual logic handling, where we just need a simple response, like a boolean, and not an entire paper of 50 pages talking about why our customer is angry with us.
LLMs are quit prone to creating mistakes, or hallucinating since they're not really built for this purpose: they are made to take a problem, or prompt, provided by the user, do some shenanigans behind the scenes and provide a human-readable response. But a computer is not a human. It only understands 1s and 0s.
How To Use Jev?
Using Jev is very straightforward. You can always expect a correct, not broken result, so no need to use a try-catch:
...That's it, it's that simple! No need to write a giant prompt, and no need to sanitize and verify an LLM's response; it just works.
Behind The Scenes
Most LLMs generate text one token at a time, where every token depends on the ones before it. Jev doesn't work like that. You send it an input (text or JSON) along with a set of questions, and it reads the input once and produces every answer in a single parallel pass, as probabilities over the options you defined. Nothing gets written word by word, and one answer never becomes context for another. This makes it so Jev is MUCH faster than standard LLMs and its output is way more consistent and reliable.
To make those probabilities trustworthy, TypeSafe trains Jev with a method it calls RLCD, short for Reinforcement Learning for Calibrated Decisions. The idea is calibration: when Jev says 80%, it should be right about 80% of the time. That's what lets you set thresholds in your own code, like acting automatically above 0.9 and sending anything below 0.5 to a human. This way you are still in control, which is something standard LLMs can't give you reliably.
Use Cases
So what can we as developers do with this tool? Automation. Specifically, automating things that are unpredictable or dynamic.
Let's imagine we have a small CRM system. It works great, the client is super happy, but every now and then we receive an error message. Since we have a lot of projects to work on, having to figure out the problem while you're already breaking your head on another just causes extra stress and costs time.
The solution? Every time an error occurs, we send it to Jev. We can then ask what exactly the error is (is this likely a bug in our code, is this likely a complex issue that needs human intervention, how urgent is it, etc.) and let it generate us a response. Based on the response we then get we can instantly take action (give a human a notification about this error, explain what the error is, or even prompt an LLM to resolve this issue).
This way, we don't have to manually do everything ourselves, and we can use Jev to automate the boring, dynamic, unpredictable work for us, while we continue working on new projects that generate revenue.
Let's Talk Money
If you compare it to a classic LLM, you might think it's quite expensive to prompt this thing. Actually, it's cheaper, by a landslide: about 20 - 400 times cheaper, wowza!
How come it's cheaper? While an LLM writes text for itself behind the scenes in order to provide you an answer, Jev just outputs predictable, structured responses. The tasks are also much, much simpler and specific. It does not generate natural-language tokens or "think out loud" with conversational filler, output tokens are thus basically free.
Incredibly affordable for a smart if-else engine!
Wrapping Up
This is as you might have noticed not something for the general public: this is where LLMs dominate and rightfully so. They provide a human-readable interface for responses.
Jev is something for the developers. It lets you take something unpredictable like an email from a customer, and turn it into something you can actually properly and reliably act upon, with no need for human intervention.
Definitely something worth taking a look at!
Happy coding!
