New Programming Languages versus Rust / C++
Imagine asking a voice assistant for directions. You can judge the wait before it says “turn left” long before it has finished the sentence. At Inworld, that interval became a design problem. The speech team worked with Modular, the creator of Mojo, to run its voice model on an inference platform called MAX. One small part of the job explains the choice of language particularly well. Mojo let the engineers write their own silence detector and run it on the graphics processor, where the audio was already being processed.
The creator
Modular’s language and compiler team
- Where the idea came from
- Modular was building software that had to work across several kinds of processor. Its designers wanted to express a model operation once and still be able to decide how data would move through a particular chip.
- Earlier work
- The design draws on experience with LLVM and Swift. In a recorded interview, the project’s co-founder explains why the ease of introducing a new language into an existing application mattered as much as the syntax of the language itself.
Why Inworld put the silence detector on the GPU
Inworld’s synthesizer has two main stages. A speech language model predicts a stream of tokens, compact numbers representing sound. An audio decoder turns those tokens into samples that can be played through a loudspeaker. If the first stage runs ahead while the second is busy, a faster prediction alone may do little for the person waiting to hear a word. Both stages have to keep pace as more people request speech.
The teams moved the model description from PyTorch to the Python interface for MAX. Within that system, Mojo let them add a custom kernel, a routine executed directly by the GPU. They used one to identify silence while processing the output on the device. This is the specific engineering choice that Inworld documents. Its account gives a reason to extend MAX with Mojo. It reports a platform comparison with vLLM, and gives no account of a selection trial between Mojo, Rust and C++ for that detector. Inworld’s engineering account and test conditions.
Who owns the next fragment of sound?
Consider a temporary buffer holding a fragment of audio. One routine writes the samples, another reads them and a later request wants to reuse the space. Mojo gives a value one owner at a time. A routine may borrow access or take ownership. When the owner’s lifetime ends, the value is destroyed. A type that manages an external resource must still implement the appropriate cleanup. For a large buffer, the practical question is whether handing the value to another routine transfers the existing storage or requires another copy.
Rust users will recognise the distinction between a move and a borrow. In C++, an owning container can also release storage when it is destroyed, and its contents can be moved. A separate issue arises on a GPU. A host routine may finish issuing work before the device has finished reading its input. The program must respect that completion point before reusing the buffer. A rule about ownership and a measurement of device completion answer different engineering questions. Mojo ownership · Lifetimes and references · Rust ownership · C++ resource management.
From a model description to instructions on a chip
A Mojo application can be compiled ahead of time into a native executable. For an AI service, the unit a developer supplies may instead be a single operation inside a graph. The graph specifies which calculations depend on one another. A Mojo kernel specifies how one calculation runs. The compiler can specialise it for facts known before execution, such as a data type or a tile size. The platform then schedules work and arranges access to memory. This division lets an engineer change one expensive operation while keeping the surrounding model expressed in Python. Native compilation · Custom operations in MAX.
When doubling the matrix changes the winner
A study presented at RTNS 2025 ran equivalent programs on an isolated performance core of an Intel Core i9-13900H under Ubuntu 24.04. Its authors compiled the programs at three optimisation levels and reported means over 20 runs. The figure below reads the most aggressive setting, -O3, for Mojo, Rust and both C++ compilers. The numbers are approximate readings of the published bars. The paper supplies the variation across runs and additional C baselines. Original figure 2 · Experimental programs.

| CPU workload | Mojo | Rust | C++ Clang | C++ GCC |
|---|---|---|---|---|
| Mandelbrot, size 512 | 259 | 382 | 394 | 396 |
| Matrix, 256 × 256 | 14 | 28 | 30 | 27 |
| Matrix, 512 × 512 | 350 | 219 | 239 | 219 |
| Text search, larger input | 45 | 5.5 | 7.1 | 2.8 |
Mojo finished first in the arithmetic loop that draws the Mandelbrot set. For the smaller matrix it also finished ahead of Rust and both C++ builds, although the paper’s C program compiled with Clang finished sooner still. Doubling each matrix dimension reversed Mojo’s position. For text search, the paper summarises the gap as around four to eight times. Its larger-input bars at -O3 imply about six times the C++/Clang time, eight times the Rust time and sixteen times the C++/GCC time. The choice of compiler matters even within C++. The experiment measured those programs on one CPU before Mojo 1.0. It leaves the speed of the current compiler, and the latency of Inworld’s GPU service, open to separate measurement.
Who maintains the language?
Qualcomm announced the completion of its acquisition of Modular in July 2026 and continued the Mojo and MAX product lines. In its August announcement of Mojo 1.0, Modular counted nearly 200 people who had contributed to the standard library. The compiler source and public release history make it possible to examine how the language is changing. Those contributions describe participation in development. Inworld supplies the distinct evidence of use by a team operating a speech service. Qualcomm’s announcement · The 1.0 release and contributor count · Public source.
What this language can contribute to artificial intelligence
Mojo gives a model developer a way to add an operation the serving software does not already provide. Modular says it writes its own CPU and GPU graph operations in Mojo. The developer can specify how neighbouring values are laid out in memory and how work is divided among processing units. When a model spends much of its time moving data, keeping intermediate results on the device is a practical reason to use that control. Inworld’s silence detector shows what such an operation can do for a deployed application. Modular’s account of internal use · How model developers extend MAX.
For its streaming service, Inworld reported that the first two seconds of audio arrived about 70% faster on average with MAX 25.4 than with its baseline implementation on vLLM 0.9.1. Both ran on the same NVIDIA B200. Scheduling, the decoder’s number formats and the overlap between operations changed as well. The experiment measures the experience of obtaining audio from the complete system. A test of the detector’s own contribution would hold those choices fixed and time the same inputs with each implementation. Inworld’s result and comparison method.
This evidence concerns inference, the use of an already trained model. Inworld also describes training its voice model, but identifies Mojo specifically in the system that serves the resulting speech. Ownership rules can help an engineer reason about the lifetime of a buffer. Whether a voice model resists impersonation or reproduces sensitive material is a question about the model and its safeguards. Neither the speech experiment nor the CPU timings evaluated those outcomes.
The measurement a listener would notice
Start a clock when the request arrives and stop it at the first playable sound. Repeat with short and long passages, quiet openings and many simultaneous listeners. Check whether the detector preserves a deliberate pause and the first soft syllable as well as it handles clear speech. Then compare the median wait and the wait for the slowest listeners at the same output quality. That experiment would connect the language decision to the promise a voice service makes each time somebody presses play.
Liked this post? Follow this blog to get more.