· via dev.to (home feed)
SiMa.ai Palette 3.0 lets Modalix edge chips run multiple AI models and containers at once
SiMa.ai's Palette 3.0 software update lets one Modalix edge AI chip run several models and Docker containers concurrently, with company-reported gains of up to 3.5x faster token generation.
SiMa.ai has released Palette 3.0, a software update for its Modalix edge AI processors designed to let a single chip serve several AI workloads at once rather than act as a single-purpose accelerator. According to the company's announcement, as covered by dev.to, the release runs language, vision and vision-language models concurrently — including GPT-OSS-20B — and targets systems such as robots, drones, industrial cameras and autonomous vehicles.
One accelerator, many workloads
Palette 3.0 allows multiple models and Docker containers to draw on the same Modalix accelerator resources, with applications assignable to different priority levels and separate hardware engines operating in parallel. SiMa.ai demonstrated a language model running alongside three vision-language models, plus six Docker containers handling jobs that included object detection and 3D pose estimation. The company also reported that one Modalix chip sustained 60 simultaneous 720p object-detection streams at an aggregate 600 frames per second — a density that matters where dozens of cameras run continuously and extra servers would add cost, space and power draw.
For system designers, the practical change is consolidation. Rather than dedicating a processor to each AI function, one chip can be split across several services with priorities adjusted as conditions change. SiMa.ai's community release notice says existing Modalix customers can adopt the update by recompiling their applications, meaning hardware already in the field could gain new functions without being physically replaced — a relevant point for fleets of robots or cameras spread across large facilities.
Benchmark numbers come with caveats
The most eye-catching figure, up to 3.5 times faster token generation, comes from speculative decoding, a technique in which a smaller process drafts tokens that a larger model then verifies. SiMa.ai's technical paper, which examined the EAGLE3, DFlash and Gemma 4 MTP approaches on Modalix hardware, reported token-generation gains between 2.21 and 3.49 times versus conventional decoding, along with up to 2.06 times faster actionable responses and 51% lower compute energy per request.
As dev.to notes, the top-end result reflects the company's own testing on one model under favorable conditions. A more product-like evaluation, a 900-request automotive-assistant workload, produced the smaller range of 1.35 to 2.06 times faster response times. Darius Popa of The Next Web drew the same distinction, and no independent benchmark or third-party assessment has been published. The energy figures also describe chip computation only; motors, sensors, cooling and networking can dominate the total power budget of a physical machine, though lower accelerator energy could still extend battery life or free thermal headroom.
Tooling and cost
The release also upgrades the Model Browser, exposing measurements for throughput, latency, power, memory and individual model layers. That visibility may prove as valuable as raw speed, since engineers building physical AI systems constantly trade response time against power draw and stream count — goals that often conflict. A developer might pair a small, fast model for obstacle detection with a heavier language assistant, for example, and tooling that shows those trade-offs reduces guesswork. On the commercial side, VentureBeat reported a $1,499 developer kit available in the United States and Europe, while production pricing remains undisclosed.
Why it matters
Edge systems live under tight memory, energy and thermal constraints, and they need predictable timing — a model that performs well in a data center can be too power-hungry or too slow inside a drone or beside moving machinery. If orchestration software can genuinely make one chip juggle perception, reasoning and conversation, companies could cut hardware counts, extend the life of installed equipment and keep more intelligence running locally without depending on cloud connectivity. The open questions are operational rather than promotional: how performance holds up when every workload is busy, how quickly lower-priority requests are served, and whether a time-sensitive control loop can tolerate the same delay as a conversational reply. Those answers will come from long-running deployments, not laboratory demonstrations.
- #edge-ai
- #ai-chips
- #docker
- #embedded-systems
- #robotics