Remote
Backend Engineer
About this role
About Jan Jan is an open source AI assistant built by Menlo. It began as a private, offline alternative to ChatGPT and has grown into a general purpose agent that runs open models locally on your own hardware, or connects to frontier models like GPT, Claude, and Gemini when you want them. Give Jan a task and it works on its own until the job is done. Built in public around one belief: AI should be open, private, and answer only to you.
The Role We are building Jan Router, an enterprise LLM gateway. Think OpenRouter, but for organizations: one gateway in front of many model providers that routes traffic intelligently and gives an org a single place to manage keys, budgets, quotas, usage, and billing. You will bootstrap it from scratch and own the architecture, the first production deploy, and everything from the request proxy to the billing ledger. This is a lean, agent-native setup.
Rather than leaning on a large team, you will orchestrate coding and execution agents to ship at team-scale velocity, while keeping the hard backend judgment to build a billing-grade gateway that does not fall over. The router is your primary charge, but not your only one: you will also contribute to the Jan Agent, wiring agent and tool traffic through the same routing, metering, and controls. You are fluent in trade-offs.
You know which parts can ship fast and be hardened later, and which parts, like the money path and auth, have to be right the first time. This role suits a Backend or Software Engineer ready to open a new chapter: working closely with a frontier research team and building real AI services from the bottom up, starting at the model level. What You'll Do - Build the core gateway: a unified, OpenAI-compatible API in front of multiple providers (OpenAI, Anthropic, Google, plus self-hosted and OSS models).
- Own provider routing and reliability: load balancing, automatic failover, and cost and latency-aware routing. - Build billing and metering that is correct, not approximate: per-request token accounting, usage ledgers, cost attribution per team, user, and key, budgets, and spend limits. - Ship org controls: API key management, per-team and per-user quotas, rate limiting, and RBAC. - Handle streaming and performance: low-overhead proxying, streaming responses, connection handling, and caching where it helps.