Platform · 2026
A private “OpenAI” for an external team
An external team uses the lab’s models with their own key, as if it were the OpenAI API, with limits, traceability and no per-token fees.
- tokens consumed by a single user
- ≈2.24 B
- tokens in half an hour at peak
- 14.7 M
- stable model roles
- 8
- per token
- €0
The approach
Several friends and collaborators wanted to use powerful models with their own agents, but without depending on a paid API or sending their data elsewhere. The solution was to open up the lab to them in a controlled way.
How it works
- An OpenAI-compatible endpoint, published with TLS. Any tool that talks to OpenAI works by changing only the URL and the key.
- Eight stable roles (
spark,coder,embeds,tts…). When I change the model behind them, nobody has to touch their configuration. - Accounts, teams and budgets, with the list of allowed models for each team.
- Usage dashboard per person and model, reconciled with the gateway logs.
- Chaining for a client with their own server: their gateway uses mine as just another provider, with a restricted key.
The numbers
One of the users, with their own Hermes agent, consumed about 2.24 billion tokens, with peaks of 14.7 million in half an hour and prompts of up to 180,000 tokens. With a commercial API it would have been a hefty bill; here the cost is electricity.
What I learned
- Timeouts have to be designed. A 200,000-token prompt takes time to process, and cutting it off at 30 seconds only triggers retries.
- Real IPs are visible at the edge, not at the gateway. You have to look for them in the right place.
- No hidden fallbacks. If the local model fails, the user knows.