AI API Relay

Fix its nature first, because that decides whether you need licences, risk controls and paperwork:

1. Your own relay: one entry point across your devices and apps. Least compliance pressure
2. Shared with a small group: friends or a small team. Needs key distribution and quota control
3. Sold to others: the API as a product. This is the heavy one — business registration, invoices, refunds, complaints. Think hard before doing it as an individual

Three things to settle outside the technology:

• Does the upstream's terms allow forwarding? Most providers forbid public resale
• Who carries the cost, and who covers an overrun? Set a hard ceiling
• What are you liable for when something breaks? Write the disclaimer

What this site recommends: do the first two as an individual and get "controllable, measurable, stoppable" working before thinking commercially. Don't take money on day one — the responsibility afterwards is ten times harder than the technology.

Cost first: unit price × expected call volume, plus the server and traffic. Most people overestimate revenue and underestimate abuse.

Option A: write the smallest relay yourself — about 60 lines, maximum control

import express from 'express';
const app = express();
app.use(express.json({ limit: '1mb' }));
app.post('/v1/chat/completions', async (req, res) => {
  const upstream = await fetch(UPSTREAM + '/v1/chat/completions', {
    method: 'POST',
    headers: { 'Authorization': `Bearer ${UPSTREAM_KEY}`, 'Content-Type': 'application/json' },
    body: JSON.stringify(req.body)
  });
  // streaming: pipe it through; otherwise forward the json
  upstream.body ? upstream.body.pipe(res) : res.json(await upstream.json());
});

Option B: an open-source gateway — try this first
Projects in this family ship with multi-channel management, key distribution, quotas and rate limiting, usage statistics and a web admin UI. Install and go, rather than building a wheel.

Four things that are not optional:

• The key lives only in server-side environment variables — never in the front end, never in Git
• Cap request body size and max_tokens, so a single request can't drain the credit
• Rate-limit per key: requests per minute plus a daily total
• Turn timeouts and upstream errors into a friendly message. Don't dump the raw upstream error on the user

Don't: expose an unauthenticated endpoint so others can use it "free and unlimited". It gets found and drained within days.

One key per user — that's the floor. Share a key and you'll never find out who caused the incident.

Minimal tables (SQLite or a JSON file is fine):

keys:  key_hash | name | quota_total | quota_used | rpm_limit | enabled | created_at
usage: ts | key_hash | model | in_tokens | out_tokens | status | latency_ms

Four capabilities you must implement:

• POST /admin/keys issues a key — return the plaintext once, store only the hash
• Add to quota_used after every call; return 429 once the quota is gone
• /admin/usage shows usage per key, by day and by model
• enabled=false stops one key in one action. That's your emergency brake

Fix the metering definition: record usage.prompt_tokens and completion_tokens exactly as upstream reports them. Never estimate tokens yourself — change models and your tokeniser is wrong.

Alerting: notify yourself when the day's total passes 80% of budget, by chat webhook or email. Discovering it on the invoice is too late.

Access control:

• Every /v1/* endpoint validates a key; missing or invalid returns 401
• Rate-limit per key on both requests per minute and concurrent requests
• Admin endpoints go on an internal network or a separate port, never on 80/443, behind a second password

What the log must contain — this decides whether it's useful:
time | key | model | status | latency | input/output tokens | error

What the log must never contain:

• The full request body, which carries the user's private content — record length and a redacted head/tail fragment only
• The upstream key, or user keys in plaintext
• Anything that could re-identify a person

Retention and compliance: 7–30 days is enough for debugging, then prune. State in the docs which data you record and for how long.

Handling an anomaly: a key's volume spikes → enabled=false to stop the bleeding → read the log → only then decide whether to re-enable. The order isn't negotiable.

Launch:

• Domain plus HTTPS, with Nginx reverse-proxying to the local service
• Set the request body limit and timeouts in Nginx — AI requests can run long, so a read timeout of 120 s or more
• Add a /healthz endpoint and point an uptime monitor at it

Three tables decide whether this survives:

1. Usage: tokens and request counts by day, key and model
2. Cost: usage × unit price, reconciled against the actual invoice. A gap over 10% means your metering is wrong
3. Quality: error rate, average latency, timeout share

Events to instrument (into data/conversion.json): api_call, api_error, quota_hit. Rising quota_hit means either abuse or quotas set too low.

Monthly review, 20 minutes: cost versus budget, error-rate trend, top three consumers. If cost exceeds expectations two months running with nothing to show for it, scale down — the most dangerous shape for a relay is "someone else uses it, you pay for it".

Compliance to close: the API docs state the usage limits and the disclaimer, and you disable abuse on sight.