HTTP API Standard

How every HTTP API we build looks from the outside, and what makes sure it does.

MUST, SHOULD and MAY are used as in RFC 2119. MUST is security, correctness, data integrity and production reliability. SHOULD is performance and maintainability: break it only with a reason written in the merge request. MAY is an option a service takes when it fits.

The goal is that nobody has to remember this document. Each rule names the thing that enforces it: a shared library that does it by default, a CI check that fails without it, or a review item when neither can. A developer chooses; the library and the pipeline do the rest.

---

How the rules are enforced

Layer	What it does here	Where
This standard	Defines the expected behaviour	This file; playbook-rest-api for agents
Shared library	Does it by default, on echo and gin alike	aioz-common httpapi, httpapi/echoapi, httpapi/ginapi; plus apierr, httpsec, clientip, ratelimit, httpclient
CI	Fails or warns when the spec breaks a rule	swagger-verify (spec is fresh), validate:openapi (Spectral with templates/openapi/.spectral.yaml), lint, test
Review	Catches what neither can see	definition-of-done.md "New Endpoint", security.md
Observability	Shows whether it works	Per-route latency, status and failure-reason series from the stack; 304 rate on revalidated routes

Rule	Level	Enforced by
Success and error bodies	MUST	httpapi renders both; Spectral warns on an operation without the error schema
Status codes	MUST	apierr constructors; review
Pagination with a hard cap	MUST	Page (default 25, over 100 is a 400), List; Spectral warns on a list GET without limit
Strict request bodies and limits	MUST	Bind, Setup's body limit; review
Timeouts	MUST	Service template server config; httpclient; review
Cache policy declared on every GET	SHOULD	Cache/NoStore; Spectral warns on a GET without x-cache-policy
Sensitive responses never stored	MUST	Default no-store from httpsec; errors always no-store; review
Conditional requests	SHOULD	CheckETag; test
Idempotency	SHOULD	Review
Versioning and compatibility	MUST	Spectral path rule; review
OpenAPI spec	MUST	swagger-verify, validate:openapi
Naming	MUST	Spectral; review

---

URIs and naming

• A URI MUST name a resource with a noun, not an action: /users, not /create-user.
• Collections SHOULD be plural: /v1/companies/123.
• A sub-resource MUST be reflected in the path: GET /v1/users/123/posts/1.
• Paths MUST be lowercase, with - between words: /v1/payment-methods.
• JSON fields and query parameters MUST be snake_case.
• The HTTP method MUST match the action, and a safe method (GET, HEAD) MUST NOT change state.

An action that is not create, read, update or delete MAY be a sub-resource verb on POST when no noun fits: POST /v1/videos/123/publish, POST /v1/companies/batch-delete. HTTP DELETE carries no body, so a batch delete is POST /<collection>/batch-delete with {"ids": [...]}.

Versioning and compatibility

• Every path MUST start with its major version: /v1/... or /api/v1/.... The version is never a header.
• Within a version, changes MUST be backward compatible: add an optional field, an endpoint or an enum value a client can ignore; never remove or rename a field, change its type or meaning, or make an optional input required.
• An incompatible change MUST ship as a new version beside the old one.
• The old version MUST keep working for at least three months after the new one is released, or longer when the BA says a client needs it; the date is announced and the old operations are marked deprecated in the spec.

Requests

• A JSON body MUST be Content-Type: application/json (otherwise 415), one JSON value, and MUST be decoded strictly: an unknown field is a 400 naming the field. echoapi.Bind / ginapi.Bind do this.
• Every body MUST be bounded (413 over the limit); the stack's body limit does this for every route.
• Every field MUST be validated at the boundary: length, range, format, enum. A failure is a 400 (or 422 for a well-formed request that breaks a business rule) with details naming each field.
• The caller's identity MUST come from the verified token, never from a body, query or header field it controls (see security.md).

Filtering, searching and sorting

Type	Example
int	GET /v1/companies?size=10
float	GET /v1/products?price=20.99
string	GET /v1/users?name=John
enum	GET /v1/orders?status=completed
list (repeated key)	GET /v1/events?location=nyc&location=la
time (ISO 8601, UTC)	GET /v1/logs?start_time=2023-08-01T00:00:00Z&end_time=2023-08-15T23:59:59Z

• Free-text search uses search: GET /v1/companies?search=Mckinsey.
• Sorting uses sort_by (the field) and order_by (asc or desc). A sort field MUST come from a documented allowlist (postgres.SortColumns); an unknown one is a 400.

Lists and pagination

• Every list endpoint MUST paginate.
• Offset pagination uses offset and limit. Cursor pagination uses cursor and limit; a request MUST NOT mix offset and cursor.
• limit defaults to 25 and MUST be capped at 100 unless the endpoint documents another cap. A missing or non-positive limit gets the default, as aioz-stream does. A limit over the cap is a 400 (limit-too-large), not a silent clamp: a client that asked for 1000 and got 100 believes it has the whole list.
• A non-integer limit or offset, or a negative offset, is a 400.
• A cursor MUST be opaque (base64url of whatever the store needs to resume), and one the API did not issue is a 400.
• A list MUST answer with data holding the items under the collection's plural snake_case name and total across every page, as aioz-stream does, and query beside data at the top level: the request as served (defaults applied, filters included). A cursor list has next_cursor instead of total, omitted on the last page.
• An empty list is [], never null.

echoapi.Page / ginapi.Page and httpapi.EncodeCursor do all of this.

Responses

Every response body is one of these shapes, written by httpapi, never by hand: OK and Created for data, List(c, "videos", items, total, query) and CursorList for a page, Send(c, status, httpapi.Message("...")) for a message alone, and httpapi.Success(v).WithMessage("...") for both.

{"status": "success", "data": {"id": 1, "name": "Dwarves", "slug": "dwarvesf"}}

{
  "status": "success",
  "data": {
    "videos": [{"id": 1, "title": "Intro"}, {"id": 2, "title": "Demo"}],
    "total": 295
  },
  "query": {"offset": 20, "limit": 2, "sort_by": "created_at", "order_by": "desc"}
}

{"status": "success", "data": {"events": [...], "next_cursor": "eyJpZCI6NDJ9"}, "query": {"limit": 25}}

{"status": "success", "message": "password changed"}

{"status": "success", "message": "video published", "data": {"id": "v1"}}

{
  "status": "error",
  "reason": "video-not-found",
  "message": "no such video",
  "request_id": "0199c3a2-5d7e-7c41-9a7b-2f1e3c4d5e6f"
}

{
  "status": "error",
  "reason": "invalid-input",
  "message": "some fields are invalid",
  "request_id": "...",
  "details": [{"field": "email", "reason": "invalid-format"}]
}

• A success MAY carry a message: a sentence for a person that a client shows as-is (a toast, a banner). It MAY be the whole body when there is nothing else to return; when nobody reads the answer, 204 with no body is the better choice. A client MUST NOT branch on a success message: what happened is the status code and the data. data and message are each omitted when empty, except a list, whose data is always present, as [] when empty.
• reason is a lowercase kebab-case slug naming the cause (name-too-long, not bad-request). It is the field clients branch on, and it MUST NOT change meaning within a version.
• message is for a person. It MUST NOT carry a cause written for an operator: a driver error, a host, a query, a stack, a file path. Every 5xx says an unexpected error occurred; the detail goes to the log under the same request_id.
• Every response carries X-Request-Id. A caller's own sane id is kept, so one id follows a request across services.

Request log

Every service MUST write one access log record per request in the same format, aioz-stream's request line, because the logger's Graylog and Elasticsearch sinks parse it into fields and the dashboards are built on those fields:

GET status:200 latency:012ms url:/v1/videos/42

• Info level, in the log group api, with request_id, route, remote_ip, bytes and user_agent as attributes, plus reason when the request failed and err for a 5xx.
• The method is three letters (GET, PST, PUT, PTC, DEL), and the method and status are ANSI-colored; the sinks strip the colors.
• The url is the path only, never the query string: a query can carry a token.

The stack's access log writes exactly this; a service MUST NOT add a second request logger.

Status codes

Situation	Status	apierr
Read	200
Created	201
Accepted, done later	202
Updated or deleted, nothing to return	204, or 200 with only a message when a person reads it
Client already has the current version	304	CheckETag
Malformed or invalid input	400	BadRequest
No or invalid credentials	401	Unauthorized
Known caller, not allowed this action	403	Forbidden
Does not exist, or exists but is not the caller's	404	NotFound
Method not supported on this path	405	(stack)
Conflicts with current state (duplicate, version mismatch)	409	Conflict
Body over the limit	413	(stack)
Body not JSON	415	(stack)
Well-formed but breaks a business rule	422	New(reason, 422, ...)
Rate limited, with Retry-After	429	TooManyRequests
Unexpected fault	500	Internal, or any unclassified error
Dependency down, starting, or draining	503	Unavailable

• A batch operation that partly succeeds answers 200 with what happened to each item.
• A 5xx MUST NOT be returned for a client's mistake, and a 4xx MUST NOT hide a server fault.

Caching

Every GET SHOULD declare one cache policy. It does not have to cache; it has to choose. A GET with no policy gets no-store, which is safe and wastes bandwidth.

Endpoint type	Policy	Cache-Control
Public, rarely changing (catalogs, static metadata)	public + max-age	public, max-age=N
Public, changes any time	public, no max-age, with an ETag	public, no-cache
The caller's own, fine to reuse briefly	private + max-age	private, max-age=N, Vary: Authorization
The caller's own, changes any time	revalidate, with an ETag	private, no-cache, Vary: Authorization
Tokens, credentials, personal data, money, anything financial	no-store	no-store

• A response with tokens, credentials, personal data or financial data MUST be no-store.
• An error response MUST NOT be cached; the stack sends every error as no-store with no ETag, whatever the route's policy.
• The policy is declared twice and the two MUST agree: in code with echoapi.Cache / ginapi.Cache, and in the spec as x-cache-policy (see OpenAPI).

Conditional requests

• A revalidate or public, no-cache endpoint SHOULD send an ETag and answer a matching If-None-Match with 304 Not Modified and no body.
• The ETag MUST come from what the resource is (a row version, updated_at in nanoseconds, a content hash), not from the rendered body, and MUST change whenever any field the response shows changes, including fields joined from other tables.
• An ETag on a response that depends on the caller MUST only be sent with Vary: Authorization (the private and revalidate policies do this).
• A test MUST show the 304 for every endpoint that sends an ETag.

Idempotency

• GET, HEAD, PUT and DELETE MUST be idempotent: repeating one changes nothing further.
• A POST that moves money, places an order or starts paid or long work SHOULD accept an Idempotency-Key header and answer a repeat with the first result instead of acting twice. The key is stored with the result for at least 24 hours.

Timeouts and limits

• The HTTP server MUST set ReadHeaderTimeout, ReadTimeout, WriteTimeout (per-route deadlines for streaming) and IdleTimeout; Go's zero values mean none.
• Every outbound call MUST have a deadline: an aioz-common httpclient with explicit timeouts, and a context deadline per request.
• Login, OTP, password reset and expensive endpoints MUST be rate limited with ratelimit, answering 429 with Retry-After.
• Details and the rest of the input limits are in security.md.

OpenAPI

• Every service MUST publish an OpenAPI document generated from its code (swaggo), committed, and checked fresh in CI (GENERATE_VERIFY_CMD: make swagger-verify).
• The document MUST pass validate:openapi (Spectral with the team ruleset, templates/openapi/):

Rule	Severity
Every operation has a unique lowerCamel operationId, a summary and a tag	error
Every operation declares a 2xx and an error response (4xx, 500 or default)	error
Every path starts with /v<n>/ or /api/v<n>/, is lowercase and is not a verb	error
A GET declares x-cache-policy	warn
x-cache-policy.type is public, private, revalidate or no-store	error
A list GET has limit, and offset or cursor	warn
Properties and query parameters are snake_case	warn

With swaggo, the policy is one annotation beside the route's others:

// @Summary  Get a video
// @Tags     videos
// @Success  200 {object} VideoResponse
// @Failure  404 {object} httpapi.ErrorBody
// @Router   /v1/videos/{id} [get]
// @x-cache-policy {"type":"revalidate"}

---

Sources

• RFC 9110 HTTP Semantics (methods, status codes, conditional requests, section 13), RFC 9111 HTTP Caching.
• RFC 2119 requirement levels.
• OWASP API Security Top 10 2023.