Logs and metrics configuration

How a Go service loads the settings for its logs and metrics. It covers the aioz-common logger section (level, format, sampling, the Elasticsearch sink) and the metrics push (aioz-common stats/telemetry). The key words MUST, MUST NOT, SHOULD and MAY are used as in RFC 2119.

The rule

Setting	Source
Log level, format, sampling, stack traces	Config file
Log shipping: enabled, url, data-stream, credentials	Config file
Metrics push: url, credentials, app, instance, interval	Config file
Debug listener address (/metrics, pprof)	Config file
Deployment label (staging / production)	DEPLOY_ENV environment variable
Role label (api, core, consumer)	The subcommand name, in code

• Every log and metric setting MUST come from the config file, except the deployment label.
• The deployment label MUST come from one environment variable, DEPLOY_ENV. It feeds both logger.elasticsearch.env (service.environment on every log document) and metrics.env (the env label on every series), so one deployment's logs and metrics can never carry different labels.
• The config file MAY also set both env keys, for a local run. A non-blank DEPLOY_ENV overrides them; a blank one leaves the file in charge.
• The role label MUST come from the subcommand that started the process. metrics.role in the config file MAY override it for a deployment that runs several processes of one role and wants them told apart.
• A service MUST NOT bind any other environment variable to a log or metric setting: no LOG_LEVEL, no METRICS_URL, no <PREFIX>_LOGGER_ELASTICSEARCH_* override. With a viper AutomaticEnv loader that means excluding the logger and metrics keys from it, not just leaving them out of the compose file.

Why

• The config file is already per host. It is mounted into the container, not baked into the image, so staging and production each have their own. An environment variable on top of it is a second source for the same setting, and an operator reading one of them cannot tell what the process will use.
• An environment variable silently beats the file. parser-service passed AIOZ_MAP_LOGGER_ELASTICSEARCH_ENABLED: ${LOG_ELASTICSEARCH_ENABLED:-false} from its compose file. The literal false default overrode enabled: true in the config file, and log shipping stopped on every host whose .env did not set the variable.
• The deployment label is the one value the deploy owns. It is the same for every service on a host and is what a dashboard selects by exact match, so it is set once per host by the deploy, not per service file.
• Not AIOZ_ENV. The host monitoring agent exports AIOZ_ENV for its own series, and its value is outside the service's control. A host exporting prod would otherwise label records no filter finds, or fail every container at boot.

Values and validation

• DEPLOY_ENV MUST be blank, staging or production. Any other value MUST fail startup, before any role starts work. Blank writes no label.
• Each env key in the config file follows the same rule.
• Metrics push is enabled exactly when metrics.url is set; log shipping when logger.elasticsearch.enabled is true. When enabled, the URL MUST be validated at startup (http:// or https://), not on the first push.
• An empty config file MUST produce a working process that logs JSON to stdout and pushes nothing, so a host that has not been onboarded runs unchanged.

Credentials

• Bearer tokens, API keys and passwords for the log sink and the metrics receiver MUST live in the host's config file (or the secrets.yaml merged over it), with mode 0600.
• They MUST NOT appear in the compose file, .env, the image, or the committed example config, which keeps them blank.
• In code they SHOULD be held in secret.Token (aioz-common), so they cannot reach a log line.

Compose

• The telemetry environment block MUST be exactly:

  x-telemetry: &telemetry
    DEPLOY_ENV: ${DEPLOY_ENV:-}

• Never write a literal default (${X:-false}, ${X:-info}) for a telemetry variable. A blank value is unset (viper's default AllowEmptyEnv(false)), so the config file decides.

• deploy/deploy.env (the .env template) documents DEPLOY_ENV and nothing else about logs or metrics.

Tests

A service SHOULD guard the rule with two tests next to its config loader:

• Load a config with DEPLOY_ENV=staging and assert both logger.elasticsearch.env and metrics.env are staging; load with DEPLOY_ENV blank and assert the file's values stand.
• Parse the deploy compose file and assert the telemetry block passes DEPLOY_ENV and no other log or metric variable.

Reference

parser-service (internal/app/config.go, internal/app/log_env_test.go, docker-compose.deploy.yml).