tako.Secret, metrics, and tako logs
Configuration you must not commit, numbers you want to watch, and the command that reads a deployed app's logs.
Declare
From examples/gallery/app.go:
var (
notifier = tako.Secret("notifier-key")
ups = tako.Counter("uploads")
thumbMs = tako.Histogram("thumb_ms")
)
Every name must be a string constant (analyzer-enforced). None of the three take further options today.
Use
A secret resolves once at startup — boot refuses to start a
service with any declared secret unset, which is what lets Value()
take no Ctx and never fail at request time:
// Value never fails here: boot refused to start if the secret was unset.
key := notifier.Value()
Metrics are fire-and-forget — recording never fails and never blocks the handler:
ups.Add(ctx, 1)
thumbMs.Observe(ctx, float64(time.Since(start).Milliseconds()))
Logging
ctx.Log() returns a *slog.Logger — the standard library's, not a
tako type, so anything that takes an slog.Logger takes this one:
ctx.Log().Info("thumbnail written", "photo", req.PhotoID, "ms", took)
ctx.Log().Error("notify failed", "err", err)
The logger is pre-tagged with the execution it belongs to, which is
the reason to prefer it over a package-level slog: requests carry
req=<id>, crons carry cron=<name>, jobs carry job=<name> (and
task=<i> under RunnerConfig.Parallel), singletons carry singleton=<name>,
and push deliveries carry handler and source. Every line from one
request is therefore greppable as a unit without you passing an id
around.
Where the line goes depends on where the service runs, and you write it the same way either way. Locally it is Go's default human-readable output next to the dev console's request lines. Deployed, tako installs a JSON handler and the same call arrives as a Cloud Logging record:
{"time":"2026-08-01T20:52:19.418113Z","severity":"ERROR","message":"notify failed",
"req":"r-8f2c1d","err":"dial tcp: i/o timeout",
"logging.googleapis.com/trace":"projects/my-project/traces/105445aa7843bc8bf206b12000100000"}
Three things follow from that, and each is the point:
severityis a real field, soseverity>=ERRORin the Logs Explorer finds exactly the errors. Note the spelling: slog'sWARNis written out as Cloud Logging'sWARNING, because those are the only level names it recognizes.- Your attributes stay yours.
errabove is the handler's; only slog's three built-ins (level,msg,time) are renamed to the keys Google reads. - The trace field nests the line under its request. Cloud Run sends
X-Cloud-Trace-Context; every line a handler logs during that request carries the matching trace, so the Logs Explorer groups them under the request log rather than scattering them. Locally, and on push deliveries where there is no trace,reqdoes the correlating on its own.
Request logs are Cloud Run's, not tako's. The platform writes one
entry per request to run.googleapis.com/requests — full URL, request
and response sizes, the client IP, nanosecond latency, the trace, and a
severity already derived from the status (a 404 arrives as WARNING).
Tako does not write a second one; locally, where there is no platform to
do it, you get the dev console's GET /path → 200 (14.2ms) line.
tako logs renders those platform entries in the same shape, so the
output reads the same in both places.
tako owns the default logger in a deployed service. A generated main
calls Boot() before any of your code runs, so a slog.SetDefault in
your own init() is replaced there. That is deliberate: the log format
in the cloud is derived the same way the service's shape is, and it is
what makes severity mean anything without configuration. Locally
nothing is installed, so slog.SetDefault is yours.
Cron, job, singleton and push-handler lines get severity and the same
treatment; they carry a cron=/job=/handler= attribute naming what
ran, since an execution is not a request from a client and has no
platform request entry to nest under.
A refused delivery or cron trigger says so, at WARNING. The
platform's request log shows a 403 and cannot say whose it was — Cloud
Run refusing a caller with no run.invoker grant and tako refusing a
token whose audience is stale look identical from outside the container,
and they have different fixes. So tako narrates its own:
{"severity":"WARNING","message":"tako refused this delivery",
"handler":"countClick","status":403,
"reason":"delivery rejected: token audience is not this service",
"expected_audience":"https://shortlink-countclick-….run.app",
"expected_invoker":"shortlink-countclick-sa@….iam.gserviceaccount.com"}
The two expectations are in the line because comparing them against
the deliverer's own OIDC configuration is the fix; neither is a secret.
A 403 in the request log with no such line beside it is the
platform's refusal, not tako's.
Local
Secrets come from .env in the module dir (name=value lines),
loaded once at boot: echo 'notifier-key=nk_dev_123' > .env before
tako dev. A missing one stops the service before it serves.
Locally every Add/Observe is one structured log line, plus a 📈
line in the dev console: 📈 uploads +1 (total 3) for counters,
📈 thumb_ms 42 (n=7) for histograms.
Cloud
Secret values are data the user owns: tako never creates, writes,
or deletes secrets. The physical Secret Manager name is env-prefixed
like every logical name — except prod, which uses the bare name
(notifier-key; a pr-42 env reads pr-42-notifier-key). Deploy's
preflight checks each one exists and, when it doesn't, prints the
exact create command:
secret pr-42-notifier-key does not exist in my-project — create it: printf '<value>' | gcloud secrets create pr-42-notifier-key --data-file=- --project my-project
Every service resolves every declared secret from Secret Manager at boot — retrying briefly on a first deploy while IAM grants propagate — and the deploy derives the accessor grants before any service starts. Rotation is yours: add a new secret version; instances pick it up as they restart.
Metrics in the cloud
Locally, every Add and Observe is a structured log line and a 📈
line in the dev console. Deployed, the same call sites also export to
Cloud Monitoring as user-defined metrics:
custom.googleapis.com/tako/uploads # tako.Counter("uploads")
custom.googleapis.com/tako/thumb_ms # tako.Histogram("thumb_ms")
The two metrics you never declare
Every deployed service also records one counter and one histogram per endpoint, without being asked:
custom.googleapis.com/tako/requests # count, labelled route + status
custom.googleapis.com/tako/request_ms # latency, labelled route
route is the declared pattern, method included — GET /api/photos,
not /api/photos/8f2c. That is what keeps the series count bounded: one
photo or a million is one series either way, where a path label would
mint one per id. An unmatched path records as <unmatched>, so a
scanner probing random URLs cannot turn into a metrics bill. status is
the class (2xx…5xx), never the raw code.
Crons and jobs are in here too, under /_tako/api/cron/<Name> — the mux
pattern alone is POST /_tako/api/cron/{cron}, which would merge every cron
in a service into one series, so the dispatcher names what it actually
ran.
These exist because Cloud Run's own request_count cannot answer a
question about one endpoint. It publishes a route label and leaves it
empty on every series (measured across 88 of them), so
Erroring(someEndpoint, …) and SlowerThan(…) have nothing to filter
on. Tako writes the label itself, which is what makes per-endpoint
alerting possible at all.
Both are generic_task series like the rest, so the job label carries
the service and task_id the instance — an endpoint's error rate is
therefore sliceable by service without any extra declaration.
Find them in Metrics Explorer by pasting that metric type. From the
command line there is no gcloud surface for reading time series —
gcloud monitoring covers dashboards, policies and uptime checks, not
points — so read the API:
curl -s -G -H "Authorization: Bearer $(gcloud auth print-access-token)" \
--data-urlencode 'filter=metric.type="custom.googleapis.com/tako/uploads"' \
--data-urlencode "interval.startTime=$(date -u -v-1H +%Y-%m-%dT%H:%M:%SZ)" \
--data-urlencode "interval.endTime=$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
https://monitoring.googleapis.com/v3/projects/my-project/timeSeries
Recording is still free at the call site. Add takes a lock and
returns; nothing in a request path ever waits on Cloud Monitoring. A
background flush ships what accumulated every 60 seconds, so a
metric you just recorded takes up to a minute to appear.
Both kinds are written cumulatively — the running total since the instance started, not the delta since the last flush. That is what makes a failed flush harmless: the total lives in the instance, so the next flush carries the truth rather than leaving a gap. The one flush that cannot be recovered is the last one, which is why a service drains before it exits.
Metrics are filed under generic_task, not cloud_run_revision.
Google allows user-defined metrics to be written against only a set of
monitored resource types, and Cloud Run's own is not among them — a
write there is refused with "metric cannot be written to
cloud_run_revision". generic_task is the type for this, and its
labels map cleanly:
| label | value |
|---|---|
namespace |
<app>-<env> — prod and a preview env never merge |
job |
the tako service name, as you wrote it — unprefixed, because namespace already carries the env |
task_id |
the instance |
location |
the region |
job being the logical name is load-bearing rather than cosmetic:
alert queries filter on it, and a query that spells the
label differently from the writer matches nothing — which reads
identically to a service that has stopped serving. Tako's own alerts
therefore build the matcher from the same name the deploy injects.
task_id is load-bearing, not decoration: every instance of a revision
would otherwise share one cumulative series, and concurrent writers to a
single series is either a rejection or nonsense. Aggregate across
task_id when you chart — sum for counters, percentile aggregation
for histograms. The revision rides along as a revision metric label,
which adds no series (task_id is already finer) and makes a shift in a
metric traceable to a deploy.
Histogram buckets are exponential, doubling, from 1 to 262144 — one
layout for every histogram, because tako.Histogram takes a name and a
float64 and no unit. Samples below 1 and above 262144 land in the
under/overflow buckets rather than being dropped. At 2× per bucket this
is honest about magnitude and coarse about precision: read it for "did
thumbnails get slower", not for a tight p99.
Cost: user-defined metrics bill on ingestion, and one instance writing two metrics once a minute is negligible. It grows with metrics × instances, so it is worth a glance if you declare many metrics on a service that scales wide.
Shutdown
Cloud Run replaces a revision by sending the old instance SIGTERM and
killing it 10 seconds later. That window is fixed — there is no API
field for it and no gcloud flag — so tako divides it instead:
- The listener closes. New requests go to the new revision; this instance accepts nothing more.
- Requests already executing run to completion — the drain ends the moment the last one returns, with a 7-second ceiling for the ones that take their time.
- Shutdown hooks flush, which is where the final metrics write goes.
- The process exits.
Step 2 is adaptive, and there is no setting. tako watches the in-flight count rather than spending a budget: an instance that is idle at SIGTERM — the common case, because Cloud Run stops routing to it before signalling — drains in milliseconds and hands nearly the whole window to the flush. A busy one takes what it needs, up to the ceiling, and the flush keeps a one-second floor regardless. Time the drain does not use is repaid, not lost.
Locally, Ctrl-C does exactly the same, with the same 7-second ceiling — a hanging request behaves in dev the way it will in Cloud Run, which is the one moment the difference would matter. An idle server still stops instantly. A second Ctrl-C exits immediately, and that, not a shorter local cap, is the escape hatch for a wedged handler.
A handler that outlives the ceiling is cut off, and says so rather than leaving you to find it as a mystery 502:
tako: 1 request did not finish within 7s — cutting it off (SIGKILL follows in ~3s)
and locally, where there is no SIGKILL to name, it says where the number came from instead:
⏻ 1 request did not finish within 7s — cutting it off
(Cloud Run allows 10s from SIGTERM; the rest flushes metrics)
The count covers HTTP/2 as well as HTTP/1.1. That is not free:
http.Server.Shutdown stops tracking a connection once h2c hijacks it,
so the drain waits on tako's own counter, which sits above the
protocol.
Jobs flush on exit too, on both the success and the failure path — a failing task's metrics are usually the ones worth having.
tako logs [service] [--since 15m] (plus the standard
--project/--region/--env) is cloud read-only: it analyzes the
app to map logical service names to deployed ones, accepts either
form, fetches up to 200 recent entries, and prints
time service severity text per line. No matching service lists the
real names; no entries prints no log entries in the last 15m0s.
A structured entry prints its fields, not just its message. The
handler line ctx.Log().Error("handler error", "err", err) reaches you
as the error, with the request id that ties it to the request line
above:
14:12:20 x1-gallery ERROR handler error [req=759268]
err: no matching index found. recommended index is:
- kind: photos
properties:
- name: ready
A multi-line value keeps its own shape, because for the errors worth
reading — a Datastore index recommendation, a stack — the shape is the
remediation. Whatever else a handler attaches rides along as
key: value.
Test it
From examples/gallery/app_test.go — tests
provide secrets on the app, no .env involved:
app := tako.NewTestApp(t)
app.SetSecret("notifier-key", "nk_test_123")
Metrics need no test wiring: the test app carries its own sink.
Errors you'll meet
At analysis time, verbatim:
app.go:7:28: tako.Secret name must be a string constant — names are read at analysis time, not run time; got variable "keyName" (tako/const-name)
At boot with a secret unset:
secret "notifier-key" is not set — add it to .env (local) or Secret Manager (cloud). Reading an unset secret outside boot's guarantee
(tests without SetSecret) panics:
tako: secret "notifier-key" is not set — add it to .env (tako dev, generated services) or call app.SetSecret (tests).
From tako logs: --since wants a duration like 15m, got "soon", and
no service named "web" — services: gallery, gallery-makethumb, ….
Not yet
- No labels on your metrics:
AddandObservetake only a value, so you cannot breakuploadsdown by user or status. Tako's ownrequests/request_msdo carryrouteandstatus, but that is the framework labelling what it already knows; there is no way to add a dimension of your own yet. (Therevisionlabel and the resource'stask_idare likewise tako's — they keep instances from colliding, not a slicing surface.) - No dashboards are created for you; tako writes the metrics, you build the views. Alert policies are declared — see alerts.md — but nothing charts them.
- Histogram bucket bounds are fixed (doubling, 1 to 262144) and not declarable.
tako logsfetches once — no--follow/streaming, and no--severityflag yet (the severities are there; the CLI does not filter on them). The Logs Explorer does.- No log sampling and no level knob: a deployed service logs everything
INFOand above, and there is no field to change that. - No log-based metrics and no log-based alerting:
tako.Alertreads metrics only, so "page me when this line appears" has no expression. - Log volume is billed by Cloud Logging like any other. Structured lines are slightly larger than the text they replaced; a chatty handler on a wide-scaling service is the case to watch.
- No secret versioning surface: services read
latestat boot, and tako will not write versions for you (by design, not a gap).