JSON logging best practices

JSON logs are structured objects rather than free form strings. I started using JSON logs while building an analytics pipeline that called the Instagram Insights API. The difference in day to day operations was immediate. Searches became faster and alerts stopped firing on noise.

This guide shares practical rules I use often and includes an Instagram Insights API example so you can map fields to real requests.

Quick setup note: adding JSON logging to a small service takes a couple hours. For a medium service plan a day. For large fleets expect several days of staged rollout and validation.

Why JSON logging

JSON logs are machine friendly and they make troubleshooting less painful. Fields can be indexed and queried directly which removes brittle parsing from the equation and speeds searches so you find the relevant events faster.

JSON also preserves types. Numbers remain numbers and booleans remain booleans which avoids costly conversion errors during aggregation and alerting.

Best practices

Use a stable minimal schema

Define a small set of required keys for every entry. At minimum include timestamp, level, service, and message. Add requestId or traceId for correlation.
Keep a human message field

Include a concise message string for humans and tools that fall back to text. The message should summarize the event so it is readable without parsing all fields.
Prefer typed values

Emit numbers, booleans, and arrays instead of encoded strings. This improves aggregation and avoids conversion errors in pipelines.
Standardize level names and timestamp format

Use a canonical set of levels such as debug, info, warn, error. Use ISO 8601 or RFC 3339 timestamps in UTC to avoid timezone ambiguity.
Avoid logging sensitive data

Never log raw credentials, full tokens, or unredacted personal data. Apply redaction rules before shipping entries and use the pipeline to mask secrets consistently.
Keep entries small at hot paths

Avoid serializing large payloads synchronously on high throughput code paths. Emit minimal fields and optionally attach references to stored payloads.
Add versioning for evolving schemas

Include a logSchemaVersion when you plan to change field names or types. That makes rolling upgrades and backfills safer.
Correlate with traces and metrics

Add traceId and requestId fields. Ensure the same identifiers are used in traces, metrics and logs to enable end to end investigation.
Validate logs early

Validate JSON shape at emission time in staging. Use a schema validator or lightweight contract tests to catch missing keys or type drift.
Use sampling and enrichment

Sample verbose debug traffic and enrich sampled entries in a pipeline with deployment, region, and environment metadata to reduce cost while keeping context.

Instagram Insights API example

This example shows how to log a request to the Instagram Insights API. The entry includes correlation ids, request metadata, and outcome fields useful for alerts and dashboards.

Example request success

{
  "timestamp": "2026-05-05T14:30:45Z",
  "level": "info",
  "service": "analytics-api",
  "environment": "production",
  "requestId": "req_8f3a2b",
  "traceId": "trace_12ab34",
  "userId": "user_42",
  "endpoint": "/instagram/insights",
  "httpMethod": "GET",
  "statusCode": 200,
  "responseTimeMs": 235,
  "metricsRequested": ["impressions","reach","profile_views"],
  "message": "Fetched instagram account insights"
}

Example rate limit error

{
  "timestamp": "2026-05-05T14:31:02Z",
  "level": "warn",
  "service": "analytics-api",
  "requestId": "req_8f3a2c",
  "endpoint": "/instagram/insights",
  "httpMethod": "GET",
  "statusCode": 429,
  "retryAfterSeconds": 60,
  "apiErrorCode": "rate_limit",
  "message": "Instagram API rate limit exceeded"
}

Implementation patterns

Use a logging library that emits JSON natively

Pick a library that supports structured fields and async flushing. Avoid hand rolling JSON serialization in multiple places.
Enrich logs at the edges

Add request level context from middleware where requestId, userId, and traceId are available so downstream code does not need to fill them in.
Batch and buffer to reduce overhead

Buffer log writes or send them asynchronously to avoid blocking application threads on IO.
Centralize redaction and enrichment rules

Implement redaction and enrichment in a single pipeline stage so rules are consistent across services.

Common pitfalls

Schema drift

Changing field names or types without versioning breaks queries and dashboards. Use logSchemaVersion and migration plans.
Logging PII by accident

Inspect logs after deployment and run automated checks to detect sensitive patterns.
Overlogging

High cardinality fields such as raw user agents or full URLs create expensive indices. Hash or sample high cardinality values.

FAQ

What timestamp format should I use? Use ISO 8601 in UTC such as 2026-05-05T14:30:45Z to ensure consistent sorting and parsing.

What keys are required for every entry? At minimum include timestamp, level, service, and message. Add requestId or traceId when available for correlation.

How do I handle large request or response bodies? Avoid serializing full bodies. Store the payload in object storage and reference it by id or sample a subset of requests for full capture.

Can I use JSON logs in plain text pipelines? Yes many tools accept JSON lines. If you must support plain text fallback include a stable message field and key=value pairs for critical identifiers.

How do I monitor logging costs? Track ingestion and index costs by environment and service. Use sampling and retention policies to control volume from debug and batch jobs.

JavaScript middleware example

The following Express middleware attaches requestId and traceId to each request and exposes a child JSON logger on req.log. It uses pino for structured output but can be adapted to other libraries.

const pino = require('pino');
const crypto = require('crypto');

const logger = pino();

function requestLogger(req, res, next) {
   const requestId = req.headers['x-request-id'] || crypto.randomUUID();
   const traceId = req.headers['x-b3-traceid'] || requestId;

   req.requestId = requestId;
   req.traceId = traceId;

   req.log = logger.child({ requestId, traceId, service: 'analytics-api', environment: process.env.NODE_ENV || 'development' });

   const start = Date.now();

   res.on('finish', () => {
      req.log.info({
         method: req.method,
         url: req.originalUrl,
         statusCode: res.statusCode,
         responseTimeMs: Date.now() - start
      }, 'request completed');
   });

   next();
}

// Example usage in a route performing an Instagram Insights API call
async function handleGetInsights(req, res) {
   const metrics = ['impressions', 'reach', 'profile_views'];

   const start = Date.now();
   try {
      const data = await fetchInstagramInsights(metrics); // implement your fetch

      req.log.info({
         endpoint: '/instagram/insights',
         httpMethod: 'GET',
         statusCode: 200,
         responseTimeMs: Date.now() - start,
         metricsRequested: metrics
      }, 'Fetched instagram account insights');

      res.json(data);
   } catch (err) {
      req.log.error({ err: err.message, statusCode: err.status || 500 }, 'instagram insights call failed');
      res.status(500).send('internal error');
   }
}