Observability and Logging

How to use structured logging, configure observability, and inspect traces and logs with the iii Console.

Goal

Make your iii application fully observable: correlate every log entry to the exact trace that produced it, inspect execution timelines to find bottlenecks, and optionally export all telemetry to third-party tools like Grafana, Jaeger, or Datadog.

The iii Logger

Every iii SDK ships a Logger class that emits logs as OpenTelemetry LogRecords. Each log call automatically captures the active trace ID and span ID, linking the log entry to the distributed trace that produced it.

Language-native logging functions such as console.log in Node, print() in Python, tracing::info! in Rust write to stdout but are not connected to iii traces. This means you cannot find them in the iii Console’s trace detail view, and they are invisible to any OTLP-based observability backend.

Approach	Where it appears	Trace correlation
console.log("Order created")	stdout only	None
print("Order created")	stdout only	None
tracing::info!("Order created")	stdout only	None
logger.info("Order created", { orderId })	stdout, iii Console, OTLP backends	Automatic — linked to the active trace and span

Avoid using console.log, print(), or tracing::info! for application logs. These bypass the OpenTelemetry pipeline and will not appear in the iii Console or any connected observability tool.

Trace-correlated logs in iii Console

When you use the iii Logger, every log entry is attached to the trace that was active when the log was emitted. In the iii Console, clicking a trace in the waterfall chart opens a detail drawer. The Logs tab shows every log entry from that exact execution — with severity, timestamp, message, and any structured data you attached.

This is the core value of using the iii Logger: you can go from a slow trace in the waterfall chart directly to the logs that explain what happened, without grepping through stdout or cross-referencing timestamps manually.

Using the Logger

1. Import and instantiate

   Create a Logger instance in your function handler. No configuration is required — trace context is injected automatically.

   Node / TypeScript

   import { Logger } from 'iii-sdk'
   
   const logger = new Logger()
   
   logger.info('Worker connected')
   

   Python

   from iii import Logger
   
   logger = Logger()
   
   logger.info("Worker connected")
   

   Rust

   use iii_sdk::Logger;
   
   let logger = Logger::new();
   
   logger.info("Worker connected", None);
   
2. Attach structured data

   Pass a second argument with key-value data. Structured data becomes filterable attributes in the iii Console and in any OTLP-compatible backend.

   Node / TypeScript

   logger.info('Order processed', { orderId: 'ord_123', amount: 49.99, currency: 'USD' })
   logger.warn('Retry attempt', { attempt: 3, maxRetries: 5, endpoint: '/api/charge' })
   logger.error('Payment failed', { orderId: 'ord_123', gateway: 'stripe', errorCode: 'card_declined' })
   logger.debug('Cache lookup', { key: 'user:42', hit: false })
   

   Python

   logger.info("Order processed", {"order_id": "ord_123", "amount": 49.99, "currency": "USD"})
   logger.warn("Retry attempt", {"attempt": 3, "max_retries": 5, "endpoint": "/api/charge"})
   logger.error("Payment failed", {"order_id": "ord_123", "gateway": "stripe", "error_code": "card_declined"})
   logger.debug("Cache lookup", {"key": "user:42", "hit": False})
   

   Rust

   use serde_json::json;
   
   logger.info("Order processed", Some(json!({ "order_id": "ord_123", "amount": 49.99, "currency": "USD" })));
   logger.warn("Retry attempt", Some(json!({ "attempt": 3, "max_retries": 5, "endpoint": "/api/charge" })));
   logger.error("Payment failed", Some(json!({ "order_id": "ord_123", "gateway": "stripe", "error_code": "card_declined" })));
   logger.debug("Cache lookup", Some(json!({ "key": "user:42", "hit": false })));
   

   Prefer key-value objects over string interpolation. Structured fields let you filter, aggregate, and build dashboards — string-interpolated messages do not.

3. Use inside a function handler

   The Logger works anywhere inside a function handler. Trace context is captured from the active invocation automatically.

   Node / TypeScript

   import { registerWorker, Logger } from 'iii-sdk'
   
   const iii = registerWorker(process.env.III_URL ?? 'ws://localhost:49134')
   
   iii.registerFunction('orders::create', async (req) => {
     const logger = new Logger()
     const { customerId, amount } = req.body
     const orderId = `order-${Date.now()}`
   
     logger.info('Order created', { orderId, customerId, amount })
   
     // ... business logic ...
   
     logger.info('Order processing complete', { orderId })
     return { status_code: 201, body: { orderId } }
   })
   

   Python

   import os
   from iii import register_worker, Logger
   
   iii = register_worker(os.environ.get("III_URL", "ws://localhost:49134"))
   
   
   def create_order(req):
       logger = Logger()
       body = req.get("body", {})
       customer_id = body.get("customerId")
       amount = body.get("amount")
       order_id = f"order-{int(__import__('time').time() * 1000)}"
   
       logger.info("Order created", {"order_id": order_id, "customer_id": customer_id, "amount": amount})
   
       # ... business logic ...
   
       logger.info("Order processing complete", {"order_id": order_id})
       return {"status_code": 201, "body": {"orderId": order_id}}
   
   
   iii.register_function("orders::create", create_order)
   

   Rust

   use iii_sdk::{register_worker, InitOptions, Logger, RegisterFunction};
   use serde_json::{json, Value};
   
   let iii = register_worker("ws://127.0.0.1:49134", InitOptions::default());
   
   let reg = RegisterFunction::new_async("orders::create", |req: Value| async move {
       let logger = Logger::new();
       let customer_id = req["body"]["customerId"].as_str().unwrap_or("");
       let amount = req["body"]["amount"].as_f64().unwrap_or(0.0);
       let order_id = format!("order-{}", chrono::Utc::now().timestamp_millis());
   
       logger.info("Order created", Some(json!({ "orderId": order_id, "customerId": customer_id, "amount": amount })));
   
       // ... business logic ...
   
       logger.info("Order processing complete", Some(json!({ "orderId": order_id })));
       Ok(json!({ "status_code": 201, "body": { "orderId": order_id } }))
   });
   iii.register_function(reg);
   

Logger API reference

All three SDKs expose the same four methods:

Method	Node / TypeScript	Python	Rust
Info	logger.info(msg, data?)	logger.info(msg, data=None)	logger.info(msg, Option<Value>)
Warning	logger.warn(msg, data?)	logger.warn(msg, data=None)	logger.warn(msg, Option<Value>)
Error	logger.error(msg, data?)	logger.error(msg, data=None)	logger.error(msg, Option<Value>)
Debug	logger.debug(msg, data?)	logger.debug(msg, data=None)	logger.debug(msg, Option<Value>)

When OpenTelemetry is not initialized (e.g. in unit tests), the Logger falls back to console.* (Node), Python logging (Python), or tracing::* (Rust) — your logs still appear in stdout.

Configuring observability

The iii engine’s Observability worker (iii-observability) controls how traces, logs, and metrics are collected and exported. There are two main configurations depending on your environment.

Local development

For local development, use the memory exporter. Traces and logs are stored in the engine’s memory and can be inspected through the iii Console. This is the simplest setup and requires no external infrastructure.

iii-config.yaml

workers:
  - name: iii-observability
    config:
      enabled: true
      exporter: memory
      logs_enabled: true
      memory_max_spans: 1000

Field	Purpose	Default
exporter	Where to send traces: memory, otlp, or both	otlp
memory_max_spans	Max spans kept in memory	1000
logs_enabled	Enable structured log storage	true (always initialized)
logs_max_count	Max log entries kept in memory	1000
logs_console_output	Also print logs to the terminal via tracing	true

With exporter: memory, all data lives in the engine process. This is ideal for development — no collector, no database, just start the engine and open the console.

Exporting to third-party tools

For production or when you want to send telemetry to an external system (Grafana, Jaeger, Datadog, or any OTLP-compatible collector), use the otlp exporter with an endpoint.

iii-config.yaml

workers:
  - name: iii-observability
    config:
      enabled: true
      exporter: otlp
      endpoint: "http://collector.example.com:4317"
      service_name: my-service
      service_version: 1.0.0
      metrics_enabled: true
      logs_enabled: true
      logs_exporter: otlp

To keep local visibility through iii Console and export to a collector simultaneously, use exporter: both:

iii-config.yaml

workers:
  - name: iii-observability
    config:
      enabled: true
      exporter: both
      endpoint: "http://collector.example.com:4317"
      service_name: my-service
      logs_enabled: true
      logs_exporter: both

The endpoint field can also be set via the OTEL_EXPORTER_OTLP_ENDPOINT environment variable. See the Observability worker reference for the full list of configuration fields and environment variable overrides.

Viewing Logs in iii Console

Once you’ve instrumented your code with the Logger, view trace-correlated logs in the iii Console.

See Use the Console for step-by-step instructions on navigating traces and viewing span details.

Navigate to Traces, select a trace, and open the Logs tab on any span to see every log entry from that execution — with severity, timestamp, message, and structured attributes you attached.

The dedicated Logs page provides a full log viewer with severity filters, full-text search, and time-range controls. If a log entry has a trace_id, click it to jump directly to the corresponding trace.

For the full console feature set, see the Console reference.

Result

Your iii application is now fully observable:

• Structured logs are correlated to distributed traces automatically — no manual wiring.
• Local visibility is available through iii Console with the memory exporter — no external infrastructure needed.
• Third-party export sends traces, logs, and metrics to any OTLP-compatible backend via the otlp or both exporter.
• Bottleneck identification is possible through waterfall, flame graph, and service breakdown views in the console.

Next steps

Console

Full iii Console feature reference

OpenTelemetry Integration

Custom spans, metrics, and telemetry utilities

Observability Worker

Full configuration reference for traces, logs, metrics, alerts, and sampling

Observability Example

End-to-end multi-step workflow with trace correlation