We have all been there, debugging across different services, each with their own logging format. The additional overhead of querying and parsing each different format reduces our visibility and slows us down when finding the root cause.

Here is an example standard log format:

{
  "version": "1",
  "level": 10,
  "timestamp": "2021-04-04T12:49:27.3831583+00:00",
  "traceId": "924c1365-c014-4bd9-88f7-efe9a3d06242",
  "spanId": "e5d222c5-49d5-4371-898f-61948e52779a",
  "message": "Request failed POST /login",
  "context": {
    "method": "POST",
    "path": "/login"
  },
  "exception": {
    "type": "AuthenticationException",
    "message": "Invalid credentials",
    "stacktrace": "at SomeMethod in SomeClass:1:13 \n at SomeOtherMethod in SomeOtherClass:2:24",
    "context": {
      "username": "teddy"
    }
  }
}

• version to track changes to the format

• level is the indicate the severity

  • example scale

    level	ordinal
    debug	10
    info	20
    warn	30
    error	40
    fatal	50

  • Numeric type enables range query

    SELECT * FROM log
    WHERE level >= 40

• timestamp for ordering events

• traceId for correlating logs for a request across services

  • This is created once at the origin of the request and passed to downstream services

    • e.g. generate the traceId when a user clicks a button and triggers a request on the UI

  • Useful for getting a big picture across services

    SELECT * FROM log
    WHERE traceId = '924c1365-c014-4bd9-88f7-efe9a3d06242'

• spanId for correlating logs for a request in the same service

  • Created at start of request to a service
  • Useful for zoning on the logs of a single service
  • Necessary if logs for all applications are delivered to the same sink
    • e.g. a single request enqueues multiple jobs, all of which will have the same traceId

• message and context is the message and extra context data attached to the log depending on language

• exception is a subset of the exception object

  • Only relevant for the error and fatal log levels
  • type is the class or name of the error depending on language
  • message and context is the message and extra context data attached to the exception
  • stacktrace will be the stack trace specific to the language

Examples

Microsoft.Extensions.Logging

• Information

  logger.LogInformation("message {param} {complexParam}", "something", new {
      One = 1,
      Two = 2
  });

  • Expected log

    {
        // ...
        "message": "message something { One: 1, Two: 2}",
        "context": {
        "param": "something",
        "complexParam": {
            "One": 1,
            "Two": 2
        }
        }
    }

• Error

  class AuthenticationException extends Exception {
      public AuthenticationException(string username) : base("Invalid credentials") {
          Data.add("username", username);
      }
  }

  var ex = new AuthenticationException("teddy");
  
  logger.LogError(ex, "message {param} {complexParam}", "something", new {
      One = 1,
      Two = 2
  });

  • Expected log

    {
      // ...
      "message": "message something { One: 1, Two: 2}",
      "context": {
        "param": "something",
        "complexParam": {
          "One": 1,
          "Two": 2
        }
      },
      "exception": {
        "type": "AuthenticationException",
        "message": "Invalid credentials",
        "context": {
          "username": "teddy"
        },
        "stacktrace": " at StandardLog.TestBench.AspNetCore.Controllers.WeatherForecastController.Post() in /home/tt/projects/standard-log/dotnet/StandardLog.TestBench.AspNetCore/Controllers/WeatherForecastController.cs:line 63\n   at Microsoft.Extensions.Internal.ObjectMethodExecutor.\u003C\u003Ec__DisplayClass33_0.\u003CWrapVoidMethod\u003Eb__0(Object target, Object[] parameters)\n"
      }
    }