unlog

Unlog - Unstructured Logging for Go log/slog

This package provides support for removing the structure from structured go logs. It uses Go templates to expand structured logs into text, and applies colorization using github.com/liamg/tml and line wrapping using github.com/manifoldco/ansiwrap.

It supports both short and long unstructuring templates present as comments in the code, and the tool cmd/unlog can be used to extract those templates, do linting across a source based, and manually unstructure input logs.

Installing the tool

go install bitbucket.org/fufoo/unlog/cmd/unlog@latest

Collecting a message catalog

For example, to collect messages across an entire repo

unlog collect -l -c messages.unl $(find . -name '*.go')

Using unstructurable errors

import (
    "bitbucket.org/fufoo/unlog/errors"
)

func DoIt(x int) error {
    if x < 0 {
        // FOO-1234 the value {{.x}} is negative, which
        // is not a supported configuration.
        //
        // Calling DoIt({{.x}}) is not supported; this functionality
        // only applies to non-negative values
        return errors.New("it doesn't work",
            slog.String("msgid", "FOO-1234"),
            slog.Int("x", x))
    }
}

Using a message catalog

From the command line

Using the command line tool:

unlog format -c messages.unl somelogs.json

Programatically

To programmatically make use of a message catalog, the most convenient approach is to use the new "embed" feature of Go to embed the message catalog into the executable, and then load it into an in-memory message catalog something like this:

import (
    _ "embed"
    "os"

    "bitbucket.org/fufoo/unlog"
)

//go:embed messages.unl
var ourCatalog []byte

func explain(err error) {
    cat := unlog.New()
    cat.Load(ourCatalog)
    wr := unlog.NewWriter(os.Stdout, unlog.WithCatalog(cat))
    wr.Explain(err)
}

Working with git

If you are committing your message catalog to the source repository using git, then add a .gitattributes file to mark it as binary:

*.unl binary

Usage with bazel

Since this is a bazel repo as well as a golang module, when consuming this functionality in bazel, instead of using gazelle (TBH I don't know how to use gazelle to load this module; if you figure it out, let me know; it seems that since this is also a golang module, either approach should work but I get an error about not being able to resolve @rules_go when using gazelle 🤷), add this to your MODULE.bazel:

bazel_dep( name = "fufoo_unlog", version = "0.0.1" )

git_override(
    module_name = "fufoo_unlog",
    remote = "[email protected]:fufoo/unlog.git",
    commit = "c7278db5c7b403f630d2873b7eb7f3d0c6f55f75",
)

then you can express a dependency in a BUILD file like so:

    deps = [
        "@fufoo_unlog//:unlog",
    ],

For embedded message catalogs, don't forget to include the embedsrcs in your build, e.g.:

load("@rules_go//go:def.bzl", "go_binary")

go_binary(
    name = "app",
    srcs = ["app.go"],
    visibility = ["//visibility:public"],
    deps = [
        "//:unlog",
    ],
    embedsrcs = ["messages.unl"],
)

Usage outside of bazel

For non-bazel users - i.e., basically everybody - we include the generated protobuf files.

Local development

To test local changes on another repo, first compile a local binary using bazel. From the root of this project, run:

bazel build //cmd/unlog

Then symlink this for ease of use:

mkdir bin
ln -s <your-path-to-project>/bazel-bin/cmd/unlog/unlog_/unlog bin/unlog

From here you can call the binary on other repos. From the root of this project:

./bin/unlog collect ../reka-streams/pkg/