Keeping logs clean, rich and useful
With modern logging systems able to parse JSON out of the box, we need to define easily jsonable logs. The alternative to define regex for parsing logs is a rather dreadful fragile experience limited in its nature. Known solutions do not do it consistently and in a type safe way. It is easy to shoot into your own legs with them.
Structured logging became part of std library for golang , as well part of official documentation for python. Golang managed to reach a rather big comfort out of the box, but it has some parts missing… like type safety. U can insert Any objects into logging msgs easily. Python std way is in even more raw state. And they all lack a way to do it in a type safe way.
For reading what is type safety, check hereachivehtmlpdf
Typelog comes to bring best parts out of different solutions, and having added typing safety against shooting into your own legs! Presence of type Any is as maximum as possible removed from its public interface parts.
if u will try inserting invalid to log objects, u will get errors before runtime!
package types
type TaskID string
type WorkerID int
package typedlogs
import (
"log/slog"
"github.com/darklab8/go-typelog/examples/types"
"github.com/darklab8/go-typelog/typelog"
)
func TaskID(value types.TaskID) typelog.LogType { return typelog.String("task_id", string(value)) }
func WorkerID(value types.WorkerID) typelog.LogType {
return typelog.Int("worker_id", int(value))
}
package examples
var logger *typelog.Logger = typelog.NewLogger("worker")
func TestTypedLogs(t *testing.T) {
worker_id := types.WorkerID(5)
logger.Info("Worker was started", typedlogs.WorkerID(worker_id))
logger.Info("Worker was started", 123123, "asdasd") // TYPING ERROR
logger := logger.Log.WithFields(typedlogs.WorkerID(worker_id), typedlogs.TaskID("abc"))
logger.Info("Worker started task")
logger.Info("Worker finished task")
}
In order to use py-typelog in python to its full capacity u will need to turn on mypy or pyright in preferably strict mod. py-typelog comes with mypy typelog-stubs as part of its package.
# types.py
from dataclasses import dataclass
from typing import NewType
TaskID = NewType("TaskID", int)
@dataclass(frozen=True)
class Task:
smth: str
b: int
# logtypes.py
from typing import Any, Dict
from typelog import LogType
from . import types
def TaskID(value: types.TaskID) -> LogType:
def wrapper(params: Dict[str, Any]) -> None:
params["task_id"] = str(value)
return wrapper
def Task(value: types.Task) -> LogType:
def wrapper(params: Dict[str, Any]) -> None:
params.update(value.__dict__)
return wrapper
import logging
import unittest
import typelog
from typelog import LogConfig, Loggers, get_logger
from typelog.types import LibName, LogLevel, RootLogLevel
from . import logtypes, types
logger = get_logger(__name__)
class TestExamples(unittest.TestCase):
def setUp(self) -> None:
Loggers(
RootLogLevel(logging.DEBUG),
LogConfig(LibName("examples"), LogLevel(logging.DEBUG)),
add_time=True,
).configure()
def test_basic(self) -> None:
logger.warn("Writing something", logtypes.TaskID(types.TaskID(123)))
def test_another_one(self) -> None:
task = types.Task(smth="abc", b=4)
logger.warn("Writing something", logtypes.Task(task))
logger.warn("Writing something", 123123, "324234") # TYPING ERROR
Once you tap into the power of context rich logging, you gain another powerful tool for easier program debugging on runtime. The power of it should not be understimated. Some problems are detectable only at runtime, only in production. It is univetable. It is very nice to have useful logs as source of data to investigate when such problems appear. Also, better logging brings easier time debugging unit tests as well, especially if turning third-party libraries to a warning level and turning on your logging level to debug during test runs.