Can structured logging be done with Pythons standard library?

Issue

I recently read about structured logging (here). The idea seems to be to log not by appending simple strings as a line to a logfile, but instead JSON objects. This makes it possible to analyze the logfile by automatic tools.

Can Pythons logging library do structured logging? If not, is there a “mainstream” solution for it (e.g. like numpy/scipy is the mainstream solution for scientific calculations)? I found structlog, but I’m not sure how widespread it is.

Solution

If you install python-json-logger (288 stars, 70 forks) and have a logging configuration (YAML) like the following, you will get a structured logging file.

version: 1
formatters:
    detailed:
        class: logging.Formatter
        format: '[%(asctime)s]:[%(levelname)s]: %(message)s'
    json:
        class: pythonjsonlogger.jsonlogger.JsonFormatter
        format: '%(asctime)s %(levelname)s %(message)s'
handlers:
    console:
        class: logging.StreamHandler
        level: INFO
        formatter: detailed
    file:
        class: logging.FileHandler
        filename: logfile.log
        level: DEBUG
        formatter: json
root:
    level: DEBUG
    handlers:
        - console
        - file

Exceptions

You might also want to make exceptions / tracebacks use the structured format.

See Can I make Python output exceptions in one line / via logging?

Answered By – Martin Thoma

This Answer collected from stackoverflow, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0

Leave a Reply

(*) Required, Your email will not be published