Adding python logging to FastApi endpoints, hosted on docker doesn't display API Endpoints logs

Question

I have a fastapi app on which I want to add python logging. I followed the basic tutorial and added this, however this doesn't add API but just gunicorn logging.

So I have a local server hosted using docker build so running server using docker-compose up and testing my endpoints using api client (Insomnia, similar to postman). Below is the code where no log file is created and hence no log statements added.

My project str is as follows:

project/
  src/ 
    api/
      models/ 
        users.py
      routers/
        users.py
      main.py
      logging.conf

"""
main.py Main is the starting point for the app.
"""
import logging
import logging.config
from fastapi import FastAPI
from msgpack_asgi import MessagePackMiddleware
import uvicorn

from api.routers import users


logger = logging.getLogger(__name__)


app = FastAPI(debug=True)
app.include_router(users.router)


@app.get("/check")
async def check():
    """Simple health check endpoint."""
    logger.info("logging from the root logger")
    return {"success": True}

Also, I am using gunicorn.conf that looks like this:

[program:gunicorn]
command=poetry run gunicorn -c /etc/gunicorn/gunicorn.conf.py foodgame_api.main:app
directory=/var/www/
autostart=true
autorestart=true
redirect_stderr=true

And gunicorn.conf.py as

import multiprocessing
bind = "unix:/tmp/gunicorn.sock"
workers = multiprocessing.cpu_count() * 2 + 1
worker_class = "uvicorn.workers.UvicornWorker"
loglevel = "debug"
errorlog = "-"
capture_output = True
chdir = "/var/www"
reload = True
reload_engine = "auto"
accesslog = "-"
access_log_format = '%(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"'

This is my output terminal for the above API endpoint on docker:

Could anyone please guide me here? I am new to FastApi so some help will be appreciated.

seems like the logger isn't enabled.

JPG
– JPG

2020-08-20 19:22:32 +00:00
Commented Aug 20, 2020 at 19:22 — JPG
– JPG, Commented Aug 20, 2020 at 19:22
@ArakkalAbu How do you enable logger?

Atihska
– Atihska

2020-08-21 00:36:02 +00:00
Commented Aug 21, 2020 at 0:36 — Atihska
– Atihska, Commented Aug 21, 2020 at 0:36
Also look at loguru.

Yash Nag
– Yash Nag

2021-09-15 17:47:52 +00:00
Commented Sep 15, 2021 at 17:47 — Yash Nag
– Yash Nag, Commented Sep 15, 2021 at 17:47

Dom · Accepted Answer · 2024-01-03 17:31:28Z

72

Inspired by JPG's answer, but using a pydantic model looked cleaner.

You might want to expose more variables. This config worked good for me.

from pydantic import BaseModel

class LogConfig(BaseModel):
    """Logging configuration to be set for the server"""

    LOGGER_NAME: str = "mycoolapp"
    LOG_FORMAT: str = "%(levelprefix)s | %(asctime)s | %(message)s"
    LOG_LEVEL: str = "DEBUG"

    # Logging config
    version: int = 1
    disable_existing_loggers: bool = False
    formatters: dict = {
        "default": {
            "()": "uvicorn.logging.DefaultFormatter",
            "fmt": LOG_FORMAT,
            "datefmt": "%Y-%m-%d %H:%M:%S",
        },
    }
    handlers: dict = {
        "default": {
            "formatter": "default",
            "class": "logging.StreamHandler",
            "stream": "ext://sys.stderr",
        },

    }
    loggers: dict = {
        LOGGER_NAME: {"handlers": ["default"], "level": LOG_LEVEL},
    }

Then import it into your main.py file as:

from logging.config import dictConfig
import logging
from .config import LogConfig

dictConfig(LogConfig().dict())
logger = logging.getLogger("mycoolapp")

logger.info("Dummy Info")
logger.error("Dummy Error")
logger.debug("Dummy Debug")
logger.warning("Dummy Warning")

Which gives:

edited Jan 3, 2024 at 17:31

Dom

133 bronze badges

answered Jun 11, 2021 at 12:30

Yash Nag

1,26513 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Young Emil Over a year ago

how do you then log to a particular file or directory? Any Documentation?

Yash Nag Over a year ago

@YoungEmil See this answer. Just add the file handler and assign it to your logger.

previous_developer Over a year ago

Why would you use a pydantic model for something that's essentially a static dict?

Yash Nag Over a year ago

@previous_developer Used it with BaseSettings rather than BaseModel later. Picks up env vars, easy to write custom ops, it's great.

revy Over a year ago

I'm using pydantic 2.5.2 and I get the following error:

pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `version = 1`. All model fields require a type annotation; if `version` is not meant to be a field, you may be able to resolve this error by annotating i t as a `ClassVar` or updating `model_config['ignored_types']`.

I had to add the type for each class attribute.

JPG · Accepted Answer · 2020-08-21 03:09:04Z

24

I would use dict log config

create a logger config as below,

# my_log_conf.py

log_config = {
    "version": 1,
    "disable_existing_loggers": False,
    "formatters": {
        "default": {
            "()": "uvicorn.logging.DefaultFormatter",
            "fmt": "%(levelprefix)s %(asctime)s %(message)s",
            "datefmt": "%Y-%m-%d %H:%M:%S",

        },
    },
    "handlers": {
        "default": {
            "formatter": "default",
            "class": "logging.StreamHandler",
            "stream": "ext://sys.stderr",
        },
    },
    "loggers": {
        "foo-logger": {"handlers": ["default"], "level": "DEBUG"},
    },
}

Then, load the config using dictConfig function as,

from logging.config import dictConfig
from fastapi import FastAPI

from some.where.my_log_conf import log_config

dictConfig(log_config)

app = FastAPI(debug=True)

Note: It is recommended to call the dictConfig(...) function before the FastAPI initialization.

After the initialization, you can use logger named foo-logger anywhere in your code as,

import logging

logger = logging.getLogger('foo-logger')

logger.debug('This is test')

answered Aug 21, 2020 at 3:09

JPG

89.4k24 gold badges164 silver badges235 bronze badges

4 Comments

Atihska Over a year ago

It doesn't work for me. I only see gunicorn request based HTTP logs and not the ones I print using logging module on path functions.

JPG Over a year ago

Do you see any logs if you are running without docker? btw, this example should work as-is with both gunicorn as well as with uvicorn

JPG Over a year ago

BTW, I did test this solution in the Docker environment too, and it seems fine. You can test the same with this demo project of mine, with branch so-63510041. You will find the commands in the README.md

ScipioAfricanus Over a year ago

is there a way to add response time to the logs?

Collectives™ on Stack Overflow

Adding python logging to FastApi endpoints, hosted on docker doesn't display API Endpoints logs

2 Answers 2

5 Comments

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related