Skip to content

When using follow=True option to read pod logs, the response stream sometimes returns many log lines at once #2426

@gjohnston9

Description

@gjohnston9

What happened (please include outputs or screenshots):
I'm using read_namespaced_pod_log, setting follow=True and _preload_content=False:

response = client.read_namespaced_pod_log(follow=True, ...)
for log in response.stream():
  ...

We expect each line to be valid JSON. However something changed recently, so that sometimes, a single item returned from the stream contains multiple log lines, so that we fail to parse it. When I run the exact same command but without setting follow=True, and call splitlines() on the whole response, each line is separated correctly.

With Follow=True: (removing a lot of details from these logs, keeping the structure and newlines)

> log_lines = get_logs_for_pods()
(Pdb) !l=next(log_lines)
(Pdb) p l
b'{"level":"info","ts":...,"caller":"...","msg":"...","rel":"...","tr":{"tid":"...","sid":"...","op":"..."}}\n{"level":"info","ts":...,"caller":"...","msg":"...","rel":"...","tr":{"tid":"...","sid":"...","op":"..."}}\n{"level":"info","ts":...,"caller":"...","msg":"...","rel":"snip","tr":{"tid":"...","sid":"...","op":"..."}}\n'

Without:

{"level":"info","ts":...,"caller":"...","msg":"...","rel":"...","tr":{"tid":"...","sid":"...","op":"..."}}

What you expected to happen:
Each item from the stream would be a single log line.

I am not 100% sure if it's expected for the SDK to behave this way or not, based on the documentation. I can split the lines myself, but it would be a lot more intuitive if each item were a single log line.

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Environment:

  • Kubernetes version (kubectl version):
Client Version: v1.30.2
Server Version: v1.32.4-gke.1698000
  • OS (e.g., MacOS 10.13.6): Sequoia 15.5
  • Python version (python --version): python 3.8.13
  • Python client version (pip list | grep kubernetes): 20.13.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.kind/bugCategorizes issue or PR as related to a bug.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions