Force Python’s print function to flush the stream

A useful parameter in Python’s print function to consume stream output in a subsequent program.

Say we have a python program that prints to stdout and we want to write that output into a file.
We can achieve that like this:

%%bash
python -c 'print("great output\nas always")' > file.txt

Which works fine:

%%bash
cat file.txt

great output
as always

Now, if we want to do the same while the program is still running, things are a bit different.
For example, this program will not write to file until we interrumpt it:

%%bash
python -c '\
while True:
    print("great output\nas always")' \
> newfile.txt

That is, the command on the left side of > will be run until it’s done and only then the file will be written.
We want to fix that.
In other words, stream the program output directly into the file while it’s being generated.

I found this solution to work, passing flush=True to the print function:

%%bash
python -c '\
import time
while True:
    print("great output\nas always", flush=True) # <----- HERE
    time.sleep(2)' \
> newfile.txt

As the docstrings say, that forces the stream to be flushed:

print?

Docstring:
print(value, ..., sep=' ', end='\n', file=sys.stdout, flush=False)
Prints the values to a stream, or to sys.stdout by default.
Optional keyword arguments:
file:  a file-like object (stream); defaults to the current sys.stdout.
sep:   string inserted between values, default a space.
end:   string appended after the last value, default a newline.
flush: whether to forcibly flush the stream.
Type:      builtin_function_or_method

An alternative

It turns out that Python has a parameter we can pass in the command line to make it unbuffered and thus achieving the same results. The parameter is -u, as per official documentation: > Force the stdout and stderr streams to be unbuffered. This option has no effect on the stdin stream.

This is how the code would look like:

%%bash
python -u -c '\
import time
while True:
    print("great output\nas always") # <-- No need for flush=True here anymore
    time.sleep(2)' \
> newfile.txt

/Fin

Any bugs, questions, comments, suggestions? Ping me on twitter or drop me an e-mail (fabridamicelli at gmail).
Share this article on your favourite platform: