Geek asks and answers

Real-time carriage return removal in the shell

In context, I am trying to create a shell script that simplifies ffmpeg console output in real time by only displaying the current encoded frame. My ultimate goal is to use this information in some kind of progress indicator for batch processing.

For those unfamiliar with ffmpeg, it displays encoded video information on stdout and console information on stderr. Also, when it actually gets the encoding information, it uses a carriage return to prevent the console screen from filling up. This makes it impossible to simply use grep and awk to capture relevant line and frame information.

The first thing I tried was to replace the carriage return with tr:

$ ffmpeg -i "ScreeningSchedule-1.mov" -y "test.mp4" 2>&1 | tr '\r' '\n'

This works in that it displays the output in real time to the console. However, if I then pass this information to grep or awk or something else, tr output is buffered and no longer works in real time. For example: $ ffmpeg -i "ScreeningSchedule-1.mov" -y "test.mp4" 2>&1 | tr '\r' '\n'>log.txtleads to a file that is immediately filled with some information, and then after 5-10 seconds more lines get into the log file.

At first I thought that it would be great for this: $ # ffmpeg -i "ScreeningSchedule-1.mov" -y "test.mp4" 2>&1 | sed 's/\\r/\\n/'but he gets in line with all the carriages returned and waits until the processing is complete before she tries to do anything. I assume that this is because sed works on a “one-by-one” basis and needs to complete the entire line before it does anything else, and then it does not replace the carriage return anyway. I tried a different regex for carriage return and newline and have not yet found a solution that replaces carriage return. I am running OSX 10.6.8, so I am using BSD sed, which may explain this.

I also tried to write information to a log file and use tail -fit to read it, but I still run into the problem of replacing carriage returns in real time.

, python perl, . -, python perl. -, , , , python/perl. , , , , . , bash, OSX .

, , , , , .

+5
3

. gawk ( BSD awk) mawk, . , :

... | gawk '1;{fflush()}' RS='\r\n' > log.txt

, awk , , ...

... | awk '{sub(/\r$/,x); print>>f; close(f)}' f=log.out

, , bash:

... | while IFS= read -r line; do printf "%s\n" "${line%$'\r'}"; done > log.out
+4

Libc , stdout stderr ( 4 ) . , , : ffmpeg , , tr.

unbuffer ffmpeg -i "ScreeningSchedule-1.mov" -y "test.mp4" 2>&1 | tr '\r' '\n'
stdbuf -e0 -o0 ffmpeg -i "ScreeningSchedule-1.mov" -y "test.mp4" 2>&1 | tr '\r' '\n'

unbuffer stdbuf, .

+4

, (Fedora 17) :

$ ulimit -a | grep pipe
pipe size            (512 bytes, -p) 8
$ ulimit -p 1
bash: ulimit: pipe size: cannot modify limit: Invalid argument
$

, , , , ( ).

, , . , , / , stdin/stdout, .

perl script, :

#!/usr/bin/perl
use strict;
use warnings;

use Term::ReadKey;
$ReadKeyTimeout = 10; # seconds

$| = 1; # OUTPUT_AUTOFLUSH

while( my $key = ReadKey($ReadKeyTimeout) ) {
        if ($key eq "\r") {
                print "\n";
                next;
        }
        print $key;
}

However, as a pointer already, you must make sure that ffmpeg does not buffer its output if you want to respond in real time.

+1
source

More articles:


All Articles