I have a question, and I could not find help anywhere on stackoverflow or the Internet.
I have a program (distributed task with celery), and I have several instances (workers), each of which has a log file (celery_worker1.log, celery_worker2.log).
Important errors are stored in the database, but I like to tighten these logs from time to time when starting new operations to make sure everything is in order (the log level is lower).
My problem: these logs take up a lot of disk space. What I would like to do: be able to "watch" the logs (tail -f) only when I need it, without them taking up a lot of space.
My ideas so far:
- log output to stdout, not to a file: this is impossible, since I have many workers that output to different files, but I want to delete them immediately (tail -f celery_worker * .log)
- using logrotate: this is an OK solution for me. I do not want this to be a daily task, but would rather not set a crontab minute for this, and more, the server is not mine, so that would mean some work on the administrator side.
- using named pipes: it looked good at a glance, but I didn't know that named pipes (linux FIFOs) are blocking. Therefore, when I do not find the tail -f of ALL pipes at the same time, or when I just leave my tail, write operations from the recorder are blocked.
, stdout , /dev/null, ?
? , ?
!