0

I am writing some logs in a file using logging module

#filename : demo.py
import logging
#other imports as well
logging.basicConfig(filename="myfile.log",
                    format='%(asctime)s %(funcName)s %(levelname)s %(message)s',
                    filemode='w')
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
logger.debug("Writing logs in this way")

But sometimes my module writes lots of nullvalues

^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@

So I have written a script to remove these values. But when I modify myfile.log my further logs are not stored. I know that this is due to change/close of file stream of myfile.log. Is it possible to modify my log file without hurting or losing my already existing stream which demo.py has. I am using ubuntu : 18.04 and python : 3.6.9
For modification I tried the following ways

  1. I am using script sed -i -e 's/\o00//g' myfile.log to modify file
  2. I manually edit file using vim : vim myfile.log To delete line I used dd then closed using :wq

From a comment https://stackoverflow.com/questions/64875452/modify-a-file-without-changing-its-file-stream I know that both method over write new file over old file. Is there a way to edit file inplace.

Pranjal Doshi
  • 101
  • 3
  • I don't know Python, but I can connect the dots. You're using `filemode='w'` instead of `a`. It's like `>` vs `>>` in [this answer of mine](https://superuser.com/a/1342490/432690). If something truncates the file then your logger will make the file "regain its old size and grow further, missing data will be filled with zeros (in a sparse way, if possible)". What process truncates the file? I don't know, `logrotate` maybe. Also see [how this is flawed in general](https://jdebp.eu/FGA/do-not-use-logrotate.html). – Kamil Maciorowski Nov 17 '20 at 15:29
  • This also means that if you manage to remove these null bytes in place and you will reduce the size of the file, your logger will keep its ever-growing offset inside the file and null bytes will soon appear again. If I'm right, then what you want to do is futile because of `filemode='w'`. – Kamil Maciorowski Nov 17 '20 at 15:36
  • @KamilMaciorowski so if I use `filemode='a'` then it should solve the issue.. right? – Pranjal Doshi Nov 17 '20 at 15:40
  • Probably. Still (from the linked article): "having the log-rotater program separate from the log-writer program results in several unavoidable potential problems". – Kamil Maciorowski Nov 17 '20 at 15:45
  • @KamilMaciorowski Thanks for your help :) – Pranjal Doshi Nov 17 '20 at 15:53

0 Answers0