22

I use RAM for my tmpfs /tmp, 2GB, to be exact. Normally, this is enough but sometimes, processes create files in there and fail to cleanup after themselves. This can happen if they crash. I need to delete these orphaned tmp files or else future process will run out of space on /tmp.

How can I safely garbage collect /tmp? Some people do it by checking last modification timestamp, but this approach is unsafe because there can be long-running processes that still need those files. A safer approach is to combine the last modification timestamp condition with the condition that no process has a file handle for the file. Is there a program/script/etc that embodies this approach or some other approach that is also safe?

Incidentally, does Linux/Unix allow a mode of file opening with creation wherein the created file is deleted when the creating process terminates, even if it's from a crash?

Syncopated
  • 443
  • 1
  • 3
  • 9

8 Answers8

21

You might want to try something like that:

find /tmp -mtime +7 -and -not -exec fuser -s {} ';' -and -exec echo {} ';'

find is used to find files that match certain criteria.

  • -mtime +7 only selects files that are older than 7 days (you may use any other value)
  • -exec fuser -s {} ';' calls fuser in silent mode for every file that matches the oldness criteria. fuser returns 0 (=true) for every file that's been accessed right now and 1 (= false) for the unaccessed ones. As we are only interested in the unaccessed ones, we put a -not in front of this -exec
  • -exec echo {} ';' just prints all file names matching the criteria. you might want use -exec rm {} ';' instead here, but as this may delete some still-in-use files, I think it's safer to do a simple echo first.
  • edit: You might want to add something like -name 'foo*.bar' or -uid 123 to limit the effects of the cleanup to specific file patterns or user IDs to avoid accidental effects.

To the last point: Consider that there might be files that are only written once (e.g. at system boot) but read frequently (e.g. any X-session-cookie). Therefore I recommend adding some name checks to only affect files created by your faulty programs.

edit2: To your last question: A file won't get deleted from disk until no process has an open handle to it (at least for native linux filesystems). The problem is that the directory entry is removed immediately which means that from the time you remove the file no new processes can open the file anymore (as there's no filename attached to it).

For details see: https://stackoverflow.com/questions/3181641/how-can-i-delete-a-file-upon-its-close-in-c-on-linux

edit3: But what if I wanted to automate the whole process?

As I said, there might be files that are written once and then read every once in a while (e.g. X session cookies, PID files, etc.). Those won't be excluded by this little removal script (which is the reason why you might wanna do a test run with echo first before actually deleting files).

One way to implement a safe solution is to use atime.
atime stores the time each file was last accessed. But that file system option often is disabled because it has quite some performance impact (according to this blog somewhere in the 20-30% region). There's relatime, but that one only writes the access time if mtime has changed, so this one won't help us.

If you want to use atime, I'd recommend to have /tmp on a separate partition (ideally a ramdisk) so that the performance impact on the whole system isn't too big.

Once atime is enabled, all you have to do is to replace the -mtime parameter in the above command line with -atime.
You might be able to remove the -not -exec fuser -s {} ';', but I'd keep it there just to be sure (in case applications keep files open for a long period of time).

But keep in mind to test the command using echo before you end up removing stuff your system still needs!

Community
  • 1
mreithub
  • 381
  • 2
  • 4
  • nice. What about files closed by a long-running process while it is not updating them? If they are context files, you could loose process context (admittedly, not a very smart process this; but one needs to know expected side-effects of a 'lateral' `/tmp/` cleanup) . – nik Nov 03 '12 at 17:56
  • That is the problem of this approach (as I was pointing out in the last paragraph). The best approach here would afaik be to add uid/gid or file pattern checks (edited the answer accordingly) – mreithub Nov 03 '12 at 18:06
  • Should this be placed into a cron script...? – CMCDragonkai Jun 05 '14 at 06:55
  • @CMCDragonkai Of course you can put this into crontab. But as I mentioned there might be files that are accessed but not written and therefore might not be filtered out by this little script. That's why it's safer to print the list of affected files first and then decide yourself whether to delete them or not. If your `/tmp` is on a separate partition (e.g. a ramdisk), you could enable `atime` for it and use the `-atime` parameter of `find`. – mreithub Jun 05 '14 at 10:15
  • I'm planning to do this on a server. Therefore I can't be there to count all the files in tmp all the time. Would there be any issues? Also I thought we were meant to use relatime not atime? – CMCDragonkai Jun 05 '14 at 12:19
  • @CMCDragonkai I can't say I'm an expert on `atime`. It was just meant as a hint to another possible solution for your problem. – mreithub Jun 05 '14 at 12:22
  • @CMCDragonkai `relatime` doesn't seem to work in our case as it only updates `atime` once after each file change. I've updated the answer to give you a little more information. – mreithub Jun 05 '14 at 12:54
  • I guess atime isn't too much of an issue if you're using tmpfs or ramfs. I'm using tmpfs of course. Although I wish there would be some benchmarks showing the trade offs. – CMCDragonkai Jun 05 '14 at 13:03
  • When you shutdown with files inside tmpfs/ramfs, does this mean the file exists on disk? – CMCDragonkai Jun 24 '14 at 06:23
  • @CMCDragonkai No, those filesystems are RAM-only. As soon as it's unmounted, all the data is lost – mreithub Jun 24 '14 at 06:45
  • Right and it gets unmounted during a shutdown! – CMCDragonkai Jun 24 '14 at 08:19
5

Don't roll your own.

Debian/Ubuntu have tmpreaper, it's probably available in other dists as well.

# tmpreaper - cleans up files in directories based on their age

sudo apt-get install tmpreaper

cat /etc/tmpreaper.conf
Gringo Suave
  • 1,309
  • 11
  • 10
  • In the `/etc/tmpreaper.conf` file, if I set both `/tmp` and `/var/tmp` as the cleanup directories, can long do you recommend for the `TMPREAPER_TIME` parameter or the max ago of the tmp files to be removed? I have heard it's better to keep a longer age for `/var/tmp` files than `/tmp` files. But if they can only be set up with the same max age, I have no clue. – Xiaodong Qi Apr 19 '17 at 05:30
  • A typical interval would be two weeks, but one or two months is safer and probably fine unless you are low on space. – Gringo Suave Jun 20 '22 at 23:15
2

Regarding the last part of your question:

While I don’t think that an ‘delete-this-if-I-die’ open/creation mode exists, a process can safely delete a file directly after creating it, as long as it keeps a handle to said file open. The kernel will then keep the file on disk and as soon as the last process who had opened the file exits (be it by crash or normally), the space occupied by the file will be freed.

For a general way around the problem that some processes sometimes don’t clean up /tmp, I would suggest having a look at mount namespaces, described, for example here or here. If the process in question is a system daemon, systemd and its native feature to allow private /tmp filesystems might be of interest.

Community
  • 1
Claudius
  • 8,808
  • 1
  • 15
  • 13
0

You could just do rm -rf /tmp/* and hope nothing breaks...

Solomon Ucko
  • 103
  • 1
  • 5
  • 3
    Suggesting to do something "and hope nothing breaks" doesn't really answer OP's "is there are a **safe way** to do this. Perhaps you could elaborate as to why your suggestion is safe? – bertieb Jul 03 '18 at 21:57
  • @bertieb Good point. I guess it's probably safe if it's not run as root, but... – Solomon Ucko Jul 03 '18 at 23:03
0

use built-in tmpfiles.d instead of. Check this answer: https://askubuntu.com/a/857154/453746

I'm checking this on Ubuntu 16.10. I can certify that editing /etc/default/rcS has no effect at all anymore and the files in tmp are wiped out by reboot no matter what you put in that file. As others mention, tmpreaper is no longer used.

I think the right answer is that Ubuntu 16.10 has a new setup. There is a folder /etc/tmpfiles.d, documented in the man page "tmpfiles.d". In that folder, one should place a configuration file to control whether the /tmp is to be erased. This is what I am doing to stop reboots from erasing files in /tmp unless they are 20 days old:

#/etc/tmpfiles.d/tmp.conf

d /tmp 1777 root root 20d Replace "20d" by "-" if you never want files deleted. This is my best effort, that man page is nearly impenetrable with detail.

The advantage of the new setup is that a file cleaner can still run even if the system is not rebooted (as in the case of an always on server). That's a big plus, I think.

meotimdihia
  • 134
  • 1
  • 7
  • Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Jun 23 '22 at 09:36
0

for a GUI try this; http://bleachbit.sourceforge.net/

cleans and scrubs. preview mode.

wuxmedia
  • 231
  • 2
  • 5
0

Obtain a list of files older than so an so, exclude files that are open by anything from that list: 

find /tmp -mtime +7 |\
    egrep -v "`lsof -n +D /tmp | awk 'NR>1 {print $9}'| tr \\n \|`"

lsof -n +D /tmp: look for open files in /tmp
awk 'NR>1 {print $9}': print only the ninth column of lsof output, excluding the headers
tr \\n \|: replace new-line by bar (OR in egrep)
egrep -v "foo|moo|bar": print lines NOT containing foo or moo or bar

Ярослав Рахматуллин
  • 11,096
  • 5
  • 42
  • 74
0

I agree with the above, to add to it though- I always run lsof +L1 | grep tmp and either kill or restart the processes holding on to "deleted" tmp files: EXAMPLE-

# lsof +L1 | grep tmp
xfce4-ter  1699  user   32u   REG    8,6      192     0 818552 /tmp/vte966VLX (deleted)
chrome     3301  user  138u   REG    8,6    16400     0 818547 /tmp/etilqs_Z0guKD7p6ork9iG (deleted)
SeaPhor
  • 1