1

I have a Proxmox VM running Ubuntu, which I use to self host some web apps and as a media server.

I've run into a problem whereby the disk has become full. The virtual disk assigned to the VM by Proxmox is 85GB in size.

When I run df -h I get the following output which clearly shows that / is 100% full.

df -h

However, when I run ncdu (with some mounted cifs shares excluded) to try and find the files that are taking the space I get the following output indicating that the total disk usage is only 30.3GB. What's with the discrepancy, and how can I find the missing 50GB of files?

ncdu --exclude /mnt/zpool /

I have run lsof -nP +L1 to see if there's some pending file deletes but can't see anything out of the ordinary there:

lsof -nP +L1

I've tried rebooting but that doesn't seem to help.

Any help/advice appreciated.

Ahmed Ashour
  • 2,350
  • 2
  • 14
  • 21
bananamana
  • 131
  • 1
  • 4
  • Hit Win key, then type `disk usage` and click on the icon that appears. Select `/ ` and let the software churn on it... Find the `Treemap chart` icon in the lower border of the window and click on it... now browse the map for large files (large blobs are interesting, but may be folders, not files). – Hannu Aug 28 '20 at 18:20
  • It's runing on a server so it doesn't have a desktop GUI. I would also guess that ```disk usage``` is using ```du``` in the backend so would have the same results. – bananamana Aug 29 '20 at 09:02
  • "Disk usage Analyzer" <==> apt install baobab, it displays size of files and folders graphically. – Hannu Aug 29 '20 at 15:39
  • Open a terminal/bash -> `find -type f -printf "%16s %p\n" | sort -n | less` will list files and their sizes, largest last - type a `>` to go to the end of the list, when `less`displays the list. – Hannu Aug 29 '20 at 15:41
  • Host OS == Linux? Yes: Do you have a `sshd` running in that VM? Try `ssh -Y ...` into the vm and launch `baobab` from the bash prompt in ssh. – Hannu Aug 29 '20 at 16:05
  • compare `df -h /` with `du -hsx /` . If du is really lower than df, there's probably a deleted file kept by a process, or files kept invisible by a mountpoint – A.B Aug 29 '20 at 16:06

1 Answers1

2

I've figured out my problem. I had mounted a CIF share to /mnt/my_cif_share/ but before mounting this folder already had 50GB of files in it. As far as I can tell mounting over it essentially 'hides' those files from du and similar tools.

To rectify, you can either unmount the share, and delete the 'hidden' files. Or to fix this without unmounting, you can remount your root folder using mount -o bind / /mnt and you will be able to delete the files from there.

I found the answer here: https://serverfault.com/questions/275206/disk-full-du-tells-different-how-to-further-investigate

bananamana
  • 131
  • 1
  • 4