81

On ubuntu server here and I'm trying to unpack 300M image files from a compressed file format using Java.

My unpack rate is 0.5Mbytes/sec, abysmal (34 days to unpack 1.5TB at this rate).

I'm trying to figure out why, and the only oddity I notice is that updatedb.mlocate is always working when I'm doing the unpack process. I want to turn it off to see if it's getting in the way, but I don't understand much about what it is.

top

top - 05:16:52 up 1 day,  5:15,  3 users,  load average: 2.00, 2.01, 1.83
Tasks:  83 total,   1 running,  82 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.4%us,  0.8%sy,  0.0%ni,  8.4%id, 90.2%wa,  0.0%hi,  0.0%si,  0.2%st
Mem:   1737420k total,  1722680k used,    14740k free,  1241260k buffers
Swap:   917500k total,      160k used,   917340k free,   165448k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
22901 davidpar  20   0 1051m  56m 4992 S    3  3.3   0:47.84 java
 2221 root      20   0 32348  26m  268 D    1  1.6  27:57.86 updatedb.mlocat
   25 root      20   0     0    0    0 S    0  0.0  10:10.77 kswapd0
  678 root      20   0 15864  444  268 S    0  0.0   0:19.45 irqbalance
  849 davidpar  20   0 26560 1676  332 S    0  0.1  17:17.49 screen

iotop

Total DISK READ: 4.07 M/s | Total DISK WRITE: 789.62 K/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND
 2221 idle root     1556.98 K/s    6.36 K/s  0.00 % 99.61 % updatedb.mlocate
22902 be/4 davidpar    2.54 M/s  671.93 K/s  0.00 % 96.96 % java -cp /home/davidparks21/fruggutils/lib/FruggMapreduceJobs.~educe.UnpackImages /mnt/local/imagebinaries-r-00010 /mnt/ebs1/
  547 be/3 root        0.00 B/s   87.47 K/s  0.00 %  0.30 % [jbd2/xvdf-8]
  177 be/3 root        0.00 B/s    3.98 K/s  0.00 %  0.15 % [jbd2/xvda1-8]
Stefano Palazzo
  • 85,787
  • 45
  • 210
  • 227
David Parks
  • 2,486
  • 10
  • 38
  • 48
  • 1
    This problem has been affecting me for years and I can not understand why updatedb.mlocate run at high IO priority when the script says a idle mode... – Ferran Basora Jun 12 '13 at 23:31
  • 8
    it's a bug. see https://bugs.launchpad.net/ubuntu/+source/mlocate/+bug/1190696 – Thufir Jul 07 '14 at 10:57
  • Why it still the default enabled? Over many years it happens to me every time I create a new system. :-/ – Fernando Kosh Sep 20 '17 at 13:49
  • 2
    For anyone wandering why is this thing needed and whether you break something by switching it off: https://unix.stackexchange.com/a/273283/126119 – Ufos Nov 01 '17 at 13:23
  • mlocate does not seem to be included in Ubuntu 20.04 by default. On the other hand, in Ubuntu 20.10 you could try [plocate](https://plocate.sesse.net/) as an alternative. – jarno Mar 20 '21 at 08:21

4 Answers4

89

It can be killed with:

sudo killall updatedb.mlocate

Or:

sudo kill -9 <PID>

It runs every day by cron. Disable it with:

sudo chmod -x /etc/cron.daily/mlocate

And if you want to re-enable it:

sudo chmod +x /etc/cron.daily/mlocate
Eric Carvalho
  • 53,609
  • 102
  • 137
  • 162
23

I did not want to totally eliminate the process but I did want to make it happen less frequently so I worked out how to set it to run weekly instead of daily. This is based on the accepted answer above but probably best listed as its own answer as it's not disabling it.

That said... It's rather simple and appears to work just fine.

sudo chmod -x /etc/cron.daily/mlocate
sudo cp /etc/cron.daily/mlocate /etc/cron.weekly/mlocate
sudo chmod +x /etc/cron.weekly/mlocate

The first one disables the cron job. The second moves it to the weekly tasks. The third command sets the permissions so that it is enabled. Daily, hourly, weekly, and monthly are all options.

KGIII
  • 3,932
  • 3
  • 20
  • 32
  • 2
    See "[How do I get mlocate to only index certain directories](https://askubuntu.com/q/160424/79344)" for another way to speed it up. – Amir Ali Akbari Oct 10 '17 at 08:50
15

I never used locate, so I removed it. sudo dpkg -P mlocate See Also: https://wiki.archlinux.org/index.php/File_system_search

andras.tim
  • 584
  • 5
  • 18
ant-san
  • 151
  • 1
  • 2
6

If you uninstall locate, mlocate and plocate with e.g.:

sudo apt remove locate mlocate plocate

You can prevent them from being reinstalled with:

cat <<EOF | sudo tee /etc/apt/preferences.d/do-not-install-locate.pref
Explanation: Do not install locate nor mlocate nor plocate in order to avoid excessive Disk I/O
Package: locate mlocate plocate
Pin: origin *
Pin-Priority: -1
EOF
Ganton
  • 385
  • 2
  • 11
davidvandebunte
  • 251
  • 2
  • 7