I am new to Linux. Is it always this slow when moving files?

Question

I am setting up a Linux machine to as a Plex server. I was also hoping to use it to stream steam games.

I am running the following:

Ubuntu 18.04.4 LTS
4 GB RAM 2400mhz
Intel i3-8100
integrated graphics
6 TB hdd

I also have a 1 tb HDD

Whenever I move large large video files from one hard drive to the other, the machine grinds to a halt. The cursor won't even move for seconds or minutes at a time.

I have adjusted the following:

Swappiness to 10 from the default 60
Enabled Drive Caching
Switched I/O Scheduling to [BFQ] (do I have to do this for each drive)

What other information would be useful. I ran "Top" and there was plenty of CPU and Memory available.

Any thoughts? do you need any more information?

Any help would be greatly appreciated. I have an SSD, but I'd rather not put it in this server.

EDIT:

I did Read benchmarks on each of the HDD: they average about 160MBs

free -h. AND -i swap /etc/fstab

Solution: It was a hardware issue.

see https://askubuntu.com/questions/995946/decrease-of-transfer-rate-when-copying-large-amount-of-data/996010#996010 There are many questions about copy speed here, but looks like you found most suggestions with swap and scheduler tweaks. I find moving the copy buffers into userspace helpful (3x speed improvment over worst case). tar and cpio have buffer options, try different sizes, 0.5-2M to see which is best for you. — ubfan1, Apr 04 '20 at 04:39
@ubfan1 too many times I see users setting vm.swappiness to some **crazy** value based on some article they read some where. This is another example. — heynnema, Apr 04 '20 at 14:44
What file system are you using. Post this: `lsblk -o name,fstype,size,fsused,label,mountpoint` — oldfred, Apr 04 '20 at 21:23
i3 CPUs are not designed for fast file service on any OS; Linux is the least worst. You may analyze where your system is being choked to confirm this is the issue. — K7AAY, Apr 10 '20 at 17:28

heynnema · Accepted Answer · 2020-04-10T02:30:22.570

RAM vs swap

Setting vm.swappiness=10 is wrong wrong wrong for 4G RAM with an i3. This is why your file copies fail. Set it back to 60 now... and we may change it more later.
Set drive caching and I/O Scheduling back to default.
free -h and grep -i swap /etc/fstab show me 4G RAM and a 2G /swapfile. The /swapfile is probably too small for your system, so we'll make it twice the size, like this:

Bigger /swapfile

Warning: Be careful. Improper use of the dd command can cause file loss. Best to copy/paste.

free -h                   # confirm 4G RAM and 2G swap
sudo swapoff -a           # turn off swap
sudo rm -i /swapfile      # remove old /swapfile

sudo dd if=/dev/zero of=/swapfile bs=1M count=4096

sudo chmod 600 /swapfile  # set proper file protections
sudo mkswap /swapfile     # init /swapfile
sudo swapon /swapfile     # turn on swap
free -h                   # confirm 4G RAM and 4G swap
reboot                    # reboot and verify operation

Update #1:

Although the larger swap file helped, it didn't entirely fix the problem. User swapped out HDD for SSD, and it's all working ok now. Maybe some bad blocks on the HDD?

Update #2:

To bad block your old HDD...

Note: do NOT abort a bad block scan!

Note: do NOT bad block a SSD

Note: backup your important files FIRST!

Note: this will take many hours

Note: you may have a pending HDD failure

Boot to a Ubuntu Live DVD/USB in “Try Ubuntu” mode.

In terminal...

sudo fdisk -l # identify all "Linux Filesystem" partitions

~~sudo e2fsck -fcky /dev/sdXX # read-only test~~

or

sudo e2fsck -fccky /dev/sdXX # non-destructive read/write test (recommended)

The -k is important, because it saves the previous bad block table, and adds any new bad blocks to that table. Without -k, you loose all of the prior bad block information.

The -fccky parameter...

   -f    Force checking even if the file system seems clean.

   -c    This option causes e2fsck to use badblocks(8) program  to  do  a
         read-only  scan  of  the device in order to find any bad blocks.
         If any bad blocks are found, they are added  to  the  bad  block
         inode  to  prevent them from being allocated to a file or direc‐
         tory.  If this option is specified twice,  then  the  bad  block
         scan will be done using a non-destructive read-write test.

   -k    When combined with the -c option, any existing bad blocks in the
         bad blocks list are preserved, and any new bad blocks  found  by
         running  badblocks(8)  will  be added to the existing bad blocks
         list.

   -y    Assume  an answer of `yes' to all questions; allows e2fsck to be
         used non-interactively.  This option may not be specified at the
         same time as the -n or -p options.

Thank you for taking a look at this. I ran the commands you requested. (See Screenshot at link in question). The I/O schedules reverted on reboot. And I reset the swappiness. — tzarreptar, Apr 04 '20 at 21:07
@ heynnema I followed your recommendation and I confirmed 4G RAM and 4G swap after reboot. While there was a considerable increase in functionality, the computer still freezes when transferring 5gb files. I am gong to try an SSD. — tzarreptar, Apr 09 '20 at 23:13
I changed over the the SSD and the Computer is working perfectly. it must have been my hardware, — tzarreptar, Apr 10 '20 at 00:26
@tzarreptar You must have had some bad blocks on the disk. If you'd like instructions on how to bad block the disk, let me know, and I'll add it to my answer. I updated my answer with your SSD fix. Also, please remove the "Solved" from the subject title, as they really frown about that here. Thanks! — heynnema, Apr 10 '20 at 02:26
@tzarreptar I also went ahead and added Update #2 with bad block instructions, just in case you want to go there. — heynnema, Apr 10 '20 at 02:31
@ heynnema I cannot thank you enough for the time you took to look into this. I'd be surprised if the problem was bad blocks. The HDD was a brand new drive I took out of an iMac I upgraded. In fact, that's where the i3 came from too. — tzarreptar, Apr 11 '20 at 04:40

I am new to Linux. Is it always this slow when moving files?

1 Answers1