1

I'm using NetApp device as a NAS storage. I have a lot of small files (30-60k) and dirs. For example I have: 30000 dirs with another 1000-2000 dirs.

A lot of dirs are empty, but most of them have many files (30k).

Should I change block size on Ext3?

paszczak000
  • 57
  • 8
  • 2
    See http://serverfault.com/questions/6711/filesystem-for-millions-of-small-files and http://serverfault.com/questions/10698/what-are-the-differences-between-ext3-ext4-reiserfs for possible duplicates. – Richard Keller Jun 01 '12 at 10:41
  • 1
    Possible duplicate of [filesystem for millions of small files](https://serverfault.com/questions/6711/filesystem-for-millions-of-small-files) – phuclv Apr 21 '17 at 06:26

2 Answers2

7

If you're using the NetApp as a NAS, i.e. using NAS protocols such as NFS, CIFS/SMB, http etc. then you don't get to choose the filesystem, it uses NetApp's WAFL file system, you can't change it.

If you're using the NetApp to provide block-level sharing such as iSCSI/Fibre-Channel/FCoE then it's irrelevant that it's on a NetApp - you can assume it's on local disk for all intents. In which case please use our search function as this kind of question has been asked many times before.

Chopper3
  • 101,299
  • 9
  • 108
  • 239
  • This is right however regardless of access type the netapp is designed to use WAFL as the underlying filesystem. Netapp considers insulating block level access as a feature: http://partners.netapp.com/go/techontap/matl/sample/0206tot_resiliency.html – Jim B Jun 01 '12 at 12:29
  • @JimB - yep totally with you there but if it's block at least you get to partition/format those LUNs as you wish, even though that actual block file is indeed on top of WAFL. – Chopper3 Jun 01 '12 at 12:56
  • Yes, I'm using NetApp AS NAS. So I can't change file system. I have only 190 i-nodes free and 70% free space on volume. This is hudge problem. NetApp suggest to not increase free i-nodes over 40%. So I must create new volume? – paszczak000 Jun 02 '12 at 21:17
  • 2
    You can increase the max inodes for a volume using the `maxfiles` command -- you can increase but never decrease, so be careful. The default maxfiles value depends on the volume size, but it should generally be safe to increase by say 10% if you've never done it before. – carillonator Jun 07 '12 at 23:02
1

Based on your requirements, I would recommend ReiserFS. Benchmarks show that ReiserFS performs optimally with a large number of small files compared to other file systems. Ext3 is far from optimal for your scenario, even with a smaller block size.

However, as has already been mentioned in one of the other answers, if you're using a NAS protocol you probably won't be able to choose the file system. If you're using it as a block device, take a look at the following extract from the Linux File System Primer:

EXT2

  • Recommended to move to EXT3
  • Not Journaled
  • POSIX access control

EXT3

  • Most popular Linux file system, limited scalability in size and number of files
  • Journaled
  • POSIX extended access control

JFS

  • High performance and scalability
  • Journaled
  • POSIX extended access controls

ReiserFS

  • Best performance and scalability when number of files is great and/or files are small
  • Journaled
  • POSIX extended access controls

XFS

  • Best for extremely large file systems, large files, and lots of files
  • Journaled (an asymmetric parallel cluster file system version is also available)
  • POSIX extended access controls
Richard Keller
  • 2,040
  • 2
  • 19
  • 31