Find files from different directories with inclusion and exclusion of some

Question

I want to find files from three different folders with certain files included & excluded.
Folder 1:

• Path: /var/www/app/var/log/
• Exclude only: file1.log & file2.log

Folder 2:

• Path: /var/log/web/log/
• Include only: error.log

Folder 3:

• Path: /var/log/service/log/
• Include only: app1.error.log & app2.error.log

How would I search files from three different folders in this case?

Answer 1

You can do this with find. Read about general usage of find in the context of excluding and including: Recursively search files with exclusions and inclusions.

Your problem involves three paths and different requirements for each. One can specify many starting points in a single invocation of find and build an expression (e.g. with -path) so different tests are performed for different paths. It's possible but rather cumbersome. A straightforward way is to run three separate find commands in sequence.

find /var/www/app/var/log ! -name file1.log         ! -name file2.log
find /var/log/web/log       -name error.log
find /var/log/service/log   -name app1.error.log -o   -name app2.error.log

Notes:

• Multiple spaces in the above commands are only to emphasize their structure and similarities.

• You say "I want to find files". The above commands will find files including files of the type directory; in particular the first command will find its own starting point (…/log). To find regular files only you need -type f. To find non-directories only you need ! -type d.

• If file1.log or app1.error.log is of the type directory then it's not clear if "included" or "excluded" refers to the directory alone or to the directory and everything below. This is probably not an issue because most likely file1.log and such are regular files in your setup.

  For the record, each of the above commands traverses the entire directory tree under its starting point. E.g. if file1.log is of the type directory then ! -name file1.log will not stop find from testing file1.log/foo. To exclude a subtree you need -prune.

• find works recursively. The simplest way to make GNU find not descend into subdirectories is -maxdepth 1. Non-GNU implementations may not support it. With purely POSIX find it's not that simple.

  You tagged ubuntu; Ubuntu provides GNU find out of the box so use -maxdepth 1 freely.

• Proper usage of -o is not as straightforward as you may wish.

• If there are symbolic links involved, read about -H and -L options in man 1 find. You may want to use one of them.

---

With all this in mind I can craft an example variant of the third command that maybe fits your needs better:

find /var/log/service/log/ -maxdepth 1 -type f \( -name app1.error.log -o -name app2.error.log \)

Or POSIX-ly (to make the answer more useful for a general audience):

find /var/log/service/log/. \( -name . -o -prune \) -type f \( -name app1.error.log -o -name app2.error.log \)

The last component of the starting point in the latter command ends is .. This allows us to use a generic solution (I mean the form -name . -o -prune does not depend on the really meaningful part of the starting point path). On the other hand find will include /. in every pathname, you may not want this.

---

Running three finds instead of one does not prevent you from piping the results to a single destination (e.g. xargs):

{ find … ; find … ; find … ; } | xargs …

Note in Ubuntu you can do find … -print0 | xargs -r0 …. This is more robust than find … -print | xargs …, not portable though.