79

How can I discard the last n lines of a file with a unix command line filter?

That would be sort of the opposite of tail: tail discards the first n lines but pipes the rest through, but I want the command to pipe everything through except the last n lines.

Unfortunately I haven't found anything like that - head doesnt help, too. EDIT: At least in Solaris it does not take negative arguments.

Update: I'm mostly interested in a solution that works for big files, i.e. logfiles, where you might want to inspect what happened except in the last minutes.

Dr. Hans-Peter Störr
  • 2,169
  • 4
  • 19
  • 24

6 Answers6

81

If you have GNU head, you can use

head -n -5 file.txt

to print all but the last 5 lines of file.txt.

If head -n takes no negative arguments, try

head -n $(( $(wc -l file.txt | awk '{print $1}') - 5 )) file.txt
slhck
  • 223,558
  • 70
  • 607
  • 592
35
head file.txt               # first 10 lines
tail file.txt               # last 10 lines
head -n 20 file.txt         # first 20 lines
tail -n 20 file.txt         # last 20 lines
head -20 file.txt           # first 20 lines
tail -20 file.txt           # last 20 lines
head -n -5 file.txt         # all lines except the 5 last
tail -n +5 file.txt         # all lines except the 4 first, starts at line 5
Kjetil S.
  • 549
  • 4
  • 6
14

Here's a simple way to delete the last line, which works on BSD, etc.

sed '$d' input.txt

The expression reads "on the last line, delete it". The other lines will be printed, since that is sed's default behavior.

You could chain them together to remove multiple lines

sed '$d' input.txt | sed '$d' | sed '$d'

Which is a little heavy-handed, admittedly, but does only one scan through the file.

You can also take a look at this, for more answers: https://stackoverflow.com/questions/13380607/how-to-use-sed-to-remove-last-n-lines-of-a-file

Here's a one-liner adapted from one of my favorites there:

count=10
sed -n -e ':a' -e "1,$count"'!{P;N;D;};N;ba'

I had fun deciphering that one, and I hope you do, too (: It does buffer count lines as it scans, but otherwise is pretty efficient.

Note: the ! is quoted using single-quotes to avoid it being interpreted by Bash (if that's your shell). Plain double-quotes around the whole thing works for sh, possibly others.

jwd
  • 3,562
  • 2
  • 22
  • 15
12

On MAC the solution of getting all lines except the last N lines:

head -n -5 file.txt

will not work, as you will get the following error:

head: illegal line count -- -5

One of the solutions is installing coreutils and running ghead:

brew install coreutils
ghead -n -5 file.txt

The GNU Core Utilities or coreutils is a package of GNU software containing reimplementations for many of the basic tools, such as cat, ls, and rm, which are used on Unix-like operating systems.

MikeL
  • 221
  • 2
  • 4
4

I'm curious why you think head is not an option:

~$ man head
...
-n, --lines=[-]K
        print the first K lines instead of the first 10; 
        with the leading `-', print all but the last K lines of each file

This seems to fit your purpose, using, for example:

head -n -20 yourfile.txt
slhck
  • 223,558
  • 70
  • 607
  • 592
3

Another way to do it if tail -n won't take negative arguments is

tac file.txt | tail -n +6 | tac

This would remove the last 5 lines

atw31337
  • 31
  • 1
  • Thanks! That's a nifty idea nobody came up with so far. Unfortunately that would be quite inefficient for the usecase I had in mind with this question: if that's a large file, it would not only be completely read through one or more times, as with the other solutions, but it would probably also be written to disk to temporary files by tac if it doesn't fit into memory. – Dr. Hans-Peter Störr Jan 06 '19 at 19:14
  • @Hans-Peter Very true. Decided to write a python3 script for it. Try this https://github.com/atw31337/donkey. I recommend using the output options. They run much faster than using redirects. – atw31337 Feb 16 '19 at 21:06
  • Nicely written! It does, however, read through the file twice, which isn't really necessary if you buffer the last n lines, and this is a problem on large files. Personally, I don't need that anymore, but in case you have fun improving it and other people need that... There were, after all, a few likes and bookmarks on the question. – Dr. Hans-Peter Störr Feb 19 '19 at 08:50
  • @Hans-Peter. The buffer size would be dependent on the number of lines to be removed. This could be an issue if a very large number of lines needed to be dropped from the file. In order to avoid memory related issues, I rewrote the script to use the line count method with high n values and a buffering method with lower n values; however, after testing it on a very large file, the original line count method is still faster. It seems the buffer management overhead outweighs the line count overhead...or I'm just missing something. – atw31337 Feb 26 '19 at 22:29
  • 1
    Nice, but for BSD-variant Mac OS X there is no tac command by default. :( This kind of defeats the use case. – ingyhere Apr 16 '19 at 04:48
  • Hi, me again. So in Mac OS X `tac` = `tail -r`. So, try this ugliness: `tail -r file.txt | tail -n +6 | tail -r` ... – ingyhere Apr 16 '19 at 04:53
  • On MacOS: bash: tac: command not found – Shai Alon Jul 01 '21 at 10:36