0

I have around 58,000 txt files that have the prefix ppdb- in a directory. I want to merge all of them into one file.
Wgen I try to merge them using cat ppdb-* >> out.txt it gives me an error saying "-bash: /usr/bin/cat: Argument list too long".
Is there a way to merge all the files together efficiently?

Rumesh Madhusanka
  • 152
  • 1
  • 2
  • 8
  • 2
    Have a look at [Cannot merge files using cat ; Argument list too long](https://unix.stackexchange.com/questions/167137/cannot-merge-files-using-cat) – Gounou Sep 20 '21 at 00:53
  • 1
    This answer is covered by this similar question with the same bash error: https://askubuntu.com/a/949552/167115 – mchid Sep 20 '21 at 01:02
  • _Is there a way to merge all the files together efficiently?_ Your question is not a duplicate of the link a give. Maybe there is a way of doing the concatenation more efficiently (the two answers use `cat`). – Gounou Sep 20 '21 at 01:04

1 Answers1

2

Read man find xargs and do something like: (UNTESTED, remove echo when your tests succeed)

find . -maxdepth 1 -name 'ppdb-*' -print0 |\
  xargs -0 -r echo cat >out.txt
waltinator
  • 35,099
  • 19
  • 57
  • 93