Hey There,
Here's a question that gets asked a lot (and, consequently, answered a lot ;) on the boards. How do you go about dealing with a situation in which you're trying to take care of some business on your Linux or Unix box and you get stopped with the "Argument list too long" error message? It's probably happened to all of us at some point, but it's fairly simple to avoid, and in more than one way. I'm almost positive, even though I know this stuff, that I'll get that error message again at some point in the future. My mind is a terrible thing ;)
The way I try to keep on top of avoiding that error (which, admittedly, isn't all that terrible, assuming its not a predicating factor in a script - like, you don't have any action is your script that might do something destructive if you don't check for errors) is a basic two-prong approach. First, I'll try to figure out what kind of limitations my system can stand. Secondly, I try to stick to some basic methods that guarantee I won't run into the issue no matter what my system's maximum argument length is. Of course, the second part obviates the first part, but I'm a curious guy (take that whatever way you want ;)
1. How can you figure out your systems command line argument length limitations?
The good news is that there a number of ways (and, most probably, much more than I'm going to list here today):
On almost all Unix and Linux systems you can use getconf to find out the answer very simply, like so (all examples today from OpenBSD 4.4-stable):
host # getconf ARG_MAX
262144
RedHat, SUSE, and BSD Linux (just to name a few) also support figuring this out using the sysctl command (for a more in-depth look at this command, check out our old post on using sysctl to work with kernel tunables on Linux:
host # sysctl kern.argmax
kern.argmax=262144
2. How can you avoid ever getting this error in the first place (not to be used as a substitute for doing error checking in your scripts to "make doubly sure" :)
This boils down to, very basically, using three methods: find, xargs or using your shell's basic looping constructs.
For the purposes of all of our examples below we'll assume that you've done something like:
host # ls *
in a directory, with 564,321 files in it, and have received the titular error.
xargs and find work because, used properly, they can reduce that huge list of file names to 564,321 separate lists of one filename each (that's putting it a little simply, but I'm trying to paint a picture here. And, believe me, painting is a lot harder to do while you're typing than it looks ;)
The find command can be used with the -exec option to iterate through the elements of your ls output, like so:
host # find /that/directory -exec ls {} \;
The xargs command will pretty much do the same thing. Some common wisdom (which doesn't work) is to do this:
host # echo /that/directory/*|xargs ls
however this will still give you the error, unless your version of echo can subvert the OS globbing rules and bypass the ARG_MAX without error.
The only real reason to use xargs to get around this issue is if you run into a situation more complicated than find's -exec flag can deal with. This situation doesn't apply, in that respect, so the following solution is, admittedly redundant and overcomplicated:
find /that/directory|xargs ls
Thirdly, you can use shell while looping to get around the error, again using find to make sure you don't get dinged by the maximum argument length error:find /that/directory|while read x
do
ls $x
done
Yee-hah :)
Of course, this discourse has been limited to the basics of the shell and one particular situation. If you use Perl, tcl or any other language (under any number of situations) your options will multiply. As with anything else in Linux or Unix, the possible solutions are, to a large degree, limited only to your imagination.
With these little fires lit, go on out there and (to use a dangerous colloquialism) "burn that mother down" ;)
Cheers,
, Mike
Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.
