Delete all files except the newest 3 in bash script
This will list all files except the newest three:
ls -t | tail -n +4
This will delete those files:
ls -t | tail -n +4 | xargs rm --
This will also list dotfiles:
ls -At | tail -n +4
and delete with dotfiles:
ls -At | tail -n +4 | xargs rm --
But beware: parsing ls
can be dangerous when the filenames contain funny characters like newlines or spaces. If you are certain that your filenames do not contain funny characters then parsing ls
is quite safe, even more so if it is a one time only script.
If you are developing a script for repeated use then you should most certainly not parse the output of ls
and use the methods described here: http://mywiki.wooledge.org/ParsingLs
How to delete all files except the N newest files?
You can use this command,
ssh -t xxx.xxx.xxx.xxx "cd /directory_wanted; ls -t *.tgz | tail -n
+11 | xargs rm -f; bash"
In side quotes, we can add what ever the operations to be performed in remote machine. But every command should be terminated with semicolon (;)
Note: Included the same command suggested by silentMonk. It is simple and it is working. But verify it once before performing the operation.
how to delete all files except the latest three in a folder
Alright, there are a few things wrong with your script.
First, and most problematically, is this line:
ls -t /var/path/to/folder |head -n 3;
ls -t
will return a list of files in order of their last modification time, starting with the most recently modified. head -n 3
says to only list the first three lines. So what this is saying is "give me a list of only the three most recently modified files", which I don't think is what you want.
I'm not really sure what you're doing with the second ls
command, but I'm pretty sure that's just going to concatenate all the files in the directory into your list. That means when it gets sort
ed and uniq
'ed, you'll just be left with an alphabetical list of all the files in that directory. When this gets passed to something like xargs rm
, you'll wipe out everything in that directory.
Next, sort | uniq
doesn't need the uniq
part. You can just use the -u
switch on sort
to get rid of duplicates. You don't need this part anyway.
Finally, the actual removal of the directory. On that part, you had it right in your question: just use rm -r
Here's the easiest way I can think to do this:
ls -t1 /var/path/to/folder | tail -n +4 | xargs rm -r
Here's what's happening here:
ls -t1
is printing a list, one file/directory per line, of all files in/var/path/to/folder
, ordering by the most recent modification date.tail -n +4
is printing all lines in the output ofls -t1
starting with the fourth line (i.e. the three most recently modified files won't be listed)xargs rm -r
says to delete any file output from thetail
. The-r
means to recursively delete files, so if it encounters a directory, it will delete everything in that directory, then delete the directory itself.
Note that I'm not sorting anything or removing any duplicates. That's because:
ls
only reports a file once, so there are no duplicates to remove- You're deleting every file passed anyway, so it doesn't matter in what order they're deleted.
Does all of that make sense?
Edit:
Since I was wrong about ls
specifying the full path when passed an absolute directory, and since you might not be able to perform a cd
, perhaps you could use tail
instead.
For example:
ls -t1 /var/path/to/folder | tail -n +4 | xargs find /var/path/to/folder -name $1 | xargs rm -r
Delete all but the most recent X files in bash
For Linux (GNU tools), an efficient & robust way to keep the n
newest files in the current directory while removing the rest:
n=5
find . -maxdepth 1 -type f -printf '%T@ %p\0' |
sort -z -nrt ' ' -k1,1 |
sed -z -e "1,${n}d" -e 's/[^ ]* //' |
xargs -0r rm -f
For BSD, find
doesn't have the -printf
predicate, stat
can't output NULL bytes, and sed
+ awk
can't handle NULL
-delimited records.
Here's a solution that doesn't support newlines in paths but that safeguards against them by filtering them out:
#!/bin/bash
n=5
find . -maxdepth 1 -type f ! -path $'*\n*' -exec stat -f '%.9Fm %N' {} + |
sort -nrt ' ' -k1,1 |
awk -v n="$n" -F'^[^ ]* ' 'NR > n {printf "%s%c", $2, 0}' |
xargs -0 rm -f
note: I'm using bash
because of the $'\n'
notation. For sh
you can define a variable containing a literal newline and use it instead.
Solution for UNIX & Linux (inspired from AIX/HP-UX/SunOS/BSD/Linux ls -b
):
Some platforms don't provide find -printf
, nor stat
, nor support NULL
-delimited records with stat
/sort
/awk
/sed
/xargs
. That's why using perl
is probably the most portable way to tackle the problem, because it is available by default in almost every OS.
I could have written the whole thing in perl
but I didn't. I only use it for substituting stat
and for encoding-decoding-escaping the filenames. The core logic is the same than the other solutions above and is implemented with POSIX tools.
BTW, perl
's default stat
has a resolution of a second, but starting from perl-5.8.9
you can get sub-second resolution with the the stat
function of the module Time::HiRes
(when the OS and the filesystem support it). That's what I'm using here; if your perl
doesn't provide it then you can remove the ‑MTime::HiRes=stat
from the command line.
n=5
find . '(' -name '.' -o -prune ')' -type f \
-exec perl -MTime::HiRes=stat -l -e '
foreach (@ARGV) {
@st = stat($_);
if ( @st > 0 ) {
s/([\\\n])/sprintf( "\\%03o", ord($1) )/ge;
print sprintf( "%.9f %s", $st[9], $_ );
}
else { print STDERR "stat: $_: $!"; }
}
' {} + |
sort -nrt ' ' -k1,1 |
sed -e "1,${n}d" -e 's/[^ ]* //' |
perl -l -ne '
s/\\([0-7]{3})/chr(oct($1))/ge;
s/(["\n])/"\\$1"/g;
print "\"$_\"";
' |
xargs -E '' sh -c '[ "$#" -gt 0 ] && rm -f "$@"' sh
Explanations:
For each file found, the first
perl
gets the modification time and outputs it along the encoded filename (eachnewline
andbackslash
characters are replaced with the literals\n
and\\
respectively).Now each
time filename
is guaranteed to be single-line, so POSIXsort
andsed
can safely work with this stream.The second
perl
decodes the filenames and escapes them for POSIXxargs
.Lastly,
xargs
callsrm
for deleting the files. Thesh
command is a trick that preventsxargs
from runningrm
when there's no files to delete.
Delete all folders execpt the last 10 versions
To preserve 10 directories (or fewer) and assuming GNU tools are available, you could us this null-terminated pipeline if there are no other files in this directory:
printf '%s\0' * | sort -zVr | tail -zn+11 | xargs -r0 rm -r --
Related Topics
Sort Logs by Date Field in Bash
Need to Intercept Hid Keyboard Events (And Then Block Them)
How to Limit File Size on Commit
Increase of Virtual Memory Without Increse of Vmsize
How to Check If Ssh-Agent Is Already Running in Bash
Using Curl in a Bash Script and Getting Curl: (3) Illegal Characters Found in Url
How to Forward Localhost Port on My Container to Localhost on My Host
How to Signal Handler to Survive After "Exec"
What Is the Linux Built-In Driver Load Order
Docker Alpine Executable Binary Not Found Even If in Path
How to Use Linux Software Watchdog
Linux: Block Until a String Is Matched in a File ("Tail + Grep with Blocking")
Apache Server Keeps Crashing, "Caught Sigterm, Shutting Down"
How to Get Time Since File Was Last Modified in Seconds with Bash
Position of a String Within a String Using Linux Shell Script
Sed: -I May Not Be Used with Stdin on MAC Os X
Maintain File and Folder Permissions Inside Archives
How to Connect to the Fitbit Zip Over Bluetooth 4.0 Le on Linux with Bluez