2

I have a directory structure as follows:

backup-2018-01-12
backup-2018-01-13
backup-2018-01-14
backup-2018-01-15
backup-2018-01-16
backup-2018-01-17
backup-2018-01-18
backup-2018-01-19
backup-2018-01-20
backup-2018-01-21
backup-2018-01-22
backup-2018-01-23
backup-2018-01-24
backup-2018-01-25
backup-2018-01-26
backup-2018-01-27
backup-2018-01-28
backup-2018-01-29
backup-2018-01-30
backup-2018-01-31
backup-2018-02-01
backup-2018-02-02
backup-2018-02-03
backup-2018-02-04
backup-2018-02-05
backup-2018-02-06
backup-2018-02-07
backup-2018-02-08
backup-2018-02-09
backup-2018-02-10
backup-2018-02-11
backup-2018-02-12
backup-2018-02-13
backup-2018-02-14
backup-2018-02-15

How can I use a script to save the latest 7 days and the the last backup of each week for 4 weeks long

E.g

So I keep

backup-2018-01-25  <-- this is 3 waeks from now and so on. 
backup-2018-02-02  <-- this is 2 weeks earlier from now
backup-2018-02-09
backup-2018-02-10
backup-2018-02-11
backup-2018-02-12
backup-2018-02-13
backup-2018-02-14
backup-2018-02-15

So I tried

find -type d -name 'backup-*' -mtime +7 -exec rm -v {} \;

and this does keep the latest 7 but removes everything older than 7 days.

Improve this question
3
  • 1
    Some use dedicated backup software for these types of tasks (i.e. backing up data and keeping backups of certain ages), like borgbackup or borgmatic (which uses borgbackup). Commented Feb 15, 2018 at 13:14
  • I see not obvious tools for the jobs, facing similar problem i wrote a c program. Commented Feb 15, 2018 at 13:22
  • This is the first hit on Google for this type of backup retention which is my primary requirement. However, I'm looking for something with a GUI, unless I'm forced to use a script so I created a post in SE-software-reqs here: softwarerecs.stackexchange.com/questions/84066/… Commented Sep 22, 2022 at 2:19

3 Answers 3

Reset to default
2

I haven't been able to find a straightforward solution. The easiest way I found to solve it was to use a "keep" list rather than a "remove" list. Then I could add the last N backups to the list, and the newest of each week's range for M weeks. (In my own code, written some time ago, I extended to months, quarters, and years.) Obviously there will be some overlap but because it's a "keep" list that doesn't matter. Then you delete everything that isn't in the list - having first checked the list isn't empty.

#!/bin/bash
#
dirs=(
    backup-2018-01-12
    backup-2018-01-13
    backup-2018-01-14
    backup-2018-01-15
    backup-2018-01-16
    backup-2018-01-17
    backup-2018-01-18
    backup-2018-01-19
    backup-2018-01-20
    backup-2018-01-21
    backup-2018-01-22
    backup-2018-01-23
    backup-2018-01-24
    backup-2018-01-25
    backup-2018-01-26
    backup-2018-01-27
    backup-2018-01-28
    backup-2018-01-29
    backup-2018-01-30
    backup-2018-01-31
    backup-2018-02-01
    backup-2018-02-02
    backup-2018-02-03
    backup-2018-02-04
    backup-2018-02-05
    backup-2018-02-06
    backup-2018-02-07
    backup-2018-02-08
    backup-2018-02-09
    backup-2018-02-10
    backup-2018-02-11
    backup-2018-02-12
    backup-2018-02-13
    backup-2018-02-14
    backup-2018-02-15
)
declare -A list
declare -A keep


####################################################################################
# Go
#
prefix="backup-"
today=$(date +%Y-%m-%d)


# Populate list from set of directories
#
for item in "${dirs[@]}"
do
    list[$item]="$item"
done


# Build keep list from business criteria
#
for dayback in {0..6}
do
    dateback=$(date --date "$today - $dayback day" +%Y-%m-%d)
    keep[$prefix$dateback]=1
done

for weekback in {0..3}
do
    dateback1=$(date --date "$today - $weekback week" +%Y-%m-%d)
    dateback2=$(date --date "$today - $((weekback + 1)) week" +%Y-%m-%d)

    # Look for newest in range
    #
    newest=
    dateback="$dateback1"
    while [[ $dateback != $dateback2 ]]
    do
        [[ -n ${list[$prefix$dateback]} ]] && newest=$dateback && break
        dateback=$(date --date "$dateback - 1 day" +%Y-%m-%d)
    done

    [[ -n $newest ]] && keep[$prefix$newest]=1
done


# We now have a keep list, so delete anything not in that list
#
for item in "${list[@]}"
do
    [[ -n ${keep[$item]} ]] && echo "KEEP $item" || echo "DELETE $item"
done


# All done
#
exit 0

When you're happy, change or extend the echo statements towards the end to perform the actual deletion.

Improve this answer
7
  • I have used a similar solution in the past, where I kept directories for daily, weekly, and monthly backups, and just moved files around, and kept various numbers of files in each directory. I like the idea of a keep list, though. Commented Feb 15, 2018 at 15:52
  • @roaima awesome! It almost working. What if directory is /some/path/backup-* how to dael with this? Commented Feb 17, 2018 at 18:14
  • I think I fixing the problem. Thanks Roaima Commented Feb 18, 2018 at 14:05
  • @roaima , How can i make this work when dir is like dirs= ( /var/lib/somepath/data/*/*/*/backups/backup-*) where under data there are different directories each with backups/backup-* . This work when the script is located in the same location as backups-* the above. Commented Feb 21, 2018 at 20:35
  • @holasz I'd discard the concept of a $prefix entirely, and instead just build a set of dates to keep. I'd then compare the date component of the backup directory paths to the set of dates I'd generated, deleting the directories for which there was no match. I suspect it's a harder problem to solve than the one you described in your question, though. Commented Feb 21, 2018 at 23:20
1

Whats about this?

for i in $(find . -maxdepth 1); do 
    if ! (( $((($(date +%s) - $(date +%s -r $i )) / 86400)) % 7)); then 
        echo $i # Print out the folder. Can be replaced with your action
    fi
done

or as oneliner:

for i in $(find . -maxdepth 1); do if ! (( $((($(date +%s) - $(date +%s -r $i )) / 86400)) % 7)); then echo "$i $i_weekly" ;fi; done

This should just print all folders which should be 7,14,21,28.... days old. Instead of "echo" just make a mv or what ever...

Improve this answer
2
  • @Marcc, doesn't seem to show nothing Commented Feb 17, 2018 at 19:56
  • may be you need to change the path of the find command. The "." means use the current working directory, so change in the parent dir of your backups first. Commented Feb 19, 2018 at 10:06
0

Here is an example script to rotate backups but keep last 7 days, 12 monthes backup-*-*-01 and 10 years backup-*-01-01

all=$(ls backup-*)
days=$(ls backup-* | tail -n 7)
monthes=$(ls backup-*-01 | tail -n 12)
years=$(ls backup-*-01-01 | tail -n 10)
keep=" $days $monthes $years "

for item in $all
do
    if ! [[ "$keep" =~ "$item" ]]
    then
        echo "$item will be deleted"
        # rm $item
    fi
done
Improve this answer
3
  • 2
    Don't use ls - it isn't needed or wanted. Try running this BUT replacing the 'rm $item' with 'echo $item'. Commented Sep 15, 2021 at 14:38
  • @JeremyBoden thank you for the comment, I have replaced unsafe rm with an echo example, also I'm not sure what did you mean by saying that ls isn't needed, can you please describe it a little bit more, I have tried something like all=$(backup-*); echo $all with no luck Commented Sep 16, 2021 at 15:10
  • You are probably OK with ls in this case since you know that all names won't have spaces etc in them. Also you (hopefully) will get filenames in date sequence (which would allow the tail commands to work properly). I always try to avoid using ls in scripts... Commented Sep 16, 2021 at 23:10

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.