Re: [Duplicity-talk] Simple multi-generation backup

[Felix Natter]

hello Edgar,

thanks for your reply, answers are inline:

On 08.12.22 13:37, edgar.soldin--- via Duplicity-talk wrote:

> hey Felix,
>       
>       
>       On 08.12.2022 08:10, Felix Natter via Duplicity-talk wrote:
>       
> > Dear duplicity community,
> >         
> >         
> >         I would like to set up a _simple_ multi-generation backup using
> >         duplicity
> >         
> >         to a cloud service:
> >         
> >         
> >         - target $CLOUD/daily/: backup once per day, keep last X
> >         
> >         - target $CLOUD/weekly/: backup once per week, keep last Y
> >         
> >         - target $CLOUD/monthly/: backup once per month, keep last Z
> >         
> >         
> >         My daily script (to be called by cron job) looks like this:
> >         
> >         
> >         duplicity --encrypt-key KEY --full-if-older-than 5D /repos
> >         $CLOUD/daily/
> >         
> >         duplicity --encrypt-key KEY remove-all-but-n-full 2
> >         $CLOUD/daily/
> >         
> >         
> >         And my weekly cron job would call this:
> >         
> >         
> >         duplicity --encrypt-key KEY --full-if-older-than 3W /repos
> >         $CLOUD/weekly/
> >         
> >         duplicity --encrypt-key KEY remove-all-but-n-full 2
> >         $CLOUD/weekly/
> >         
> >         
> >         and so on for the monthly backup.
> >         
> >         
> >         Q1: Is that a valid/good way to do it?
> >         
> >       
>       
>       kind of wasteful, but sure. why are your fulls not located at the
>       time frame's start?
>       
>

Sorry I do not understand. I do think that it is "wasteful" recomputing weeklies/monthlies
instead of moving some backups from daily to weekly or monthly ("rotation"),
but what do you mean by "fulls not located at the time frame's start"?

Is there a better way to do multi-generation backups with duplicity (or duply)?

>  
>       
> > Q2: I tried to make a daily backup. Then I
> >         added a file /repos/bla.txt, then
> >         
> >         made a daily backup. I removed it and made another daily backup.
> >         
> >         Then I tried to restore /bla.txt from the backup:
> >         
> >       
>       
>       list the backup contents with action command 'list-current-files'
>       . as you deleted it, it will probably not be there. use '--time'
>       to point to a time where it was not deleted, it should be listed. 
>       from the man page https://duplicity.us/stable/duplicity.1.html
>       
>       "
>       
>       list-current-files [--time <time>] <url>
>       
>       Lists the files contained in the most current backup or backup at
>       time. The information will be extracted from the signature files,
>       not the archive data itself. Thus the whole archive does not have
>       to be downloaded, but on the other hand if the archive has been
>       deleted or corrupted, this command will not detect it.
>       
>       "
>       
>        
>       
> > duplicity restore --file-to-restore
> >         /bla.txt $CLOUD/daily /tmp/bla.txt
> >         
> >         # (I tried -t 1D/2D/3D as well)
> >         
> >       
>       
>       restore needs a _relative_ path to the backup root. from the man
>       page https://duplicity.us/stable/duplicity.1.html
>       
>       "
>       
>       restore [--file-to-restore <relpath>] [--time <time>]
>       <url> <target_folder>
>       
>       You can restore the full monty or selected folders/files from a
>       specific time. Use the relative path as it is printed by
>       list-current-files. Usually not needed as duplicity enters restore
>       mode when it detects that the URL comes before the local folder.
>       
>       "
>       
>       
>

-> this was a typo, of course I used a relative path. Thanks for the hint with list-current-files
and collection-status!

So the workflow for restoring something in the latest possible version is roughly:

    - use duplicity collection-status --file-changed=FILE and then
    choose the appropriate backup time.

- if it is not found at all in $CLOUD/daily, repeat for $CLOUD/weekly and $CLOUD/monthly

- do restore with the same time and path

> > only looks at the last full/incremental
> >         backup of the set in $CLOUD/daily.
> >         
> >         But I need to be able to restore a file that was deleted a few
> >         days ago.
> >         
> >       
>       
>       list your backup chains to find out which backups exist
>       
>       "
>       
>       collection-status [--file-changed <relpath>]
>       [--show-changes-in-set <index>] <url>
>       
>       Summarize the status of the backup repository by printing the
>       chains and sets found, and the number of volumes in each.
>       
>       The --file-changed option summarizes the changes to the file (in
>       the most recent backup chain). The --show-changes-in-set option
>       summarizes all the file changes in the index:th backup set (where
>       index 0 means the latest set, 1 means the next to latest, etc.).
>       
>       "
>       
>       from the man page https://duplicity.us/stable/duplicity.1.html
>       
>

Thanks, very helpful!

>       
> > I think I am misunderstanding things,
> >         could you please point me to good
> >         
> >         documentation?
> >         
> >       
>       
>       https://duplicity.us/stable/duplicity.1.html
>       
>

This looks very good, I will make sure to read it completely.

Thanks and Best Regards,

Felix

--

SIDACT GmbH
Simulation Data Analysis and
Compression Technologies

Felix Natter
Software Developer

Auguststraße 29
53229 Bonn
Germany

Phone	:   +49 228 5348 0430
Direct	:   +49 228 4097 7118
Email	:   felix.natter@sidact.com
Web	:   http://www.sidact.com/

OpenPGP_signature
Description: OpenPGP digital signature