Page 1 of 2

Guarding against backup loss/damage

Posted: Sun Aug 26, 2018 3:03 pm
by davewilk
I am just starting with FBackup, and am planning to use it in mirror mode with fast mirror, removal of deleted and excluded files, and copy empty folders. I had thought that this would maintain a mirror of my selected source(s) in the backup location. However I have come to realize that this does not guard against damage or accidental removal of the backup files, because with fast mirror FBackup only backs up files that have changed on the source; it does not back up files that are missing in the destination.

To guard against such damage or removal, I am thinking of occasionally doing a manual backup by un-checking the fast mirror option and running the backup, followed by re-checking the fast mirror option for subsequent scheduled backups . Would this work?

Re: Guarding against backup loss/damage

Posted: Tue Aug 28, 2018 6:59 am
by Adrian (Softland)
Hi,

Yes, you can do that.
You can also schedule a backup test, to make sure there are no errors/warnings (also including missing files). This way you will run the complete mirror only when the test returns warnings/errors.

Re: Guarding against backup loss/damage

Posted: Tue Aug 28, 2018 6:39 pm
by davewilk
Softland wrote:You can also schedule a backup test, to make sure there are no errors/warnings (also including missing files). This way you will run the complete mirror only when the test returns warnings/errors.
You mean use the "Test" button that is on the main window? What does this do exactly? Is it the same as the test that is run after a backup job completes? If so, how does it handle files that have been deliberately removed or moved in the source since the backup in question?

Re: Guarding against backup loss/damage

Posted: Wed Aug 29, 2018 10:04 am
by Adrian (Softland)
Hi,

I am not talking about the Test button from the toolbar.
When you create a scheduled task in Backup Properties->Scheduler, you can select the Test action to be executed.

The test after backup will test only the files from that backup, but a complete test you schedule will test all the files from the catalog.

Re: Guarding against backup loss/damage

Posted: Wed Aug 29, 2018 2:59 pm
by davewilk
Softland wrote:I am not talking about the Test button from the toolbar.
When you create a scheduled task in Backup Properties->Scheduler, you can select the Test action to be executed.

The test after backup will test only the files from that backup, but a complete test you schedule will test all the files from the catalog.
That is great. But this needs to be explained in the manual. In fact there are three kinds of test: (a) the test performed after each backup, (b) the test available on the main window, and (c) the scheduled test. All of these need to be more fully explained in the manual.

If I am correct:

Test (a) checks that files copied in the current backup were copied correctly. This is obviously worth doing (even essential) in my opinion.

Test (c) tests all the files that are currently in the backup. Worth doing occasionally, in my opinion, to check that the backup has not been accidentally (or maliciously) removed or damaged.

Test (b) I not quite sure what it does, because I do not understand how it treats files that were legitimately removed form the source (and hence the backup) subsequent to the backup in question. In my opinion, it ought to be possible to perform a type (c) test from this window, perhaps as an extra "Current status" line in the table. Why can this full type (c) test only be scheduled, and not run immediately, just as the Backup task can be either run immediately or scheduled?

Also what should the user do when a test fails? For example, I just did an initial backup of about 300 GB of data on my Windows 10 workstation to a shared drive attached to a different Windows 10 machine (which acts as a kind of HomeServer). It took about 5 hours, and the test took about 2 hours. The test revealed a CRC error on just one (binary) file. Indeed, checking with fc.exe revealed an error in one byte. I tried touching the file (using git bash) and re-running the backup, but FBackup did not recopy the file.

Re: Guarding against backup loss/damage

Posted: Thu Aug 30, 2018 10:12 am
by Adrian (Softland)
Hi,

The differences between (b) and (c) are:
- (b) can be run manually and you can select which version will be tested (you can select all versions too)
- (c) can be executed only scheduled and you cannot select which versions to be tested

Re: Guarding against backup loss/damage

Posted: Thu Aug 30, 2018 6:34 pm
by davewilk
Softland wrote:The differences between (b) and (c) are:
- (b) can be run manually and you can select which version will be tested (you can select all versions too)
- (c) can be executed only scheduled and you cannot select which versions to be tested
Yes, I understand that. What is bothering me about the type (b) test is what happens when I am removing deleted and excluded items from the backup.

Suppose on Day 1. I create a File SomeFile.txt and backup job Backup1 completes without errors, so that SomeTile.txt is in the backup. Then on Day 2 I delete SomeFile.txt and the backup job Backup2 removes SomeFile.txt from the backup. If I now do a type (b) test of Backup1, then it will fail because SomeFile.txt is missing from the backup. But there is nothing wrong. So I do not see the purpose of a type (b) test (at least in the case where deleted files are always removed from the backup).

For me, the type (c) test is far more useful because it tests whether all the files that are currently supposed to be in the backup set are actually there, and still match the one in the source tree. So I am puzzled by the design decision not to allow this more useful test to be made on demand.

Or am I missing something?

But in any case, I think that all tests (a), (b)) and (c) need to be more carefully described in the manual.

Re: Guarding against backup loss/damage

Posted: Sun Sep 02, 2018 1:40 am
by davewilk
Maybe I am still not understanding exactly what tests (a) (b) and (c) do.

Re: Guarding against backup loss/damage

Posted: Tue Sep 04, 2018 12:38 pm
by Adrian (Softland)
Hi,

The tests will not compare the source and destination. FBackup uses the backup catalog to store information about the backed up files. It is faster to access the information from the backup catalog than scanning the source and calculate the information again.
For tests, FBackup will compare the information from the catalog with the files in destination.

When a file in destination is deleted by FBackup because the file was removed from sources, that file will be also removed from catalog, so it will be not searched when running a test.
If you manually delete a file from the backup destination, it will be reported in a warning at the end of the test.

Re: Guarding against backup loss/damage

Posted: Tue Sep 04, 2018 4:03 pm
by davewilk
Softland wrote:When a file in destination is deleted by FBackup because the file was removed from sources, that file will be also removed from catalog, so it will be not searched when running a test.
If you manually delete a file from the backup destination, it will be reported in a warning at the end of the test.
Yes, but in a type (b) test FBackup allows (even encourages) the user to test previous backups. The backup catalog of such a backup may contain files that were removed from the destination is later backups (because they were deleted from the source, or added to the exclusion list). Won't such a test fail? Or does the test of an older backup pass if a file in the catalog is no longer in the source, or is in the current state of the exclusion list? Perhaps that is what I am missing.

But anyway, I just don't see the point of doing a test of an older backup. Wouldn't the user always want to test that the current state of the backup is consistent with the source?

Another question: if I select all the backups from the type (b) screen, is that the same as what is done with a scheduled backup?