Nextcloud Backup App - Mini-tutorial and review [April 2022]
Wed, Apr 27, 2022
21-minute read
Table of Contents
In November, 2021, Nextcloud released a much anticipated feature: a backup app for a less technical user! I hadn’t been following the project closely, but decided to give it a spin as I see it recommended on occasion. This is my quick review as of April 2022 using the Nextcloud Backup App version 1.0.6.
Overview
A backup App baked right into Nextcloud.
The concept is that anyone can backup their Nextcloud instance from inside the GUI. This is a great feature, especially for those who are using managed Nextcloud instances like those from Hetzner, Webo, Tab.Digital, The Good Cloud, etc. In these scenarios, one does not have “root” access to their server, meaning they are unable to use solutions like the one I recommend in my 3-part backup tutorial.
In this quick tutorial/review, I will be using Nextcloud 23.0.4 installed directly on a VM with 1 vCPU and 1GB RAM, running on Ubuntu 20.04. I am also using a basic Redis install and MariaDB 10.3.34 and PHP 8.0.
Note:
The Nextcloud documentation still recommends the older method of simply copying the relevant files as shown in my 3-part tutorial (linked above).
How does it work?
By default, the Nextcloud Backup App will create 2 different types of backups based upon the schedule shown in the settings. The two backups types are:
Full Backup: This is a FULL copy of your Nextcloud, including Nextcloud files, Apps, your database, and data files. This is quite time consuming and roughly AS LARGE AS your entire Nextcloud. For this reason it is only run roughly once per month, on the weekend, and during off-hours.
Partial Backup: This is a copy of your Nextcloud, including Nextcloud files, Apps, your database, and any NEW or MODIFIED data files since the last time you did a Full Backup above.
The Nextcloud Backup App works in two stages once it is setup correctly.
The first step is to create a copy of all Nextcloud data and make a copy (dump) of your database. This is done by first putting Nextcloud into maintenance mode, which will make Nextcloud unusable for the duration. It is recommended to set the app to work during off-hours.
The second step is to compress the backup (optional), encrypt the backup (optional), and copy the backup to an external source if desired. This is done in the background and will not require your Nextcloud to be in maintenance mode.
Install and Configure Backup App
The Backup App found in Nextcloud’s App Hub.
Simply head to the Apps section of your Nextcloud as an “admin” user. The Backup app can be found under “Tools”, or you can simply find it by using the search bar. Click Download and Enable. You may be asked to enter your admin password to complete the installation.
Configure the Backup App
The Backup App has a number of configuration settings.
Once installed, configuration settings can be found in your Settings. Scroll down the left-hand side until you see the Backup section. Clicking on it will bring you to the screen shown above.
Most of the settings are pretty straight forward and are explained in the window itself.
A few notes from me:
Enable background tasks to…: This will only work if you are using cron based background jobs. If you are using AJAX, it will not work! I highly recommend using cron with Nextcloud.
Backup Schedule: I find the defaults to be pretty good, but you may want to modify them based upon your own usage.
Compress Restore Points: A good way to save some space, these backups will be BIG.
Encrypt Restore Points: A good choice, especially if you are saving them in the cloud or on an external drive (see below for some critical considerations).
Encrypting Your Backups
This isn’t the place to discuss the benefits/risks of encrypting your data, but I believe it’s a critical component if you are storing your data anywhere in the cloud. I would also argue it’s a good idea to encrypt your backups on external hard drives as well. If anything is ever stolen, at least there’s a lesser chance for someone to steal valuable data you have stored in your cloud/external drive.
Export your config file and store the decryption key in safe places.
CRITICAL!!! YOU MUST EXPORT THE CONFIGURATION FILE ASAP. Without it your backups will be useless. While the configuration file is technically encrypted, I DO NOT recommend storing it in the same location as your backups. Use a USB key or something small you can secure in a safe or otherwise.
You must also keep note of the long random string of letters that shows onscreen as well. This is known as your encryption key for that configuration file. Without it, the above file is useless.
I would recommend either hand-writing it on a piece of paper and keeping it somewhere safe, or at the very least using a proper password manager (KeePassXC, pass, or BitWarden/ValutWarden).
Store Backups Externally (Optional)
As mentioned above, the backups will occur in two stages, both being stored locally by default. The first stage will create copies on your machine, in your “appdata” folder. This will be roughly the same size as your entire Nextcloud setup, so make sure you have sufficient space available.
Afterwards, during the second stage, that raw copy will be packed, compressed, and sent to your desired storage location in the background. By default, this will also be kept in your “appdata” folder.
As you can probably tell by now, following this method, you will end up with backups far out-sizing Nextcloud itself. Fortunately, the sections as the bottom of the settings page allow us to alleviate some pressure on our storage space.
Install and Configure the External Storage App
The External Storage Support App found in Nextcloud’s App Hub.
As a Nextcloud Admin user, go to your Apps section. The easiest way to find the app is to simply type “external” in the search bar. This will show the “External storage support” app which simply needs to be enabled.
External storage settings found in the Admin settings section.
After the app has been enabled, head to your Settings and scroll down to find the External storage section.
From here we can configure a locally attached hard drive, an S3 storage (Wasabi, Backblaze, Amazon), (S)FTP, OpenStack, WebDAV, or even another Nextcloud instance.
Here I’ve added an S3 compatible storage. Your settings will vary.
In my example, I’m using an S3 Minio instance on another VM. We add the connection settings, credentials, and select “Admin Group” as the only ones with access to this bucket.
Click the check mark at the far right side of the screen and you should see a green check mark appear to the left indicating a successful connection (shown above).
Set External Restore Point Storage
Our external storage is now available for a backup location.
Back in our Backup settings, we scroll to the bottom where we find our external storage already listed as an option in the drop-down menu.
External storage has been added as a storage location for our packed/compressed/encrypted backups.
If you so choose, you may create a subfolder like I have simply called “backups”. Click the blue Add new external location button to the right. And that’s it! After a restore point is created, it will upload automatically to your selected location(s).
Set External App Data
As discussed above, the Nextcloud Backup App will use a LOT of storage even during the “first stage” of the backup. If your Nextcloud server has very limited storage resources, you can have it do this process on the external storage location as well.
BUT! There’s always a but isn’t there… Beware that by using this option, all of the data will be sent TO the external storage, read back for processing, and then re-uploaded to the “External Restore Point” set above! This could incur massive charges depending on the service you are using! Be aware of the consequences of your upload/download fees!
If you decide that this is still an option for you, scroll to the bottom of the Backup page where you can set the “App Data” location.
Click the drop-down menu and select your external storage. Give it a name like app_data - this folder will be created at the root of the selected external storage.
Changing App Data location will remove backups stored in app_data.
Then, after clicking Set as App Data, a warning window will pop up letting you know that anything that’s been backed up to the original local storage will be deleted. Checking the box and selecting Change the App Data should take you back to the main screen with a little box acknowledging the change.
Creating Restore Points (Backups)
Completed and upcoming backups shown in the ‘Restoring points history’ sub-section.
Once the above has been set, the backups should be occurring automatically during the schedule set. However, if you want to trigger a full backup sooner than scheduled, you have two ways to accomplish that.
Manually Trigger a Full Restore Point (Backup)
You can trigger an additional full backup from the web interface, or from the CLI using the occ command.
From the Web Interface
From the Backup settings page, scroll down and click the Create full restore point button. After which, you will see a notification that the request has been made and the backup will start soon.
We must be patient for the backup to start.
Note:
The Restoring points history queue doesn’t change even after clicking the restore point button.
At first, I was a bit frustrated that the app wasn’t creating a full restore point, nor was it telling me when it might occur. However, after walking away for a few minutes, the job was eventually run during a cron session.
During a later running of cron, the backup was health checked, packed, compressed, encrypted, and uploaded to the external storage.
From the CLI with occ
Fortunately, there is a way to create a full backup immediately. However, you must have access to the occ command, either via terminal or the Occ Web app.
occ backup:point:create
Using occ we can immediately create a backup.
Note:
This will make a full restore point immediately, putting your instance into maintenance mode and rendering it unusable for a time (depending on the amount of data you have and speed of your system).
One thing to keep in mind, is that this will create a full backup, but it will not be uploaded immediately to your external storage location. It remains unpacked, un-encrypted, and in the “appdata” directory. That second set of steps will only occur via a cron job.
Put the Backup to the Test
As stated in my other backup related articles: A backup is completely useless if you can’t actually restore anything from it!
Before we dive into how to restore things with the Nextcloud Backup App, I must point out one major issue. Creating the backups from the web interface is quite easy, but there’s no restore functionality anywhere to be found in the GUI! Furthermore, there’s no indication or obvious documentation on how to restore anything, or even where to begin looking for help. I see this as a huge oversight as I see this app as targeting newer Nextcloud users and those who don’t have access to the command line.
Restore a Single File
Oops! Deleted an important file.
Restoring a single file is a bit of a process, and unfortunately undocumented as of writing this. I will walk you through step-by-step. To test a single file, I have deleted one of the example files: Example.md in the Documents folder.
1. Download the Relevant Restore Point(s)
Finding files only works with those restore points which exist on the server itself. If they are stored externally (which I recommend), they will need to be transferred first. Make sure you have enough local disk space for this!
occ backup:point:list
List of restore points stored both locally and externally. Note that I cut off the bottom of the screen as the rest was poorly formatted and very long output.
From this list, I know that the file lives in the oldest one (20220428111858...). I also note that it is located on external:3.
Note:
If you don’t have the space to transfer multiple backup points, you might have to search for the file you are looking for one-by-one.
Download the restore points you want to search through.
Now that it has been transferred, I can use the backup:point:list command above, I can see that it is located both on the external drive, as well as locally.
2. Unpack the Backup(s)
If the backup is packed (and encrypted), you will need to use the unpack command to make it available for searching and restoring.
Unpacking the restore point allows us to work with it by decrypting, decompressing, and putting chunks back together. Note the output was long so I truncated to show start and finish.
3. Find the file
Hopefully you know the name of the file you are looking for. Fortunately, the search feature works quite well, and does partial matching. Therefore, you only need to know a part of the name of the file.
occ backup:file:search example.md
The find command helps us locate the file and any other versions.
Here we can see that the file exists in our backup. If we had multiple versions of the backup downloaded, the utility would search for the file in each of them.
If there are old versions of the file, you will also see those under the files_versions directory, but might have a slightly different name like: Example.md.v1651162117.
Another way to look at a file and which versions exist is to use the history command.
occ backup:file:history data admin/files/Documents/Example.md
We get a bit more information with the history command.
Note:
After the backup:file:history command, we have to use the word data if we want to look at the history in our user data backup. After, we must give the full file path to the file, which is easy to find with the search command shown above.
Here we can see the history of the file between two different backups. Note how the file sizes are different, which might give us an indication of which one we want to restore.
4. Restore the File
Now that we know which version of the file we want to restore, we can actually restore it. Once again we must use the --data switch as explained below.
occ backup:point:restore --file admin/files/Documents/Example.md --data data 20220428111858-full-ThxNrSKclmh2FQM
Restoring the file is a command but happens quickly.
Note:
After our restore command, we give the full file path, followed by --data data. Why we need this switch is still a bit of a mystery to me. Another option to use instead of --data would be to use the --chunk data-f5cae31a-b933-43af-94fb-b3f67c23546c command (replacing with your correct chunk).
The file has been restored, but it won’t show up in your Nextcloud yet. I’m not entirely sure why this is the case.
5. Notify Nextcloud of the File Change
As with all file changes made to the system, Nextcloud isn’t aware and needs to update the database with the restored file’s info.
You would think this step should happen automatically after the previous step but alas, we must run it by hand.
Note:
We notify the database using the files:scan command. While we can easily scan the entire folder structure, we only need to scan the directory we just modified. This will greatly speed up the scan.
The second command comes from the Nextcloud documentation:
And now we see our file is back in its original form!
Refreshing the page shows that we have successfully restored the deleted file.
Restore an Entire Installation
Similar to the process above, we will follow a series of steps. Fortunately, these steps are more clearly defined by the software’s authors. However, it might not be obvious at first where the documentation exists. Heading to the project’s GitHub page will lead you part way down the page where we can see references to restoring an entire installation. As this is well documented, I will be giving a cursory summary.
Keep in mind that this will restore EVERYTHING from the point in time the backup was made. This means whatever version of Nextcloud you are on is unimportant, as the backup will also restore your Nextcloud files, database, configuration, data, EVERYTHING as it was at that point in time. Any files created or changed since then will be deleted!!
Last chance to double check your intentions and which restore point you’ve chosen.
At this point the Nextcloud Backup App checks the health of the backup (awesome), and verifies with you that you really want to roll back your installation to the given time.
Next you will be lead through a series of questions about what you’d like to restore. This is a handy feature, but beware that saying yes to some and no to others could theoretically break your Nextcloud. For example: if your Nextcloud system files (let’s say version 23.0.0) don’t match what’s currently in the database (version 23.0.4).
At the end, you will see a small summary of what occured.
Nice summary of what we restored.
5. Not done yet!
Although I restored the old database file, for some reason the recent changes I had made (putting a few items in the trash) still persisted. I’m curious as to whether the database was properly restored or not. Anyways, I would recommend running the same two commands as above as well.
At this point you should have a fully restored Nextcloud instance. There are a few more details you might want to look into (like re-scanning your backups to add any that were made between the original state and the restore you just made).
Review
Well if you made it to this point I assume you’re either tired from reading the novel I wrote above, or you skipped ahead to see my thoughts on the App. Either way, the above wasn’t intended to be so long, but it felt unfair to give a critical review without trying out as much of the software as possible myself.
While I think the concept is great, I think there’s still work to be done. Or, better yet, the App should scrap the backend, and rely on an already existing tool like restic or borg to do the heavy lifting.
This would mean the app simply acts as a GUI “middleware” to take care of the Nextcloud specific details. Well that’s just my two cents.
So for the most part the Nextcloud Backup App works… However…
It Wasn’t All Smooth Sailing
While testing out the App I ran into a number of problems which I was hesitant how to document. I didn’t write them all down but for example:
When it was time to find a file to restore, I used the simple backup:file:search command:
Hmmmmmm…
OK… That’s odd - the file couldn’t be found under my “admin” user’s files (but you can see it found it for user “kit”). I decided to use the file:history command to see if the file could be found another way. Indeed the file exists in the backup.
As you can see below, during my earlier testing while preparing to write this article, the backup:file:search command worked fine.
The original backup worked flawlessly.
Anyways, since I started messing with so many settings I will say I messed something up or it is a bug of some sort. I haven’t taken the time to investigate further. Either way, the file is indeed there - just out of curiosity, I copied and unzipped the data folder to see.
Unzipping and searching through the data file shows the file exists.
After restoring the file, I searched again just out of curiosity. It worked once or twice, then went back to the old behavior of not showing me the file anymore. Very strange behavior.
So to give a quick TL;DR, I’ve compiled a small list of my Pro’s and Con’s while using the App
Pro’s and Con’s
Pro’s
Backups can be created from GUI - Some find the CLI to be overwhelming when learning new software, and Nextcloud is a great entry into the self-hosted world. This tool helps to manage many factors that can be confusing at first.
Easy scheduling - Effectively replaces cron and makes the options VERY clear.
Setup and configuring - No technical knowledge required.
Partial backups are differential - This saves some space.
Backups can easily be exported to external sources - Critical features for good backup plans.
Backups get regular TLC - The software includes regular health checks.
Compression and Encryption - Important for backups (especially externally or in the cloud).
Full Nextcloud restore or single file restore - Flexibility for finding and restoring as much or little as you want.
Full restore health check - The software performs a full health check on your backup before you restore it.
Con’s
No restore from GUI - While the backups are great and can be done by any non-technical user, restoring is an entirely different matter.
Backups are HUGE - Even with differential backups and compression, the backups are massive and include unnecessary files.
Can’t restore a directory - While restoring a single file is possible, if you accidentally deleted a folder with hundreds of files, it will take some more creative methods to restore it so you don’t have to enter the backup:point:restore --file hundreds of times.
Files can be difficult to recover - Tracking down files took a while to figure out, and won’t be found if a backup is encrypted or not stored locally.
Documentation severely lacking - Some of the issues above could be solved with some solid documentation. Some of my complaints and time running around were due to minimal (or non-existent) documentation. If you end up finding their GitHub page, you can try to scroll your way though the walls of text to find some information. To be fair they do say it’s a work in process, but that’s a bit disappointing considering it was RELEASED as a feature 5 months ago.
Buggy - While most things mostly work, it unfortunately follows many of Nextcloud’s apps that all seem to have constant bugs and a growing number of breaking issues.
Does Not Work With S3 backend - While this is more an issue with Nextcloud’s server implementation, it should be one of the #1 issues for them to tackle considering S3 backends are often associated with enterprise-level implementations.
Other Considerations
If you even glanced at my home page or anything related to Nextcloud, you can see that I have spent quite a lot of time working out how to effectively create a good backup solution. I love the idea of having an App integrated into Nextcloud that takes care of this for you. In fact, I believe it should be part of the Installation process!
One of my main concerns is the longevity of the project. Too many of Nextcloud Apps face heavy development at the start, and then fall off quickly. Sometimes this isn’t a big deal, but a solid backup is Critical to running Nextcloud. I truly hope the Nextcloud team puts more resources towards this side of Nextcloud.
Another major question I have is why they are trying to reinvent the backup wheel? There are incredible open source backup solutions available which would integrate very well into the way Nextcloud works. Their backup methods, encryption, and functionality are well-tested and have great longevity.
Many of the “cons” and frustrations listed above are easily taken care of with a program like restic or borg. You can constantly do FULL backups, but they are all incremental and reduce space drastically. On top of that, you can mount the ENTIRE backup repository to the local file system. This means you can easily search for a file, but in EVERY single backup made. And copying lost files from your backup is a normal file system operation. This is all done without taking up an extra byte of storage on your local drive.
Conclusion
I must admit I was both pleasantly surprised and sorely disappointed while trying out the Nextcloud Backup App. I see that it is being actively developed by the Nextcloud team, but I still have a lot of questions before I would consider using it long term.
I look forward to the point in time where I can completely re-write this review to be 100% positive, and remove my other 3-part article about backing up Nextcloud outside of the software.
Once again, Nextcloud is a good sync software, BUT IT IS NOT A BACKUP. You MUST setup something more robust outside of Nextcloud to ensure you don’t lose everything. The comment section is open below for your thoughts or criticisms!
Thank you for visiting my site and checking out this post! I hope you found it helpful. If you are feeling generous, please consider a small donation to help keep the server running.
