Skip to content

Nextcloud Backup pt. 1

Last updated on June 8, 2021

You may have never heard it, or you may have heard it echoed 10,000 times in an online forum: Syncing solutions are not a backup solution! Nextcloud is a fantastic tool to keep your files and folders in-sync across multiple platforms, but that doesn’t mean it is “backing-up” or saving your data from catastrophic failure. It does, admittedly, have some features built-in to help mitigate accidental deletions – Trashbin and File Versioning.

There is an app for Nextcloud (available through the App hub) called NextBackup which reportedly works quite well. However, Nextcloud version 21 appears to have made a breaking change that does not allow the app to work and it is unknown when (if) it will be fixed.

Even if you have just installed your Nextcloud, it’s a good idea to implement a backup solution early and know how to use it.

What is a backup?

There are books written on the topic and more online articles than one cares to count. So I will keep my version simple. We want a perfect copy of all the data which we cannot replace. Example: You want a backup of your personal photos, but probably do not need a backup of your digitized film collection. While it is quite annoying to re-digitize a film, they are generally easier to find. If you have the space, go ahead and backup everything, but make the priority your irreplaceable data.

This data should be stored separately from your Nextcloud instance. Use an external hard drive, an offsite location, or even better – both. While sending your data to an offsite provider is great, they are also not immune to catastrophic failures. In this article, we will cover a basic backup of your Nextcloud onto a local device. For more, follow along to part two of the article where we will cover some offsite options.

Restic

So let’s create a proper backup for Nextcloud using a popular new tool: restic. The following method focuses on Nextcloud, but could really be used for any application. There are other options such as borg and duplicati among others. To be honest, all are great options and the most important at this point is to MAKE A BACKUP. I don’t have experience with the other two, but from a quick search, they accomplish more or less the same task in slightly different ways.

Restic is a great tool that works quickly and efficiently. One of the ways to increase its efficiency is by using “snapshots”. After your first backup, restic creates an index of the files in the backup repository. If you change a few files with Nextcloud, the next restic backup begins by looking at the old index. It then performs the next backup “snapshot” by only uploading the changed files. This greatly reduces space needed for a backup and time taken for subsequent backups!

Installing and Updating Restic

This article is written on a Debian 10 server, but the same applies for an Ubuntu server. CentOS and others can easily be adapted. For this first part, restic must run using sudo to ensure access to files and directories owned by different users. Part 2 will include a way around this. But for now, let’s begin:

sudo apt update && sudo apt install restic
sudo restic self-update

As of writing this, the restic .deb repository has version 0.9.6, but with the self-update, it quickly moves to the most up-to-date version (0.12.0).

Let’s create a directory to hold our restic repository:

sudo mkdir /srv/nc-backup

To create a repository, we use the following restic command to initiate a new repository at the location of our new directory:

sudo restic init -r /srv/nc-backup
Creating our restic repository

Restic asks for a password; I recommend using a secure password of 20-30 characters. This can be saved in a password manager and will be used to encrypt our backup. Now that it’s setup, let’s prepare Nextcloud for a full backup.

Backup our Nextcloud database

The first thing we want to do is put Nextcloud into maintenance mode. From this point forward, we will keep it “ON” to prevent any further changes to the database while we are performing the backup.

docker exec -u www-data nextcloud php occ maintenance:mode --on

Replace “nextcloud” with your Nextcloud container name

sudo -u www-data php /var/www/html/occ maintenance:mode --on

Replace the /var/www/html with your actual path to Nextcloud

Nextcloud in maintenance mode to prevent changes in the background.

Now let’s make a directory to hold the database “dumps”. A database dump is a better way to create a backup than just copying the directory structure itself. The “Why?” is a bit outside the scope of this article, but we are taking a “snapshot” of the database, so no other writes during this time (should be disabled by maintenance mode) are recorded, potentially corrupting our backup.

docker exec nextcloud-db mkdir /var/lib/mysql/backup

Replace nextcloud-db with the name of your Nextcloud Database container.

sudo mkdir /var/lib/mysql/backup

This folder could really be anywhere, but for simplicity I keep it in the same directory.

Now let’s actually dump the database. I have provided a few examples below depending on your setup. There’s no way I could include them all, so please let me know if I missed a popular choice and I will add it! The following are for MariaDB (and likely MySQL but I haven’t tested it).

docker exec nextcloud-db bash -c 'mysqldump --single-transaction \
-u nextcloud -p`cat "$MYSQL_PASSWORD_FILE"` nextcloud \
> /var/lib/mysql/backup/nextcloud-sqlbkp_`date +"%Y%m%d"`.bak'

Replace nextcloud-db with the name of your Nextcloud database container. Note that there is no space after the “-p” switch!!

Using docker secrets is a good security practice, but definitely adds a bit of complexity to the above command. We need to begin our docker exec command with bash -c so that it correctly reads the following environment variable (starts with a $). The –single-transaction flag ensures that nothing else is written during the dump. “-u” Indicates the database user which can also be loaded from an environment variable. “-p” is the password which we read from the secrets file. “nextcloud” is the name of our database. The final piece says that we push the data into a .bak file which is tagged with today’s date.

docker exec nextcloud-db bash -c 'mysqldump --single-transaction \
-u nextcloud -p$MYSQL_PASSWORD nextcloud > \
/var/lib/mysql/backup/nextcloud-sqlbkpdate +"%Y%m%d"`.bak'

Replace nextcloud-db with the name of your Nextcloud database container. Note that there is no space after the “-p” switch!!

We need to begin our docker exec command with bash -c so that it will correctly read the following environment variable (starts with a $). The –single-transaction flag ensures that nothing else is written during the dump. “-u” Indicates the database user which can also be loaded from an environment variable. “-p” is the password which we will read from the environment variable. “nextcloud” is the name of our database. The final piece says that we will push the data into a .bak file which is tagged with today’s date.

sudo mysqldump --single-transaction -u nextcloud -pMYSECRETPASSWORD \
 nextcloud > /var/lib/mysql/backup/nextcloud-sqlbkp_`date +"%Y%m%d"`.bak

Replace the string after “-p” with your database password. This can also be read from an environment variable like the two other examples above.

The mysqldump command is included with MariaDB and might need to be run as the mysql user. The –single-transaction flag ensures that nothing else is written during the dump. “-u” Indicates the database user which can also be loaded from an environment variable. “-p” is the password which we will read from the environment variable. “nextcloud” is the name of our database. The final piece says that we will push the data into a .bak file which is tagged with today’s date.

If everything worked correctly you won’t get any output on screen. Take a look in the folder to make sure the backup is there and labeled correctly.

Restic to backup Nextcloud files and DB

For simplicity (and ease of a full working restore), we will backup the entire Nextcloud directory and the database dump file we just created above. This part could not be simpler:

sudo restic backup -r /srv/nc-backup /opt/volumes/nextcloud/
sudo restic backup -r /srv/nc-backup /opt/volumes/nextcloud-db/backup

Replace /opt/volumes/nextcloud and /opt/volumes/nextcloud-db/backup with the directories of your Nextcloud app and database respectively. For Docker setups this would be where you mount your volumes.

sudo restic backup -r /srv/nc-backup /var/www/html
sudo restic backup -r /srv/nc-backup /var/lib/mysql/backup

Once again remember to replace with the correct directories.

Nextcloud restic backups complete!

The commands must be run as root (unless you have access to the files and directories in question). The “-r” switch tells restic which repository to use. We can have multiple if so desired. And the final piece tells restic which directory we wish to add to our repository.

After entering each command above, the repository will open, ask you for your password, and begin reading the data and creating a snapshot of the data. And that’s it! Backup is successfully completed. But wait… a backup is only as valuable as its ability to actually restore the data in question!! So let’s do a little verification check.

We can see the status of our backup with the following set of commands:

  • restic stats – a quick overview of the statistics about your repository
  • restic check – a quick integrity check to make sure there are no errors in your repository
  • restic snapshots – a view of your individual backups, and one of the most useful features

One quick reminder: You must disable maintenance mode in Nextcloud after finishing your backup. The command is exactly the same as above, but just change the last switch to --off.

Put your Nextcloud backup to the test!

Oops! I deleted my favorite Nextcloud folder! My Nextcloud broke after upgrading to version X! My Nextcloud is corrupted!

Fear not fellow backup aficianado! Let’s have a look at how we can restore data to our Nextcloud using the restic backup with two scenarios: a complete restore, and restoring a particular directory.

For the sake of the following example, I will delete the Documents folder from a basic default Nextcloud installation:

Deleting a folder.
Oops! Permanently deleted!!

Completely Restore Nextcloud Backup

For this, we will follow along with the suggested steps in the Nextcloud Documentation. Once again, let’s turn maintenance mode “on”. This prevents any issues during our restoration.

The first step is to restore a backup from one of our restic snapshots. Let’s find which one we want to use:

sudo restic snapshots -r /srv/nc-backup

This shows us a list of all our snapshots in the /srv/nc-backup repository.

A view of our restic snapshots.

In this case we only have one snapshot, so we take note of the “ID” – 553671eb. In this article, I cover just one of a few ways to accomplish this next step. We create a directory separate from our original installation to restore everything to.

sudo mkdir /tmp/nc-restore

Note that for a large installation you would need enough free space on the disk to accomplish this. We use the /tmp directory as a temporary location – it will be deleted upon a server reboot.

And now to restore the snapshot to our newly created directory:

sudo restic restore 553671eb -r /srv/nc-backup --target /tmp/nc-restore

Replace with your snapshot ID, and notice how we are using –target to tell restic WHERE to put the files.

Successful restore of our files.

Now that we have unpacked the entire backup, we can either search the directory for our missing files, or we can copy everything over using a tool such as rsync:

sudo rsync -Aax /tmp/nc-restore/opt/volumes/nextcloud \
/opt/volumes/nextcloud

Replace the first directory string with your restored backup, and the second with the location of your mounted Nextcloud volume.

sudo rsync -Aax /tmp/nc-restore/var/www/html /var/www/html

For a look at why we are using the -Aax switches, have a look at the rsync manual. The short version is that they tell rsync to recursively (full directory depth) replace the files and preserve permissions.

Ok so we are only halfway there. If the original backup was made while maintenance mode was not on, it could have popped your Nextcloud out of maintenance mode. I suggest refreshing your Nextcloud page in the browser to confirm we are still in maintenance mode. Next comes the scary part: we must drop the old database first before we can restore from the backup. If you are restoring an old backup or are unsure of the quality of your backup, make another dump of the current database!!!

Nextcloud Database Restore

The first thing to do is to copy the database dump (.bak) file into our database directory for easier access. This is especially critical for docker installations, and simply easier to find on bare metal. Follow the same steps above to recover your database file from the snapshot we took of the database directory.

The following commands in succession DROP (delete) our existing database, CREATE a new blank one to be filled, and finally fill it from our .bak dump file:

docker exec nextcloud-db bash -c 'mysql -u nextcloud -p`cat \
"$MYSQL_PASSWORD_FILE"` -e "DROP DATABASE nextcloud"'

This should be familiar to the command earlier in this article. The main difference is we are now using the “-e” switch to execute the DROP command.

docker exec nextcloud-db bash -c 'mysql -u nextcloud -p`cat \
"$MYSQL_PASSWORD_FILE"` -e "CREATE DATABASE nextcloud"'
docker exec nextcloud-db bash -c 'mysql -u nextcloud -p`cat \
"$MYSQL_PASSWORD_FILE"` nextcloud < /var/lib/mysql/nextcloud-sqlbkp_20210416.bak'

Here we are reading in the data from our .bak file into our newly created blank database. Replace with the name of your desired backup file.

docker exec nextcloud-db bash -c 'mysql -u nextcloud \
-p$MYSQL_PASSWORD -e "DROP DATABASE nextcloud"'

This should be familiar to the command earlier in this article. The main difference is we are now using the “-e” switch to execute the DROP command.

docker exec nextcloud-db bash -c 'mysql -u nextcloud \
-p$MYSQL_PASSWORD -e "CREATE DATABASE nextcloud"'
docker exec nextcloud-db bash -c 'mysql -u nextcloud \
-p$MYSQL_PASSWORD nextcloud < /var/lib/mysql/nextcloud-sqlbkp_20210416.bak'

Here we are reading in the data from our .bak file into our newly created blank database. Replace with the name of your desired backup file.

sudo mysql -u nextcloud -pMYSECRETPASSWORD \
-e "DROP DATABASE nextcloud"'

This should be familiar to the command earlier in this article. The main difference is we are now using the “-e” switch to execute the DROP command. Replace with your password.

sudo mysql -u nextcloud -pMYSECRETPASSWORD \
-e "CREATE DATABASE nextcloud"'
sudo mysql -u nextcloud -pMYSECRETPASSWORD \
nextcloud < /var/lib/mysql/nextcloud-sqlbkp_20210416.bak'

Here we are reading in the data from our .bak file into our newly created blank database. Replace with the name of your desired backup file.

Phew! Stressful part is finished. I wasn’t given any output after each command above, but you might see an “OK”. The final steps are to turn off maintenance mode and set an option that tells all clients to look for a change in the file system:

docker exec -u www-data nextcloud php occ maintenance:data-fingerprint
sudo -u www-data php /var/www/html/occ maintenance:data-fingerprint

According to the Nextcloud Documentation:

“After restoring a backup of your data directory or the database, you should always call maintenance:data-fingerprint once. This changes the ETag for all files in the communication with sync clients, allowing them to realize a file was modified.”

https://docs.nextcloud.com/server/stable/admin_manual/configuration_server/occ_command.html#maintenance-commands
And the directory is back! Great success.

Restore specific files or directories only

Here we have a few options which I will not cover in as deep of detail since most can be deduced from the above steps. The key differences are how we might find a specific file or directory, and how we go about restoring it. In the above example, we restored the entire Nextcloud app, data, and database. Let’s explore a few other options for restic:

  • restic find Documents – this command allows us to find the exact directory path to the item we wish to restore. To restore only the Documents folder, we can include the follwing switch in our restic restore statement (shortened for the example):
sudo restic restore ... --include /path/to/my/Documents
  • restic mount – this command will mount your snapshot(s) to be viewed as if they are a mounted external drive. More information in the official documentation.

Besides rsync (usually the better option) we can also use cp -a to move files back and retain their attributes:

sudo cp -a /tmp/nc-restore/opt/volumes/nextcloud/data/user/files/Documents \
/opt/volumes/nextcloud/data/user/files/

Once we have returned the files to the correct user directory, we need to let Nextcloud know that something has changed. We do that with another occ command:

docker exec -u www-data nextcloud php occ files:scan user

Replace “nextcloud” with the name of your container and “user” with the name of your user. Alternatively, you can use files:scan --all

sudo -u www-data php /var/www/html/occ files:scan user

Replace “user” with the name of your user. Alternatively, you can use files:scan --all

Next Steps (preview of pt. 2 in the series)

This first part of the series has been a simple start to make sure you have a good backup of your Nextcloud instance and know how to restore it properly. However, there’s a good deal of improvement and automation we can do to modify the above steps. Here’s a quick preview of what I will cover in the next article:

  • Use an s3 offsite bucket (Wasabi, Backblaze, Amazon, other)
  • Using an environmental file to read in some repetetive variables for restic
  • Using –exclude-file and –files-from tags with restic to simplify backups and reduce their size.
  • Create a non-root user to execute the backups
  • Rotate our database backups and restic backups
  • Automate the process with a script + cron

Conclusion

In this article I walked you through the importance of making a proper backup of your data and how it differs from a syncing solution like Nextcloud. We also looked at restic as a tool to easily and efficiently create a full backup for us. Finally we restored a “broken” Nextcloud installation from a restic snapshot.

This is just the first step in backups, as regular backups and keeping them in more than a single location is critical for your most valuable files. How you make that backup and the tool you use is not as critical as making the backup itself. But keep in mind that it should be verifiable! There’s no use in a backup if you can’t actually restore data from it!

Thus far I have been satisfied with the results of restic as a backup solution. On that note, before you start to play with your working Nextcloud application, MAKE A BACKUP!

Published inNextcloud

Be First to Comment

Leave a Reply

Your email address will not be published. Required fields are marked *