Today I would like to once again walk through the edge-on backup to the cloud. I will not discuss whether it is good or bad, but I want to share examples of implementations of solutions for this most cloud-based backup - from finished software to crutches on bicycles.

Don't back up to the cloud yet or want to read about solution options? I ask for cat.

3-2-1, let's go

It is believed that the history of the 3-2-1 backup rule begins with Peter Krogh, who outlined it in his book Digital Asset Management for Photographers. I briefly recall this principle:

• There must be at least 3 copies of the data.
• At least 2 copies must be on physical media of different types. For example, one copy is working data on a disk array, the second copy is data on magnetic tape.
• At least one backup must be stored outside the office.

Personally, I most often use slightly different rules in creating backups.

The classic 3-2-1 circuit.

Firstly, I take backup copies as the initial data, and secondly, it is not always convenient and budgetary to store them on various types of media - especially for small and medium-sized businesses. My usual backup storage strategy is this:

• Online backups. Their main goal is to ensure the fastest recovery in case of a small failure. Depending on the infrastructure, these backups can even be stored on the server being backed up - only on a separate disk.
• Archive backups. They are always stored at least on another server and with history (most often - 6 daily backups, 4 weekly and 4 quarterly).
• Remote backups. Backups are necessarily stored in another place - on a server in a remote data center or in the cloud. A good option is to synchronize the archive backup directory with the remote storage, if possible.

With operational and archive backups, usually everything is quite simple, except that you should adhere to certain recommendations. One of the options for such recommendations is under the spoiler.

But with remote backups there are many questions. In particular, one must choose where to store these very copies and how to throw them there. First, I will give some examples of “where”.

Choose a cozy cloud

One option would be to simply rent a dedicated server or install your server in a data center on a collocation.

Indeed, the "cloud" that he built himself gives more control over what is happening, and the choice of a solution for storage and direct backup is up to the system administrator. You can even include the server in the domain “on the ground”, as I described in the article “ How I Hid the 1C Base in Germany ”.

On the other hand, control also means responsibility - it will be necessary to monitor the status of the server in case of hardware and software problems, while the lack of clouds in the form of Internet dependence and the issue of trust for outsiders has not been canceled.

But is it your rented server from an inexpensive host?

Another option would be to use specialized services that were created just for storing backups. The most famous example are Amazon Glacier services. They are shrouded in legends about the technology used - from tape cassettes to blu ray-disks and robotic arms. But officially this is an inexpensive HDD.

Unlike a rented server, the solution is already starting to smell like a bloody enterprise with many "nines of reliability" after the decimal point. True, like many web services at Amazon, it has a complex costing formula. To put it bluntly, uploading data to the service is free, storage is quite inexpensive ($ 1 for 1 TB per month), but you will have to pay to get the data. Like at old fairs - “admission is free, exit is 15 kopecks”.

Classic storage services like Amazon S3 and Yandex Object Storage can also be used for backups, of course, but the price tag in this case will be less humane - ~ $ 10 \ month for 1 TB for Yandex. Also, one cannot fail to mention all-inclusive solutions from manufacturers of backup systems, since only the lazy have the benefit of their cloud now. For example, Acronis Cloud Storage as an addition to Acronis products literally for $ 299 per year will give 250 GB on its servers.

The third option would be to use cloud storage, which is not very designed to store backups of the company, but more focused on ordinary users. Here are just a few of them that are heard:

• Dropbox
• OneDrive
• Google Drive
• Mega .
• Yandex.Disk .
• Cloud Mail.Ru.

I will not compare cloud platforms now, I will give it to the mercy of numerous materials on the network. For example, the article " Cloud storage for individuals: what to choose and why ." Personally, I settled on Yandex.Disk for my needs, because he is one of the few who can use WebDAV, APIs and snapshots (history) of files on disk on free plans. Well, of course, I have accumulated a certain amount of free gigabytes on it.

Of course, when choosing, you should pay attention not only to the free number and cost of gigabytes, but also to the license agreement, since backing up conditional 1C databases can violate it. Separately, it is worth noting the points on which the cloud provider does not bear any responsibility, can delete all files at any time and there will be nothing for it. But almost all of these services have software that allows you to upload files to the service, which brings us to the next point in today's story.

How to ship to a cozy cloud

Personally, I don’t really like to use the software provided by the services (unless, of course, we are talking about a specialized service like Acronis): it’s not always possible to set the synchronization schedule, and the story is still alive when Yandex.Disk arranged for Barmin’s patch when updating operating system. Fortunately, there are special software supporting various providers. As usual, I will give a few examples of mostly free and near-free solutions.

Handy Backup . It is issued on the first page of Google for the query "backup to the cloud." There are paid versions of various functionalities, separate plugins (for example, for Exchange and 1C). There is even a cloud - HBDrive . But most importantly, there is still a free version that can only backup to the cloud - Handy Backup Free for Cloud . Unfortunately, as part of the testing, I was not able to get it to work stably with Yandex.Disk - the periodically scheduled task did not work. It is difficult to want something from a free solution, but I refused to use this software.

CloudBerry Backup The product is good for everyone, there are even solutions for recovering individual Exchange objects, there is support for many different providers. The use was stopped by the lack of a free version and support of the usual Yandex.Disk, only S3 compatible Yandex Object Storage.

List of supported solution providers from CloudBerry Lab.

Duplicati 2 . Already a completely free product , even for commercial use. There are all popular platforms from Windows to GNU \ Linux, you can work both through the web interface and through the command line, there is also encryption of backups out of the box.

Duplicati interface, supported providers.

Unfortunately, Yandex.Disk is not supported “out of the box” - only in WebDAV mode. In this mode, the solution from Yandex does not work perfectly - there are problems with large files. But there is one on the valid destination list that solves this problem. There he is.

Rclone Perhaps this is my undisputed leader among other software. A command line utility for many platforms, on the official website , downloads are also available for rare operating systems like Plan9 and Solaris. The list of supported cloud providers is also impressive - it even supports Cephs and OwnCloud . And yes, Yandex.Disk is listed. Until recently, the configuration was carried out only through the interactive console menu, but relatively recently it became possible to launch the web interface and configure through it.

Rclone web interface.

The disadvantages include the lack of any built-in schedulers. The utility works exclusively as a transport to / from the clouds, but does not require installation. Including because of this, I use it in conjunction with Yandex.Disk to transfer information from one remote server to another - it turned out that large files are faster to upload to the cloud and download from the cloud than to organize direct file sharing. Yes, and upload backups is a pleasure. For example, to copy only fresh files to the cloud, you can use the command:

rclone copy --max-age 24h --no-traverse D:\backups yandex:backups
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Where yandex is the name of the config created in advance, and backups is the folder with backups.

The principles of rclone work are described in more detail in the official documentation and in the article “ Rclone: ​​rsync for clouds ”.

In principle, as already a complete backup solution, rclone can be used with Duplicati, choosing rclone as the type of storage. Then Duplicati will create backups using vss (snapshots) according to the scheduler, and the first will be responsible for downloading the backups to the cloud we need. Of course, you can use any other solution like Cobian or even do vss snapshots with the diskshadow command, archive and upload to the cloud using rclone. True, if you really reinvent the wheel, then no rclone is needed.

Create your bike script

Of course, if the cloud provider provides WebDAV access, downloading data will be easy. Example for cmd and Yandex.Disk:

 net use Z: "https://webdav.yandex.ru/backup/" /User:login@yandex.ru password rem     net use Z: /delete
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

But not all providers are good at WebDAV, and there are questions about speed and stability. Therefore, you can use the API, if, of course, the provider provides such access. Let's analyze an example with the same Yandex.

Yandex uses OAuth for authorization, so for our script we need to get a special token. First you need to create an application in the " Create an Application " section of the site.

You must not forget to give access to the application on Yandex.Disk:

Script access to Yandex.Disk API.

And substitute the development URL in the Callback URI (it will be available after setting the “Web Services” checkbox on available platforms):

Callback URI setup.

After receiving the application ID, follow the link:

 https://oauth.yandex.ru/authorize?response_type=token&client_id=12345678&display=popup
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Where 12345678 is the received ID. After granting the application access, we will receive the coveted OAuth token, which can already be used in scripts. Here, for example, uploading a file to Yandex.Disk using PowerShell:

 #   $filepath = "D:\backup.zip" $headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]" $headers.Add("Authorization" ,'OAuth Oauth') $headers.Add("Content-Type","application/json") #   URL    $UploadUrl= (Invoke-RestMethod -method GET -URI ("https://cloud-api.yandex.net:443/v1/disk/resources/upload?path=backup.zip") -Headers $headers).href #   Invoke-WebRequest -uri $UploadUrl -Method Put -Infile $filepath -ContentType 'application/zip'
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

It is proposed to organize file rotation, download control and other “body kits” on their own, since the Yandex API is well documented . But personally, I prefer not to reinvent the wheel, but to use rclone.

Well, when backing up to the cloud, I strongly recommend encrypting the archives, so as not to be in the situation as the hero of a poem by the famous poet Aiklaud Von Browser in certain circles, the line of which this article is named.

In the comments, I propose not to breed a holivar on the rationality of cloud backup, but to share my favorite backup tool.