Automatic backup and recovery of dynamic address lists on Mikrotik

RouterOS in Mikrotik does not remember dynamic elements of address lists, and in the event of a reboot or loss of power, only static elements will remain in the lists. For example, you have a rule that catches spammers or port scanners and bans their IP addresses for a month. Typically, such addresses are saved as dynamic entries in lists, but they are not saved during reboot, which means that it is easy to lose the “radish” database, collected for say a couple of months of continuous gateway operation.



For this scheme to work, you need an FTP server (I have it Windows, but the batch file is easily redone for nixes) with an account that has read / write permissions. The mechanism is based on the interaction of the following elements: the backupDynList_to_FTP and BackupDynListFromFTP scripts on the gateway, and the convertmtik.bat script on the FTP server. A third-party server is needed in order to offload the internal memory of Mikrotik (lists can be quite large, and non-volatile memory is not large for everyone) and in order to process the file with more powerful means of working with text than the built-in script language mikrotik has. The DynListExport function call is used to bypass the system limit on the size of the list. The need to convert the list is due to the fact that if the same element (say, a static record) is already present in the database, then an error occurs during import and the import is stopped. To avoid this, each time a static record is added, the error handler is reset and in case of a match, the record is simply not added, and the import process goes further.



The BackupDynList_to_FTP script is executed every hour (00:00), it collects all the data about dynamic records (which have at least half an hour to live) in all the lists into the iplist_dyn.src file and sends this file to ftp. On the server, every hour with a shift of half an hour from the first script (00:30), the convertmtik.bat batch file is executed, which converts the original script received from the gateway into a script that is protected from coincidence errors and saves it under the name iplist_dyn_done.src



Now, in the event of a fall or reboot of the gateway, at the time of launch, data about dynamic records automatically disappear, but 60 seconds after the launch, the BackupDynListFromFTP script downloads iplist_dyn_done.src from FTP and starts it to execute, restoring the lists.



The Hourly_Dynlist_Backup_on_FTP (/ sys sheduler) activator is executed at the beginning of each hour:

/system script run BackupDynList_to_FTP
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Script BackupDynList_to_FTP (do not forget to change the FTP address, username and password):

 /system script environment remove [ find where name="DynListExport" ]; :global DynListExport do={ :foreach i in=[/ip firewall address–list find where dynamic=yes and timeout>0d00h30m] do={ :local list [/ip firewall address–list get $i list]; :local address [/ip firewall address–list get $i address]; :local timeout [/ip firewall address–list get $i timeout]; :local comment [/ip firewall address–list get $i comment]; :put "/ip firewall address–list add list=$list address=$address timeout=$timeout comment=\"$comment\";"}; } :log info "Starting Backup to FTP Script..." :global iplistfile ("iplist_dyn.rsc") :if ([/file find name=$iplistfile]!= "") do={/file rem $iplistfile}; /execute script="\$DynListExport" file=$iplistfile :delay 60s /tool fetch address="_FTP_" port=21 mode=ftp src–path="iplist_dyn.rsc.txt" user= password= dst–path="iplist_dyn.src" upload=yes :delay 20s /file rem $iplistfile :log info "Finished Backup to FTP!"
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

The Activator BackupDynList_from_FTP (/ sys sheduler) is executed at the time the gateway is started (startup):

 {:delay 60s}; /system script run BackupDynListFromFTP
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Script BackupDynListFromFTP (do not forget to change the FTP address, username and password):

 :local BackupFile "iplist_dyn_done.src" /file remove [find name=$BackupFile] /tool fetch address="_FTP_" port=21 mode=ftp src–path="$BackupFile" user= password= /import file–name=$BackupFile {:delay 30s}; /file remove [find name=$BackupFile] /log info "$BackupFile imported"
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Convertmtik.bat server script - uses the port of the Linux SED utility for its work, for example from gnuwin32, which must be registered in path on the server:

 echo # ––––––––––––––––––––––––––––––––––––––––––––––––––––– — > iplist_dyn_done.src echo # %date% %time% >> iplist_dyn_done.src echo # ––––––––––––––––––––––––––––––––––––––––––––––––––––– — >> iplist_dyn_done.src echo /ip firewall address–list > iplist_dyn_done.src sed –e "s/;//" –e "s/\/ip firewall address–list //" –e "s/.*/:do { & } on–error={}/" iplist_dyn.src >> iplist_dyn_done.src