I am a danish programmer living in Bangkok.
Read more about me @ rasmus.rummel.dk.
Webmodelling Home > Bacula backup tutorial
Do good

Bacula backup tutorial - install, backup & restore

Updated 23 Apr 2013. This bacula backup tutorial is a step by step guide how to install Bacula, backup & restore Ubuntu and Windows using Bacula - the prime open source and free industrial grade backup solution.

Then you finish this bacula tutorial, you will be able to use Bacula to backup & restore multiple servers and multiple operation systems using one or multiple backup servers - Bacula easily scales to many hundreds of computers in a network.

Index :

Appendixes :

Relevant Links :



Bacula main components

Bacula consist of 5 main parts :

  • Director daemon : Controls backup & restore operations. In /etc/bacula/bacula-dir.conf you configure Jobs, FileSets, Schedules etc.
  • Bacula Console : A command line program that allows you to interact with the Director daemon, eg. then you need to run a Restore job.
  • Storage daemon : The Director hands off the responsibility for writing and reading the physical backup media to the Storage daemon - in our case the physical backup media are files (not tapes) on a harddrive, while the harddrive itself (not a tape drive) is our backup device.
  • File daemon : The File daemon is a program that must be installed on the machine to backup. The Directory communicates with the machines to backup through the File daemons. The File daemon is responsible for the file system dependent parts of the backup and restore processes, therefore the File daemon is specific to the OS - currently there is a Unix/Linux File daemon, a Windows File daemon and a Mac OSX daemon.
  • Catalog : A database that keeps records of all volumes, all jobs and all files in a way that allows the administrator to easily & quickly locate files for restore across full, differential & incremental backups of multiple servers - the Catalog is what sets Bacula apart from your own homegrown tar based backup system no matter how good you are.


Server schema

In this tutorial I am going to backup several servers, the following is the full server schema :

FQDN IP OS Bacula daemons installed Comments
backup.favouritehosting.com 27.254.33.96 Ubuntu Director, Catalog, Storage, Console All backup data are stored here and all the information about the backup data are also stored here and both Director & Console are installed here as well - this is our backup server.
web1.favouritehosting.com 27.254.33.57 Ubuntu Linux File Shows how to backup a linux web server.
db1.favouritehosting.com 27.254.33.58 Ubuntu Linux File Shows how to backup a MySQL database server.
win1.favouritehosting.com 27.254.33.65 Windows 2008r2 Windows File Shows how to backup a windows server.

Bacula can only be installed on a Linux OS (here Ubuntu), however Bacula can easily take backup of Windows and Mac OSX machines.



How to Install Bacula

We have 4 servers to install on.

On the backup server we need to have :

  • The bacula director daemon (controlling backup, restore, pools, volumes, jobs, filesets etc).
  • The bacula console program (command line utility we use to communicate with the director, eg. start a restore job or purge a volume).
  • The bacula storage daemon (responsible for reading & writing the backup media, in our case backup media is the files that contains the backups).
  • A MySQL server for the catalog database (Bacula can alternatively use postgresql or sqllite) - I will not go through how to install & configure MySQL as if you install Bacula on an empty system, MySQL will be automatically installed for you. If you already have MySQL server installed Bacula will prompt you if you want to let Bacula setup the database for you.
  • A mail sending program for sending backup & restore reports to your email account - Bacula comes with a small mail sender called bsmtp, which I will use in this tutorial (so no need to install & configure an MTA).

Let's start the installation :

  1. Install the backup server :
    1. Log on to the backup server either on a local or remote terminal. Here I use Putty on a Windows dev box to log onto the backup server.
    2. backup server shell> apt-get update : update your package information, so you will install the newest version of any package.
    3. Simulate an install of bacula to see what is going to happen :
      • First on an empty system :
        • backup server shell> apt-get -s install bacula : MySQL and email capability are automatically installed and will also be configured to some extent.
      • Second on a system already having MySQL and Postfix : (this is the actual system I am going to install on)
        • backup server shell> apt-get -s install bacula : MySQL and email capability are already available and it seems we need to configure the whole thing ourselves.
    4. backup server shell> apt-get install bacula : start the installation
    5. You are prompted whether you want to create the a database, choose Yes.
    6. You are prompted for the MySQL server root password.
    7. You can supply a password for the bacula-director-mysql, I just leave it blank to get an autogenerated password.
    8. backup server shell> dpkg -l | grep bacula : list the installed packages.
    9. DONE - that concludes the package installation on the backup server.
  2. Install a Linux File Daemon on the web server :
    1. Logon to the web server either on a local or remote terminal. Again I use Putty on a Windows dev box to log onto the web server.
    2. web1 shell> apt-get -s install bacula-fd : simulate (-s) an install of the bacula-fd (bacula file daemon) package to see what's going to happen - 2 packages are going to be installed & configured.
    3. web1 shell> apt-get install bacula-fd : start the actual installation
    4. web1 shell> dpkg -l | grep bacula : confirm the 2 packages are installed.
    5. DONE - bacula file daemon installed on backup target server.
  3. Install a Linux File Daemon on the database server :
    1. Do exactly the same as for the web server.
  4. Install a Windows File Daemon on the windows server :
    1. Log on to a windows server that you want to backup either locally or remote using eg. Remote Desktop.
    2. From SourceForge.net download the bacula windows file daemon that match the version on your backup server (in my case it is 5.0.3).
    3. On your windows server, double click the downloaded installation file to start the bacula file daemon installer.
    4. Press the "Next" button.
    5. Press the "I Agree" button.
    6. Choose "Automatic" installation type and then press the "Next" button.
    7. I choose only the "Client" component as I see no use for the windows local bacula CLI nor the BAT (I do think the BAT have great potential though). Click the "Next" button.
    8. Write the name that you later will give to the Bacula Director (I choose FH-dir for Favourite Hosting Director). Press the "Install" button.
    9. If you like, you can choose to write out a template for the Client{..} section to put in your bacula-dir.conf file on your backup server, however it is more fun less helpful. Press the "Next" button.
    10. Ok, you are finish installing the bacula file daemon on your windows server.
    11. Navigate to Bacula main folder "C:\Program Files\Bacula" and look at the files, especially note the bacula-fd.conf file.
    12. DONE - you are finish installing the bacula windows file daemon.

All necessary programs are now installed, it is time to start configuring the system.



How to Configure Bacula

Bacula configuration is done in 4 .conf files, one for each service :

  • bconsole.conf

    : /etc/bacula/bconsole.conf (on backup server) : configuring the Console program
    • Director section : Describing what Director this Console will access (where the Director is and what credentials to use for logon).
  • bacula-dir.conf

    : /etc/bacula/bacula-dir.conf (on backup server) : configuring the Director daemon
    • Director section : Describing this Director daemon.
    • Pool sections : You group multiple backup media (file volumes or tape volumes) into pools, eg. all tapes or files on which you store your full backup may be grouped into one Pool, while all tapes or files on which you store your incremental backup may be grouped into another Pool.
    • Job sections : We will have many backup & restore jobs, each of them is defined in a Job{..} section.
    • Schedule sections : A Job wil refer to one or more Schedule{..} sections defining when the Job will run.
    • Client sections : Describing the backup target machines (where they are and logon credentials). A Job will refer to one Client{..} section defining on which machine the Job will run (in our case we need Client{..} sections for web1, db1 & win1).
    • FileSet sections : A Job will refer to one FileSet{..} section defining what files to backup on the Client or what files to restore.
    • Messages section : Here you can setup notification emails to your email address so you know what have happened, eg. if a backup job succeeded.
    • Catalog section : Describing the Catalog database (where the database is and logon credentials).
  • bacula-sd.conf

    : /etc/bacula/bacula-sd.conf (on backup server) : configuring the Storage daemon
    • Storage section : Describing this Storage daemon.
    • Director section : Authentication details for the Director that is allowed to control this Storage daemon.
    • Device section : Describing the backup device (in our case the backup device is a harddrive, but it could also be a tape drive).
    • Messages section : Setup which messages are sent back to the Director.
  • bacula-fd.conf

    : /etc/bacula/bacula-fd.conf (on client server) : configuring the File daemons - one for each backup target (here web1, db1 & win1)
    • Client section : Describing this File daemon.
    • Director section : Authentication details for the Director that is allowed to control this File daemon.
    • Messages section : Setup which messages are sent back to the Director.


Bacula Configuring Connections

The first thing we want to achieve is to be able to use the Console to connect to the Director and the Director to connect to the Storage & 3 File daemons.

Here is the official connection credentials overview. I have added the tables to show also the connection addresses :

   

Console

:
bacula-dir.conf bconsole.conf
Director{..} Name==Director{..} Name
Director{..} Password==Director{..} Password
Director{..} DirAddress==Director{..} Address


Storage

:
bacula-dir.conf bacula-sd.conf
Director{..} Name==Director{..} Name
Storage{..} Password==Director{..} Password
Storage{..} Address==Storage{..} SDAddress


File Daemon

:
bacula-dir.conf bacula-fd.conf
Director{..} Name==Director{..} Name
Client{..} Password==Director{..} Password
Client{..} Address==FileDaemon{..} FDAddress

Ok, let's implement it :

  1. backup server shell> cd /etc/bacula : navigate to the Bacula main folder.
  2. backup server shell> ls -l : list all files in the Bacula main folder, especially note bacula-dir.conf, bconsole.conf & bacula-sd.conf (ignore the bacula-fd.conf which we will not use since we do not backup anything on the backup server).
  3. bacula-dir.conf : (on backup server)
    1. backup server shell> nano /etc/bacula/bacula-dir.conf : open bacula-dir.conf on the backup server in the nano editor.
      1. Edit the bacula-dir.conf Director{..} section :
        • Director {
        •     Name = FH-dir : I have changed the name to FH-dir for Favourite Hosting Director.
        •     DIRport = 9101 : Director will listen on this port, eg. the Console program must contact Director on this port.
        •     QueryFile = "/etc/bacula/scripts/query.sql"
        •     WorkingDirectory = "/var/lib/bacula"
        •     PidDirectory = "/var/run/bacula"
        •     Maximum Concurrent Jobs = 1
        •     Password = "dir-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" : if you want to logon to the Director, you need to use this password (eg. then using the Console program).
        •     Messages = Daemon
        •     DirAddress = 27.254.33.96 : changed from 127.0.0.1 (this is not strictly necessary, however it is necessary in other places, so I like to use all global IP addresses). Update : it seems that in Bacula version 5.2.5 that bacula-dir.conf Director{} DirAddres MUST be equal to bconsole.conf Director {} address, so if you change to global IP instead of localhost, you should also do so in bconsole.conf.
        • }
      2. Edit the bacula-dir.conf Catalog{..} section : (Bacula version 5.0.3 have an error in the generated bacula-dir.conf file in the Catalog{..} section)
        • Catalog {
        •     Name = MyCatalog
        •     # Uncomment the following line if you want the dbi driver
        •     # dbdriver = "dbi:sqlite3"; dbaddress = 127.0.0.1; dbport =
        •     dbname = bacula; DB Address = ""; dbuser = "bacula"; dbpassword = "catalog-zzzzzzzzzzzz" : bacula version 5.0.3 have quotes around dbname value "bacula", remove the quotes, otherwise the Director will not be able to connect to the database (keep the other quotes around the DB Address, dbuser & dbpassword values).
        • }
      3. Delete or out-comment ALL bacula-dir.conf Client{..} sections. If bacula-dir.conf have a Client{..} section that have invalid references, the Director cannot start.
      4. Create a bacula-dir.conf Client{..} section for web1 :
        • Client {
        •     Name = web1 : this Client{..} section defines the web1 server, so I think web1 is a good name.
        •     Address = 27.254.33.57 : the IP address of the web1 server.
        •     FDPort = 9102 : the port on which the file daemon on web1 will be listening (that is: the port on which this director should contact the file daemon).
        •     Catalog = MyCatalog
        •     Password = "web1-yyyyyyyyyyyyyyyyyyyyyyyyyyyyyy" : password specific for web1 and must match the password specified in bacula-fd.conf Director{..} section on the web1 server.
        •     File Retention = 30 days
        •     Job Retention = 6 months
        •     AutoPrune = yes
        • }
      5. Create a bacula-dir.conf Client{..} section for db1 :
        • Client {
        •     Name = db1
        •     Address = 27.254.33.58 : the IP address of the db1 server.
        •     FDPort = 9102
        •     Catalog = MyCatalog
        •     Password = "db1-yyyyyyyyyyyyyyyyyyyyyyyyyyyyyy" : password specific for db1 and must match the password specified in bacula-fd.conf Director{..} section on db1.
        •     File Retention = 30 days
        •     Job Retention = 6 months
        •     AutoPrune = yes
        • }
      6. Create a bacula-dir.conf Client{..} section for win1 :
        • Client {
        •     Name = win1
        •     Address = 27.254.33.65 : the IP address of the win1 server.
        •     FDPort = 9102
        •     Catalog = MyCatalog
        •     Password = "win1-yyyyyyyyyyyyyyyyyyyyyyyyyyyyyy" : password specific for win1 and must match the password specified in bacula-fd.conf Director{..} section on win1.
        •     File Retention = 30 days
        •     Job Retention = 6 months
        •     AutoPrune = yes
        • }
      7. Edit the bacula-dir.conf Storage{..} section : (there are 4 storage sections, however 3 of them are out-commented and not interesting for us)
        • Storage {
        •     Name = storage1 : I am going to use 2 devices (disks) for backup, so I like the name of this section to represent the disk number of the device this section is going to point to.
        •     # Do not use "localhost" here
        •     Address = 27.254.33.96 : change localhost to the global IP of the storage server (even if the storage server is indeed localhost).
        •     SDPort = 9103 : the port on which to connect the Storage Daemon.
        •     Password = "storage-ssssssssssssssssssssssss" : this password much match the bacula-sd.conf Director{..} section.
        •     Device = disk1 : for each device (harddrive, DVD drive, tape drive) the Storage daemon uses for storage, there will be a Device{..} section in the bacula-sd.conf file. Here we reference by name, disk1, the Device{..} section in bacula-sd.conf that we want to use for selecting storage device (disk1 is here not an actual device but just a name pointing to a Device{..} section in bacula-sd.conf).
        •     Media Type = File : Bacula supports many different media types, eg. DDS-4 tapes, DVDs etc. We are using files to store the backup data, so we specify the Media Type = File.
        • }
      8. Edit the bacula-dir.conf JobDefs{..} section : (we don't want to define it now, but we need to edit some wrong references, otherwise the Director cannot start)
        • JobDefs {
        •     Name = "DefaultJob"
        •     Type = Backup
        •     Level = Incremental
        •     Client = web1 : here we need to reference the name of one of our Client{..} sections, otherwise Director will not start.
        •     FileSet = "Full Set"
        •     Schedule = "WeeklyCycle"
        •     Storage = storage1 : here we need to reference the name of our Storage{..} section, otherwise Director will not start.
        •     Messages = Standard
        •     Pool = File
        •     Priority = 10
        •     Write Bootstrap = "/var/lib/bacula/%c.bsr"
        • }
      9. One of the bacula-dir.conf Job{..} sections contains a Client reference, set the value of that Client to web1 (we don't want to set up the whole Job{..} section now, we just want to avoid an invalid reference to a Client{..} section that does not exist). If bacula-dir.conf have a Job{..} section with invalid references, the Director cannot start.
    2. Press ctrl+x and then y to exit the nano editor and save the bacula-dir.conf file.
    3. backup server shell> service bacula-director restart : restart the Director daemon to apply the new configurations.
  4. bconsole.conf : (on backup server)
    1. backup server shell> nano /etc/bacula/bconsole.conf : open bconsole.conf on the backup server in the nano editor.
      1. Edit the bconsole.conf Director section :
        • Director {
        •     Name = FH-dir : Here you must use the same name you gave the bacula-dir.conf Director{} section (there the Director defines itself).
        •     DIRport = 9101 : The port on which the Console will contact the Director.
        •     address = 27.254.33.96 : the address on which the Director runs (I changed address from localhost which in our case is not strictly necessary, however I like to use all global IP addresses).
        •     Password = "dir-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" : the password we together with the name use to logon to the Director (must match the password in the bacula-dir Director{} section).
        • }
    2. Press ctrl+x and then y to exit the nano editor and save the bconsole.conf file.
  5. bacula-sd.conf : (on backup server)
    1. backup server shell> nano /etc/bacula/bacula-sd.conf : open bacula-sd.conf on the backup server in the nano editor.
      1. Edit the bacula-sd.conf Director section :
        • Director {
        •     Name = FH-dir : the name of the Director that can administer this Storage daemon.
        •     Password = "storage-ssssssssssssssssssssssss" : the password that the Director must use then administer this Storage daemon. This password is also in the bacula-dir.conf Storage{..} section.
        • }
      2. Edit the bacula-sd.conf Storage section :
        • Storage {
        •     Name = FH-sd : I use FH-sd for Favourite Hosting Storage Daemon. This name is also in the bacula-dir.conf Storage{..} section.
        •     SDPort = 9103 : The port on which this Storage daemon will listen (and therefore the port on which the Director must contact this Storage daemon).
        •     WorkingDirectory = "/var/lib/bacula"
        •     Pid Directory = "/var/run/bacula"
        •     Maximum Concurrent Jobs = 20
        •     SDAddress = 27.254.33.96 : change 127.0.0.1 to global IP of the storage server.
        • }
      3. Edit the bacula-sd.conf Device section : (there are many Device{..} sections, however only one is un-commented and that is the one we are interested in)
        • Device {
        •     Name = dis1 : this Device{..} section will specify a disk device to use for storage, since I use multiple disks, I think disk1 is a suitable name. Also this Device{..} section is referenced by bacula-dir.conf Storage{..} section by name, disk1.
        •     Media Type = File : on this device we will use files to store the backup data, therefore the Media Type = File.
        •     Archive Device = /media/disk1 : here you specify the mount point of the actual device (the actual harddrive I want to use is mounted as /media/disk1). You can also use just a folder to store your backup, it does not need to be a fresh disk.
        •     LabelMedia = yes;
        •     Random Access = Yes;
        •     AutomaticMount = yes;
        •     RemovableMedia = no;
        •     AlwaysOpen = no;
        • }
      4. Edit the bacula-sd.conf Messages section :
        • Messages {
        •     Name = Standard
        •     director = FH-dir = all : change to correct Director name, in my case to FH-dir.
        • }
    2. Press ctrl+x and then y to exit the nano editor and save the bacula-sd.conf file.
    3. backup server shell> service bacula-sd restart : restart the Storage daemon to apply the new configurations.
  6. bacula-fd.conf on web1 :
    1. web1 server shell> cd /etc/bacula : navigate to the Bacula main folder on the web1 server.
    2. web1 server shell> ls -l : list all files in the Bacula main folder, especially note the important bacula-fd.conf file.
    3. web1 server shell> nano /etc/bacula/bacula-fd.conf : open bacula-fd.conf on the web1 server in the nano editor.
      1. Edit the bacula-fd.conf Director section : (defining what Director is allowed to control this this file daemon)
        • Director {
        •     Name = FH-dir : the name I gave the Director in the bacula-dir.conf file.
        •     Password = "web1-yyyyyyyyyyyyyyyyyyyyyyyyyyyyyy" : this passwords needs to match the web1 Client{..} section in the bacula-dir.conf file.
        • }
      2. Note the bacula-fd.conf FileDaemon section : (identifies this file daemon, so it can be referenced by the Director)
        • FileDaemon {
        •     Name = web1-fd : the default name, web1-fd, is exactly the name I want, so no need to change it.
        •     FDport = 9102 : the port on which this file daemon will listen (so the Director needs to use this port to contact the file daemon).
        •     WorkingDirectory = /var/lib/bacula
        •     Pid Directory = /var/run/bacula
        •     Maximum Concurrent Jobs = 20
        •     FDAddress = 27.254.33.57 : changed from 127.0.0.1 as I like to use all global IP addresses (even though it is not necessary here).
        • }
      3. Edit the bacula-fd.conf Messages section :
        • Messages {
        •     Name = Standard
        •     Director = FH-dir = all, !skipped, !restored : Director name needs to be the same as specified in the first Director{..} section, in my case FH-dir.
        • }
    4. Save the bacula-fd.conf file (either directly if you use eg. nano or upload if you use ftp).
    5. web1 server shell> service bacula-fd restart : restart the file daemon to apply the configuration changes.
  7. bacula-fd.conf on db1 :
    1. db1 server shell> cd /etc/bacula : navigate to the Bacula main folder on the db1 server.
    2. db1 server shell> ls -l : list all files in the Bacula main folder, especially note the important bacula-fd.conf file.
    3. db1 server shell> nano /etc/bacula/bacula-fd.conf : open bacula-fd.conf on the db1 server in the nano editor.
      1. Edit the bacula-fd.conf Director section : (defining what Director is allowed to control this this file daemon)
        • Director {
        •     Name = FH-dir : the name I gave the Director in the bacula-dir.conf file.
        •     Password = "db1-yyyyyyyyyyyyyyyyyyyyyyyyyyyyyy" : this passwords needs to match the db1 Client{..} section in the bacula-dir.conf file.
        • }
      2. Note the bacula-fd.conf FileDaemon section : (identifies this file daemon, so it can be referenced by the Director)
        • FileDaemon {
        •     Name = db1-fd : the default name, db1-fd, is exactly the name I want, so no need to change it.
        •     FDport = 9102 : the port on which this file daemon will listen (so the Director needs to use this port to contact the file daemon).
        •     WorkingDirectory = /var/lib/bacula
        •     Pid Directory = /var/run/bacula
        •     Maximum Concurrent Jobs = 20
        •     FDAddress = 27.254.33.58
        • }
      3. Edit the bacula-fd.conf Messages section :
        • Messages {
        •     Name = Standard
        •     Director = FH-dir = all, !skipped, !restored
        • }
    4. Press ctrl+x and then y to exit the nano editor and save the bacula-sd.conf file.
    5. db1 server shell> service bacula-fd restart : restart the file daemon to apply the configuration changes.
  8. bacula-fd.conf on win1 :
    1. Navigate to C:\Program Files\Bacula, the Bacula main folder on the win1 server.
    2. In C:\Program Files\Bacula folder notice the important bacula-fd.conf file.
    3. Open and edit the bacula-fd.conf file using your favourite windows text editor :
      1. Edit the bacula-fd.conf Director section : (defining what Director is allowed to control this this file daemon)
        • Director {
        •     Name = FH-dir : the name I gave the Director in the bacula-dir.conf file.
        •     Password = "win1-yyyyyyyyyyyyyyyyyyyyyyyyyyyyyy" : this passwords needs to match the win1 Client{..} section in the bacula-dir.conf file.
        • }
      2. Note the bacula-fd.conf FileDaemon section : (identifies this file daemon, so it can be referenced by the Director)
        • FileDaemon {
        •     Name = win1 : default Bacula installer will set the name to [hostname]-fd, however I here I just use the hostname, win1.
        •     FDport = 9102 : the port on which this file daemon will listen (so the Director needs to use this port to contact the file daemon).
        •     WorkingDirectory = "C:\\Program Files\\Bacula\\working"
        •     Pid Directory = "C:\\Program Files\\Bacula\\working"
        •     # Plugin Directory = "C:\\Program Files\\Bacula\\plugins"
        •     Maximum Concurrent Jobs = 10
        • }
      3. Edit the bacula-fd.conf Messages section :
        • Messages {
        •     Name = Standard
        •     Director = FH-dir = all, !skipped, !restored
        • }
    4. Save the bacula-fd.conf file.
    5. Restart the file daemon service on win1 :
      1. On the win1 server press the Windows Key + R to open the run box, write services.msc and press the "Ok" button to launch the Services Manager.
      2. In the Services Manager right click on "Bacula File Service" and choose "Restart" from the context menu.


Bacula Configuring Connections - Test

  1. backup server shell> bconsole : start the console program and confirm that you can connect to Director.
  2. *> status storage : test if you can connect.
  3. *> status client : prompt for a client to connect to - try selecting web1-fd to see if you can connect.
  4. *> list clients : you should be able to see the 3 clients web1, db1 & win1.
  5. *> quit : quit Bacula Console to return to OS system prompt.
  6. Just for the fun of it, let's confirm the clients using the Catalog database :
    1. backup server shell> mysql -u bacula -pzzzzzzzzzzzz : logon to MySQL server using the bacula credentials specified in the bacula-dir.conf Catalog{..} section (notice there is no space between -p and the password).
    2. mysql> show databases; : lets see what databases the bacula user have access to.
    3. mysql> use bacula; : set default database to bacula (bacula is the name of the Catalog database).
    4. mysql> show tables; : lets see what tables are in the Catalog database.
    5. mysql> select * from Client : and there you get exactly the same 3 clients we saw before using the bacula list clients command.
    6. mysql> quit : quit MySQL CLI prompt to return to OS system prompt.

Congratulations - your Bacula Console, Director and File Daemons should now be able to connect (however there is no backup going on yet).



Bacula Configuring Storage

On my backup server, I have 3 harddrives, here I am going to use 2 of them as bacula backup devices (the 3. harddrive I use for OS, Bacula, ad-hoc backup etc.).

Indeed Bacula does support multiple storage devices, eg. store backup data on multiple harddrives spread over multiple backup servers. Unfortunately Bacula does not natively support to treat multiple harddrives as one harddrive (it can be setup outside of bacula, but it is too cumbersome for my taste, sere more here). Instead in bacula-sd.conf we need to specify a Device{..} section for each harddrive we want to use as a backup device - in our case 2 such Device{..} sections.

Ok, let's configure the storage :

  1. bacula-sd.conf

    1. backup server shell> nano /etc/bacula/bacula-sd.conf : open bacula-sd.conf on the backup server in the nano editor.
      1. Edit the bacula-sd.conf Device{..} section :
        • Device {
        •     Name = disk1 : on this device we will store backup of web1.
        •     Media Type = File
        •     Archive Device = /media/disk1 : mount point of the harddrive (if you use folders instead of harddrives, just specify a folder here).
        •     LabelMedia = yes; : then a new media volume (in our case a file inside which backup data is stored) is created, we want Bacula to automatically give the volume (file) a name.
        •     Random Access = Yes;
        •     AutomaticMount = yes;
        •     RemovableMedia = no;
        •     AlwaysOpen = no; : this directive is ignored for Media Type = File, therefore in our case yes or no is the same.
        • }
      2. Create a second Device{..} section in the bacula-sd.conf file :
        • Device {
        •     Name = disk2 : on this device we will store backup of db1 and win1.
        •     Media Type = File
        •     Archive Device = /media/disk2 : mount point of the harddrive.
        •     LabelMedia = yes;
        •     Random Access = Yes;
        •     AutomaticMount = yes;
        •     RemovableMedia = no;
        •     AlwaysOpen = no;
        • }
    2. Press ctrl+x and then y to exit the nano editor and save the bacula-sd.conf file.
    3. backup server shell> service bacula-sd restart : restart the storage daemon to apply the configuration changes.
  2. bacula-dir.conf

    1. backup server shell> nano /etc/bacula/bacula-dir.conf : open bacula-dir.conf on the backup server in the nano editor.
      1. Edit the bacula-dir.conf Storage{..} section :
        • Storage {
        •     Name = storage1
        •     Address = 27.254.33.96
        •     SDPort = 9103
        •     Password = "storage-ssssssssssssssssssssssss" : must match bacula-sd.conf Director{..} Password.
        •     Device = disk1 : reference to the Device{..} section in bacula-sd.conf that use the harddrive mounted on /media/disk1.
        •     Media Type = File
        • }
      2. Create a new Storage{..} section in the bacula-dir.conf file : (if you have only 1 harddrive, you can still separate your backup jobs by storing the volume files in separate folders if you should wish)
        • Storage {
        •     Name = storage2
        •     Address = 27.254.33.96
        •     SDPort = 9103
        •     Password = "storage-ssssssssssssssssssssssss" : must match bacula-sd.conf Director{..} Password.
        •     Device = disk2 : reference to the Device{..} section in bacula-sd.conf that use the harddrive mounted on /media/disk2.
        •     Media Type = File
        • }
      3. Delete or out-comment all Pool{..} sections in bacula-dir.conf.
      4. Create a new Pool{..} section in bacula-dir.conf : (here we group volumes for disk1, however we could also group volumes for full backup and another group for incremental backup)
        • Pool {
        •     Name = pool1
        •     Pool Type = Backup
        •     Recycle = yes : yes (default) means that purged Volumes will be recycled then Bacula finds no Volume that are appendable. A purged Volume is a Volume that contains only expired Jobs & files and thus is not in the Catalog.
        •     AutoPrune = yes : yes (default) means that Bacula will automatically apply the Volume Retention period then a new Volume is needed (no appendable Volume exist in the Pool). Volume Pruning means to delete expired Jobs (older than Volume Retention period) from the Catalog permitting possible recycling of the Volume.
        •     Volume Retention = 61 days : I don't want to recycle the volumes before last writing to the volume is 2 month old (that way I should be able to go 2 month back in time).
        •     Maximum Volume Bytes = 25G : this is a tricky size, if the file is corrupted, the data cannot be restored so the file size should be small, but on the other hand if you run out of backup space you will need to purge the volumes one at a time, so the volume size should be big to avoid having to manually purging too many volumes.
        •     Maximum Volumes = 19 : the harddrive is 500G, so I can have 20 volumes of 25G (but to avoid problems, I set it to 19)
        •     Label Format = pool1- : volumes are now created automatic and labelled pool1-0000, pool1-0001 and so forth (if you have different pools for your full and incremental backups, you would probably use a label format like Full- and Incr-).
        •     Catalog Files = yes : yes (default) means that file names backed up will be written to the catalog database. If setting the value to "no", then the catalog database will be much smaller, howerver it will not be possible to produce a catalog listing of files backed up for each Job and the console restore command will not work.
        •     Maximum Volume Jobs = 0 : zero (default) means no limit on the number of Jobs that can be written to the same Volume.
        •     Maximum Volume Files = 0 : zero (default) means no limit on the number of files that can be written to the same Volume.
        •     Volume Use Duration = 0 : zero (default) means no limit on the time that files can be written to the same Volume. A positive value defines a limited number of days that a Volume can be written to starting from the first write.
        • }
      5. Create a second new Pool{..} section in bacula-dir.conf : (to group volumes for incremental backup)
        • Pool {
        •     Name = pool2
        •     Pool Type = Backup
        •     Recycle = yes
        •     AutoPrune = yes
        •     Volume Retention = 62 days : Then the last writing to a volume is more than 61 days old, Bacula is allowed to purge the volume and recycle it if needed for storage space.
        •     Maximum Volume Bytes = 25G : Each volume should be 25G.
        •     Maximum Volumes = 19 : In this Pool we will have 19 volumes.
        •     Label Format = pool2- : In this Pool (group of volumes), I want to prefix all volume names with pool2-
        •     Catalog Files = yes
        •     Maximum Volume Jobs = 0
        •     Maximum Volume Files = 0
        •     Volume Use Duration = 0
        • }
    2. Press ctrl+x and then y to exit the nano editor and save the bacula-dir.conf file.
    3. backup server shell> service bacula-director restart : restart the director daemon to apply the configuration changes.

For the storage daemon we have now configured 2 backup devices and for the director we have configured references to these devices as well as 2 pools (collection of volume files), one pool for each backup device.



Bacula Configuring Schedules

At it's most basic level, a schedule defines then to execute what, in our case then to execute a job (typically only backup jobs & verify jobs, not restore jobs, are scheduled). In our case we are going to slightly complicate this very basic schedule for our backup jobs : (setting the backup level)

  1. bacula-dir.conf

    1. backup server shell> nano /etc/bacula/bacula-dir.conf : open bacula-dir.conf on the backup server in the nano editor.
      1. Create a new Schedule{..} section : (it's possible to define many Scedule{..} sections, however in our case we need only one)
        • Schedule {
        •     Name = "MonthlyCycle"
        •     Run = Level=Full on 1 at 2:05 : the 1. of every month at 2:05 I want to take a full backup.
        •     Run = Level=Incremental on 2-31 at 2:05 : all other days of the month, I want to take an incremental backup.
        • }
    2. Press ctrl+x and then y to exit the nano editor and save the bacula-dir.conf file.
    3. backup server shell> service bacula-director restart : restart the Director to apply the new configurations.

Note that in the above Schedule{..} section we are for any job definition using the schedule actually overwriting the backup level property of the job by saying that on the 1. of every month we want the level to be full and all other days we want the level to be incremental - this means that we don't need to specify one job for full backup and another job for incremental backup with each their schedule, thus saving us one Job{..} section for each client.

While the above schedule is all we need in our setup, the Schedule section is actually a much more powerful construct with the ability to overwrite also other job properties like Level, Pool, Storage & Messages. This fleksibility of the Schedule{..} section may often greatly reduce the amount of jobs that we need to define.

Say that on a single backup device, you wanted to keep full backup in one pool and incremental backup in another pool, you could then have the following schedule : (overwriting the pool to use depending on level).

  • Schedule {
  •     Name = "MonthlyCycle"
  •     Run = Level=Full Pool=MyFullPool on 1 at 2:05 : taking full backup using the MyFullPool for storage media.
  •     Run = Level=Incremental FullPool=MyFullPool IncrementalPool=MyIncrPool on 2-31 at 2:05 : for incremental backup use MyIncrPool, however if an incremental backup is upgraded to full (no prior full backup exists), then use MyFullPool.
  • }

Note that the Pool override does NOT apply if you run the job manually, in which case the Pool will be selected from the Job {} resource (or from you manually modifying the Pool then executing a run)



Bacula Configuring Messages

We want to be able to overview how the jobs are running - imagine our backup jobs failed (it can fail for many reasons, eg. no more backup space or a file daemon have crashed etc.) and we did not know about it and a month later we need to restore - that would be bad, very bad.

Bacula daemons all have the ability to send job execution information to various targets (log files, catalog, emailing, director), however Bacula also makes it possible to aggregate all the information of a job in one report per job. Here I will setup messaging so that we are emailed one report per job and also send that report to the Director so it is available using the Console program :

  1. bacula-dir.conf

    1. backup server shell> nano /etc/bacula/bacula-dir.conf : open bacula-dir.conf on the backup server in the nano editor.
      1. backula-dir.conf contains several Messages{..} sections, find the Messages{..} section with Name = Standard and edit it as following :
        • Messages {
        •     Name = Standard : the name by which a job can reference this section and thereby define the job report.
        •     mailcommand = "/usr/lib/bacula/bsmtp -h localhost -f \"\(Bacula\) \<backup@favouritehosting.com\>\" -s \"Bacula: %t %e of %c %l\" %r" : command mailing the job report, the command have the following parts :
          • /usr/lib/bacula/bsmtp -h localhost : use bsmtp on host (-h) localhost to send the email.
          • -f \"\(Bacula\) \<backup@favouritehosting.com\>\" : create from (-f) field "Bacula backup@favouritehosting.com".
          • -s \"Bacula: %t %e of %c %l\" : create subject (-s) field "Bacula: [JobType] (-t) [Exit Code] (-e) of [Client] (-c) [Level] (-l), eg. "Bacula: Backup OK of win1 Incremental".
        •     operatorcommand = "/usr/lib/bacula/bsmtp -h localhost -f \"\(Bacula\) \<backup@favouritehosting.com\>\" -s \"Bacula: Intervention needed for %j\" %r" : command mailing maintenance needs.
        •     mail = rasmusrummel@gmail.com = all, !skipped : using the mailcommand mail all message types except skipped to rasmusrummel@gmail.com all (eg. incremental backup will skip files not changed since last backup - do not send me any message that these files are not backup)
        •     operator = rasmusrummel@gmail.com = mount : using the operatorcommand mail all messages of type mount to rasmusrummel@gmail.com.
        •     console = all, !skipped, !saved : send all messages except of type skipped & saved to the console (these messages are held by the console until you request watching them using the messages command).
        •     #append = "/var/lib/bacula/log" = all, !skipped : I like to out-comment the log as I don't use it. If you keep the log file, you need to delete it periodically (it would be great if it was possible to set a maximum size on the log file so that oldest records are deleted then new records are appended).
        •     catalog = all : send all messages to the catalog (bacula) database (in table Log). The records in the Log table are pruned then the Job records are pruned so the table will not grow indefinetely. The Log table is typically used by reporting software, but can also be searched the same way as the file log I just out-commented.
        • }
    2. Press ctrl+x and then y to exit the nano editor and save the bacula-dir.conf file.
    3. backup server shell> service bacula-director restart : restart the Director to apply the new configurations.
  2. Install mail transfer agent (MTA)

    Bacula comes with a mail user agent (MUA) called bsmtp. Default your Message {} section will use /usr/lib/bacula/bsmtp to try to connect to an MTA on localhost as in the operator command above operatorcommand = "/usr/lib/bacula/bsmtp -h localhost ...". While bsmtp supports connecting to an external MTA, I have not tried how to authenticate, so I recommend installing an MTA on the same host as your Director. Ubuntu default installation does not come with any MTA, so if you have not installed an MTA on your Bacula Director host already, you need to install one now :

    1. backup server shell> nmap localhost : first check if any smtp daemon is already listening on port 25, if there are then an MTA is probably already available and you can skip the next step.
    2. backup server shell> apt-get install sendmail-bin : sendmail-bin is a small footprint well working MTA supporting both spools and sockets.

    Note that because of bsmtp capability to connect to an external host, bsmtp does not deliver messages to a spool service like /var/spool/mqueue, but instead connect to your MTA directly using TCP (like in telnet localhost 25).



Bacula Configuring Jobs

There are 3 types of jobs, backup jobs, restore jobs and verify jobs. We will create one backup job and one restore job for each client (each server to backup).

Then configuring a job, we need to specify quite a lot of job parameters :

  • On what client the job should run - we reference a Client{..} section.
  • What files to backup (or restore) - we reference a FileSet{..} section.
  • When to execute the job - we reference a Schedule{..} section (a schedule is only necessary for jobs you want to execute automatically).
  • On what device (storage) to store the backup - we reference a Storage{..} section.
  • What group of volumes to store the backup - we reference a Pool{..} section (a pool is a group of volumes).
  • What type the job is (backup, restore or verify)
  • In case of type = backup then also what level to backup (full, differential or incremental).
  • In case of type = verify then also what to verify (lots of settings there).
  • What priority the job has compared to other jobs

Also since Bacula does not natively support treating multiple physical devices as one logical device (eg. by being able to specify multiple "Archive Device" within one Device{..} section), we will need to spread the backup load by manually deciding which jobs run on which device - that is just not smooth, but that is how we are going to do it :

  1. bacula-dir.conf

    1. backup server shell> nano /etc/bacula/bacula-dir.conf : open bacula-dir.conf on the backup server in the nano editor.
      1. Setup 2 collections of job properties (one for each backup device) so we don't have to write all these properties more than once :
        1. Edit the bacula-dir.conf JobDefs{..} section : (there is default only one JobDefs{..} section)
          • JobDefs {
          •     Name = "jobdefs1" : all jobs including the definitions setup here will be backed up to disk1.
          •     Type = Backup :
          •     Schedule = "MonthlyCycle" :
          •     Storage = storage1 :
          •     Messages = Standard :
          •     Pool = pool1 : in our case we actually use this pool as we don't use the Schedule {..} section to overwrite the pool to use.
          •     Priority = 10 :
          •     Write Bootstrap = "/var/lib/bacula/%c.bsr" :
          • }
        2. Create a new bacula-dir.conf JobDefs{..} section : (to hold default values for all jobs that is associated with disk2)
          • JobDefs {
          •     Name = "jobdefs2"
          •     Type = Backup :
          •     Schedule = "MonthlyCycle" :
          •     Storage = storage2 :
          •     Messages = Standard :
          •     Pool = pool2
          •     Priority = 10 :
          •     Write Bootstrap = "/var/lib/bacula/%c.bsr" :
          • }
      2. Setup the 3 backup jobs (one backup job for each client to backup) :
        1. Create a new bacula-dir.conf Job{..} section :
          • Job {
          •     Name = "web1-backup" : this job will run on web1 and is of type backup.
          •     JobDefs = "jobdefs1" : include all job settings from jobdefs1.
          •     Client = web1 : reference to the web1 Client{..} section to specify what client to run this job on (web1 server).
          •     FileSet = "web1" : reference to the web1 FileSet{..} section to specify which files to backup on the client (which files to backup on the web1 server).
          • }
        2. Create a new bacula-dir.conf Job{..} section :
          • Job {
          •     Name = "db1-backup"
          •     JobDefs = "jobdefs2"
          •     Client = db1
          •     FileSet = "db1"
          • }
        3. Create a new bacula-dir.conf Job{..} section :
          • Job {
          •     Name = "win1-backup"
          •     JobDefs = "jobdefs2"
          •     Client = win1
          •     FileSet = "win1"
          • }
      3. Setup the 3 restore jobs : (one restore job for each client that we may need to restore)
        1. Create a new bacula-dir.conf Job{..} section :
          • Job {
          •     Name = "web1-restore"
          •     Type = Restore" : specifying that this is a restore job.
          •     Client = web1 : which client we want to restore.
          •     FileSet = "web1" : what files we are restoring from.
          •     Storage = "storage1" : on what device to find the backup files.
          •     Pool = "pool1" : in what collection of volumes the backup files are stored.
          •     Messages = "standard" : what messages to send where about the restore job.
          •     Where = "/var/FHRestore" : where to restore the files to, here I restore to a folder /var/FHRestore on the web1 server.
          • }
        2. Create a new bacula-dir.conf Job{..} section :
          • Job {
          •     Name = "db1-restore"
          •     Type = Restore"
          •     Client = db1
          •     FileSet = "db1"
          •     Storage = "storage2"
          •     Pool = "pool2"
          •     Messages = "standard"
          •     Where = "/var/FHRestore"
          • }
        3. Create a new bacula-dir.conf Job{..} section :
          • Job {
          •     Name = "win1-restore"
          •     Type = Restore"
          •     Client = win1
          •     FileSet = "win1"
          •     Storage = "storage2"
          •     Pool = "pool2"
          •     Messages = "standard"
          •     Where = "C:/FHRestore" : notice that for windows clients, we still use the forward slashes navigating a folder structure (though there is of course no beginning slash since windows does not have an equilavent to the linux root).
          • }
      4. Setup the 3 FileSet{..} sections : (one for each client)
        (A FileSet first of all consist of a list of directories & files to backup and a list of directories & files to exclude. However backup options like compression, encryption & signatures are available)
        1. Create a new bacula-dir.conf FileSet{..} section :
          • FileSet {
          •     Name = "web1" : since we are defining what files to backup on web1, I think web1 is a good name for this FileSet.
          •     Include { : the Include subsection (there are also an Exclude subsection which we will not use).
          •         Options { : different options can be specified for the Include subsection.
          •             signature = MD5 : here we specify that the Catalog database should keep a signature of each file, which I think is mostly useful with verify jobs and intrusion detection, however it is default so I keep it.
          •         }
          •         File = /var/www/clients : we want to backup the /var/www/clients folder (which is the parent folder for websites on this server). Here I only specify one File property, however we can have as many File properties as we want (see the win1 FileSet below).
          •     }
          • }
        2. Create a new bacula-dir.conf FileSet{..} section :
          • FileSet {
          •     Name = "db1"
          •     Include {
          •         Options {
          •             signature = MD5
          •         }
          •         File = /var/FHDBDumps : on db1 all databases are backup once a day to this folder - so here we are actually backup the backup.
          •     }
          • }
        3. Create a new bacula-dir.conf FileSet{..} section :
          • FileSet {
          •     Name = "win1"
          •     Include {
          •         Options {
          •             signature = MD5
          •         }
          •         File = "C:/Websites" : I have some asp.net websites here.
          •         File = "C:/Databases/Backup Auto" : I don't use a separate database server for my asp.net websites. All databases runs on the same server and are backup daily to this folder - again here we are actually backup the backup.
          •     }
          • }
    2. Press ctrl+x and then y to exit the nano editor and save the bacula-dir.conf file.
    3. backup server shell> service bacula-director restart : restart the Director to apply the new configurations.

Ok, that concludes the Bacula configuration - we should now have daily backup of our servers, however we better test if the backup jobs can actually run.



Test backup jobs

Here we want to test :

  • That we can manually execute a job (using the run command).
  • That after the job have executed a job report is written and send to the Director.
  • That we receive an email with the same job report.
  1. backup server shell> bconsole : start the Console program (to interact with the Director).
  2. *> run : start the interactive prompt for running a job, you should see all jobs defined (I have a little more than in this tutorial).
  3. Select Job resource (1-16):> 7 : I select 7 which on my system is win1-backup, you should select a number representing a backup job on your own system. You are now presented with the job properties, in my case :
    • JobName: win1-backup
    • Level: Incremental
    • Client: win1
    • FileSet: win1
    • Pool: pool2
    • Storage: storage2
    • When: NOW
    • Priority: 11
  4. OK to run? (yes/mod/no):> yes : you can change the job settings by writing mod, however if this is your first run with this FileSet on this Client, then your backup level will automatically be upgraded from incremental to full.
  5. *> messages : since we were alerted that we had messages, we wrote the messages command to display them. As we can see the Director, FH-dir, have upgraded the backup level to full and ordered the Storage daemon to write to pool2 on disk2 and the Storage daemon, FH-sd, have found a writable volume pool2-0008 in pool2 and is preparing the volume to receive data.
  6. *> status dir : check out the current situation - showing scheduled, running and terminated jobs. Here we can see that the win1-backup job is currently running. We can also see that there are more messages.
  7. *> messages : get the latest messages which is from the win1 file daemon stating that a VSS (Volume Shadow Copy) snapshot is being creating of the files to backup.
  8. Wait a long time ...
  9. *> messages : success, we got the last of the job report and it even terminates OK
  10. *> status dir : and we can see the job in the "Terminated Jobs" table.
  11. success, I got the report sent to my gmail account in which I have setup a Bacula label so I fast can select all emails from Bacula and delete them (mostly you will just want to fast see the termination codes, OK or ERROR).

Congratulations - you are now able to execute backup jobs, being notified with execution reports and most likely your scheduling also works (though that you can only test by waiting until next day or setting the clock appropriate in the schedule).



Restore files

The restore command is very fleksible enabling you to restore from backup latest version or older version of one or many files/folders for different clients restoring the files to different targets. However I find that the 2 most common restore schenarios are :

  • We need to restore the latest version of one or more files - (eg. if the files have been deleted or overwritten by accident).
  • We need to restore an older version of one or more files - (eg. if the files have become virus infected unnoticed for some time).

Note that if you want to cancel an action, you press the dot key '.' - dot is the standard cancel command throughout the Console program.

Restore latest version

Restore type 5 "Select the most recent backup for a client".

  1. backup server shell> bconsole : start the Bacula console program.
  2. *> restore : start the interactive restore process. The Catalog MyCatalog is automatically selected and you are presented with 12 different ways to select from which JobIds you want to restore.
  3. Select item (1 - 13):> 5 : select method 5 "Select the most recent backup for a client" which will prompt you to select what client to restore.
  4. Select the Client (1 - 12):> 9 : here I select the client web1 and the following is now happening :
    1. The only appropriate fileset, web1, is autoselected.
    2. The latest full backup JobId (84) and all subsequent incremental JobIds (87, 98, 107) are selected.
    3. A folder/file tree with the latest version of each file is build - this tree is now ready for navigation and full or partial selection of folders & files.
  5. $> dir : we use dir to see what is in current folder, here we find a subfolder called var.
  6. $> cd var : we use cd to change directory.
  7. Continue browse the file tree until you arrive to some files/folders you want to restore.
  8. $> mark web : we use mark to mark files or folders we want to restore, here we mark the "web" folder which contains 14,052 files. We could now continue to browse the file tree and mark more folders or single files if we would like.
  9. $> done : we use done to tell that we are finished selecting files. The restore dialog will now show what volumes and devices must be online to fetch the files selected, however this have no meaning for disk based storage as volumes are (in theory) always online. Also we are presented with a list of available restore jobs - the restore job we select will set various restore parameters, most importantly where to restore.
  10. Select Restore Job (1-11):> 3 : here we select "web1-restore" which will define several appropriate properties of the restore job, the most important are :
    • Restore Client: web1 : target server on which the restored files will be copied to.
    • Where: /var/FHRestore : path on the target server to where the restored files will be copied.
  11. OK to run? (yes/mod/no):> yes : execute the job with its default parameters as we have defined them in the bacula-dir.conf file (otherwise write mod to modify the parameters). Here the job gets the JobId 116.
  12. *> messages : SUCCESS I expected to see only the preparing setup in the messages, however the job was terminated very fast
  13. The job was also emailed my gmail account even though it was not clear from the above job parameter output that any messages were associated with the job.
  14. web1 server shell> cd /var/FHRestore : the part of the file tree restored (/var/www/clients/client1/web88/web) is copied to /var/FHRestore. All you now need to do is to overwrite the existing files - done.

Restore an older version

Restore type 6 "Select backup for a client before a specified time".

  1. backup server shell> bconsole : start the Bacula console program.
  2. *> restore : start the interactive restore process. The Catalog MyCatalog is automatically selected and you are presented with 12 different ways to select from which JobIds you want to restore.
  3. Select item (1 - 13):> 6 : select method 6 "Select backup for a client before a specified time" which will prompt you to specify a time before which the newest JobIds should be selected.
  4. Enter date as YYYY-MM-DD HH:MM:SS :> 2012-06-07 04:00:00 4 days ago after the nightly backup at 2:05
  5. Select the Client (1 - 12):> 9 : here I select the client web1 and the following is now happening :
    1. The only appropriate fileset, web1, is autoselected.
    2. The latest full backup JobId (84) and all subsequent incremental JobIds (87, 98) up to the specified time are selected. (notice that the incremental backup running 2012-06-08 with JobId 107 is NOT selected if you compare with the JobIds from latest backup)
    3. A folder/file tree with the latest version of each file is build - this tree is now ready for navigation and full or partial selection of folders & files.
  6. $> dir : we use dir to see what is in current folder, here we find a subfolder called var.
  7. $> cd var : we use cd to change directory.
  8. Continue browse the file tree until you arrive to some files/folders you want to restore.
  9. $> mark web : we use mark to mark files or folders we want to restore, here we mark the "web" folder which contains 14,052 files. We could now continue to browse the file tree and mark more folders or single files if we would like.
  10. $> done : we use done to tell that we are finished selecting files. The restore dialog will now show what volumes and devices must be online to fetch the files selected, however this have no meaning for disk based storage as volumes are (in theory) always online. Also we are presented with a list of available restore jobs - the restore job we select will set various restore parameters, most importantly where to restore.
  11. Select Restore Job (1-11):> 3 : here we select "web1-restore" which will define several appropriate properties of the restore job, the most important are :
    • Restore Client: web1 : target server on which the restored files will be copied to.
    • Where: /var/FHRestore : path on the target server to where the restored files will be copied.
  12. OK to run? (yes/mod/no):> yes : execute the job with its default parameters as we have defined them in the bacula-dir.conf file (otherwise write mod to modify the parameters). Here the job gets the JobId 116.
  13. *> messages : SUCCESS I expected to see only the preparing setup in the messages, however the job was terminated very fast
  14. The job was also emailed my gmail account even though it was not clear from the above job parameter output that any messages were associated with the job.
  15. web1 server shell> cd /var/FHRestore : the part of the file tree restored (/var/www/clients/client1/web88/web) is copied to /var/FHRestore. All you now need to do is to overwrite the existing files - done.

Restore a database

Restoring a database is straightforward :

  1. Restore the appropriate database backup file using either of the approaches above.
  2. Use the restored database file to restore your database the way you have always restored your database.


Bacula Console Commands & Management

Here are the commands I use the most : (full command list)

  • Daemon management :
    • backup server shell> service bacula-director restart : instead of restart you can use status, stop or start.
    • backup server shell> service bacula-sd restart : : instead of restart you can use status, stop or start.
    • client server shell> service bacula-fd restart : : instead of restart you can use status, stop or start.
  • backup server shell> bconsole : start the console
    • *> help : display the help menu.
    • *> version : display current Bacula version.
    • *> run : prompt for job to run - very important command.
    • *> restore : start the restore interactive prompt - very important command.
    • status-command :
      • *> status dir : displays todays terminated jobs, scheduled jobs and current jobs.
      • *> status storage :
      • *> status client : prompts to select a client to connect to and then displays list of jobs for that client.
      • *> status clients : connects to Storage daemon.
      • *> status jobs :
    • list-command : (used to show the content of the Catalog)
      • *> list jobs : list execution status of all jobs for some time back (eg. time run, how many files, how many bytes, what type of job and exit code)
      • *> list job=JOBNAME list execution status of a specific job for some time back
      • *> list volumes :
      • *> list pools
      • *> list clients
      • *> list volumes
      • *> list volumes jobid=JOBID
      • *> list volumes jobname=JOBNAME
      • *> list volumes pool=POOLNAME
      • *> list files jobid=JOBID : list all files backed up for a specific job
    • show-command : (used to show the content of the Directors resource records as defined in bacula-dir.conf)
      • *> show devices
      • *> show storages
      • *> show pools
      • *> show clients
      • *> show jobs
      • *> show filesets
      • *> show schedules
    • Management commands :
      • *> purge : prompts for what to purge, most common a volume for which all records should be pruned and VolStatus changed to Purged. Cannot be undone. After purging a volume, the jobs & files will be deleted from the catalog database and the volume record itself will change VolStatus to Purged. Data is still in the volume, however Bacula may now recycle the volume (whiping out the data) then a new volume is needed.
      • *> cancel :
      • *> update : prompts for what to update, eg. update pool will among other update volume retention periods for new volumes (existing volumes will keep their retention periods, bacula.Media.VolRetention, however retention period for individual volumes can be updated using update volume).
      • *> delete : prompts for what to delete (volume, pool or jobid). If deleting eg. a volume, the record will disappear in the catalog database, however the volume file itself will not be deleted. If you have some test pools, this is also a consistent way to delete the pool records from the database.
  • Delete Clients :
    1. backup server shell> mysql -u USERNAME -pPASSWORD : log on to the local mysql server.
      1. mysql> use bacula; : change database to bacula (the Catalog database).
      2. mysql> delete from Client where Name = 'CLIENTNAME'; : delete the relevant record from the Client table.
      3. mysql> quit : exit to system prompt.
    2. backup server shell> dbcheck -f -b -c /etc/bacula/bacula-dir.conf : remove any orphane records from the Catalog database.
  • Delete FileSets :
    1. backup server shell> mysql -u USERNAME -pPASSWORD : log on to the local mysql server.
      1. mysql> use bacula; : change database to bacula (the Catalog database).
      2. mysql> delete from FileSet where FileSet = 'FILESET'; : delete the relevant record from the FileSet table.
      3. mysql> quit : exit to system prompt.
    2. backup server shell> dbcheck -f -b -c /etc/bacula/bacula-dir.conf : remove any orphane records from the Catalog database.
  • Debugging bacula : (eg. why a daemon will not start)
    • backup server shell> tail -15 /var/log/bacula/log : shows the last 15 lines from the bacula log file.
    • backup server shell> service bacula-sd start -d100 : will start the Storage daemon with debug level 100 (default is zero).
  • Testing a new configuration file :
    • backup server shell> /etc/bacula/bacula-dir -t -c bacula-dir.conf : shows if the bacula director configuration file is syntactically correct (the same can be done for SD & FD).


Bacula Concepts

In my experience the most difficult concepts to grasp are : pruning, purging and rentention periods. Let me fast explain these concepts here :

  • Purge >< Prune >< Delete : A purged Volume is a Volume that contains only expired Jobs & files and thus is not in the Catalog. Then pruning a Volume, it means to delete expired jobs & files in the Catalog associated with that Volume. Since not all files & jobs within a Volume may be expired, pruning a Volume does not guarantee the Volume will be purged. You can though purge a Volume which will delete all files and jobs from the Catalog associated with that Volume whatever the timestamps on these records - that will guarantee a purged Volume.
  • Retention Periods : regards how long time Bacula keeps records in the Catalog database. Deleting Catalog file, job & volume records from the Catalog database is called pruning. Then eg. a file record is deleted, it is not any more possible to browse the file using the standard restore command.
    • File Retention : the period that Bacula will keep file records in the Catalog database. If AutoPrune is set to yes, then Bacula will delete file records from the Catalog database after the file retention period (actually these records may be deleted earlier if the Job record to which they belong have been deleted).
    • Job Retention : the period that Bacula will keep job records in the Catalog database. If AutoPrune is set to yes, then Bacula will delete job records from the Catalog database after the job retention period (actually a job record may be deleted earlier if the volume on which the job have been stored have been pruned).
    • Volume Retention : the period that Bacula will keep the volume records in the Catalog database after end time of the last job written to the volume. If AutoPrune is set to yes, then Bacula will delete volume records from the Catalog database after the volume retention period and also delete all jobs and file records associated with the volume. The physical volume can now be recycled (but will first be recycled if no other volume is free to write).
  • Volume Status : (*> list volumes will show all your volumes and their status)
    • Append : volume is appendable so backup data can be written to this volume.
    • Full : volume is full so no more backup data can be written to this volume.
    • Purged : volume contains only expired jobs & files (no content in the catalog points to this volume) and so this volume can be recycled for new backup data.
    • Error : some error have occured, the volume cannot be used to store backup data.
    • Recycle : I am not sure, I once had this status which prohibited backup. I tried to run a backup several times without luck, but then after maybe an hour a backup changed the status to Append and everything worked again.
    • Used : volume cannot be used to store additional backup data and even if jobs & files stored is all expired, the volume will not change status (to Append).
    • Archive :
    • Busy :
    • Cleaning :
    • Disabled :
    • Read-Only :


Appendix : Bacula utility programs

Sometimes disaster strucks and you may not be able to access the director using bconsole or the Catalog have been destroyed etc. In such cases there are different utilities you can use to list and retrieve data from the raw volumes (full list of Bacula utility programs).

  • Disaster utilities :
    • bscan : Can scan a volume and rebuilding the catalog database with the files from the volume. Standard restore of the files within the volume will then be possible. This is usefull in at least 2 scenarios :
      • The Catalog database have become damaged.
      • Information about the volume have been pruned from the Catalog database.
    • bextract : can extract files or file lists from volumes in case restore cannot be used.
    • bls : can do an ls-like listing of a bacula volume.
  • Other utilities :
    • bacula-web : a read-only web based interface to Director & Catalog showing real-time progress of jobs (Bacula-web homepage).
    • bweb : a web based interface to Director as a GUI alternative to the CLI Console.

Read more about Bacula disaster recovery here



Appendix : Bacula folders

  • /etc/bacula : bacula main folder on both backup server and linux backup clients. On backup server containing bacula-dir.conf, bacula-sd.conf, bconsole.conf etc. On backup clients containing bacula-fd.conf.
  • /var/log/bacula : bacula log file is here.
  • /var/lib/bacula : bacula keeps email to be sent here.
  • /usr/sbin : seems to keep several daemons here.
  • /usr/lib/bacula : the bacula smtp daemon is here.
  • C:\Program Files\Bacula : bacula main folder on windows backup clients containing most notably the bacula-fd.conf file.


Appendix : Bacula alternatives

This comparision is a work in progress, email me if you want to help develop it.

Name Free DB Integration Email Integration OS'es Comment
Bacula yes no no Unix, Linux, Windows, Mac
Amanda yes Amanda is free, but comes in a commercial enterprise version sold by Zmanda.
Commvault no Likely among the best enterprise backup software
Quest Netvault no
Arkeia
Crashplan no
Symantic Netbackup no
CA ARCserve Backup no
Tivoli Storage Manager no Is likely very expensive
EMC Networker no
HP Data Protector no
PerfectBackup+ no
FreeFileSync yes Linux, Windows Seems to be intended for personal use as it seems to be controlled from each computer instead of a central management.
Robocopy yes
SyncToy yes Windows Part of Microsoft Power Toys.
Retrospect no

A more detailed comparison of Bacula and other backup solutions here and here.

Another award based comparison by techtarget

Lastly I found this exceptional comparison half down the page by a guy called Mark : what-is-the-best-enterprise-backup-software.



Appendix : Common errors and solutions

  1. shell> bconsole displays Connecting to Director xxx.com:9101 and then returns to system prompt.
  2. shell> tail /var/log/bacula/log shows error Warning: Cannot bind port 9101: ERR=Cannot assign requested address: Retrying ...

Reason : Different problems may lead to error 1, however if you also have error 2, it is most likely because the directory address specified in /etc/bacula/bconsole.conf does not correctly resolve.

Example : In bconsole.conf I had the address of the bacula director specified by a domain name, favouritehosting.com, but then I changed the IP of favouritehosting.com and so bconsole would try to find the director on that new IP (which was a different server).
Say your bconsole.conf file look like this :

  • Director {
  •     Name = localhost-dir
  •     DIRport = 9101
  •     address = favouritehosting.com : favouritehosting.com does not any longer resolve to the IP of the server that bacula director is installed on.
  •     Password = "secret"
  • }

Solution : In /etc/bacula/bconsole.conf change the address field to the correct IP, eg like this :

  • Director {
  •     Name = localhost-dir
  •     DIRport = 9101
  •     address = 27.254.33.96 : ok, now I am sure to get the correct IP.
  •     Password = "secret"
  • }
  1. Bacula director service seems to crash less than a minute after starting it

Reason : There can be a lot of different reasons for this, so the most important is to try do debug the problem.

Debug example :

  • shell> tail -30 /var/log/bacula/log : check for any entry in the bacula log file
  • shell> /usr/sbin/bacula-dir -d 9 -c /etc/bacula/bacula-dir.conf start bacula-dir with debug (-d) maximum (9) using config file (-c) /etc/bacula/bacula-dir.conf.
  1. Director server cannot connect to client server JobId 0: Fatal error: bsock.c:134 Unable to connect to Client: web1-fd on 27.254.33.57:9102. ERR=Connection refused.

Reason 1 : There is a firewall on the client server blocking port 9102 - go to Solution 1.

Reason 2 : no software is listening on port 9102 :

  • Reason 2a : bacula file daemon is not running - go to Solution 2a.
  • Reason 2b : bacula-fd.conf on the client server specifies localhost or 127.0.0.1 for FDAddress in the FileDaemon{..} section - go to Solution 2b. Note that this problem cannot happen on a windows server, but is a very common problem on a linux server.

Solution 1 : unblock port 9102 on the client server

  • Unblock port 9102 on a windows server
    1. Click on "Start | Administrative Tools | Windows Firewall with Advanced Security"
    2. Select "Inbound Rules" and then click on "New Rule..." to create a new rule for Bacula.
    3. Select "Port" for Rule Type and then click the "Next" button.
    4. Set the port to 9102 and then click the "Next" button.
    5. Select the default "Allow the connection" and then click the "Next" button.
    6. Select the default which is all Profiles and then click the "Next" button.
    7. Name the new rule Bacula and then click the "Finish" button.
    8. Ok, the new rule called Bacula should appear in the list of inbound rules.
  • Unblock port 9102 on a Ubuntu server : (UFW : Uncomplicated FireWall : default iptables frontend in Ubuntu)
    1. shell> ufw status verbose : display among other whether the UFW firewall is enabled and if it is also display all the the firewall rules.
    2. shell> ufw allow 9102 : unblock port 9102 from all IP sources (this only have meaning if ufw is enabled).

Solution 2a : start the bacula file daemon : (I assume that bacula file daemon is actually installed on the client server).

  • Start the bacula file daemon on a windows server
    1. On the win1 server press the Windows Key + R to open the run box, write services.msc and press the "Ok" button to launch the Services Manager.
    2. In the Services Manager right click on "Bacula File Service" and choose "Restart" from the context menu.
  • Start the bacula file daemon on a Ubuntu server
    1. client server shell> netstat -anp | grep 9102 : show what if anything is listening on port 9102 (it should show bacula-fd).
    2. client server shell> service bacula-fd status : show whether bacula file daemon is running.
    3. client server shell> service bacula-fd restart : this will start or restart the bacula file daemon.

Solution 2b : edit bacula-fd.conf FileDaemon{..} section on the client server to specify global IP address:

  • On Windows : the problem cannot happen on a windows client.
  • On Linux :
    1. client server shell> nmap localhost : this may show that something is listening on port 9102.
    2. client server shell> nnap 27.254.33.57 : this should show that nothing is listening on port 9102.
    3. client server shell> nano /etc/bacula/bacula-fd.conf : open bacula-fd.conf in the nano editor.
    4. Edit the FileDaemon section :
      • FileDaemon {
      •     Name = web1-fd
      •     FDport = 9102
      •     WorkingDirectory = /var/lib/bacula
      •     Pid Directory = /var/run/bacula
      •     Maximum Concurrent Jobs = 20
      •     FDAddress = 27.254.33.57 : change FDAddress to the global IP address (not localhost or 127.0.0.1).
      • }
    5. Press ctrl+x and then y to close nano and save the changes to bacula-fd.conf.
    6. client server shell> service bacula-fd restart : restart the file daemon to apply the configuration changes.
    7. client server shell> telnet 27.254.33.57 9102 : this time you should get through (remember to use your own IP address)
    8. client server shell> telnet localhost 9102 : this should be refused.
  1. 09-May 14:51 FH-Backup-dir JobId 0: Fatal error: Unable to authenticate with File daemon at "27.254.33.57:9102". Possible causes:
    Passwords or names not the same or
    Maximum Concurrent Jobs exceeded on the FD or
    FD networking messed up (restart daemon).

Reason : Most likely your bacula-fd.conf Director{..} section (Password & Name) does not exactly match the values specified in bacula-dir.conf.

Solution : the following must be true : (values are case sensitive, also the Name value)

  • bacula-dir.conf Director{..} Name == bacula-fd.conf Director{..} Name
  • bacula-dir.conf Client{..} Password == bacula-fd.conf Director{..} Password
  1. Cannot find any appendable volume.
  2. *> status jobs displays Device xxx is not open and Device is blocked waiting to create a volume for:.

Reason 1 : If you get above error 6 or 7, the reason is most likely that there are no more space available on the backup device. Current volume (file) is full, there is no space to create new volumes and due to volume retention periods no existing volume can be purged to allow for new data to be written.

Reason 2 : especially the above error 7 can have other reasons than disk space shortage, eg. problems with the storage daemon.

Solution 1 : Long term solutions to this problem is to either increase the size of your device (bigger harddrive) or to reduce the volume retention period (keep backup data for a shorter period before discarding them). However the here and now solution that will allow you to continue your backup is to manually purge your oldest volumes :

  1. shell> bconsole : start the Console if not already started.
  2. *> purge : start the interactive purge program.
  3. Select Volumes
  4. You will now be allowed to select a volume to purge. Select your oldest volume and continue purging volumes until you have enough space. Purging a volume means to remove information of the content of the volume from the Catalog, not actually deleting the content within the volume. Also then purging a Volume, the VolStatus of the volume in the Catalog changes to Purged. If the volume you purge belong to a Pool that have Recycle=Yes, then Bacula will automatically continue the backup not long after a volume have been purged.

Solution 2 : restart the bacula daemons :

  1. shell> service bacula-sd restart
  2. shell> service bacula-director restart
  1. JOB-NAME is waiting for Client CLIENT-NAME to connect to Storage STORAGE-NAME

Reason 1 : Your firewall on the storage server is not allowing incoming traffic on port 9103 from your file daemon server (remember that the file daemon will have to initiate contact to the storage daemon).

Reason 2 : The file daemon does not know the IP address of the storage daemon - this IP address I think is given to the file daemon by the director from bacula-dir.conf Storage {} address, which under installation could be set to localhost (the Director does NOT translate localhost to actual IP then giving the value to the file daemon).

Solution 1 : Be sure your firewall allow tcp traffic on port 9103 on the storage daemon server : here using UFW

  • storage daemon server shell> ufw status verbose : first see if the UFW firewall is enabled and if it is then list the rules.
  • storage daemon server shell> ufw allow proto tcp to any port 9101:9103 : here I allow all the bacula ports to any IP on on the storage daemon server (I don't specify from what IP, so that is also any IP).

Solution 2 : Be sure to use global IP's and not localhost for daemon addresses :

  1. Write your bacula configuration files so the following is true :
    • bconsole.conf Director {} address = GlobalIP of director server
    • bacula-dir.conf Director {} DirAddress = GlobalIP of director server
    • bacula-dir.conf Storage {} Address = GlobalIP of storage daemon server
    • bacula-sd.conf Storage {} SDAddress = GlobalIP of storage daemon server
  2. Restart your daemons to apply the configuration changes : (normally director machine and storage daemon machine is the same machine)
    1. director machine shell> service bacula-director restart
    2. storage daemon machine shell> service bacula-sd restart

Comments

You can comment without logging in
 
 B  U  I  S 
Words: Chars: Chars left: 
 Captcha 
 Nickname
Facebook
    
rex_ray
User type : Standard
Register : 2012-Dec-30
Topics : 2
Replies : 4
--------------
      report  reply  
Thank you very much for this exhaustive guide on Bacula. As a result of your guide, I was able to understand and install this wonderful software.

Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  

Hi rex_ray, you are welcome.

If you have any bacula related problems, post the problem here and I will help if I can.

web fiddler by nature

rex_ray
User type : Standard
Register : 2012-Dec-30
Topics : 2
Replies : 4
--------------
      report  reply  
Thanks for your offer. Am just taking my baby steps with Bacula as I have no previous experience working with a Backup Software. By the way, Wish you a Very Happy & Prosperous New Year.

Jimmy
--------------
      report  reply  
Hi,

Thanks for the awesome guide.

I am backing up across a three drives. When the first drive gets full, how do i get get the job to continue on a second drive?

Thanks
Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Hi Jimmy,
To my knowledge it is unfortunately NOT possible for Bacula to automatically continue writing on a second harddrive then the first harddrive is full.

I think there are 2 solutions to involve more than 1 physical harddrive as backup device for Bacula :
  1. Merge multiple physical harddrives into 1 virtual harddrive using eg. raid, OR
  2. In bacula-sd.conf define a Device section for each harddrive.
I think you can say that method 1 is the most space efficient method and if it was easy I think I would have used method 1, however in the above guide I have used method 2, please see the Bacula Configure Storage section.

For more info about backup to multiple disks, here are some explanation from the Bacula manual.

web fiddler by nature

Jimmy
--------------
      report  reply  
Hi,

Thanks for your response.

I have tested using 2 disks listed in the device section. 

The disks are 1TB each and the data being backed up is 1.8TB total.

I started a full backup job however once the 1st disk gets full, the job stops, waiting for more space.

Do you know how i can manually continue the backup to disk2(device2)?

Thanks
Jimmy
Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Hi Jimmy
I am sorry, I don't know if it is possible to continue manually on a second drive.

However, if your 1.8 TB data to backup is from multiple clients, then you could define your .conf files so that some of your clients backup on the first device and the other clients backup on the second device.

If your 1.8 TB data to backup is from a single client, then it is less nice, but you could define 2 FileSets, the first FileSet include half of your client data and the second FileSet include the other half of your client data. You would then need 2 Job definitions, one for each FileSet, and then let each Job backup to each their device - I am unsure how that will look in a restore scenario, but my guess is that it will be transparent.

web fiddler by nature

BvsDark
User type : Standard
Register : 2013-Jan-31
Topics : 0
Replies : 1
--------------
      report  reply  
Hey Im thankful, this is a great job, im having a big problem, when I perform a Back up all is good, but when im trying to restore it, and I browse the file, nothing appears to be restores, back up says that 11.4 mb wash backed but none of the files appear in from linux client or Windows, I dont have any issues related to conections or anything, everything just perform great but this! thank you ill be really glad if you can help me!
BvsDark - Learning is what matters!
Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  

Hi BvsDark

I understand your post as :
1) you actually have a backup
2) you have run a restore that terminated with success
3) you cannot find the files that you have restored

Ad 3) Then you restore, you need to select a client, that is the machine to which you want to restore the files. You also have to select Where on that client to restore. Say that you took a backup of "/var/www" and you now restore the files setting the Where value to "/myRestores" - your restored files will now be in "/myRestores/var/www" on the selected client. If you want to restore directly to the same place as from where you backup the files, you will need to set the Where value to "/".

Here is a picture from a restore job ready to run :


Also if you have not done already, be sure to go through the #bacula-restore section.

web fiddler by nature

rex_ray
User type : Standard
Register : 2012-Dec-30
Topics : 2
Replies : 4
--------------
      report  reply  
Hi,
My Backup Setup is as follows,

OS: CentOS 6.3 64 bit
Bacula Version: 5.2.12 (Compiled from Source with MySQL support)
Backup Media: Hard Drive

Am trying to backup a couple of folders on a Windows 2008 (64 Bit) Server. I have installed the latest 64 bit version of the file client
on the Windows server. Manually running a backup works perfectly. The problem arises when running the backup through the scheduler.
I get the dreaded "Bacula cannot find any appendable volumes" error. There is more than enough space in the harddrive so space shouldn't be a
problem. I new to Bacula can you please point out what am doing wrong here,

My Bacula Configuration,

Only the Relevant sections of 'bacula-dir.conf',


# Backing up single client machine only

Client {
  Name = fin-hp-fd
  Address = 10.10.0.91
  FDPort = 9102
  Catalog = MasterCatalog
  Password = "password"          # password for FileDaemon
  File Retention = 30 days            # 30 days
  Job Retention = 6 months            # six months
  AutoPrune = yes                     # Prune expired Jobs/Files
}

# File based Back up to hard drive

Storage {
  Name = storage1
  Address = 10.10.0.61                # N.B. Use a fully qualified name here
  SDPort = 9103
  Password = "password"
  Device = hd1
  Media Type = File
}

# My single Job defs config

JobDefs {
  Name = "InternalStorage1"
  Type = Backup
  Client = fin-hp-fd
  FileSet = "Full Set"
  Storage = storage1
  Schedule = "BackupCycle"
  Messages = Standard
  Write Bootstrap = "/var/bacula/working/%c.bsr"
  Priority = 10
}

# Backing up only a couple of folders
FileSet {
  Name = "Full Set"
Include {
Options {
      signature = SHA1
    }
    File = E:/test
    File = E:/test2
    }
}

Job {
Name = "FinBackup"
JobDefs = "InternalStorage1"
Client = fin-hp-fd
FileSet = "Full Set"
Pool = "Full-Pool"
}

Schedule {
  Name = "BackupCycle"
  Run = Level=Full Pool=Full-Pool fri at 13:30  
  Run = Level=Differential Pool=Diff-Pool mon-thu at 13:30
  Run = Level=Incremental Pool=Inc-Pool hourly at 0:05
  Run = Level=Incremental Pool=Inc-Pool hourly at 0:45
}

# Recycle Full Volumes after 3 months

Pool {
  Name = Full-Pool
  Pool Type = Backup
  Recycle = yes           # automatically recycle Volumes
  AutoPrune = yes         # Prune expired volumes
  Volume Retention = 3 months
  Maximum Volume Files = 0
  Maximum Volume Jobs = 1
  Volume Use Duration = 0
  Label Format = "Weekly-Full-${Year}-${Month:p/2/0/r}-${Day:p/2/0/r}-${Hour:p/2/0/r}-${Minute:p/2/0/r}"
  Maximum Volumes = 12
}

# Recycle Differential Volumes after 14 days

Pool {
  Name = Diff-Pool
  Pool Type = Backup
  Recycle = yes
  AutoPrune = yes
  Volume Retention = 14 days
  Maximum Volume Files = 0
  Maximum Volume Jobs = 1
  Volume Use Duration = 0
  Label Format = "Daily-Diff-${Year}-${Month:p/2/0/r}-${Day:p/2/0/r}-${Hour:p/2/0/r}-${Minute:p/2/0/r}"
  Maximum Volumes = 7
}

# Recycle Incremental Volumes after 1 month

Pool {
  Name = Inc-Pool
  Pool Type = Backup
  Recycle = yes           # automatically recycle Volumes
  AutoPrune = yes         # Prune expired volumes
  Volume Retention = 1 month
  Maximum Volume Files = 0
  Maximum Volume Jobs = 1
  Volume Use Duration = 0
  Label Format = "Daily-Inc-${Year}-${Month:p/2/0/r}-${Day:p/2/0/r}-${Hour:p/2/0/r}-${Minute:p/2/0/r}"
  Maximum Volumes = 20
}

My Bacula Storage Daemon Configuration, (only the relevant part)

Device {
  Name = hd1
  Media Type = File
  Archive Device = /media/Linux
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
}
   

Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  

Hi rex_ray

I can see you set maximum Volumes in your Pool definitions, so if your scheduler tries to start eg. a full backup and there already are 12 Volumes in the full Pool and none of them are expired, then Bacula cannot continue and you will get the "Bacula cannot find any appendable volumes" error.

If the maximum Volumes setting is not the problem, then please :

  1. Specify if it is a full, incremental or differential job that fails through the scheduler or if it is all of them.
  2. If you have a report on the failed job, please post that report
  3. Post the exact job properties then you run manually (so we can compare with the failed job report)

web fiddler by nature

Anonymous
--------------
      report  reply  
Hi,
        Thank you for your help. This problem arises due to the 'Volume Retention Period = 1 month' specified in the 'Inc-Pool'. With the incremental backup running every 15 mins and the retention period set too long, the volumes are not getting recycled and thus am getting this error. Secondly am dumping all the backups (full,diff,inc) into a single folder (/media/Linux) which also adds to this problem. Now I have addedd different storages to full, differential and incremental backups and tweaked the 'Volume Retention Period' appropriate to the setup. But still I have many doubts with Bacula and hope to ask your help in clearing them in the
coming week. Once again thank you very much for your help.
Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Hi rex_ray
You are right, I realize now that in your Inc-Pool, you have :
  • Maximum volume jobs = 1
  • Maximum volumes = 20
  • Volume retention = 1 month
The above configuration means that if you run more than 20 jobs in 1 month, then you will run out of appendable volumes.

Some solutions :
  1. Reduce the retention period (not recommended, I think 1 month is good)
  2. Increase maximum volumes (this may be the best solution, I want to try it out next time I upgrade our backup system)
  3. Set maximum volume jobs to zero and set a maximum volume bytes value (this is the solution I currently use) :
    • Maximum volume jobs = 0
    • Maximum volume bytes = 25G #This size works for me
    • Maximum volumes = 20 #Here you need to adjust for how much data you have
    • Volume retention = 1 month
Solution 3 will give you the ability to run as many jobs as needed as long as the data does not exceed 500 GB (25G x 20). 

web fiddler by nature

rex_ray
User type : Standard
Register : 2012-Dec-30
Topics : 2
Replies : 4
--------------
      report  reply  
Hi,
       Sorry for being late. I removed the 'Maximum Volumes' option from the 'Inc-Pool'  Pool but kept the Volume Retention at 7 days (i.e recycle the volumes after 7 days) The total size of the backup data is under 3.5 GB so I thought there is no need for the 'Maximum Volume Bytes' option. Also I want a new volume for each backup job so I used 'Maximum Volume Jobs = 1'.  I read from a website that you have to keep Job, File, Volume Retention periods the same like shown below for reliable file restoration. Can you please tell me your opinion on this.

Pool {

Job Retention = 1 month
File Retention = 1 month
Volume Retention = 1 month

}

Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  

Hi rex_ray

The documentation is not clear on how retention periods exactly works, especially it is unclear to what extent the volume retention period guarantees a volume is not recycled. The following is what I believe to hold true :

  • File retention period sets then file records in the Catalog database will expire.
  • Job retention period sets the minimum time for job records in the Catalog database will expire. A job record in the Catalog database cannot expire if there are file records referencing it.
  • Volume retention period sets the minimum time before physical volumes will expire (however it may recycle before that, see below). A physical volume cannot expire if there are job records in the Catalog database referencing it.

Bacula can only write backup to volumes with status Append. If Bacula needs to write backup and no existing volumes are appendable, the following happens :

  1. If auto labeling is true, Max volumes are higher than existing number of volumes and there are sufficient space on the harddrive, Bacula will create a new volume and set the status to Append.
  2. If no new volume could be created, there is at least one volume with status Purged and Recyle is set to Yes, then Bacula will recycle one of the purged volumes WHATEVER the volume retention (that is a fact) and set the status to Append.
  3. If no volume is purged and AutoPrune=Yes then Bacula will one last time prune all records in the Catalog database to try to set a volume to status Purged, subsequently recycle it and set the status to Append.
  4. If no appendable volumes could be found or created, Bacula will throw the "Bacula cannot find any appendable volumes" error.

To answer the question whether Volume, Job & File retention periods should be the same, I think it is best as it is more easy to understand, however in large scenarios it gives meaning to lower the File retention to avoid the Catalog database growing intolerably big (you can still restore jobs but not individual files).

 

web fiddler by nature

Anonymous
--------------
      report  reply  
hello,

thank you very much for the awesome guide on bacula. anyway i have question, before completing install "apt-get install"  package. i have error for bacula-directory-mysql, like "error 2002 (hy000): can't connect to local mySQL server through socket '/var/run/mysqld/mysqld.sock' (2) "
it's that normal ??

i'm new for installing bacula. i'm trying install in ubuntu 10.04 on VMware now
 
 thank you
Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Hi Anonymous (remember to logon before posting so you can get an email notification as soon as anyone replies to your post)

Default Bacula will use MySQL server to host the Catalog database. If MySQL is not already installed on the system, then Bacula installation script should install MySQL server as part of the Bacula installation process - I have not tried this option myself and this information, https://bugs.launchpad.net/ubuntu/+source/bacula/+bug/712291, suggest it may be error prone.

So you should answer this question : Did you install MySQL server BEFORE installing Bacula ? or did you rely on the Bacula installation script to install MySQL for you ?

I will recommend to :
  1. Upgrade Ubuntu from your old 10.04 to a newer version if possible (be sure to have backup of ALL your user files & data before upgrading so that in case of disaster, you can restore the server), see how to upgrade eg. here : http://www.liberiangeek.net/2012/04/upgrade-to-ubuntu-12-04-from-ubuntu-10-04-via-the-terminal/
  2. Install MySQL server :
    • shell> apt-get install mysql-client mysql server (you will be prompted to create a MySQL administrative (root) password)
  3. Check that MySQL is running : my guess is that a non-running MySQL server is the reason for your error.
    • shell> service mysql status
  4. Check that you can login to Mysql :
    • shell> mysql -u root -p
  5. Install Bacula :
    • shell> apt-get install bacula
    • The Bacula installation script will now bypass installing MySQL server, however you will be prompted :
      1. Configure database for bacula-directory-mysql with dbconfig-common <- answer Yes
      2. The password of the mysql administrative (root) account <- type in the root password
      3. To create a password for bacula-directory-mysql (the bacula user) <- type in a password
Note that installing on VMware should not affect your installation, but I am not sure if a virtual machine is a good choice for a backup server.

web fiddler by nature

lennie
User type : Standard
Register : 2013-Mar-18
Topics : 0
Replies : 1
--------------
      report  reply  
Thanks for the guide, when i first looked at it, i thought "there's no way i'll get my head around that!!", but you laid it out quite simply.

I am, however encountering a problem. When I run the command "status storage" or "status client" (in bconsole), i get the following error:

"status storage
Automatically selected Storage: storage1
Connecting to Storage daemon storage1 at 127.0.0.1:9103

Failed to connect to Storage daemon storage1."


and

status client=
Automatically selected Client: xxxx-fd
Connecting to Client xxxx-fd at 127.0.0.1:9102
Failed to connect to Client xxxx-fd.

====

As you can see, I've been using 127.0.0.1 as my DirAddress (in all the config files) because bconsole refused to connect using my global ip, so this may be the problem, but i've replaced 127.0.0.1 with both my local ip address and my global ip, but to no avail.

Please help




Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  

Hi lennie

Let's try to solve the client connection. Please post the following 2 things here :

  1. the full bacula-fd.conf file content (for your xxxx-fd client)
  2. the bacula-dir.conf Client {} section (for your xxxx-fd client)

Also be sure to disable any firewall on the client machine and to restart both the director on the director machine and the file deamon on the client machine :
director machine shell> service bacula-director restart
client machine shell> service bacula-fd restart

web fiddler by nature

Anonymous
--------------
      report  reply  
Hi Rasmus,

Here's my bacula-fd.conf file
#
# Default  Bacula File Daemon Configuration file
#
#  For Bacula release 5.2.5 (26 January 2012) -- ubuntu 12.04
#
# There is not much to change here except perhaps the
# File daemon Name to
#

#
# List Directors who are permitted to contact this File daemon
#
Director {
  Name = presbyserver-dir
  Password = "xxxx"
}

#
# Restricted Director, used by tray-monitor to get the
#   status of the file daemon
#
Director {
  Name = presbyserver-mon
  Password = "xxxx"
  Monitor = yes
}

#
# "Global" File daemon configuration specifications
#
FileDaemon {                          # this is me
  Name = presbyserver-fd
  FDport = 9102                  # where we listen for the director
  WorkingDirectory = /var/lib/bacula
  Pid Directory = /var/run/bacula
  Maximum Concurrent Jobs = 20
  FDAddress =127.0.0.1
}

# Send all messages except skipped files back to Director
Messages {
  Name = Standard
  director = presbyserver-dir = all, !skipped, !restored
}
and the bacula-dir.conf content:
Client {
 Name  = presbyserver-fd
 Address = 127.0.0.1
 FDPort = 9102
 Catalog = MyCatalog
 Password = "xxxx"
 File Retention = 30 days
 Job Retention = 6 months
 AutoPrune = yes
}

Thanks for assisting


Anonymous
--------------
      report  reply  
update:

The command "status client" is now producing the following:

*status client=
Automatically selected Client: presbyserver-fd
Connecting to Client presbyserver-fd at 127.0.0.1:9102

presbyserver-fd Version: 5.2.5 (26 January 2012)  i686-pc-linux-gnu ubuntu 12.04
Daemon started 19-Mar-13 09:45. Jobs: run=0 running=0.
 Heap: heap=135,168 smbytes=13,265 max_bytes=13,352 bufs=48 max_bufs=49
 Sizeof: boffset_t=8 size_t=4 debug=0 trace=0
Running Jobs:
Director connected at: 19-Mar-13 09:45
No Jobs running.
====

Terminated Jobs:
====

Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Hi lennie

Congratulations, your director now connects to your client (the last line "Terminated Jobs:" will start to fill up then you have run some backup/restore jobs). Also note that your client "presbyserver-fd" get automatically selected because you have only one client defined (if you had more than one client section in your bacula-dir.conf file, then you would get a list of clients to select from)

If you have still not got your director to connect to your storage daemon, then post the following 4 things here :
  1. bacula-dir.conf Director {} section
  2. bacula-dir.conf Storage {} section
  3. bacula-sd.conf Director {} section (you likely have 2, I am interested in the section that does NOT have the monitor property)
  4. bacula.sd.conf Device {} section (if you have more than 1, then post them all)
Also be sure to restart your storage daemon :
storage machine shell> service bacula-sd restart

web fiddler by nature

Anonymous
--------------
      report  reply  
Hi Rasmus,

Thanks a lot, it seems like i'm now able to connect to the storage daemon. Here's my output when i run "status storage":
status storage
Automatically selected Storage: storage1
Connecting to Storage daemon storage1 at 127.0.0.1:9103

presbyserver-sd Version: 5.2.5 (26 January 2012) i686-pc-linux-gnu ubuntu 12.04
Daemon started 19-Mar-13 14:06. Jobs: run=0, running=0.
 Heap: heap=135,168 smbytes=16,068 max_bytes=16,186 bufs=61 max_bufs=63
 Sizes: boffset_t=8 size_t=4 int32_t=4 int64_t=8 mode=0,0

Running Jobs:
No Jobs running.
====

Jobs waiting to reserve a drive:
====

Terminated Jobs:
 JobId  Level    Files      Bytes   Status   Finished        Name
===================================================================
    73  Full          0         0   Error    13-Mar-13 23:05 BackupClient1
    74  Full          0         0   Error    13-Mar-13 23:10 BackupCatalog
    75  Full          0         0   Error    14-Mar-13 23:05 BackupClient1
    76  Full          0         0   Error    14-Mar-13 23:10 BackupCatalog
    77  Full          0         0   Error    15-Mar-13 23:05 BackupClient1
    78  Full          0         0   Error    15-Mar-13 23:10 BackupCatalog
    79  Full          0         0   Error    16-Mar-13 23:05 BackupClient1
    80  Full          0         0   Error    16-Mar-13 23:10 BackupCatalog
    81  Full          0         0   Error    17-Mar-13 23:05 BackupClient1
    82  Full          0         0   Error    17-Mar-13 23:10 BackupCatalog
====

Device status:
Device "disk1" is not open or does not exist.

Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Hi lennie

Congratulations again, it seems you have solved your connection problems (at least on localhost, then you start to connect to remote File Daemons, be sure to remember to start by disabling any firewall on the client then testing the connection)

The error status for the Terminated Jobs is expected for the dates shown, since your Director could not connect to your File Daemon nor your Storage Daemon.

The Device status : Device "disk1" is not open or does not exist. I think is also expected. First of all, a device not in use will have the status : Device "disk1" is not open. However since at that time (17-Mar) your Director had not yet been able to connect to your Storage Daemon, the Director have not been able to confirm whether disk1 exists - I guess that is the reason for the last part : or does not exist.

web fiddler by nature

Anonymous
--------------
      report  reply  
Thanks Rasmus, you are a star! I'll go through the rest of the steps and let you know how it goes.

Thanks again!
alain
--------------
      report  reply  
Great tutorial. Thank you for sharing your knowledge with us
terraltech.com
Riyaz
--------------
      report  reply  
Best Bacula config tutorial ever available!!!!!!!!!!!!!!!
Haxkers
--------------
      report  reply  
Hi, it's very good tutorial! I have got school project and I must backup Pc. If I use "Run" all is ok, but when I use "status storage" it's 
*status storage
Automatically selected Storage: storage1
Connecting to Storage daemon storage1 at 192.168.1.100:9103

Failed to connect to Storage daemon storage1.
====
You have messages.
*message
18-dub 12:46 FH-dir JobId 0: Fatal error: bsock.c:135 Unable to connect to Storage daemon on 192.168.1.100:9103. ERR=Spojení odmítnuto

Can you help me please? propably I have problem with configuration.. My PC is DIR and SD.
Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  

Hi Haxkers

The error "Failed to connect to Storage daemon" is likely because of one of the following reasons :

  1. The storage daemon is not running
  2. Firewall on the storage daemon server does not allow incoming traffic on port 9103 (default SD port)
  3. Director cannot authenticate with storage daemon


ad1) Test if the storage daemon is running :
storage daemon machine shell> service bacula-sd status
If the storage daemon is NOT running then start it :
storage daemon machine shell> service bacula-sd start

ad2) Test if your firewall allows incoming traffic on port 9103 : (here is an example for UFW)
storage daemon machine shell> ufw status verbose

ad3) Be sure you have configured bacula-dir.conf and bacula-sd.conf correctly with regard to authentication :

  • bacula-dir.conf Director{..} Name == bacula-sd.conf Director{..} Name
  • bacula-dir.conf Storage{..} Password == bacula-sd.conf Director{..} Password
  • bacula-dir.conf Storage{..} Address == bacula-sd.conf Storage{..} SDAddress

And then restart your storage daemon :
storage daemon machine shell> service bacula-sd restart

If you still cannot solve the connection problem then post the following 4 things here :

  1. bacula-dir.conf Director {} section
  2. bacula-dir.conf Storage {} section
  3. bacula-sd.conf Director {} section (you likely have 2, I am interested in the section that does NOT have the monitor property)
  4. bacula-sd.conf Device {} section (if you have more than 1, then post them all)

web fiddler by nature

Haxkers
--------------
      report  reply  
Ok. I had problem with IP adress.
Now, when I use "status storage" :
Running Jobs:
No Jobs running.
====

Jobs waiting to reserve a drive:
====

Terminated Jobs:
 JobId  Level    Files      Bytes   Status   Finished        Name 
===================================================================
     1  Full          0         0   Error    30-Mar-13 18:29 win1-backup
     3  Full          0         0   Error    25-Apr-13 12:22 win1-backup
====

Device status:
Device "disk1" is not open or does not exist.
====

Used Volume status:
====

I want to backup data on my one HDD.

and here are configs:

Director {                            
  Name = FH-dir
  DIRport = 9101                
  QueryFile = "/etc/bacula/scripts/query.sql"
  WorkingDirectory = "/var/lib/bacula"
  PidDirectory = "/var/run/bacula"
  Maximum Concurrent Jobs = 1
  Password = "dir-123"         # Console password
  Messages = Daemon
  DirAddress = 192.168.1.100

Storage {
  Name = storage1
  Address = 192.168.1.100                
  SDPort = 9103
  Password = "storage-123"
  Device = disk1
  Media Type = File
}

Director {
  Name = FH-dir
  Password = "storage-123"
}

Device {
  Name = disk1
  Media Type = File
  Archive Device = /media/disk1
  LabelMedia = yes;                   
  Random Access = Yes;
  AutomaticMount = yes;               
  RemovableMedia = no;
  AlwaysOpen = no;

Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  

Hi Haxkers

Your posted configuration details look valid and also the result of the status storage command is expected :

  • You have tried to run 2 jobs but could not connect to storage daemon, so they failed, hence the 2 Error status codes
  • Device "disk1" is not open or does not exist is not an error message. Then your device is not in use it will always display the message Device "disk1" is not open, however since your storage daemon have never actually used disk1, it's existence may never have been confirmed, I think that is the reason for the last part or does not exist.

Since your Director can now connect to your storage daemon, try to run a job by issuing the run command and see how it goes. If the job succeed, then try to issue the status storage command again, you should then get the device status Device "disk1" is not open.

web fiddler by nature

HAxkers
--------------
      report  reply  
root@MrKaviar:/home/haxkers#
bconsole
Connecting to Director 192.168.1.100:9101
1000
OK: FH-dir Version: 5.0.2 (28 April 2010)

Enter a period to cancel a command.


* run

Automatically selected Catalog:
 MyCatalog
Using Catalog "MyCatalog"

A job name must be specified.

The defined Job resources are:
    
1: win1-backup
    
2: win1-restore

Select Job resource (1-2): 1


Run Backup job

JobName:  win1-backup

Level:    Incremental

Client:   win1

FileSet:  win1

Pool:     pool1 (From Job resource)

Storage:  storage1 (From Job resource)

When:     2013-04-25 22:35:22

Priority: 10
OK to run? (yes/mod/no): y

Job queued. JobId=4
You have messages.


Anonymous
--------------
      report  reply  
And i Message:
Error: dev.c:120 Unable to stat device /media/disk1:
ERR=No such file
Warning: Device "disk1" requested by DIR could not be opened or does not exist.r directory
Error: dev.c:120 Unable to stat device /media/disk1: ERR=No such file or directory
Now... How Can I open my dikc 1?
Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  

To be sure it is not a permission problem, try to do the following :

  1. shell> chmod 777 /media
  2. shell> chmod 777 /media/disk1

, and then try to run a job again and see if the error persist.

(If you solve the problem, it would be kind of you to post here what the problem was)

web fiddler by nature

aristotleahito
User type : Standard
Register : 2013-May-10
Topics : 0
Replies : 2
--------------
      report  reply  

how are you going to restore a database server..

let say my fileset = /

then use restore command > select a client > and the file name what if i just mark* and then type done

is it possible? to restore the whole database server?

take your time!! coz time is gold!!
Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  

Hi aristotleahito

I don't know if this is possible or on what operating system it may be possible, however it is a very interesting question.

I guess it could be testet to some extent by creating a couple of virtual servers, installing different operating systems with different databases and then try it out.

Until you know more, I would advise against this strategy. You better run some backup scripts for your database creating dump files and then have Bacula backup these dump files. (Schedule the database backup scripts using crontab on Linux or Task scheduler on Windows)

web fiddler by nature

Sam
User type : Standard
Register : 2013-May-28
Topics : 1
Replies : 1
--------------
      report  reply  
Hello Friend,

Many thanks for a great tutorial. I am a newbie for bacula and your tutorial helped a lot in understanding.

I have installed bacula on Ubuntu 12.04 Server and tried running a backup job and here is the messages I have received. Please help me.

backup-sd JobId 11: Warning: mount.c:217 Open device "backup" (/backup) Volume "pool1-0001" failed: ERR=dev.c:568 Could not open: /backup/pool1-0001, ERR=No such file or directory


---
Sam
Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  

Hi Sam

The error message states that /backup/pool1-0001 does not exists.

1) Confirm that /backup/pool1-0001 exists on the server on which your storage daemon is running.

1.1) If /backup/pool1-0001 does not exist, then it is possible that the file have been manually deleted from the file system, so that the record is still existing in the Catalog database - you will then need to use the delete command (from within bconsole) to get pool1-0001 out of the Catalog database.

1.2) If /backup/pool1-0001 exits, then confirm :

1.2.1) that everyone have full access to the /backup directory.

1.2.2) that Bacula have read & write access to the /backup/pool1-0001 file :

1.2.2.1) storage daemon server shell> ps aux | grep bacula-sd : will show you what user and what group the bacula-sd process is executed under.

1.2.2.2) storage daemon server shell> ls -l /backup : will show you user & group access on all files under /backup.

 

web fiddler by nature

ariez
--------------
      report  reply  
hi can you help me fix this error during backupcatalog

 Could not stat "/etc/bacula/bacula.sql": ERR=No such file or directory

Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  

Hi ariez

It seems the error Could not stat "/etc/bacula/bacula.sql": ERR=No such file or directory often is a result of bacula not having permissions to read bacula-dir.conf or to execute make_catalog_backup.pl.

There is a fine walkthrough of this problem here : http://dan.langille.org/2010/03/23/baccula-errors-with-500-backup-catalog/

web fiddler by nature

Renier
User type : Standard
Register : 2013-Jun-25
Topics : 0
Replies : 1
--------------
      report  reply  
Rasmus - You're the man !! Thanks for such great assistance on Bacula .... No job so big or to small for you :-) Respect !!

Ish
--------------
      report  reply  
me accounting a problem , that when i hit the run command the default resources are shown.  no updations 

Plz do help...
Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  

Hi Ish

Then you execute the run command, you should see a list of Job resources - do you see such a list ?

If you see the Job resources list, you can then select which Job to execute by writing the number corresponding to the Job and then hit Enter - what happens after you have selected a Job resource ?

web fiddler by nature

RUK
--------------
      report  reply  
Hi Rasmus,

Thanks for posting a nice bacula document. It really helped me.

I have some question, please help me finding my questions answer.

1. i want to know how to clear the JobIB queue in bacula. i use delete command using bconsle, it
tells me that deletion is done but when i check the status the jobid still present.
2. i want to set the maximum volume size to 100M ans maximum volume to 9. So now my question is that when this 9 voulme finish does the bacula start over right the first backup or we need to provide the more backup space so that it can write data in other file.
Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Hi RUK

ad 1) To clear a job from the job queue use the cancel command. After you execute the cancel command from bconsole, bconsole will return a list of jobs in the queue (including current running jobs) - you can then select which job to cancel. If you choose a currently running job, it can take several hours for the job to cancel. You can make it faster by exiting bconsole and restart the storage daemon (shell> service bacula-sd restart), however you will then lose the job queue (you can then run jobs manually, though keep in mind that manually runned jobs does not use the schedule and any backup level and pool overriding in the scedule therefore does not apply). 

As a side note, you cannot use the delete command to cancel jobs (the delete command is used to delete entries in the Catalog database and have nothing to do with running or scheduled jobs).

ad 2) The answer to your question whether Bacula will overwrite existing volumes then all volumes are used and no more volumes can be created depends on your pool settings. Especially you will need to set the following in your Pool {} section :
  • Volume Retention = 61 days: Bacula will not recycle a volume before the retention period counting from the last writing to the volume (the last writing is calculated using the Catalog database not the filesystem, therefore purging a volume (delete all related records from the Catalog database) will make a volume recyclable. You may want to accommodate the 61 days to something that fit's your data size and your storage size.
  • Recycle = yes : so that Bacula will automatically recycle purged volumes belonging to the pool (that is : delete the volume content and set the volume status to Append)
  • AutoPrune = yes : so that Bacula will automatically apply the retention period then a new volume is needed and no volume can be created or change status to Append.
These 3 settings does NOT guarantee that Bacula will reuse your oldest volume then all volumes are full. If your backup fails because Bacula is not able to provide any appendable volume, you can manually purge the volumes you want to be overwritten, just execute the purge command in bconsole and take it from there.

If you want to understand the exact Bacula procedure, please refer to this comment above : http://webmodelling.com/webbits/miscellaneous/bacula.aspx?ml-d-mpids=22

web fiddler by nature

RUK
--------------
      report  reply  
Really Thanks to you !

I will keep posting you as i start working on bacula ....i must be having lots of questions.

Will you please explain me one more thing

In Pools function :
how these 2 values work
  • Maximum Volume Bytes
  • Maximum Volumes
Will you please explain and what you think before setting these 2 values.
Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Hi RUK

The files you backup are kept within volumes. If you backup to harddrives, then the harddrive is called a backup device and the volumes are themselves just files on that harddrive - files into which the backup files are stored (if you backup to tape, then a volume is a tape and the tape drive is the backup device)
A pool is a group of volumes. You typically set volume properties on a group of volumes - that is : you set the volume properties using the pool {} section. If you want some volumes with different properties, you then create a second pool. Also the volume group (the pool) itself can have properties, eg. auto creating volumes then needed.
 
  • Maximum Volume Bytes is a volume property specifying the maximum size of the volume file - as backup files are stored within a volume file, the volume file will keep growing until Maximum Volume Bytes.
  • Maximum Volumes is a volume group (pool) property specifying the maximum number of volumes allowed in the group - as backup files are stored within volume files belonging to the group and these volume files fill up new volume files needs to be added and in case of auto creating volumes Bacula will keep adding them until either no more space on the harddrive or until Maximum Volumes.
Sensible values for Maximum Volume Bytes and Maximum Volumes depend very much on different factors and different storage strategies - an understanding of this can only come with experience. However I will show you my full pool {} for one of my current backup solutions :
  • Pool {
    •   Name = full
    •   Pool Type = Backup
    •   Recycle = yes                 # Automatically delete volume content and change volume status to Append on pruned volumes
    •   AutoPrune = yes               # If no other volume can be create or appended then automatically apply retention periods by deleting DB file, job & volume records
    •   Volume Retention = 93 days    # Wait 93 days after last writing to the volume file before auto pruning the volume
    •   Maximum Volume Bytes = 25G    # Volumes sizes are not allowed to grow beyond 25 GB
    •   Maximum Volumes = 30          # There can never be more than 30 volumes in the pool
    •   Label Format = full- # Automatically create new volumes then needed (unless there are 30 volumes)
    •   Catalog Files = yes # Also write all files stored to the Catalog database (otherwise individual files cannot be restored only jobs)
    •   Maximum Volume Jobs = 0 # No restriction on how many jobs in a single volume
    •   Maximum Volume Files = 0 # No restriction on how many files in a single volume 
    •   Volume Use Duration = 0 # No restriction on over how long time the volume can be written
  • }

web fiddler by nature

RUK
--------------
      report  reply  
Thank you Very much Rasmus,

I have one more issue working with bacula.
1.     I want to know that do we need to comment all the pools{} parameter session, before creating our own pools{} session.
2.     Could you please explain me, what is "BackupCatalog" ?,
    how to access this backupcatalog ?
    Is it mendatory to have backupcatalog, if i dont want to have backupcatalog, can i comment out Job Catalog{} parameters.
    
    Here is the file job catalog section.
    # Backup the catalog database (after the nightly save)
    Job {
        Name = "BackupCatalog"
        JobDefs = "jobdefs1"
        Level = Full
        FileSet="Catalog"
        Schedule = "WeeklyCycleAfterBackup"
        # This creates an ASCII copy of the catalog
        # Arguments to make_catalog_backup.pl are:
        #  make_catalog_backup.pl <catalog-name>
        RunBeforeJob = "/etc/bacula/make_catalog_backup.pl MyCatalog"
        # This deletes the copy of the catalog
        RunAfterJob  = "/etc/bacula/delete_catalog_backup"
        Write Bootstrap = "/opt/bacula/working/%n.bsr"
        Priority = 11                   # run after main backup
        }
Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Hi RUK

Let me first suggest you to login then posting, that way you will get an email notification then a reply is written to your post.

Ad 1) You don't need to delete or out-comment the example pool {..} sections, the only harm they do is to clutter your bacula-dir.conf file (I prefer to delete them myself though).

Ad 2) I always delete the BackupCatalog example Job from the bacula-dir.conf file. Instead my standard database backup system will backup the catalog database. As my database backup files are not only transfered to Bacula but are also directly available for the last 7 days locally on the database servers, this system allows me to restore the catalog database in less than 2 minutes using a standard routine (I have had many databases in need of restore but never the Bacula catalog).

web fiddler by nature

rukender_singh
User type : Standard
Register : 2013-Jul-23
Topics : 0
Replies : 3
--------------
      report  reply  
Hi Rasmus,

Now i have account :P

Thanks for your reply.
Will you please suggest me any good GUI tool for bacula monitoring.

Anonymous
--------------
      report  reply  
Also Rasmas,

I have some problem understanding Pool concept.
This is my bacula-dir.conf file Pool{} parameter configuration in my test environment.
Pool {
  Name = pool1
  Pool Type = Backup
  Recycle = yes                       # Bacula can automatically recycle Volumes
  AutoPrune = yes                     # Prune expired volumes
  Volume Retention = 61 days         # one year
  Maximum Volume Bytes = 1G
  Maximum Volumes = 20
  Label Format = pool1-
  Catalog Files = yes
  Maximum Volume Jobs = 0
  Maximum Volume Files = 0
  Volume Use Duration = 0
}
Here in this pool configuration my "Maximum Volume Bytes is 1G" and "Maximum Volumes is 20". It means that i would be having 1GB backup file. once this 1GB fills then it will creat another pool label and start writing in the new pool label.
Ex: my current pool label is pool1-0002, once backup file in this pool level exceeds to more than 1GB, it will create another label i.e pool1-0003, am i right ?
But, in my test environment i m facing some problem i.e

[root@rukender]# ls -ltrh /backup/
total 1.2G
-rw-r-----. 1 root root 1.2G Jul 23 01:05 pool1-0002

As u can see, in my test machine, pool1-0002 exceed to more than 1GB, but still i have only one label, but there must be another pool label once it axceed to 1GB, why there is no other new pool label i.e pool1-0003?
Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Hi Anonymous

You have understood the concepts correctly and I would be equally confused seeing the 1.2GB pool1-0002 file. I have never experienced a similar situation myself, the volume sizes have always stopped exactly on the byte at maximum volume bytes. I think there is something at play cheating us, eg. maybe the volume size was above 1 GB before your pool definition was recognized by Bacula or something like that.

web fiddler by nature

rukender_singh
User type : Standard
Register : 2013-Jul-23
Topics : 0
Replies : 3
--------------
      report  reply  
ya, you are right i m still confused.

i want to tell you that initially when i start configuring the bacula-dir.conf time at that time i put the Maximum Volume Bytes as 20GB. But after clearifying the concept of this i changed it to 1GB. Maybe this is the reson ?, also if it is the reson than could you please help me to fix it.

one more thing is that why my pool cycle start from pool1-0002 why not form pool1-0001?

Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Indeed an earlier bacula-dir.conf file have most likely been the reason. To clean up the mess, I would suggest to use the delete command from within bconsole to delete all your volumes (don't delete your volume files directly on the filesystem - your Catalog database would then not be consistent with your files). Also keep in mind all backup done so far will be lost.

After cleaning up restart your daemons :
shell> service bacula-director restart
shell> service bacula-sd restart
client shell> service bacula-fd restart (different if your client is windows) 

Then run a full backup.

web fiddler by nature

rukender_singh
User type : Standard
Register : 2013-Jul-23
Topics : 0
Replies : 3
--------------
      report  reply  
Hi Rasmus,

I tried to follow you, I tried Delete option from my bconsole. first it ask me what i want to delete then i gave it my Volume name, then it gave me 4 option in between i choose to delete pool and then it was asking for pool name which i want to delete. All this task performed sucessfully but still i m able to see my pool in my storage location, its not deleted from the storage area.

Will you tell me the way how to fix this problem, so that once the backup pool reach to its limit it will start writing to the new pool ID automatically

Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Hi Ruk

Account good 

Unfortunately Bacula is not strong on the no-pay GUI side or at least it was not 3 years ago. If I were to explore the no-pay GUI options for Bacula, I would start here : http://www.bacula-web.org/ 

web fiddler by nature

Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Hi rukender_singh

Your original problem was that you had a volume file bigger than the maximum volume bytes set for the pool to which the volume belong. Most likely that had happened because you have had a different maximum volume bytes setting earlier. To remove the volume that was too big, you should from inside bconsole have issued the delete command and then deleted the volume (by selecting volume and then the pool to which the volume belongs). Instead it seems you have tried to delete a pool. I am not sure what it is you see in your storage area, I guess it is your volume files you can see, however also note that even if you delete pool records from the database, the pool will be recreated from your bacula-dir.conf if not deleted from there (I don't think such a recreated pool will pick up the old volume files though, but I am not sure).

Right now your bacula system is likely in a mess. However I think that you should try again from within bconsole to delete all volumes and then AFTER that delete all volume files from the filesystem - this will hopefully make your catalog database and filesystem consistent again.

web fiddler by nature

Rukender
--------------
      report  reply  
Hi Rasmus,

I have a problem in bacula. When i m trying to take a first time backup bacula took full backup. Till here its good.
But, when i m trying to take incermental backup by adding 1MB file in the backup folder at that time bacula is again taking full backup instead of backuping only 1MB file it backup all the files in incremental backup.
Please help me to solve this problem.
here output of status client command form bconsole.
     1  Full          0         0   Error    06-Mar-13 08:23 snmpagent2-Backup
     3  Full        992    197.7 M  OK       06-Mar-13 08:27 snmpagent2-Backup
     4  Incr        993    199.0 M  OK       06-Mar-13 08:33 snmpagent2-Backup
     5  Incr        992    199.0 M  OK       06-Mar-13 08:40 snmpagent2-Backup
Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Hi Rukender

I am a couple of days late with the answer. If you still have the problem, then compare datetime stamps on the files you backup using :
1) shell> ls -l
2) shell> ls -c : especially if ls -c shows identical datetime stamps on your files, you may have another process running that touch your files making Bacula include all files in the incremental backup.

web fiddler by nature

Rukender
--------------
      report  reply  
Hi Rasmus,
Thanks for your reply!

I have some questions please try to help me.
1. If i start talking backup for a system and after sometime i made changes in bacula-dir.conf file and change some pool setting, then in this case how can i start taking full backup with the new confiuration and deleting all the previous backup. How it will be possible and how to perfom this task sucessfully.Please explain for me in little brief.
I tried it from my side but not sucessfull to perfom.

2. how to clean all my previous JOBID using DELETE command from bconsole, i tried it from side it ask me for which jobid i want to delete but if i gave the jobid nothing happen, that jobid still shows up in the list.

olop
User type : Standard
Register : 2013-Jul-19
Topics : 0
Replies : 2
--------------
      report  reply  
Help..
When i start the console program (bconsole), i can't connected to director.
root@olop-Aspire-V3-471G:~# bconsole
Connecting to Director 192.168.1.4:9101
I use ubuntu 12.04

Can you help me? Thanks before..


Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Hi olop

To resolve connection problems between bconsole and Bacula Director, you can try the following steps :

1) If bconsole is NOT installed on the same machine as Director, then try to disable any firewall on both the bconsole machine and especially on the Director machine to determine whether the connection problem lay with the firewall - in fact : don't enable any firewall before you have solved the connection problem.

2) Your bconsole must contact the Director using the Director Name, Password and IP - if any of these mismatch, then the bconsole will be unable to connect. The 2 config files that you need to compare are bacula-dir.conf (on the machine on which the Director is installed) and bconsole.conf (on the machine on which the bconsole is installed - most often that is the same machine). In both config files you need to look for the Director {..} section :
  • bacula-dir.conf Director {..} Name == bconsole.conf Director {..} Name
  • bacula-dir.conf Director {..} Password == bconsole.conf Director {..} Password
  • bacula-dir.conf Director {..} DirAddress == bconsole.conf Director {..} Address

3) Director shell> service bacula-director status : test if the Director service is actually started (if not, then shell> service bacula-director start)

4) telnet the Director from your bconsole machine :
bconsole shell> telnet 192.168.1.4 9101 : if you can telnet, then there is no firewall problem and you have also double confirmed that Director is running (you quit telnet by holding ctrl and pres ])

5) run nmap to see what ports are available :
bconsole shell> nmap 192.168.1.4
director shell> nmap localhost

web fiddler by nature

aristotleahito
User type : Standard
Register : 2013-May-10
Topics : 0
Replies : 2
--------------
      report  reply  
hi rasmus.. have you tried using webacula? thanks using postgresql? thanks
take your time!! coz time is gold!!
Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Hi aristotleahito. I have never tried to use webacula nor postgresql with bacula. While I am a happy CLI cowboy myself, I definitely think a web based GUI could be valuable for many - if you try it you are most welcome to post your experience with webacula here.

web fiddler by nature

olop
User type : Standard
Register : 2013-Jul-19
Topics : 0
Replies : 2
--------------
      report  reply  
Hi..
I want to make my bacula email notifications on backup and restore

My configuration is
messages {
   Name = Standard
mailcommand = "/ usr / lib / bacula / bsmtp-h localhost-f \" \ (Bacula \) \ <bacula.situmorang@gmail.com\> \ "-s \" Bacula:% t% e of% c% l \ "% r"
   operatorcommand = "/ usr / lib / bacula / bsmtp-h localhost-f \" \ (Bacula \) \ <bacula.situmorang@gmail.com\> \ "-s \" Bacula: Intervention needed for% j \ "% r "
   mail = situmorang.olop @ gmail.com = all,! skipped
   operator = situmorang.olop @ gmail.com = mount
   console = all,! skipped,! saved
# append = "/ var / lib / bacula / log" = all,! skipped
   catalog = all
}

for mailcommand and operatorcommand i use bacula.situmorang@gmail.com,
for mail and comman i use situmorang.olop@gmail.com

but it not works, email not sending..

can you help me?



Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Hi olop

I can see you have configured the operatorcommand to connect to an MTA (Mail Transfer Agent) on localhost - you should be sure that you actually do have an MTA on localhost, to test you can try the following :

  • shell> dpkg -l | grep mail : look for mail related packages (likely only works on debian based OS)
  • shell> telnet localhost 25 : test if any daemon is listening on port 25
    • shell> helo localhost : to be sure that the daemon is an MTA issue the helo command
  • shell> nmap localhost : another way to test what daemons are listening on what ports
  • shell> mail "test body" -s "test subject" situmorang.olop@gmail.com : sending an email from the command line to confirm that mail actually can be sent

web fiddler by nature

Nacho
User type : Standard
Register : 2013-Sep-06
Topics : 0
Replies : 1
--------------
      report  reply  
Hi Rasmus,

First of all, thank you very much for this wonderful tutorial that helped me a lot in my installation and configuration of Bacula backup system. After doing some tests successful backup and restoration I found a problem on a Windows server, which approximately two hours after the start of Job, it disrupts the connection. I solved this problem by adding a Heartbeat Interval in the bacula-sd.conf and bacula-fd.conf ... So my question is, if it is possible to resume a Job that has been interrupted by this or another cause.

PS: Thanks again and sorry for my basic english.

Cheers!
Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Hi Nacho

It is to my knowledge NOT possible to resume a job and I would recommend NOT to fiddle with the bacula database JobStatus fields in order to get Bacula to resume a job.

Better to accept you have a failed job and start again. If your heatbeat solution does not solve the problem with lost connections, then try to split the bacup job up in 2 or 3 backup jobs each having less to backup.

I have no installations with more than  2 TB of data to backup, so I work with relatively small data, however if your data are  big, maybe it would be a good idea to look into virtual full backup, see more here : http://www.bacula.org/de/dev-manual/New_Features.html.

web fiddler by nature

Joe
--------------
      report  reply  
Hello. I followed your tutorial everything seems to be working fine. I using VM to make tests.I am using a Windows & VM and A Ubuntu Server as clients. For the ubuntu backup I used label and I performed the backup with sucess but there is a file with the label and I can't cd into it to see the files that have been backed up. On the windows client I did not used label and I am using a mounted drive but I don't see any backup files for it. Can you explain how Bacula stores the backup files and how can have access to it and restore? Thannks.
Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Hi Joe

The files you want to backup from various clients are stored within volumes. In case you use a harddrive or directory for backup device, your volumes are themselves files - so your backup files are stored within volume files and these volume files are stored on your backup device (most typically the backup device will be just a directory).

The storage daemon is responsible for handling backup devices. In case the backup device is either a harddrive or a directory (most typically), the volumes will be files, while if the backup device is a tape drive, the volumes will be tapes.

Your backup device is appointed in bacula-sd.conf in the Device{..} section as "Archive Device".

Here is a device section from one of my own bacula-sd.conf files: 
Device {
  Name = disk1
  Media Type = File
  Archive Device = /bacula
  LabelMedia = yes;
  Random Access = Yes;
  AutomaticMount = yes;
  RemovableMedia = no;
  AlwaysOpen = no;
}
, you can see that the volume files are stored in the directory "/bacula" on the server on which the storage daemon is running (the storage daemon is running on our backup server of course).

backup server shell> cd /bacula : navigate to the backup device folder (your backup folder may be in a different path).
backup server shell> ls -lh : list all the volume files.

(Note that you cannot directly use the volume files for restore, you will of course need to issue the restore command from within the bconsole prompt).

web fiddler by nature

pquesinb
User type : Standard
Register : 2014-Jan-07
Topics : 0
Replies : 2
--------------
      report  reply  
I have a question about the possibility of expiring jobs.  Not in the sense of a job that has run successfully and needs to be aged out of the catalog, but a job in the queue that's ready to be run.

I've got Bacula backing up a number of virtual machines to LTO tape but occasionally the backup doesn't run for some reason, office is closed so the employee doesn't change the tape, etc.  What subsequently happens is the job gets queued up and then runs later when the tape is changed, several jobs might start running during the day while files are open and in use instead of at night when they're supposed to.  Is there a way to keep from queuing those jobs up or expire them if they don't start within say, an hour or so?

Thanks and keep the great info coming.

- Phil

Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Hi pquesinb

I am not aware of a setting to automatically cancel a job in case the job have not started within a set time after it is sceduled to run.

However, I think you can setup a cron job that executes a script to cancel any queue (eg. running 7 o'clock each morning). Here is a link to a guy who have written such a script : http://www.cyberwizzard.nl/gentoo-linux/144-kill-all-waiting-backup-jobs-in-bacula.html.

web fiddler by nature

Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Hi pquesinb

Ok, I just found the setting, it is called Max Start Delay and needs to be set in the Job resource (or JobDefs).

You can find the description here : http://www.bacula.org/5.2.x-manuals/en/main/main/Configuring_Director.html#SECTION001430000000000000000 (search the page for Max Start Delay)

web fiddler by nature

pquesinb
User type : Standard
Register : 2014-Jan-07
Topics : 0
Replies : 2
--------------
      report  reply  
Hi Rasmus,

Thank you!

That setting works quite well, although it appears to have one limitation:
The first job will start even if the right tape isn't available, then will complete once the tape is changed (the next day or whenever).  The rest of the jobs will then cancel out.  The cron job would likely be a good way to deal with that.

This isn't a big problem for us since the first job queued is a very small backup job, only a couple-hundred MB.  The next job after that is the big one, which will use a significant chunk of the tape's remaining capacity.

Thanks for finding that!

Cheers,

- Phil

Anonymous
--------------
      report  reply  

Very good tutorial !

All is clear, you help me so much !

Thanks a lot

best regards

a new bacula's user

avecsi
User type : Standard
Register : 2014-Feb-26
Topics : 0
Replies : 3
--------------
      report  reply  

Dear Rasmus,

26-Feb 15:05 FH-dir JobId 3: No prior Full backup Jobrecord found.

26-Feb 15:05 FH-dir JobId 3: No prior or suitable Fullbackup found in catalog. Doing FULL backup.

26-Feb 15:05 FH-dir JobId 3: Start Backup JobId 3,Job=BackupClient1.2014-02-26_15.05.26_06

26-Feb 15:05 FH-dir JobId 3: Using Device"disk1"

26-Feb 15:05 FH-sd JobId 3: Volume "pool1-0001"previously written, moving to end of data.

26-Feb 15:05 FH-sd JobId 3: Ready to append to end ofVolume "pool1-0001" size=616 26-Feb 15:05 lmic-w-p-app01-fd JobId 3:Fatal error: VSS API failure calling "InitializeForBackup".ERR=Unexpected error. The error code is logged in the error log file.

26-Feb 15:05 lmic-w-p-app01-fd JobId 3: Fatal error: VSSwas not initialized properly. ERR=The operation completed successfully.

 26-Feb 15:05 lmic-w-p-app01-fd JobId 3: Error: VSS APIfailure calling "BackupComplete". ERR=Object is not initialized;called during restore or not called in correct sequence.

26-Feb 15:05 lmic-w-p-app01-fd JobId 3: Fatal error: VSSAPI failure calling "GatherWriterStatus". ERR=Object is notinitialized; called during restore or not called in correct sequence.

26-Feb 15:05 FH-sd JobId 3: Job write elapsedtime = 00:00:05, Transfer rate = 0 Bytes/second 26-Feb 15:05 FH-dir JobId 3: Error: Bacula FH-dir 5.2.5

  Build OS:               x86_64-pc-linux-gnu ubuntu 12.04

  JobId:                  3

  Job:                   BackupClient1.2014-02-26_15.05.26_06

  Backup Level:           Full (upgraded from Incremental)

  Client:                 "lmic-w-p-app01-fd"5.2.10 (28Jun12) Microsoft Windows Server 2008 R2 Standard Edition Service Pack1 (build 7601), 64-bit,Cross-compile,Win32

  FileSet:                "Full Set" 2014-02-2614:20:59

  Pool:                   "pool1" (From Jobresource)

  Catalog:                "MyCatalog" (FromClient resource)

  Storage:                "storage1" (From Jobresource)

Elapsed time:          5 secs

  Priority:               10

  FD FilesWritten:       0

  SD FilesWritten:       0

  FD BytesWritten:       0 (0 B)

  SD BytesWritten:       0 (0 B)

Rate:                  0.0 KB/s

  SoftwareCompression:   None

  VSS:                    yes

Encryption:            no

  Accurate:               no

  Volumename(s):        

  Volume SessionId:      2

Andrew
avecsi
User type : Standard
Register : 2014-Feb-26
Topics : 0
Replies : 3
--------------
      report  reply  

  Volume SessionTime:    1393425613

  Last VolumeBytes:      1,034 (1.034 KB)

  Non-fatal FDerrors:    2

  SD Errors:              0

  FD terminationstatus:  Error

  SD terminationstatus:  Canceled

  Termination:            *** Backup Error ***

 I got this message in email. This is the first time I would like to test the backup. The agent has access to the files .

Could you please help me out where should I search?

/var/log/bacula/log has the same message,what i received in email


Thanks,

Andrew
Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Hi avecsi

Your windows OS is probably 64 bit and the VSS (Volume Shadow Copy) service on a 64 bit system cannot be used from a 32 bit process. If therefore your Bacula win-client is the 32 bit version, you will get the above error (I think).

Be sure you have installed the Bacula 64 bit win-client.

web fiddler by nature

avecsi
User type : Standard
Register : 2014-Feb-26
Topics : 0
Replies : 3
--------------
      report  reply  
Thanks. That was the problem.
Great tutorial.

Without bacula how can I restore the backup?
Bacula created a pool1-0001.gz file.
Andrew
Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Good to hear you got the problem solved.

With regard to restoring, you are under normal circumstances not supposed to restore from Bacula volumes by other means than Bacula. However there are several Bacula tools that can look inside your volumes, especially "bextract" is a tool that can restore files from Bacula volumes.

web fiddler by nature

Rock
--------------
      report  reply  
Quite honestly, this is the best basic tutorial I have read for Bacula. There are so many articles that teaches you how to install but as soon as the article gets thru running services, you are left off not knowing config file descriptions. Because of this article I was able to understand Bacula in single day. 1 or more days i'll be able to go deep down to advance configuration. Thanks for this sample tutorial....
mmar
User type : Standard
Register : 2014-Jul-23
Topics : 0
Replies : 1
--------------
      report  reply  
Hi,
I have a problem when restore a file from bconsole:

23-jul 09:47 bacula-dir JobId 2611: Fatal error: Socket error on Store end command: ERR=Conexión reinicializada por la máquina remota

23-jul 09:47 bacula-sd JobId 2611: Error: bsock.c:382 Socket is terminated=1 on call to client x.x.x.x:9102


and then the bacula service on client disable so I have to start again.


Could you help me?


Thanks!



Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Hi mmar
From your error messages, it apparently look like a network problem.
Please follow this procedure :
  1. Disable all firewalls on director server, storage server and client machine
  2. Director server shell> bconsole
  3. *> status storage (depending on your configuration you will have different options)
  4. *> status client (depending on your configuration you will have different options)
If you get any error, please post here. If you get no errors, then try to re-enable any firewall and see if the error is because of a firewall setting.

web fiddler by nature

siddhartha0111
User type : Standard
Register : 2014-Sep-29
Topics : 0
Replies : 1
--------------
      report  reply  
29-Sep 16:11 bacula-dir JobId 0: Fatal error: postgresql.c:227 Unable to connect to PostgreSQL server. Database=bacula User=bacula
Possible causes: SQL server not running; password incorrect; max_connections exceeded.
29-Sep 16:11 bacula-dir ERROR TERMINATION
Please correct configuration file: /etc/bacula/bacula-dir.conf


Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Hi siddhartha0111
I have never tried to install bacula with PostgreSQL and I have never seen the error you got. 

Your error also suggest that you need to edit your bacula-dir.conf file, however this file does NOT contain the credentials for your catalog nor is the bacula-dir.conf file related to any of the 3 possible causes mentioned in the error message.

Also I think the debugging and best course of action would depend on whether your bacula installation is new.
If your bacula installation is new, I would advice you to remove your installation and then reinstall using MySql, please follow the installation instructions carefully 

web fiddler by nature

bella@UTM
--------------
      report  reply  
Hi, im using bacula 5.2.6, 
im trying to connect bconsole with director and its display..
connecting to director x.x.x.x:9101, 
and then its return to system prompt.

what is the problem and solution?

thanks..
viroteb
User type : Standard
Register : 2014-Dec-18
Topics : 0
Replies : 2
--------------
      report  reply  
Hi Rasmus
I am a newbie in Linux. I am trying to backup two servers, Alfresco and Moodle in my office. (I live in Bangkok too!) 
I really appreciate your great easy to understand guild on bacula. I have followed you guild and succeeded in backup job. However when I tried to restore, some errors occured. 
The restore job description  in   bacula-dir.conf  is as follows:

Job {

  Name ="alfresco-restore"

  Type = Restore

  Client=alfresco

 FileSet="alfresco"

  Storage ="storage1"

  Pool ="pool1"

  Messages = Standard

  Where =/var/bacula-restores

}

Job {

  Name ="moodle-restore"

  Type = Restore

  Client=moodle

 FileSet="moodle"

  Storage ="storage2"

  Pool ="pool2"

  Messages = Standard

  Where =/var/bacula-restores

}

 # List of files to be backed up

FileSet {

  Name ="alfresco"

  Include {

    Options {

      signature = MD5

    }

 

    File =/opt/alfresco-5.0.b/alf_data

  }

 

}

 

FileSet {

  Name ="moodle"

  Include {

    Options {

      signature = MD5

    }

    File =/var/www/moodledata

  }


The error messages are as follows:

5-Dec 16:51 bacula-dir JobId 52: Error: Bacula bacula-dir 5.2.6 (21Feb12):
  Build OS:               x86_64-pc-linux-gnu ubuntu 14.04
  JobId:                  52
  Job:                    alfresco-restore.2014-12-15_16.51.02_04
  Restore Client:         alfresco
  Start time:             15-Dec-2014 16:51:04
  End time:               15-Dec-2014 16:51:04
  Files Expected:         0
  Files Restored:         0
  Bytes Restored:         0
  Rate:                   0.0 KB/s
  FD Errors:              2
  FD termination status:
  SD termination status:
  Termination:            *** Restore Error ***

Could you guild me how to solve this problem?

viroteb
User type : Standard
Register : 2014-Dec-18
Topics : 0
Replies : 2
--------------
      report  reply  
Some additional information related to my previous problem, 
when I run restore command as follows:
Select the Client (1-3): 1
Automatically selected FileSet: alfresco
+-------+-------+----------+-------------+---------------------+------------+
| JobId | Level | JobFiles | JobBytes    | StartTime           | VolumeName |
+-------+-------+----------+-------------+---------------------+------------+
|    33 | F     |    4,250 | 180,981,416 | 2014-12-07 23:05:03 | pool1-0004 |
|    47 | D     |    1,087 | 101,980,894 | 2014-12-14 23:05:03 | pool1-0004 |
|    50 | I     |      480 |  59,747,632 | 2014-12-15 12:45:18 | pool1-0004 |
|    53 | I     |      358 |  39,140,893 | 2014-12-15 23:05:03 | pool1-0004 |
|    55 | I     |      475 |  62,895,786 | 2014-12-16 23:05:02 | pool1-0004 |
|    57 | I     |      396 |  61,164,226 | 2014-12-17 23:05:02 | pool1-0004 |
|    59 | I     |      522 |  64,506,300 | 2014-12-18 16:58:08 | pool1-0004 |
+-------+-------+----------+-------------+---------------------+------------+
You have selected the following JobIds: 33,47,50,53,55,57,59

Building directory tree for JobId(s) 33,47,50,53,55,57,59 ...  ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
6,193 files inserted into the tree.

You are now entering file selection mode where you add (mark) and
remove (unmark) files to be restored. No files are initially added, unless
you used the "all" keyword on the command line.
Enter "done" to leave this mode.

cwd is: /
$ dir
----------   0 root     root               0  1970-01-01 07:00:00  /opt/
$ mark
No files marked.
$
It seems that there are no file in the /opt directory. Though I have defined it in the bacula-dir.conf as previously shown.
FileSet {
  Name = "alfresco"
  Include {
    Options {
      signature = MD5
    }

    File = /opt/alfresco-5.0.b/alf_data
  }
  }
Please guide me on how to solve this problem.

Viroteb


Rasmus
User type : Admin
Register : 2012-Dec-21
Topics : 0
Replies : 108
--------------
      report  reply  
Hi viroteb
Sorry for the 2 month waiting (I have been extremely busy). Hopefully you have already solved your problem, however if your question is still relevant, I would like to see your backup job definitions (not your restore job defintions) and I would also like to see your pool definitions.

web fiddler by nature

Anonymous
--------------
      report  reply  
Hi
florek
User type : Standard
Register : 2015-Jul-18
Topics : 0
Replies : 2
--------------
      report  reply  
Hello,
Very thanks to this helpful tutorial. I have configure bacula and it works well. Now, i want to back up the databases of remote servers but i don't know how to do? Please, i need your help.

florek
User type : Standard
Register : 2015-Jul-18
Topics : 0
Replies : 2
--------------
      report  reply  
Hello everyone,
I have configured my bacula to use secure communications with TLS encryption. So, I have create my own Certificate authority and certificate and keys for my bacula server and clients. I have just started by the the tls communications between bconsole and the director, in order to come out with difficulties. But I have encountered this problem (in the prompt command) when running the bconsole command:

Connecting to Director bacula:9101
TLS negotiation failed
Director authorization problem.
Most likely the passwords do not agree.
If you are using TLS, there may have been a certificate validation error during the TLS handshake.
Please see http://www.bacula.org/en/rel-manual/Bacula_Freque_Asked_Questi.html#SECTION003760000000000000000 for help.

and in the file /var/log/bacula.log, we have:
20-Jul 01:21 bacula-dir: ERROR in authenticate.c:403 TLS negotiation failed.
20-Jul 01:21 bacula-dir: ERROR in authenticate.c:418 Unable to authenticate console "*UserAgent*" at client:127.0.0.1:36131.

Here is my configuration files (only tls part):
/etc/bacula/bacula-dir.conf
Director {                          
  TLS Enable = yes
  TLS Require = yes           
  TLS Verify Peer = yes
  TLS Allowed CN = "bacula"
  TLS CA Certificate File = /etc/bacula/keys/ca.crt
  TLS Certificate = /etc/bacula/keys/director.crt
  TLS Key = /etc/bacula/keys/director.key
}

/etc/bacula/bconsole.conf
Director {
  Name = bacula-dir
  DIRport = 9101
  address = bacula
  Password = "dir_bacula"
  TLS Enable = yes
  TLS Require = yes
  TLS CA Certificate File = /etc/bacula/keys/ca.crt
  TLS Certificate = /etc/bacula/keys/console.crt
  TLS Key = /etc/bacula/keys/console.key
}

Please, I need your help.

Anonymous
--------------
      report  reply  
Hello,

I am French and i have a problem with the installation of Bacula.
I want to install Bacula Client on a Windows 8.
First, i Install bacula-win64-5.2.10, the installation doesn't go to the end. The last screan freeze.
Second, The service Bacula doesn't want to start.
Can you help me.
Thanks
Michel Escudié
Anonymous
--------------
      report  reply  
Tengo Problemas con Bacula en el estado de la ONU ranura de almacenamiento del heno Que Me Dicen: No se encuentra en el catálogo. Slot = 3 InChanger establece en cero. Tengo 2 drive de almcenamiento 24 ranuras pero no las reconoce todas. 

 Status  |     Media Type       |      Pool          |
------+------------------+-----------+----------------------+--------------------|
    1 |         VXS002L6 |      Full |                LTO-6 |           FullPool |
    2*|         VXS003L6 |    Append |                LTO-6 |            DifPool |
    3*|         VXS004L6 |         ? |                    ? |                  ? |
    4*|         VXS005L6 |         ? |                    ? |                  ? |
Anonymous
--------------
      report  reply  
Tengo Problemas con Bacula en el estado de la ONU ranura de almacenamiento del heno Que Me Dicen: No se encuentra en el catálogo. Slot = 3 InChanger establece en cero. Tengo 2 drive de almcenamiento 24 ranuras pero no las reconoce todas. 

 Status  |     Media Type       |      Pool          |
------+------------------+-----------+----------------------+--------------------|
    1 |         VXS002L6 |      Full |                LTO-6 |           FullPool |
    2*|         VXS003L6 |    Append |                LTO-6 |            DifPool |
    3*|         VXS004L6 |         ? |                    ? |                  ? |
    4*|         VXS005L6 |         ? |                    ? |                  ? |
johnny
User type : Standard
Register : 2016-Aug-04
Topics : 0
Replies : 1
--------------
      report  reply  

Hi

I want to back up microsoft office documents in chinese characters like 苹果.docx or 苹果.xlsx. My backup is working fine however it cannot backup chinese files.

The error code is Could not stat"/home/it/20mount/group/July/??.docx": ERR=No such file or directory

Any ideas? I think it has something to do with utf-8 but I am not sure how

I am using ubuntu 10.04 , bacula version 5.0.1 , mysql server 5.1.73

Anonymous
--------------
      report  reply  
Dear Sir,

I meet a problem with bacula. I am a new user who use it. I meet an error when I delete all backup and backup again I can backup only one host - other few hosts can not backup. I need you help and how can I backup it again like the first time of backup.
I am looking forward to hearing from you soon.
Thank in advance!

Ratha
Balaraju G
--------------
      report  reply  
Hi Friends i have installed the bacula along with webmin. I have configured the server and client,Now backups are working fine. I have problem with volume pool retention. I have configure volume pool as 2 days. but its not retaining. can any body suggest where am doing the mistake 


click to top Bacula Backup