\chapter{New Features in Older Bacula Enterprise Versions}
\label{blb:chapter:newfeaturesolderbacula}
This chapter presents some of the new features that have been added to the
older Enterprise versions of \bacula{}.  These features are available only
with a \baculasystems{} subscription.

\section{\baculaenterprise{} 8.8}
%% \subsection{Native S3 Cloud Support}
%% Cloud storage has been around for a number of years, but until
%% recently, it was not very interesting for backup because of the lack of
%% bandwidth for uploading and downloading the large volumes of data that
%% today's backups use.  With the advent of fibre optics and high-speed copper
%% Internet connections, the Cloud has become a viable and cost effective
%% solution for data backup.
%% \smallskip

%% \baculasystems{} has implemented in \baculaenterprise{} 8.8.0 drivers for
%% backup and restore to and from any S3 compatible cloud whether public
%% or private.  The architecture of the \baculaenterprise{} cloud backup is new,
%% and it will provide the user with an array of features to keep
%% the cloud costs to a minimum and the performance to a maximum.
%% \smallskip

%% To provide this new Cloud functionality we have made significant changes
%% to the Storage daemon backend device architecture.  The new \bacula{} ersion
%% is more modular and permits the devices to select from multiple
%% drivers to handle different Cloud technologies.  For the first release we
%% have included an S3 driver that is compatible with a large number of S3
%% cloud services.  There is also a file driver that is useful for testing
%% the Cloud architecture, and could possibly be useful for a disk media
%% device that is very slow.

\subsubsection{Cloud Backup}
A major problem of Cloud backup is that data transmission to and from the
Cloud is very slow compared to traditional backup to disk or tape.
The \bacula{} Cloud drivers provide a means to quickly finish the backups
and then to transfer the data from the local cache to the Cloud in the
background.  This is done by first splitting the data Volumes into small
parts that are cached locally, and then uploading those parts to the Cloud
storage service in the background, either while the job continues to run or
after the backup job has terminated.  Once the parts are written to the
Cloud, they may either be left in the local cache for quick restores or
they can be removed (truncate cache).

\subsubsection{Cloud Volume Architecture}

\bimageH[0.8\linewidth]{nativeCloudStorage-diagram}{\bacula{} Cloud Architecture}{fig:cloud0}


The picture \vref{fig:cloud0} shows two Volumes (Volume0001 and Volume0002) with
their parts in the cache.  Below the cache, one can see that Volume0002 has be uploaded or synchronized with the Cloud.

\paragraph{Note}: Normal \bacula{} disk Volumes are implemented as standard files that reside in the user-defined \bdirectivename{Archive Directory}.  On the other hand, \bacula{} Cloud Volumes are directories that reside in the user-defined \bdirectivename{Archive Directory}.  The directory contains the cloud Volume parts, implemented as numbered files (\bfilename{part.1}, \bfilename{part.2}, \ldots{}).

\subsubsection{Cloud Restore}
During a restore, if the needed parts are available in the local cache, they will
immediately be used. Otherwise, they will be downloaded from cloud storage as
needed.  The restore starts with parts already in the local cache but
will wait as needed for any part that must be downloaded.  The download
proceeds while the restore is running.
\smallskip

With most cloud providers uploads are free of charge, but
downloads of data from the cloud are billed.  By using the local cache and
multiple small parts, Bacula can be configured to substantially reduce
download costs.
\smallskip

The \bdirectivename{Maximum File Size} Device directive is valid within the Storage
Daemon's cloud device configuration and defines the granularity of a restore chunk. In order to minimize the number of volume
parts to download during a restore (in particular when restoring single files), it
is useful to set the \bdirectivename{Maximum File Size} to a value smaller than or
equal to the configured \bdirectivename{Maximum Part Size}.

\subsubsection{Compatibility}
Since a Cloud Volume contains the same data an ordinary \bacula{} Volume does, all existing types of \bacula{} data may be stored in the cloud -- that is client data encryption, client-side compreion, plugin usage are all available. In fact, all existing \bacula{} functionality, with the exception of deduplication, is compatible with the \bacula{} Cloud drivers.

\subsubsection{Deduplication and the Cloud}
At the current time, \bacula{} Global Endpoint Backup does not support
writing to the cloud because cloud storage would be too slow to support large
hashed and indexed containers of deduplication data.


\subsubsection{Virtual Autochangers and Disk Autochangers}
\bacula{} Virtual Autochangers are compatible with
the \bacula{} Cloud drivers.  However, if you use a third party disk
autochanger script such as \textbf{Vchanger}, unless or until it is
modified to handle Volume directories, it may not be compatible with \bacula{}
Cloud drivers.

\subsubsection{Security}
All data that is sent to and received from the cloud by default uses
the \acs{HTTPS} protocol, so data is encrypted while being transmitted
and received.  However, data that resides in the cloud is not encrypted by
default.  If that extra security of backed up data is required,
\bacula{}'s \acs{PKI} data encryption feature should be used
during the backup.

\subsection{New Commands, Resource, and Directives for Cloud}
To support Cloud storage devices, some new \bconsolename{} commands,
new Storage Daemon directives, and a new Cloud resource that is
referenced in the Storage Daemon's Device resource are available as of
\baculaenterprise{} 8.8

\subsubsection{Cache and Pruning}

The Cache is treated much like a normal Disk based backup, so that in
configuring Cloud the administrator should take care to set \bdirectivename{Archive Device} in
the \bDevice{} resource to a dirctory that would also be suitable for storing backup data.
Obviously, unless the \bcommandname{truncate}/\bcommandname{prune} cache
commands are used, the \bdirectivename{Archive Device} will continue to fill.
\smallskip

The cache retention can be controlled per Volume with the \bdirectivename{Cache Retention}
attribute. The default value is \bdefaultvalue{0}, meaning that pruning of the cache is
disabled.
\smallskip

The \bdirectivename{Cache Retention} value for a volume can be modified with the \bcommandname{update}
command, or configured via the \bPool{} directive \bdirectivename{Cache Retention} for newly created volumes.


\subsubsection{New Cloud \bconsole{} Commands}
\begin{bitemize}
\item \bcommandname{truncate cache} 
\item \bcommandname{upload}
\item \bcommandname{cloud} The new cloud \bconsole{} command allows inspecting and manipulating cloud volumes in different ways. The options are the following:
  \begin{bitemize}
  \item None.  If you specify no arguments to the command, \bconsole{} will prompt with:
    \begin{bVerbatim}
      Cloud choice: 
      1: List Cloud Volumes in the Cloud
      2: Upload a Volume to the Cloud
      3: Prune the Cloud Cache
      4: Truncate a Volume Cache
      5: Done
      Select action to perform on Cloud (1-5):
    \end{bVerbatim}
    The different choices should be rather obvious.
    
  \item \textbf{truncate} This command will attempt to truncate the local cache for the specified Volume. \bacula{} will prompt you for the information needed to determine the Volume name or names.  To avoid   the prompts, the following additional command line options may be specified:
    \begin{bitemize}
    \item {Storage=xxx}
    \item {Volume=xxx}
    \item {AllPools}
    \item {AllFromPool}
    \item {Pool=xxx}
    \item {Storage=xxx}
    \item {MediaType=xxx}
    \item {Drive=xxx}
    \item {Slots=nnn}
    \end{bitemize}
  \item \bcommandname{prune} This command will attempt to prune the local cache for
     the specified Volume. \bacula{} will respect the \bdirectivename{Cache Retention} volume
     attribute to determine if the cache can be truncated or not. Only parts
     that are uploaded to the cloud will be deleted from the cache.  \bacula{}
     will prompt you for the information needed to determine the Volume name or
     names.  To avoid the prompts, the following additional command line
     options may be specified:
     \begin{bitemize}
     \item {Storage=xxx}
     \item {Volume=xxx}
     \item {AllPools}
     \item {AllFromPool}
     \item {Pool=xxx}
     \item {Storage=xxx}
     \item {MediaType=xxx}
     \item {Drive=xxx}
     \item {Slots=nnn}
     \end{bitemize}
   \item \bcommandname{upload} This command will attempt to upload the specified Volumes. It will prompt for the information needed to determine the Volume name or names. To avoid the prompts, any of the following additional command line options can be specified:
    \begin{bitemize}
    \item {Storage=xxx}
    \item {Volume=xxx}
    \item {AllPools}
    \item \textbf{AllFromPool}
    \item {Storage=xxx}
    \item {Pool=xxx}
    \item {MediaType=xxx}
    \item {Drive=xxx}
    \item {Slots=nnn}
    \end{bitemize}
  \item \bcommandname{list} This command will list volumes stored in the Cloud. If a
     volume name is specified, the command will list all parts for the given volume.
     To avoid the prompts, the operator may specify any of the following additional command line options:
     \begin{bitemize}
     \item {Storage=xxx}
     \item {Volume=xxx}
     \item {Storage=xxx}
     \end{bitemize}
  \end{bitemize}
\end{bitemize}

\subsubsection{Cloud Additions to the DIR \bPool{} Resource}
In \bfilename{bacula-dir.conf} \bresourcename{Pool} resources,
the directive \bkeyword{Cache Retention} can be specified. It is only effective for cloud storage backed volumes, and ignored when used with volumes stored on any different storage device.

 
\subsubsection{Cloud Additions to the SD Device Resource}
\bDevice{} resource configured in  the \bfilename{bacula-sd.conf} file can use
the \bkeyword{Cloud} keyword in the \bdirectivename{Device Type} directive,
and the two directives \bdirectivename{Maximum Part Size} and \bdirectivename{Cloud}.
%%% duplicated information here, but not really bad... I leave it as is, for now. ARL

\subsubsection{New Cloud SD Device Directives}
\begin{bdescription}
\item [Device Type] The Device Type has been extended to include the new keyword \bkeyword{Cloud} to specify that the device supports cloud Volumes. Example:
  \begin{bVerbatim}
    Device Type = Cloud
  \end{bVerbatim}
\item [Cloud] The new Cloud directive references a \bresourcename{Cloud} Resource. As with other \bacula{} resource references, the name of the \bresourcename{Cloud} resource is used as the value.  Example:
  \begin{bVerbatim}
    Cloud = S3Cloud
  \end{bVerbatim}
\item [Maximum Part Size] This directive allows specification of the maximum size for each part of any volume written by the current device.  Smaller part sizes will reduce restore costs, but will cause additional but small overhead to handle multiple parts.
  The maximum number of parts permitted per Cloud Volume is~524,288.  The maximum size of any given part is approximately~17.5\,TB.
\end{bdescription}

\subsubsection{Example Cloud Device Specification}
An example of a Cloud Device Resource might be:

\begin{bVerbatim}
Device {
  Name = CloudStorage
  Device Type = Cloud
  Cloud = S3Cloud
  Archive Device = /opt/bacula/backups
  Maximum Part Size = 10000000
  Media Type = File
  LabelMedia = yes
  Random Access = Yes;
  AutomaticMount = yes
  RemovableMedia = no
  AlwaysOpen = no
}
\end{bVerbatim}
 
As can be seen above, the \bdirectivename{Cloud} directive in the \bresourcename{Device} resource contains the name (\textbf{S3Cloud}), which references the \bresourcename{Cloud} resource that is shown below.
\smallskip

Note also that the \bdirectivename{Archive Device} is specified in the
same manner as used for a File device, i.\,e. by indicating a
directory name. However, in place of containing regular files as
Volumes, the archive device for the Cloud drivers will contain the
local cache, which consists a directory per Volume, and
these directories contain the parts associated with the particular
Volume.  So with the above \bresourcename{Device} resource, and the
two cached Volumes shown in figure \vref{fig:cloud0} above,
the following layout on disk would result:
 
\begin{bVerbatim}
  /opt/bacula/backups
      /opt/bacula/backups/Volume0001
          /opt/bacula/backups/Volume0001/part.1
          /opt/bacula/backups/Volume0001/part.2
          /opt/bacula/backups/Volume0001/part.3
          /opt/bacula/backups/Volume0001/part.4
      /opt/bacula/backups/Volume0002
          /opt/bacula/backups/Volume0002/part.1
          /opt/bacula/backups/Volume0002/part.2
          /opt/bacula/backups/Volume0002/part.3
\end{bVerbatim}
 
% A good part of this section is also in storedconf.tex and the whitepaper
% Please, keep everything synchronized
% Probably a good idea to ensure we have only one location of the
% relevant documentation... for now, I just don't touch the below
% part. ARL
\subsubsection{The Cloud Resource}
The \bresourcename{Cloud} resource has a number of directives that may be specified as exemplified in the following example:

Default east USA location:
\begin{bVerbatim}
Cloud {
  Name = S3Cloud
  Driver = "S3"
  HostName = "s3.amazonaws.com"
  BucketName = "BaculaVolumes"
  AccessKey = "BZIXAIS39DP9YNER5DFZ"
  SecretKey = "beesheeg7iTe0Gaexee7aedie4aWohfuewohGaa0"
  Protocol = HTTPS
  URIStyle = VirtualHost
  Truncate Cache = No
  Upload = EachPart
  Region = ``us-east-1"
  Maximum Upload Bandwidth = 5MB/s
}
\end{bVerbatim}

For central europe location:
\begin{bVerbatim}
Cloud { 
  Name = S3Cloud 
  Driver = "S3" 
  HostName = "s3-eu-central-1.amazonaws.com" 
  BucketName = "BaculaVolumes" 
  AccessKey = "BZIXAIS39DP9YNER5DFZ" 
  SecretKey = "beesheeg7iTe0Gaexee7aedie4aWohfuewohGaa0" 
  Protocol = HTTPS 
  UriStyle = VirtualHost 
  Truncate Cache = No 
  Upload = EachPart 
  Region = "eu-central-1" 
  Maximum Upload Bandwidth = 4MB/s 
}
\end{bVerbatim} 
 
For Amazon Cloud, refer to \url{http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region}
to get a complete list of regions and corresponding endpoints and use them respectively as
\bdirectivename{Region} and \bdirectivename{HostName} directives.

or in the following example for CEPH S3 interface:
\begin{bVerbatim}
  Cloud {
    Name = CEPH_S3
    Driver = "S3"
    HostName = ceph.mydomain.lan
    BucketName = "CEPHBucket"
    AccessKey = "xxxXXXxxxx"
    SecretKey = "xxheeg7iTe0Gaexee7aedie4aWohfuewohxx0"
    Protocol = HTTPS
    Upload = EachPart
    UriStyle = Path            # Must be set for CEPH
  }
\end{bVerbatim}

For Azure:                                                                                                                                                                                                        
\begin{bVerbatim} 
Cloud {
  Name = MyCloud 
  Driver = "Azure" 
  HostName = "MyCloud" #not used but needs to be specified 
  BucketName = "baculaAzureContainerName" 
  AccessKey = "baculaaccess" 
  SecretKey = "/Csw1SECRETUmZkfQ==" 
  Protocol = HTTPS 
  UriStyle = Path 
}
\end{bVerbatim}

The directives of the above \bresourcename{Cloud} resource for the S3 driver are defined as follows:

\begin{bdescription}

  \bwebdoc{Storage:Cloud:Name}
\item [Name = \bbracket{Device-Name}]
  \index[sd]{Name}
  \index[sd]{Directive!Name}
  The name of the Cloud resource. This is the logical Cloud name, and may be
  any string up to 127 characters in length. Shown as \textbf{S3Cloud} above.
  
  \bwebdoc{Storage:Cloud:Description}
\item [Description = \bbracket{Text}]
  The description is used for display purposes as is the case with all resources.
  
  \bwebdoc{Storage:Cloud:Driver}
\item [Driver = \bbracket{Driver-Name}]
  \index[sd]{Driver}
  
  This defines which driver to use. At the moment, the only Cloud driver that
  is implemented is \bvalue{S3}. There is also a \bvalue{File} driver, which is
  used mostly for testing.

  
  \bwebdoc{Storage:Cloud:HostName}
\item [Host Name = \bbracket{Name}]
  \index[sd]{HostName}
  This directive specifies the hostname to be used in the \acs{URL}. Each Cloud
  service provider has a different and unique hostname.  The maximum size is
  \bvalue{255} characters and may contain a \acs{TCP} port specification.

  \bwebdoc{Storage:Cloud:BucketName}
\item [Bucket Name = \bbracket{Name}]
  \index[sd]{BucketName}

  This directive specifies the bucket name that you wish to use on the Cloud
 service. This name is normally a unique name that identifies where 
 you want to place your Cloud Volume parts.
 With Amazon S3, the bucket must be created previously on the Cloud service. 
 With Azure Storage, it is generaly refered as Container and it can be created 
 automatically by Bacula when it does not exist. 
 The maximum bucket name size is \bvalue{255} characters. 

  \bwebdoc{Storage:Cloud:AccessKey}
\item [Access Key = \bbracket{String}]
  \index[sd]{AccessKey}

  The access key is your unique user identifier given to you by your cloud
  service provider.

  \bwebdoc{Storage:Cloud:SecretKey}
\item [Secret Key = \bbracket{String}]
  \index[sd]{SecretKey}
  The secret key is the security key that was given to you by your cloud
  service provider. It is equivalent to a password.

  \bwebdoc{Storage:Cloud:Protocol}
\item [Protocol =  \bbracket{HTTP | HTTPS}]
  \index[sd]{Protocol}

  The protocol defines the communications protocol to use with the cloud
  service provider. The two protocols currently supported are: \acs{HTTPS}
  and \acs{HTTP}. The default is \acs{HTTPS}.

  \bwebdoc{Storage:Cloud:UriStyle}
\item [Uri Style = \bbracket{VirtualHost | Path}]
  \index[sd]{UriStyle}

  This directive specifies the \acs{URI} style to use to communicate with the cloud
  service provider.  The two \bdirectivename{Uri Styles} currently supported are:
  \bvalue{VirtualHost} and \bvalue{Path}.  The default is \bdefaultvalue{VirtualHost}.

\bwebdoc{Storage:Cloud:TruncateCache}
\item [Truncate Cache = \bbracket{truncate-kw}]
  \index[sd]{TruncateCache}

  This directive specifies when \bacula{} should automatically remove (truncate)
  the local cache parts.  Local cache parts can only be removed if they have
  been uploaded to the cloud.  The currently implemented values are:
  \begin{bdescription}
   \item [ No ] Do not remove cache. With this option you must manually delete the cache parts with a \bconsolename{} \bcommandname{truncate cache}, or do so with an \textbf{Admin} Job that runs an \bcommandname{truncate cache} command.  This is the default.
  \item [ AfterUpload ] Each part will be removed just after it is uploaded.  Note, if this option is specified, all restores will require a download from the Cloud\footnote{Not yet implemented}.
  \item [ AtEndOfJob ] With this option, at the end of the Job, every part that has been uploaded to the Cloud will be removed\footnote{Not yet implemented} (truncated).
  \end{bdescription}

  \bwebdoc{Storage:Cloud:Upload}
\item [Upload = \bbracket{upload-kw}]

  This directive specifies when local cache parts
 will be uploaded to the Cloud.  The options are:

  \begin{bdescription}
  \item [No] Do not upload cache parts. With this option you must  manually upload the cache parts with a \bconsole{} \bcommandname{upload} command, or do so with an \textbf{Admin} Job that runs an \bcommandname{upload} command.  This is the default.
  \item [EachPart] With this option, each part will be uploaded when it is complete i.e. when the next part is created or at the end of the Job.
  \item [AtEndOfJob] With this option all parts that have not been previously uploaded will be uploaded at the end of the Job.\footnote{Not yet implemented}
  \end{bdescription}
  \bwebdoc{Storage:Cloud:MaximumConcurrentUploads}
\item [Maximum Concurrent Uploads = \bbracket{number}]
  The default is \bdefaultvalue{3}, but by using this directive, you may set it to any value you want.

  \bwebdoc{Storage:Cloud:MaximumConcurrentDownloads}
\item [Maximum Concurrent Downloads = \bbracket{number}]
 The default is \bdefaultvalue{3}, but by using this directive, you may set it to any value
 you want.

 \bwebdoc{Storage:Cloud:MaximumUploadBandwidth}
\item [Maximum Upload Bandwidth = \bbracket{speed}]

  The default is \bdefaultvalue{unlimited}, but by using this directive, you may limit the
  upload bandwidth used globally by all devices referencing this \bresourcename{Cloud}
  resource.

\bwebdoc{Storage:Cloud:MaximumDownloadBandwidth}
\item [Maximum Download Bandwidth = \bbracket{speed}]

  The default is \bdefaultvalue{unlimited}, but by using this directive, you may limit the
  download bandwidth used globally by all devices referencing this Cloud
  resource.
  
  \bwebdoc{Storage:Cloud:Region}
\item [Region = \bbracket{String}]
  The Cloud resource can be configured to use a specific endpoint within a
  region. This directive is required for AWS-V4 regions. ex:
  \bvalueddirective{Region}{"eu-central-1"}

  \bwebdoc{Storage:Cloud:BlobEndpoint} 
\item [BlobEndpoint = \bbracket{String}] 
  This resource can be used to specify a custom \acs{URL}
  for Azure Blob (see \url{https://docs.microsoft.com/en-us/azure/storage/blobs/storage-custom-domain-name}). 
 
\bwebdoc{Storage:Cloud:EndpointSuffix} 
\item [EndpointSuffix = \bbracket{String}] 
  Use this resource to specify a custom \acs{URL} postfix for Azure. ex: 
 \bvalueddirective{EnbpointSuffix}{"core.chinacloudapi.cn"} 
\end{bdescription}
 
\subsubsection{File Driver for the Cloud}
As mentioned above, one may specify the keyword \bkeyword{File} on the \bdirectivename{Driver} directive of the Cloud resource.  Instead of writing to the Cloud, \bacula{} will instead create a Cloud Volume but write it to disk. The rest of this section applies to the \bresourcename{Cloud} resource directives when the File driver is specified.
\smallskip

The following Cloud directives are ignored: \bdirectivename{Bucket Name}, \bdirectivename{Access Key}, \bdirectivename{Secret Key}, \bdirectivename{Protocol}, \bdirectivename{URI Style}.
The directives \bdirectivename{Truncate Cache} and \bdirectivename{Upload} work on the local cache in the same manner as they do for the S3 driver.
\smallskip

The main difference to note is that the \bdirectivename{Host Name}, specifies the destination directory for the Cloud Volume files, and this \bdirectivename{Host Name} must be different from the \bdirectivename{Archive Device} name, or there will be a conflict between the local cache (in the Archive Device directory) and the destination Cloud Volumes (in the Host Name directory).
\smallskip

As noted above, the File driver is mostly used for testing purposes, and we do not particularly recommend using it.  However, if you have a particularly slow backup device you might want to stage your backup data into an \acs{SSD} or disk using the local cache feature of the Cloud device, and have your Volumes transferred in the background to a slow File device.


%%% ARL: cloud parts above not touched--for now :-)

\subsection{Progressive Virtual Full}
Instead of the implementation of Perpetual Virtual Full backups with a
Perl script which needs to be run regularly, with \baculaenterprise{}
version~8.8.0, a new job directive named \bdirectivename{Backups To Keep}
has been added. This permits implementation of Progressive
Virtual Fulls fully within \bacula{} itself.


\bimageH[.8\linewidth]{pvf-slidingbackups}{Backup Sequence Slides Forward One Day, Each Day}{fig:slidingbackups}
\smallskip

To use the Progressive Virtual Full feature, the
\bdirectivename{Backups To Keep} directive is added to a \bJob{}
resource. The value specified for the directive indicates the number of backup jobs that should not be merged into the Virtual Full. The default is zero and behaves the same way the prior script \btool{pvf} worked.

\subsection{Backups To Keep Directive}
The new \bdirectivename{BackupsToKeep} directive is specified in the \bJob{} Resource and has the form:

\begin{bVerbatim}
  Backups To Keep = 30
\end{bVerbatim}

where the value (\bvalue{30} in the figure \vref{fig:slidingbackups} )
is the number of backups to retain.  When this directive is present
during a Virtual Full (it is ignored for any other Job types),
\bacula{} will check if the latest Full backup has more subsequent
backups than the value specified.  In the above example, the Job would
simply terminate unless there is a Full back followed by at least 31
backups of either Differential or Incremental level.
\smallskip

Assuming that the latest Full backup is followed by 32 Incremental
backups, a Virtual Full will be run that consolidates the Full with
the first two Incrementals that were run after the Full backup.  The
result is a Full backup followed by 30 Incremental ones. The \bJob{}
Resource in \bfilename{bacula-dir.conf} to accomplish this would be:

\begin{bVerbatim}
  Job {
    Name = "VFull"
    Type = Backup
    Level = VirtualFull
    Client = "my-fd"
    File Set = "FullSet"
    Accurate = Yes
    Backups To Keep = 10
  }
\end{bVerbatim}

\subsection{Delete Consolidated Jobs}
The additional directive \bdirectivename{Delete Consolidated Jobs}
expects a \byesorno{} value that, if set to \bvalue{yes}, will cause
any old Job that is consolidated during a Virtual Full to be
deleted. In the example above we saw that a Full plus one other job
(either an Incremental or Differential) were consolidated into a new
Full backup. The original Full and the other Job consolidated would be
deleted if this directive were set to \bvalue{yes}. The default value
is \bdefaultvalue{no}.

 
\subsection{Virtual Full Compatibility}
Virtual Full as well as Progressive Virtual Full backups work with any standard backup Job including Jobs that use the Global Endpoint Deduplication.
\smallskip

However, it should be noted that Virtual Full jobs are \textbf{not compatible} with
Windows backups using \acs{VSS} writers (mostly plugins),
nor are they compatible with a number of non-Windows \baculasystems{} plugins.
Please contact \baculasystems{} Support team for more details Virtual Full
compatibility.

\subsection{TapeAlert Enhancements}
There are some significant enhancements to the TapeAlert feature of \bacula{}. Several directives are used slightly differently, and there is a minor compatibility problem with the old TapeAlert implementation.
 
\subsection{What is New}
First, the \bdirectivename{Alert Command} directive needs to be added
in the Device resource that calls the new \btool{tapealert} script
that is installed in the scripts directory (normally:
\bdirectoryname{/opt/bacula/scripts}):

\begin{bVerbatim}
  Device {
    Name = ...
    Archive Device = /dev/nst0
    Alert Command = "/opt/bacula/scripts/tapealert %l" 
    Control Device = /dev/sg1 # must be SCSI ctl for Archive Device
    ...
  }
\end{bVerbatim}

The \bdirectivename{Control Device} directive in the Storage Daemon's configuration was previously used only for the \acs{SAN} Shared Storage feature. With \bacula{} version~8.8, it is also used for the TapeAlert command to permit \bacula{} to detect tape alerts on a specific device (normally only tape devices).

Once the above mentioned two directives (\bdirectivename{Alert Command} and \bdirectivename{Control Device}) are in place in all \bDevice{} resources, \bacula{} will check for tape alerts at two points:
\begin{bitemize}
\item After the Drive is used and it becomes idle.
\item After each read or write error on the drive.
\end{bitemize}

At each of the above times, \bacula{} will call the new \btool{tapealert} script, which uses the \btool{tapeinfo} program.  The \btool{tapeinfo} utility is part of the \btool{apt} \bfilename{sg3-utils} and \btool{rpm} \bfilename{sg3\_utils} packages.  Then for each tape alert that \bacula{} finds for that drive, it will emit a Job message that is either INFO, WARNING, or FATAL depending on the designation in the Tape Alert published by the \bfootref{T10 Technical Committee on \acs{SCSI} Storage Interfaces}{https://www.t10.org}.  For the specification, please see: \url{http://www.t10.org/ftp/t10/document.02/02-142r0.pdf}
\smallskip

As a somewhat extreme example, if tape alerts~3, 5, and~39 are set, you will get the following output in your backup job:

\begin{bVerbatim}
  17-Nov 13:37 rufus-sd JobId 1: Error: block.c:287
  Write error at 0:17 on device "tape"
  (/home/kern/bacula/k/regress/working/ach/drive0)
  Vol=TestVolume001.  ERR=Input/output error.
  
  17-Nov 13:37 rufus-sd JobId 1: Fatal error: Alert:
  Volume="TestVolume001" alert=3: ERR=The operation has stopped because
  an error has occurred while reading or writing data which the drive
  cannot correct.  The drive had a hard read or write error
  
  17-Nov 13:37 rufus-sd JobId 1: Fatal error: Alert:
  Volume="TestVolume001" alert=5: ERR=The tape is damaged or the drive
  is faulty.  Call the tape drive supplier helpline.  The drive can no
  longer read data from the tape
  
  17-Nov 13:37 rufus-sd JobId 1: Warning: Disabled Device "tape"
  (/home/kern/bacula/k/regress/working/ach/drive0) due to tape alert=39.
  
  17-Nov 13:37 rufus-sd JobId 1: Warning: Alert: Volume="TestVolume001"
  alert=39: ERR=The tape drive may have a fault.  Check for availability
  of diagnostic information and run extended diagnostics if applicable.
  The drive may have had a failure which may be identified by stored
  diagnostic information or by running extended diagnostics (eg Send
  Diagnostic).  Check the tape drive users manual for instructions on
  running extended diagnostic tests and retrieving diagnostic data.
\end{bVerbatim}

Without the tape alert feature enabled, you would only get the first error message above, which is the error \bacula{} received. Notice also, in this case the alert number 5 is a critical error, which causes two things to happen: First, the tape drive is disabled, and second, the Job is failed.
\smallskip

If you attempt to run another Job using  the Device that has been disabled, you will get a message similar to the following:
\begin{bVerbatim}
17-Nov 15:08 rufus-sd JobId 2: Warning: 
     Device "tape" requested by DIR is disabled.
\end{bVerbatim}

and the Job may be failed if no other usable drive can be found.
\smallskip

Once the problem with the tape drive has been corrected, you can clear the tape alerts and re-enable the device with the \bacula{} \bconsole{} command such as the following:

\begin{bVerbatim}
  enable Storage=Tape
\end{bVerbatim}

Note, when you enable the device, the list of prior tape alerts for that drive will be discarded.
\smallskip

Since is is possible to miss tape alerts, \bacula{} maintains a temporary list of the last 8~alerts, and each time \bacula{} calls the \btool{tapealert} script, it will keep up to 10~alert status codes. Normally there will only be one or two alert errors for each call to the \btool{tapealert} script.
\smallskip

Once a drive has one or more tape alerts, they can be inspected by using the \bconsole{} status command as follows:
\begin{bVerbatim}
status storage=Tape 
\end{bVerbatim}
which produces the following output:
\begin{bVerbatim}
Device Vtape is "tape" (/home/kern/bacula/k/regress/working/ach/drive0)
mounted with:
    Volume:      TestVolume001
    Pool:        Default
    Media type:  tape
    Device is disabled. User command.
    Total Bytes Read=0 Blocks Read=1 Bytes/block=0
    Positioned at File=1 Block=0
    Critical Alert: at 17-Nov-2016 15:08:01 Volume="TestVolume001"
       alert=Hard Error
    Critical Alert: at 17-Nov-2016 15:08:01 Volume="TestVolume001"
       alert=Read Failure
    Warning Alert: at 17-Nov-2016 15:08:01 Volume="TestVolume001"
       alert=Diagnostics Required
\end{bVerbatim}
If you want to see the long message associated with each of the alerts, simply set the debug level to 10 or more and re-issue the status command:
\begin{bVerbatim}
setdebug storage=Tape level=10
status storage=Tape
\end{bVerbatim}
\begin{bVerbatim}
    ...
    Critical Alert: at 17-Nov-2016 15:08:01 Volume="TestVolume001"
      flags=0x0 alert=The operation has stopped because an error has occurred
       while reading or writing data which the drive cannot correct. The drive had
       a hard read or write error
    Critical Alert: at 17-Nov-2016 15:08:01 Volume="TestVolume001"
       flags=0x0 alert=The tape is damaged or the drive is faulty. Call the tape
       drive supplier helpline.  The drive can no longer read data from the tape
    Warning Alert: at 17-Nov-2016 15:08:01 Volume="TestVolume001" flags=0x1
       alert=The tape drive may have a fault. Check for availability of diagnostic
       information and run extended diagnostics if applicable.   The drive may
       have had a failure which may be identified by stored diagnostic information
       or by running extended diagnostics (eg Send Diagnostic). Check the tape
       drive users manual for instructions on running extended diagnostic tests
       and retrieving diagnostic data.
    ...
\end{bVerbatim}
The next time you \textbf{enable} the Device by either using \bconsole{} or you restart the Storage Daemon, all the saved alert messages will be discarded.

\subsection{Handling of Alerts}
Tape Alerts numbered 7, 8, 13, 14, 20, 22, 52, 53, and~54 will cause \bacula{} to disable the current Volume.
\smallskip

Tape Alerts numbered 14, 20, 29, 30, 31, 38, and~39 will cause \bacula{} to disable the drive.
\smallskip

Please note certain tape alerts such as 14 have multiple effects (disable the Volume and disable the drive).


\subsection{Multi-Tenancy Enhancements}
%%% patch 29

\subsection{New BWeb Management Suite Self User Restore}
The BWeb Management Suite can be configured to allow authorized users
to restore their own files on their own Unix or Linux system through BWeb. More information can be found in the BWeb Management Suite user's guide.

\subsection{New Console ACL Directives}
By default, if a \bdirectivename{Console ACL} directive is not set, \bacula{} will assume that the
\acs{ACL} list is empty. If the current \bacula{} \bdirectorname{} configuration uses restricted
Consoles and allows restore jobs, it is mandatory to configure the new
directives.

\subsubsection{Directory ACL}
\index[dir]{Directive!Directory ACL}

This directive is used to specify a list of directories that can be accessed by
a restore session.  Without this directive, the console cannot restore any
file.  Multiple directories names may be specified by separating them with
commas, and/or by specifying multiple \bdirectivename{DirectoryACL} directives.  For example,
the directive may be specified as:

\begin{bVerbatim}
  DirectoryACL = /home/bacula/, "/etc/", "/home/test/*"
\end{bVerbatim}

With the above specification, the console can access the following files:
\begin{bitemize}
\item \bfilename{/etc/password}
\item \bfilename{/etc/group}
\item \bfilename{/home/bacula/.bashrc}
\item \bfilename{/home/test/.ssh/config}
\item \bfilename{/home/test/Desktop/Images/something.png}
\end{bitemize}

But not the following files or directories:
\begin{bitemize}
\item \bfilename{/etc/security/limits.conf}
\item \bfilename{/home/bacula/.ssh/id\_dsa.pub}
\item \bdirectoryname{/home/guest/something}
\item \bfilename{/usr/bin/make}
\end{bitemize}

If a directory starts with a Windows pattern (ex: \bdirectoryname{c:/}), \bacula{} will
automatically ignore the case when checking directories.

\subsubsection{UserId ACL}
\index[dir]{Directive!UserId ACL}

This directive is used to specify a list of UID/GID that can be accessed from a
restore session.  Without this directive, the console cannot restore any
file. During the restore session, the Director will compute the restore list
and will exclude files and directories that cannot be accessed. \bacula{} uses the
LStat database field to retrieve \texttt{st\_mode}, \texttt{st\_uid} and
\texttt{st\_gid} information for each file and compare them with the
\bdirectivename{UserId ACL}
elements. If a parent directory doesn't have a proper catalog entry, access
to this directory will be automatically granted.

UID/GID names are resolved with \texttt{getpwnam()} function within the Director. The
UID/GID mapping might be different from one system to an other.

Windows systems are \textbf{not compatible} with the \bdirectivename{UserId ACL} feature. The use of
\bdirectivename{UserId ACL = *all*} is required to restore Windows systems from a
restricted Console.

Multiple UID/GID names may be specified by separating them with
commas, and/or by specifying multiple \bdirectivename{UserId ACL} directives.  For example,
the directive may be specified as:

\begin{bVerbatim}
  UserIdACL = "bacula", "100", "100:100", ":100", "bacula:bacula"
\end{bVerbatim}

\begin{bVerbatim}
# ls -l /home
total 28
drwx------ 45 bacula bacula 12288 Oct 24 17:05 bacula
drwx------ 45 test   test   12288 Oct 24 17:05 test
drwx--x--x 45 test2  test2  12288 Oct 24 17:05 test2
drwx------  2 root   root   16384 Aug 30 14:57 backup
-rwxr--r--  1 root   root   1024  Aug 30 14:57 afile
\end{bVerbatim}

In the example above, if the uid of the user test is 100, the following files
will be accessible:
\begin{bitemize}
\item \bfilename{bacula/*}
\item \bfilename{test/*}
\item \bfilename{test2/*}
%ARL: I don't think so... \item \bfilename{afile}
\end{bitemize}

The directory \bdirectoryname{backup} will not be accessible.

\subsection{Restore Job Security Enhancement}

The \bconsole{} \bcommandname{restore} command can now accept the new \texttt{jobuser=}
and \texttt{jobgroup=} parameters to restrict the restore process to a given
user account. Files and directories created during the restore session will
be restricted.

\begin{bVerbatim}
* restore jobuser=joe jobgroup=users
\end{bVerbatim}

The Restore Job restriction can be used on Linux and on FreeBSD. If the restore
Client OS doesn't support the needed thread-level user impersonation, the restore job will be
aborted.

\subsection{New Bconsole ``list''  Command Behavior}

The \bconsole{} \bcommandname{list} commands can now be used safely from a restricted bconsole
session. The information displayed will respect the \acs{ACL} configured for the Console session.
For example, if a Console has access to JobA, JobB and JobC, information about JobD will
not appear in the \bcommandname{list jobs} command.
%% --- patch 27

\section{\baculaenterprise{} 8.6.3}

\subsection{New Console \acs{ACL} Directives}
\index[dir]{Directive!BackupClientACL}
It is now possible to configure a restricted Console to distinguish Backup and
Restore jobs permissions. The \bdirectivename{Backup Client ACL} can restrict backup jobs
on a specific set of clients, while the \bdirectivename{Restore Client
  ACL} can restrict
restore jobs.

\begin{bVerbatim}
# cat /opt/bacula/etc/bacula-dir.conf
...

Console {
 Name = fd-cons             # Name of the FD Console
 Password = yyy
...
 ClientACL = localhost-fd           # everything allowed
 RestoreClientACL = test-fd         # restore only
 BackupClientACL = production-fd    # backup only
}
\end{bVerbatim}

The \bdirectivename{Client ACL} directive takes precedence over the
\bdirectivename{Restore Client ACL} and the \bdirectivename{Backup
  Client ACL} settings. In the \bresourcename{Console}
resource above, this means that the bconsole linked to the \bresourcename{Console}
named ``fd-cons'' will be able to run:

\begin{bitemize}
\item backup and restore for ``localhost-fd''
\item backup for ``production-fd''
\item restore for ``test-fd''
\end{bitemize}

At restore time, jobs for client ``localhost-fd'', ``test-fd'' and
``production-fd'' will be available.

If \bvalue{*all*} is set for \bdirectivename{Client ACL}, backup and restore will be
allowed for all clients, despite the use of \bdirectivename{Restore
  Client ACL} or
\bdirectivename{Backup Client ACL}.


\section{Bacula Enterprise 8.6.0}

\subsection{Client Initiated Backup}
\label{blb:sec:featurecib}

A console program such as the new \bcommandname{tray-monitor} or \bconsolename{}
can now be configured to connect a \bresourcename{File Daemon}.  There
are many new features available (see the New Tray Monitor \vref{blb:sec:tray-monitor}),
but probably the most important one is the ability for the user to initiate a
backup of her own machine.  The connection established by the FD to the
\bDirector{} for the backup can be used by the \bDirector{} for the backup, thus
not only can clients (users) initiate backups, but a \bresourcename{File Daemon} that is
\acs{NAT}ed (cannot be reached by the Director) can now be backed up without
using advanced tunneling techniques.
\smallskip

The flow of information is shown in the picture \vref{fig:nat3}

\bimageH{nat}{Client Initiated Backup Network Flow}{fig:nat3}

\newpage
\subsection{Configuring Client Initiated Backup}
In order to ensure security, there are a number of new directives
that must be enabled in the new \texttt{tray-monitor}, the File Daemon and in the Director.
A typical configuration might look like the following:

\begin{bVerbatim}
# cat /opt/bacula/etc/bacula-dir.conf
...

Console {
        Name = fd-cons             # Name of the FD Console
        Password = yyy

        # These commands are used by the tray-monitor, it is possible to restrict
        CommandACL = run, restore, wait, .status, .jobs, .clients
        CommandACL = .storages, .pools, .filesets, .defaults, .info
        
        # Adapt for your needs
        jobacl = *all*
        poolacl = *all*
        clientacl = *all*
        storageacl = *all*
        catalogacl = *all*
        filesetacl = *all*
}
\end{bVerbatim}

\begin{bVerbatim}
# cat /opt/bacula/etc/bacula-fd.conf
...

Console {              # Console to connect the Director
        Name = fd-cons
        DIRPort = 9101
        address = localhost
        Password = "yyy"
        }

Director {
         Name = remote-cons   # Name of the tray monitor/bconsole
         Password = "xxx"     # Password of the tray monitor/bconsole
         Remote = yes         # Allow to use send commands to the Console defined
}
\end{bVerbatim}

\begin{bVerbatim}
cat /opt/bacula/etc/bconsole-remote.conf
....

Director {
         Name = localhost-fd
         address = localhost        # Specify the FD address
         DIRport = 9102             # Specify the FD Port
         Password = "notused"
}

Console {
        Name = remote-cons         # Name used in the auth process
        Password = "xxx"
}
\end{bVerbatim}

\begin{bVerbatim}
cat ~/.bacula-tray-monitor.conf
  
Monitor {
        Name = remote-cons
}

Client {
       Name = localhost-fd
       address = localhost     # Specify the FD address
       Port = 9102             # Specify the FD Port
       Password = "xxx"
       Remote = yes
}
\end{bVerbatim}

\bimageH{conf-nat}{Relation Between Resources (bconsole)}{fig:nat}

\bimageH{conf-nat2}{Relation Between Resources (tray-monitor)}{fig:nat2}

A more detailed description with complete examples is available in the \bilink{Tray monitor chapter}{blb:TrayMonitorChapter} of this manual.

\subsection{New Tray Monitor}
\label{blb:sec:tray-monitor}

A new tray monitor has been added to the 8.6~release, which offers
the following features:

\begin{bitemize}
\item Director, File and Storage Daemon status page
\item Support for the Client Initiated Backup protocol (See \vref{blb:sec:featurecib}). To use the Client Initiated Backup option from the
tray monitor, the Client option ``Remote'' should be checked in the
configuration (Fig.~\vref{fig:tray2}).
\item Wizard to run new job (Fig.~\vref{fig:tray4})
\item Display an estimation of the number of files and the size of the next backup job (Fig.~\vref{fig:tray4})
\item Ability to configure the tray monitor configuration file directly from the \acs{GUI} (Fig.~\vref{fig:tray2})
\item Ability to monitor a component and adapt the tray monitor task bar icon if a jobs are running.
\item \acs{TLS} Support
\item Better network connection handling
\item Default configuration file is stored under \bfilename{\$HOME/.bacula-tray-monitor.conf}
\item Ability to ``schedule'' jobs  
\item Available for Linux and Windows platforms
\end{bitemize}

\bimageH[0.8\linewidth]{tray-monitor-status}{Tray Monitor Status}{fig:tray0}

\bimageH[0.9\linewidth]{tray-monitor-conf-fd}{Tray Client Configuration}{fig:tray2}

\bimageH[0.8\linewidth]{tray-monitor-run1}{Tray Monitor Run a Job (1)}{fig:tray3}
\bimageH[0.8\linewidth]{tray-monitor-run2}{Tray Monitor Run a Job (2)}{fig:tray4}

\subsection{Scheduling Jobs via the Tray Monitor}

The Tray Monitor can periodically scan a specific directory configured
as \bdirectivename{Command Directory}
and process ``\bfilename{*.bcmd}'' files to find jobs to run.

The format of the ``\bfilename{file.bcmd}'' command file is the following:
\begin{bVerbatim}
<component name>:<run command>
<component name>:<run command>
...

<component name> = string
<run command>    = string (bconsole command line)
\end{bVerbatim}

For example:
\begin{bVerbatim}
localhost-fd: run job=backup-localhost-fd level=full
localhost-dir: run job=BackupCatalog
\end{bVerbatim}

A command file should contain at least one command. The component specified
in the first part of the command line should be defined in the tray
monitor. Once the command file is detected by the tray monitor, a popup is
displayed to the user and it is possible for the user to cancel the job.
\smallskip{}

Command files can be created with tools such as \btool{cron} or the \btool{task scheduler}
on Windows. It is possible and recommended to verify network connectivity at that time to
avoid network errors:

\begin{bVerbatim}
#!/bin/sh
if ping -c 1 director &> /dev/null
then
   echo "my-dir: run job=backup" > /path/to/commands/backup.bcmd
fi
\end{bVerbatim}


\subsection{Concurrent VSS Snapshot Support}
It is now possible to run multiple concurrent jobs that use \acs{VSS} snapshots on
the File Daemon for Microsoft Windows.

\subsection{Accurate Option for Verify ``Volume Data'' Job}

As of \bacula{} version~8.4.1, it has been possible to have a Verify Job
configured with \bvalueddirective{level}{Data} that will reread all records from a job
and optionally check size and checksum of all files.
\smallskip

Starting with~8.6, it is now possible to use the \bdirectivename{accurate} option to check
catalog records at the same time.  Using a Verify job with \bvalueddirective{level}{Data}
and \bvalueddirective{accurate}{yes} can replace the \bvalueddirective{level}{VolumeToCatalog} option.

For more information on how to setup a Verify Data job, see \vref{blb:verifyvolumedata}.

To run a Verify Job with the \bdirectivename{accurate} option, it is possible to set
the option in the Job definition or set use the \bvalueddirective{accurate}{yes} on the
command line.

\begin{bVerbatim}
* run job=VerifyData jobid=10 accurate=yes
\end{bVerbatim}

\subsection{Single Item Restore Optimisation}

\bacula{} version~8.6.0 can generate indexes stored in the catalog to speed up
file access during a Single Item Restore session for VMWare or for
Exchange.  The index can be displayed in bconsole with the
\bcommandname{list filemedia} command.

\begin{bVerbatim}
* list filemedia jobid=1
\end{bVerbatim}

\subsection{FileDaemon Saved Messages Resource Destination}

It is now possible to send the list of all saved files to a \bresourcename{Messages} resource
with the \bkeyword{saved} message type. It is not recommended to send this flow
of information to the \bDirector{} and/or the \bCatalog{} when the client \bFileSet{} is
large. To avoid side effects, the \bkeyword{all} keyword doesn't include
the \bkeyword{saved} message type. The \bkeyword{saved} message type should be
explicitly set.

\begin{bVerbatim}
# cat /opt/bacula/etc/bacula-fd.conf
...
Messages {
         Name = Standard
         director = mydirector-dir = all, !terminate, !restored, !saved
         append = /opt/bacula/working/bacula-fd.log = all, saved, restored
}
\end{bVerbatim}

\subsection{BWeb New Features}

The 8.6~release adds some new BWeb features, such as:
\begin{bitemize}
\item Two sets of wizards to help users to configure Copy/Migration
  jobs (Figures \vref{fig:cjcw} and~\vref{fig:mjcw})
\item A wizard to run jobs (Fig.~\vref{fig:rjw})
\item \acs{SSH} integration in BWeb Security Center to restart components remotely (Fig.~\vref{fig:src})
\item Global Endpoint Deduplication Overview screen (Fig.~\vref{fig:dow})
\end{bitemize}

\bimageH{dedup_usage}{BWeb Global Endpoint Deduplication Overview}{fig:dow}

\bimageH{copy_job_wizard}{Copy Job Creation Wizard}{fig:cjcw}
\bimageH{migrate_job_wizard}{Migrate Job Creation Wizard}{fig:mjcw}
\bimageH{run_job_wizard}{Run Job Wizard}{fig:rjw}
\bimageH{ssh_remote_commands}{BWeb SSH Remote Control}{fig:src}

\subsection{Minor Enhancements}

\subsubsection{New Bconsole \bcommandname{.estimate} Command}

The new \bcommandname{.estimate} command can be used to get statistics about a job to
run. The command uses the database to estimate the size and the number of
files of the next job. On a \postgresql{} database, the command uses regression
slope to compute values. On \sqlite{} or \mysql{}, where these statistical functions
are not available, the command uses a simple ``average'' estimation. The
correlation number is given for each value.

\begin{bVerbatim}
*.estimate job=backup 
level=I
nbjob=0
corrbytes=0
jobbytes=0
corrfiles=0
jobfiles=0
duration=0
job=backup

*.estimate job=backup level=F
level=F
nbjob=1
corrbytes=0
jobbytes=210937774
corrfiles=0
jobfiles=2545
duration=0
job=backup
\end{bVerbatim}

\subsubsection{Traceback and Lockdump}
%%% TODO: daemons, any binary, or a particular subset? -- ARL
After the reception of a signal by any of the \bacula{} daemon binaries, \texttt{traceback} and \texttt{lockdump}
information are now stored in the same file.

\section{\baculaenterprise{} 8.4.10}

\subsection{Plugin for Microsoft \acs{SQL} Server}

A plugin for Microsoft \acs{SQL} Server (\mssql{}) is now available. The plugin uses
\mssql{} advanced backup and restore features (like \acl{PITR}, Log
backup, Differential backup, \ldots{}).

\begin{bVerbatim}
Job {
    Name = MSSQLJob
    Type = Backup
    Client = windows1
    FileSet = MSSQL
    Pool = 1Month
    Storage = File
    Level = Incremental
}

FileSet {
        Name = MSSQL
        Enable VSS = no
        Include {
                Options {
                        Signature = MD5
                }
                Plugin = "mssql"
        }
}

FileSet {
        Name = MSSQL2
        Enable VSS = no
        Include {
                Options {
                        Signature = MD5
                }
                Plugin = "mssql: database=production"
        }
}
\end{bVerbatim}

\section{\baculaenterprise{} 8.4.1}

\subsection{Verify Volume Data}
\label{blb:verifyvolumedata}
It is now possible to have a Verify Job configured with \texttt{level=Data} to
reread all records from a job and optionally check the size and the checksum
of all files.

\begin{bVerbatim}
# Verify Job definition
Job {
    Name = VerifyData
    Type = Verify
    Level = Data
    Client = 127.0.0.1-fd     # Use local file daemon
    FileSet = Dummy           # Will be adapted during the job
    Storage = File            # Should be the right one
    Messages = Standard
    Pool = Default
}

# Backup Job definition
Job {
  Name = MyBackupJob
  Type = Backup
  Client = windows1
  FileSet = MyFileSet
  Pool = 1Month
  Storage = File
}

FileSet {
  Name = MyFileSet
  Include {
    Options {
      Verify = s5
      Signature = MD5
    }
  File = /
}
\end{bVerbatim}

To run the Verify job, it is possible to use the ``jobid'' parameter of the \bcommandname{run} command.

\begin{bVerbatim}
*run job=VerifyData jobid=10
Run Verify Job
JobName:     VerifyData
Level:       Data
Client:      127.0.0.1-fd
FileSet:     Dummy
Pool:        Default (From Job resource)
Storage:     File (From Job resource)
Verify Job:  MyBackupJob.2015-11-11_09.41.55_03
Verify List: /opt/bacula/working/working/VerifyVol.bsr
When:        2015-11-11 09:47:38
Priority:    10
OK to run? (yes/mod/no): yes
Job queued. JobId=14

...

11-Nov 09:46 my-dir JobId 13: Bacula Enterprise 8.4.1 (13Nov15):
  Build OS:               x86_64-unknown-linux-gnu archlinux
  JobId:                  14
  Job:                    VerifyData.2015-11-11_09.46.29_03
  FileSet:                MyFileSet
  Verify Level:           Data
  Client:                 127.0.0.1-fd
  Verify JobId:           10
  Verify Job:q
  Start time:             11-Nov-2015 09:46:31
  End time:               11-Nov-2015 09:46:32
  Files Expected:         1,116
  Files Examined:         1,116
  Non-fatal FD errors:    0
  SD Errors:              0
  FD termination status:  Verify differences
  SD termination status:  OK
  Termination:            Verify Differences
\end{bVerbatim}

The current Verify Data implementation requires specifying the correct Storage
resource in the Verify job. The Storage resource can be changed with the \btool{bconsole}
command line and with the menu.

\subsection{Bconsole \bcommandname{list jobs} Command Options}

The \bcommandname{list jobs} bconsole command now accepts new command line options:

\begin{bitemize}
\item \textbf{joberrors} Display jobs with JobErrors
\item \textbf{jobstatus=T} Display jobs with the specified status code
\item \textbf{client=client-name} Display jobs for a specified client
\item \textbf{order=asc/desc} Change the output format of the job list. The
  jobs are sorted by start time and JobId, the sort can use ascending (asc) or
  descending (desc) order, the latter being the default.
\end{bitemize}

\subsection{Minor Enhancements}

\subsubsection{New Bconsole \bcommandname{tee all} Command}

The ``@tall'' command allows logging all input and output of a console session.

\begin{bVerbatim}
*@tall /tmp/log
*st dir
...
@tall
\end{bVerbatim}

\subsubsection{\mysql{} Plugin Restore Options}

It is now possible to specify the database name during a restore in the Plugin
Option menu. It is still possible to use the ``Where'' parameter to
specify the target database name.

\subsubsection{\postgresql{} Plugin}

We added a ``timeout'' option to the \postgresql{} plugin command line that is set
to \bdefaultvalue{60s} by default. Users may want to change this value when the \postgresql{} cluster
is slow to complete \acs{SQL} queries used during the backup.

%%% ARL: read and modified up to here

\section{\baculaenterprise{} 8.4}

\subsection{VMWare Single File Restore}

It is now possible to explore VMWare virtual machines backup jobs (Full,
Incremental and Differential) made with the \baculaenterprise{} vSphere plugin to
restore individual files and directories. The \texttt{Single Item Restore}
feature comes with both a console interface and a \bweb{} Management Suite specific
interface. See the VMWare Single File Restore whitepaper for more information.

\bimageH{bweb-vmware-sir}{\bweb{} VMWare Single File Restore}{fig:bwebvmwaresir}


\subsection{Microsoft Exchange Single MailBox Restore}

It is now possible to explore Microsoft Exchange databases backups made with
the \baculaenterprise{} \acs{VSS} plugin to restore individual mailboxes. The
\texttt{Single Item Restore} feature comes with both a console interface and a
web interface. See the Exchange Single Mailbox Restore whitepaper for more
information.

\section{\baculaenterprise{} 8.2.8}

\subsection{New Job Edit Codes \%I}
In various places such as RunScripts, you have now access to \%I to get the
JobId of the copy or migration job started by a migrate job.

\begin{bVerbatim}
Job {
  Name = Migrate-Job
  Type = Migrate
  ...
  RunAfter = "echo New JobId is %I"
}
\end{bVerbatim}

\section{\baculaenterprise{} 8.2.2}

\subsection{New Job Edit Codes \%E \%R}
In various places such as RunScripts, you have now access to \%E to get the
number of non-fatal errors for the current Job and \%R to get the number of
bytes read from disk or from the network during a job.

\subsection{Enable/Disable commands}
The \btool{bconsole} \bcommandname{enable} and \bcommandname{disable} commands have
been extended from enabling/disabling Jobs to include Clients, Schedule,
and Storage devices.  Examples:

\begin{bVerbatim}
disable Job=NightlyBackup Client=Windows-fd
\end{bVerbatim}

will disable the Job named \textbf{NightlyBackup} as well as the
client named \textbf{Windows-fd}.

\begin{bVerbatim}
disable Storage=LTO-changer Drive=1
\end{bVerbatim}

will disable the first drive in the autochanger named \textbf{LTO-changer}.

Please note that doing a \bcommandname{reload} command will set any values
changed by the enable/disable commands back to the values in the
\bfilename{bacula-dir.conf} file.

The \bresourcename{Client} and \bresourcename{Schedule} resources in the \bfilename{bacula-dir.conf} file now permit
the directive \bdirectivename{Enabled} = \bvalue{yes} or \bdirectivename{Enabled} = \bvalue{no}.


\section{\baculaenterprise{} 8.2}

\subsection{Snapshot Management}

\baculaenterprise{} 8.2 is now able to handle Snapshots on Linux/Unix
systems. Snapshots can be automatically created and used to backup files. It is
also possible to manage Snapshots from \bacula{}'s \btool{bconsole} tool through a
unique interface.

\subsubsection{Snapshot Backends}

The following Snapshot backends are supported with \baculaenterprise{} 8.2:

\begin{bitemize}
\item \acs{BTRFS}
\item \acs{ZFS}
\item \acs{LVM}\footnote{Some restrictions described in \vref{blb:LVMBackend} applies to
    the \acs{LVM} backend}
\end{bitemize}

By default, Snapshots are mounted (or directly available) under
\bdefaultvalue{.snapshots} directory on the root filesystem. (On \acs{ZFS}, the default
is \bdefaultvalue{.zfs/snapshots}).
\smallskip{}

The Snapshot backend program is called \btool{bsnapshot} and is available in
the \textbf{bacula-enterprise-snapshot} package. In order to use the Snapshot
Management feature, the package must be installed on the Client.
\smallskip{}

\label{blb:bsnapshotconf}
The \btool{bsnapshot} program can be configured using
\bfilename{/opt/bacula/etc/bsnapshot.conf} file. The following parameters can
be adjusted in the configuration file:

\begin{bitemize}
\item \texttt{trace=\bbracket{file}} Specify a trace file
\item \texttt{debug=\bbracket{num}} Specify a debug level
\item \texttt{sudo=\byesorno{}} Use sudo to run commands
\item \texttt{disabled=\byesorno{}} Disable snapshot support
\item \texttt{retry=\bbracket{num}} Configure the number of retries for some operations
\item \texttt{snapshot\_dir=\bbracket{dirname}} Use a custom name for the Snapshot directory. (\bdirectoryname{.SNAPSHOT}, \bdirectoryname{.snapdir}, etc.)
\item \texttt{lvm\_snapshot\_size=\bbracket{lvpath:size}} Specify a custom snapshot size for a given \acs{LVM} volume
\item \texttt{mountopts=\bbracket{devpath:options}} Specify a custom mount option for a give device (avaiable in 10.0.4)
\end{bitemize}

\begin{bVerbatim}
# cat /opt/bacula/etc/bsnapshot.conf
trace=/tmp/snap.log
debug=10
lvm_snapshot_size=/dev/ubuntu-vg/root:5%
mountopts=nouuid
mountopts=/dev/ubuntu-vg/root:nouuid,nosuid
\end{bVerbatim}

\subsubsection{Application Quiescing}

When using Snapshots, it is very important to quiesce applications that are
running on the system. The simplest way to quiesce an application is to stop
it. Usually, taking the Snapshot is very fast, and the downtime is only about a
couple of seconds. If downtime is not possible and/or the application provides
a way to quiesce, a more advanced script can be used. An example is
described on \vref{blb:SnapRunScriptExample}.

\subsubsection{New Director Directives}

The use of the Snapshot Engine on the FileDaemon is determined by the
new \textbf{Enable Snapshot} FileSet directive. The default is \bdefaultvalue{no}.

\begin{bVerbatim}
FileSet {
  Name = LinuxHome

  Enable Snapshot = yes

  Include {
    Options = { Compression = LZO }
    File = /home
  }
}
\end{bVerbatim}

By default, Snapshots are deleted from the Client at the end of the backup.  To
keep Snapshots on the Client and record them in the Catalog for a determined
period, it is possible to use the \textbf{Snapshot Retention} directive in the
Client or in the Job resource. The default value is \bdefaultvalue{0 secconds}. If, for a given Job,
both Client and Job \textbf{Snapshot Retention} directives are set, the Job
directive will be used.

\begin{bVerbatim}
Client {
   Name = linux1
   ...

   Snapshot Retention = 5 days
}
\end{bVerbatim}

To automatically prune Snapshots, it is possible to use the following RunScript
command:

\begin{bVerbatim}
Job {
   ...
   Client = linux1
   ...
   RunScript {
      RunsOnClient = no
      Console = "prune snapshot client=%c yes"
      RunsAfter = yes
   }
}
\end{bVerbatim}
\smallskip{}

In RunScripts, the \texttt{AfterSnapshot} keyword for the \texttt{RunsWhen} directive will
allow a command to be run just after the Snapshot creation.

\texttt{AfterSnapshot} is a synonym for the \texttt{AfterVSS} keyword.

\label{blb:SnapRunScriptExample}
\begin{bVerbatim}
Job {
 ...
  RunScript {
    Command = "/etc/init.d/mysql start"
    RunsWhen = AfterSnapshot
    RunsOnClient = yes
  }
  RunScript {
    Command = "/etc/init.d/mysql stop"
    RunsWhen = Before
    RunsOnClient = yes
  }
}
\end{bVerbatim}

\subsubsection{Job Output Information}

Information about Snapshots are displayed in the Job output. The list of all
devices used by the Snapshot Engine is displayed, and the Job summary
indicates if Snapshots were available.

\begin{bVerbatim}
JobId 3:    Create Snapshot of /home/build
JobId 3:    Create Snapshot of /home/build/subvol
JobId 3:    Delete snapshot of /home/build
JobId 3:    Delete snapshot of /home/build/subvol
...
JobId 3: Bacula 127.0.0.1-dir 8.2.0 (23Feb15):
  Build OS:               x86_64-unknown-linux-gnu archlinux 
  JobId:                  3
  Job:                    Incremental.2015-02-24_11.20.27_08
  Backup Level:           Full
...
  Snapshot/VSS:           yes
...
  Termination:            Backup OK
\end{bVerbatim}


\subsubsection{New \bcommandname{snapshot} Bconsole Commands}

The new \bcommandname{snapshot} command will display by default the following menu:
\begin{bVerbatim}
*snapshot
Snapshot choice:
     1: List snapshots in Catalog
     2: List snapshots on Client
     3: Prune snapshots
     4: Delete snapshot
     5: Update snapshot parameters
     6: Update catalog with Client snapshots
     7: Done
Select action to perform on Snapshot Engine (1-7):
\end{bVerbatim}

The \bcommandname{snapshot} command can also have the following parameters:
\begin{bVerbatim}
[client=<client-name> | job=<job-name> | jobid=<jobid>]
 [delete | list | listclient | prune | sync | update]
\end{bVerbatim}

It is also possible to use traditional \bcommandname{list}, \bcommandname{llist},
\bcommandname{update}, \bcommandname{prune} or \bcommandname{delete} commands on Snapshots.

\begin{bVerbatim}
*llist snapshot jobid=5
 snapshotid: 1
       name: NightlySave.2015-02-24_12.01.00_04
 createdate: 2015-02-24 12:01:03
     client: 127.0.0.1-fd
    fileset: Full Set
      jobid: 5
     volume: /home/.snapshots/NightlySave.2015-02-24_12.01.00_04
     device: /home/btrfs
       type: btrfs
  retention: 30
    comment:
\end{bVerbatim}

\begin{bVerbatim}
* snapshot listclient
Automatically selected Client: 127.0.0.1-fd
Connecting to Client 127.0.0.1-fd at 127.0.0.1:9102
Snapshot      NightlySave.2015-02-24_12.01.00_04:
  Volume:     /home/.snapshots/NightlySave.2015-02-24_12.01.00_04
  Device:     /home
  CreateDate: 2015-02-24 12:01:03
  Type:       btrfs
  Status:     OK
  Error:
\end{bVerbatim}
\smallskip{}

With the \emph{Update catalog with Client snapshots} option (or
\bcommandname{snapshot sync}), the Director contacts the FileDaemon, lists snapshots
of the system and creates catalog records of the Snapshots.

\begin{bVerbatim}
*snapshot sync
Automatically selected Client: 127.0.0.1-fd
Connecting to Client 127.0.0.1-fd at 127.0.0.1:9102
Snapshot      NightlySave.2015-02-24_12.35.47_06:
  Volume:     /home/.snapshots/NightlySave.2015-02-24_12.35.47_06
  Device:     /home
  CreateDate: 2015-02-24 12:35:47
  Type:       btrfs
  Status:     OK
  Error:
Snapshot added in Catalog

*llist snapshot
 snapshotid: 13
       name: NightlySave.2015-02-24_12.35.47_06
 createdate: 2015-02-24 12:35:47
     client: 127.0.0.1-fd
    fileset:
      jobid: 0
     volume: /home/.snapshots/NightlySave.2015-02-24_12.35.47_06
     device: /home
       type: btrfs
  retention: 0
    comment: 
\end{bVerbatim}

% list
% llist
% prune
% delete
% update snapshot
% sync

\subsubsection{\acs{LVM} Backend Restrictions}
\label{blb:LVMBackend}

\acs{LVM} Snapshots are quite primitive compared to \acs{ZFS}, \acs{BTRFS}, NetApp and other
systems. For example, it is not possible to use Snapshots if the \ac{VG} is full.
The administrator must keep some free space in the \ac{VG} to create
Snapshots. The amount of free space required depends on the activity of the
\ac{LV}. \btool{bsnapshot} uses \bdefaultvalue{10\%} of the \ac{LV} by default. This
number can be configured per \ac{LV} in the \bfilename{bsnapshot.conf} file (See
\vref{blb:bsnapshotconf}).

% waa - 20150316 - Not exactly sure why this vgdisplay information is here with no explanation.
\begin{bVerbatim}
[root@system1]# vgdisplay
  --- Volume group ---
  VG Name               vg_ssd
...
  VG Size               29,81 GiB
...
  Alloc PE / Size       125 / 500,00 MiB
  Free  PE / Size       7507 / 29,32 GiB   <---- Free Space
...
\end{bVerbatim}

It is also not advisable to leave snapshots on the \acs{LVM} backend. Having multiple
snapshots of the same \acs{LV} on \acs{LVM} will slow down the system.
\smallskip{}

Only Ext4, XFS and EXT3 \footnote{XFS and EXT3 are available in 8.2.7 and
  later} filesystems are supported with the Snapshot \acs{LVM} backend. 

\subsubsection{Debug Options}

To get low level information about the Snapshot Engine, the debug tag ``snapshot''
should be used in the \bcommandname{setdebug} command.

\begin{bVerbatim}
* setdebug level=10 tags=snapshot client
* setdebug level=10 tags=snapshot dir
\end{bVerbatim}

\subsection{Global Endpoint Deduplication(TM)}
\subsubsection{Storage to Storage Copy/Migration}

Copy and Migration Jobs now use the Global Endpoint Deduplication protocol if
the destination Device Type is \texttt{dedup}.

\subsubsection{Performance Enhancements}

A new automatic Deduplication index optimization has been added to the Vacuum
procedure.
\smallskip{}

Part of the Deduplication index can be locked into memory to improve
performance.
\smallskip{}

Users can now configure parameters related to the size of the Deduplication
index and the amount of memory that can be used to cache the index.

\subsection{Hypervisor Plugins}
\subsubsection{Hyper-V VSS Plugin}

Backing up and restoring Hyper-V virtual machines is supported with Full level
backups using the \acs{VSS} \acs{API}. Use of the Global Endpoint Deduplication plugin and the
\texttt{bothsides} FileSet option minimizes the amount of data transfered and the
amount of storage used.

\subsubsection{KVM Plugin}

The \acs{KVM} plugin provides the following main features:

\begin{bitemize}
\item File level backup
\item Automatic virtual machine discovery
\item Full, Differential, Incremental backup level support
\item The ability to handle inclusion/exclusion of files
\end{bitemize}

The \acs{KVM} plugin is designed to be used when the hypervisor uses local storage for virtual
machine disks and \btool{libvirtd} for virtual machine management.

\subsection{Windows Encrypted File System (EFS) Support}

The \baculaenterprise{} Windows File Daemon now automatically supports files and
directories that are encrypted on Windows filesystem.

\subsection{\bweb{} Management Suite}

\bimageH{bweb-sd-status}{\bweb{} Storage Daemon Status/Overview}{fig:bwebsdoverview}
\bimageH{bweb-tls-security-center}{\bweb{} TLS Security Center}{fig:bwebtls}
\bimageH{bweb-fileset-wizard-browser}{\bweb{} FileSet Wizard}{fig:bwebfswiz}
\bimageH{bweb-configurable-table}{\bweb{} Job/Media Table Configuration}{fig:bwebconftable}


\subsection{Minor Enhancements}
\subsubsection{Copy/Migration/VirtualFull Performance Enhancements}

The Copy, Migration and VirtualFull performance on large jobs with millions of
files has been greatly enhanced.

\subsubsection{Storage Daemon Reports Disk Usage}

The \bcommandname{status storage} command now  reports the space available on disk devices:
\begin{bVerbatim}
...
Device status:

Device file: "FileStorage" (/bacula/arch1) is not open.
    Available Space=5.762 GB
==

Device file: "FileStorage1" (/bacula/arch2) is not open.
    Available Space=5.862 GB
\end{bVerbatim}

\section{\baculaenterprise{} 8.0}

\subsection{Global Endpoint Deduplication\texttrademark{}}

The \emph{Global Endpoint Deduplication} solution minimizes network
transfers and \bacula{} Volume size using deduplication technology.
\smallskip

The new Global Endpoint Deduplication Storage daemon directives are:

\begin{bdescription}
\item [Device Type = Dedup] sets the Storage device for
deduplication. Deduplication is performed only on disk volumes.
\item [Dedup Directory =] this directive specifies where the
deduplicated blocks will be stored. Blocks that are deduplicated
will be placed in this directory rather than in the \bacula{} Volume,
which will only contain a reference pointer to the deduplicated
blocks.
\item [Dedup Index Directory] in addition to the deduplicated
blocks, when deduplication is enabled, the Storage daemon keeps an
index of the deduplicated block locations.  This index will be 
frequently consulted during the deduplication backup process,
so it should be placed on the fastest device possible (e.g.
an \acs{SSD}).
\end{bdescription}
\smallskip

See below for a FileSet example using the new \texttt{dedup} directive.

\subsubsection{Configuration Example}
In the Storage Daemon configuration file, you must define a Device with the
\textbf{DeviceType = Dedup}.  It is also possible to configure where the
Storage Daemon will store blocks and indexes.  Blocks will be stored in the
\textbf{Dedup Directory}, the directory is common for all \textbf{Dedup}
devices and should have a large amount of free space.  Indexes will be
stored in the \textbf{Dedup Index Directory}, indexes will have a lot of random
update access, and can benefit from \acs{SSD} drives.

\begin{bVerbatim}
# from bacula-sd.conf
 Storage {
   Name = my-sd
   Working Directory = /opt/bacula/working
   Pid Directory = /opt/bacula/working

   Plugin Directory = /opt/bacula/plugins
   Dedup Directory = /opt/bacula/dedup
   Dedup Index Directory = /opt/bacula/ssd  # default for Dedup Directory
 }

 Device {
    Name = DedupDisk
    Archive Device = /opt/bacula/storage
    Media Type = DedupVolume
    Label Media = yes
    Random Access = yes
    Automatic Mount = yes
    Removable Media = no
    Always Open = no

    Device Type = Dedup    # Required
 }
\end{bVerbatim}

The Global Endpoint Deduplication Client cache system can speed up restore
jobs by getting blocks from the local client disk instead of requesting
them over the network.  Note that if blocks are not available locally, the
FileDaemon will get blocks from the Storage Daemon.  This feature can be
enabled with the \textbf{Dedup Index Directory} directive in the FileDaemon
resource.  When using this option, the File Daemon will have to maintain the
cache during Backup jobs.

\begin{bVerbatim}
# from bacula-fd.conf
 FileDaemon {
   Name = my-fd
   Working Directory = /opt/bacula/working
   Pid Directory = /opt/bacula/working

   # Optional, Keep indexes on the client for faster restores
   Dedup Index Directory = /opt/bacula/dedupindex
 }
\end{bVerbatim}

It is possible to configure the Global Endpoint
Deduplication system in the Director with a FileSet directive called
\textbf{Dedup}.  Each FileSet Include section can specify a different
deduplication behavior depending on your needs.

\begin{bVerbatim}
 FileSet {
   Name = FS_BASE

   # Send everything to the Storage Daemon as usual
   # and let the Storage Daemon do the deduplication
   Include {
     Options {
       Dedup = storage
     }
     File = /opt/bacula/etc
   }

   # Send only references and new blocks to the Storage Daemon
   Include {
     Options {
       Dedup = bothsides
     }
     File = /VirtualBox
   }

   # Do not try to dedup my encrypted directory
   Include {
     Options {
       Dedup = none
     }
     File = /encrypted
   }
 }
\end{bVerbatim}

The FileSet \textbf{Dedup} directive accepts the following values:

\begin{bitemize}
  %%% Philippe: Why is the label defined as a start just for this item?
\item[*] \textbf{storage} All the deduplication work is done on the SD
  side if the device type is \bdefaultvalue{dedup} (default value). This
  option is useful if you want to avoid the extra client-side disk space overhead
  that will occur with the \textbf{bothsides} option.
\item \textbf{none} Force FD and SD to not use deduplication
\item \textbf{bothsides} The deduplication work is done on both the FD and the
SD. Only references and new blocks will be transfered over the network.
\end{bitemize}


\subsection{Storage Daemon to Storage Daemon}
\baculaenterprise{} version 8.0 now permits SD to SD transfer of Copy and
Migration Jobs.  This permits what is commonly referred to as replication
or off-site transfer of \bacula{} backups.  It occurs automatically if the
source SD and destination SD of a Copy or Migration job are different.
That is, the SD to SD transfers need no additional configuration
directives. The following picture shows how this works.

\bimageH{sd-to-sd}{SD to SD}{fig:sdtosd}
%\includegraphics[width=0.8\linewidth]{sd-to-sd}

\subsection{Windows Mountpoint Support}
\baculaenterprise{} version 8.0 is now able to detect Windows mountpoints and include
volumes automatically in the \acs{VSS} snapshot set.  To backup all local disks
on a Windows server, the following FileSet is now accepted.  It depreciates
the alldrives plugin.

\begin{bVerbatim}
  FileSet {
   Name = "All Drives"
   Include {
     Options {
       Signature = MD5
     }

     File = /
   }
  }
\end{bVerbatim}

If you have mountpoints, the \texttt{onefs=no} option should be used as it is with
Unix systems.

\begin{bVerbatim}
  FileSet {
   Name = "All Drives with mountpoints"
   Include {
     Options {
       Signature = MD5
       OneFS = no
     }
     File = C:/                # will include mountpoint C:/mounted/...
   }
  }
\end{bVerbatim}

To exclude a mountpoint from a backup when \texttt{OneFS = no}, use the Exclude block as usual:

\begin{bVerbatim}
  FileSet {
   Name = "All Drives with mountpoints"
   Include {
     Options {
       Signature = MD5
       OneFS = no
     }
     File = C:/           # will include all mounted mountpoints under C:/
                          # including C:/mounted  (see Exclude below)
   }

   Exclude {
     File = C:/mounted    # will not include C:/mounted
   }
  }
\end{bVerbatim}

% waa - 20150317- Pretty sure Migration jobs do not involve the Client, so that should be removed
%                 from this paragraph.
\subsection{SD Calls Client}
If the \bdirectivename{SD Calls Client} directive is set to \bvalue{true} in a \bresourcename{Client} resource
any Backup, Restore, Verify Job where the client
is involved, the client will wait for the Storage daemon to contact it.
By default this directive is set to \bdefaultvalue{false}, and the Client will call
the Storage daemon as it always has.  This directive can be useful if your Storage daemon
is behind a firewall that permits outgoing connections but not incoming
connections. The picture \vref{fig:sdcallsclient} shows the communications connection paths in
both cases.

\bimageH[0.8\linewidth]{sd-calls-client}{SD Calls Client}{fig:sdcallsclient}



\subsection{Data Encryption Cipher Configuration}
\baculaenterprise{} version 8.0 and later now allows configuration of the data
encryption cipher and the digest algorithm. Previously, the cipher was forced to AES 128,
but it is now possible to choose between the following ciphers:

\begin{bitemize}
\item \bdefaultvalue{AES128} (default)
\item AES192
\item AES256
\item blowfish
\end{bitemize}

The digest algorithm was set to SHA1 or SHA256 depending on the local OpenSSL
options. We advise you to not modify the PkiDigest default setting. Please,
refer to the OpenSSL documentation to understand the pros and cons regarding these options.

\begin{bVerbatim}
FileDaemon {
    ...
    PkiCipher = AES256
}
\end{bVerbatim}

\subsection{Minor Enhancements}

\subsubsection*{New  Option Letter ``M'' for Accurate Directive in FileSet}

% waa - 20150317 - is 8.0.5 correct here?
Added in version 8.0.5, the new ``M'' option letter for the Accurate directive
in the FileSet Options block, which allows comparing the modification time and/or
creation time against the last backup timestamp. This is in contrast to the
existing options letters ``m'' and/or ``c'', mtime and ctime, which are checked
against the stored catalog values, which can vary accross different machines
when using the BaseJob feature.

The advantage of the new ``M'' option letter for Jobs that refer to BaseJobs is
that it will instruct \bacula{} to backup files based on the last backup time, which
is more useful because the mtime/ctime timestamps may differ on various Clients,
causing files to be needlessly backed up.
\smallskip{}

\begin{bVerbatim}
  Job {
    Name = USR
    Level = Base
    FileSet = BaseFS
...
  }

  Job {
    Name = Full
    FileSet = FullFS
    Base = USR
...
  }

  FileSet {
    Name = BaseFS
    Include {
      Options {
        Signature = MD5
      }
      File = /usr
    }
  }

  FileSet {
    Name = FullFS
    Include {
      Options {
        Accurate = Ms      # check for mtime/ctime of last backup timestamp and Size
        Signature = MD5
      }
      File = /home
      File = /usr
    }
  }
\end{bVerbatim}

\subsubsection*{.api version 2}

In \baculaenterprise{} version 8.0 and later, we introduced a new .api version
to help external tools to parse various \bacula{} bconsole output.

% waa - 20150317 - this section needs just a little more to explain what the "43" in "s43" mean. Perhaps
%                  if it is not a good place to list the possibilities here, then list where a reference
%                  is. Also, I think   .api 2 ...   Means "use API version 2"  but that should be stated too

The \texttt{api\_opts} option can use the following arguments:
\begin{bdescription}
\item [C] Clear current options
\item [tn] Use a specific time format (1 ISO format, 2 Unix Timestamp, 3 Default \bacula{} time format)
\item [sn] Use a specific separator between items (new line by default).
\item [Sn] Use a specific separator between objects (new line by default).
\item [o] Convert all keywords to lowercase and convert all non \emph{isalpha} characters to \_
\end{bdescription}

% waa - 20150317 - I think there should either be more output listed here to give a better feeling 
%                  or, perhaps another output listing for different .status commands

\begin{bVerbatim}
  .api 2 api_opts=t1s43S35
  .status dir running
==================================
jobid=10
job=AJob
...
\end{bVerbatim}

\subsubsection*{New Debug Options}

In \baculaenterprise{} version 8.0 and later, we introduced a new \texttt{options} parameter for
the \bcommandname{setdebug} bconsole command.
\smallskip{}

The following arguments to the new \texttt{option} parameter are available to control debug functions.

\begin{bdescription}
\item [0] Clear debug flags
\item [i] Turn off, ignore bwrite() errors on restore on File Daemon
\item [d] Turn off decomp of BackupRead() streams on File Daemon
\item [t] Turn on timestamps in traces
\item [T] Turn off timestamps in traces

% waa - 20150306 - does this "c" item mean to say "Truncate trace file if one exists, otherwise append to it"   ???
\item [c] Truncate trace file if trace file is activated

\item [l] Turn on recoding events on P() and V()
\item [p] Turn on the display of the event ring when doing a backtrace
\end{bdescription}
\smallskip{}

The following command will enable debugging for the File Daemon, truncate an existing trace file,
and turn on timestamps when writing to the trace file.

\begin{bVerbatim}
* setdebug level=10 trace=1 options=ct fd
\end{bVerbatim}
\smallskip{}

It is now possible to use a \emph{class} of debug messages called \texttt{tags}
to control the debug output of \bacula{} daemons.

\begin{bdescription}
\item [all] Display all debug messages
\item [bvfs] Display BVFS debug messages
\item [sql] Display \acs{SQL} related debug messages
\item [memory] Display memory and poolmem allocation messages
\item [scheduler] Display scheduler related debug messages
\end{bdescription}

\begin{bVerbatim}
* setdebug level=10 tags=bvfs,sql,memory
* setdebug level=10 tags=!bvfs

# bacula-dir -t -d 200,bvfs,sql
\end{bVerbatim}

The \texttt{tags} option is composed of a list of tags. Tags are separated by
``,'' or ``+'' or ``-'' or ``!''. To disable a specific tag, use ``-'' or ``!''
in front of the tag. Note that more tags are planned for future versions.

\LTXtable{\linewidth}{t-main-debugtags}

\section{\baculaenterprise{} 6.6.0}
\subsection{Communication Line Compression}
\baculaenterprise{} version 6.6.0 and later now includes communication
line compression. It is turned on by default, and if the
two \bacula{} components (DIR, FD, SD, bconsole) are both
version 6.6.0 or greater, communication line compression)
will be enabled, by default. If for some reason, you do not want
communication line compression, you may disable it with the
following directive:

\begin{bVerbatim}
Comm Compression = no
\end{bVerbatim}

This directive can appear in the following resources:
\begin{bitemize}
\item \bfilename{bacula-dir.conf}: Director resource
\item \bfilename{bacula-fd.conf}: Client (or FileDaemon) resource
\item \bfilename{bacula-sd.conf}: Storage resource
\item \bfilename{bconsole.conf}: Console resource
\item \bfilename{bat.conf}: Console resource
\end{bitemize}
\smallskip

In many cases, the volume of data transmitted across the communications line can be reduced by a factor of three when this directive is \bdefaultvalue{enabled}. In the case that the compression is not effective, \bacula{} turns it off on a record by record basis.
\smallskip

If you are backing up data that is already compressed the comm line compression will not be effective, and you are likely to end up with an average compression ratio that is very small.
In this case, \bacula{} reports \bvalue{None} in the Job report. 


\subsection{Read Only Storage Devices}
This version of \bacula{} allows you to define a Storage deamon device
to be read-only. If the \textbf{Read Only} directive is specified and
enabled, the drive can only be used for read operations.
The \textbf{Read Only} directive can be defined in any \bfilename{bacula-sd.conf}
Device resource, and is most useful for reserving one or more
drives for restores. An example is:

\begin{bVerbatim}
Read Only = yes
\end{bVerbatim}

\subsection{Catalog Performance Improvements}
There is a new \bacula{} database format (schema) in this version
of \bacula{} that eliminates the FileName table by placing the
Filename into the File record of the File table.
This substantiallly improves performance,
particularly for large (1GB or greater) databases.

% waa - 20150317 - Is 1GB _really_ considered to be a large database?  Do we mean to say 100GB??

The \btool{update\_xxx\_catalog} script will automatically update the
\bacula{} database format, but you should realize that for
very large databases (greater than 1GB), it may take some
time, and there are several different options for doing the
update:
\begin{benumerate}
\item Shudown the database and update it
\item Update the database while production jobs are running.
\end{benumerate}
See the \baculasystems{} White Paper ``Migration-to-6.6'' on this subject.
\smallskip

This database format change can provide very significant improvements in
the speed of metadata insertion into the database, and in some cases
(backup of large email servers) can significantly reduce the size of the
database.

\subsection{Plugin Restore Options}
This version of \bacula{} permits user configuration of Plugins at restore
time.  For example, it is now possible to choose the \texttt{datastore}
where your VMware image will be restored, or to choose
\btool{pg\_restore} options directly.  See specific Plugin whitepapers
for more information about new restore options.

The restore options, if implemented in a plugin, will be
presented to you during initiation of a restore either by
command line or if available by a \acs{GUI} such as \bweb{}.  For
examples of the command line interface and
the \acs{GUI} interface, please see below:

\begin{bVerbatim}
*run restore jobid=11766
Run Restore job
JobName:         RestoreFiles
Bootstrap:       /tmp/regress/working/my-dir.restore.1.bsr
Where:           /tmp/regress/tmp/bacula-restores
...
Plugin Options:  *None*
OK to run? (yes/mod/no): mod
Parameters to modify:
     1: Level
...
    13: Plugin Options
Select parameter to modify (1-13): 13
Automatically selected : vsphere: host=squeeze2
Plugin Restore Options
datastore:           *None*
restore_host:        *None*
new_hostname:        *None*
Use above plugin configuration? (yes/mod/no): mod
You have the following choices:
     1: datastore (Datastore to use for restore) 
     2: restore_host (ESX host to use for restore) 
     3: new_hostname (Restore host to specified name) 
Select parameter to modify (1-3): 3
Please enter a value for new_hostname: test
Plugin Restore Options
datastore:           *None*
restore_host:        *None*
new_hostname:        test
Use above plugin configuration? (yes/mod/no): yes
\end{bVerbatim}

Or via the \bweb{} restore interface (see Fig \vref{fig:bms-restore})

\bimageH{vsphere-restore-gui}{Choose datastore, ESXi or hostname at restore time}{fig:bms-restore}

\subsection{Alldrives Plugin Improvements}
The \texttt{alldrives} plugin simplifies the FileSet creation of
Windows Clients by automatically generating a FileSet which includes all
local drives.
\smallskip

The \texttt{alldrives} plugin now accepts the \texttt{snapshot} option that
generates snapshots for all local Windows drives, but without
explicitly adding them to the FileSet.  It may be combined with the
\acs{VSS} plugin. For example:

\begin{bVerbatim}
FileSet {
 ...
  Include {
    Plugin = "vss:/@MSSQL/"
    Plugin = "alldrives: snapshot"     # should be placed after vss plugin
  }
}
\end{bVerbatim}

\subsection{New Truncate Command}
We have added a new \bcommandname{truncate} command to bconsole which
will truncate a volume if the volume is purged, and if
the volume is also marked \textbf{Action On Purge = Truncate}.
This feature was originally added in \bacula{} version 5.0.1,
but the mechanism for actually doing the truncate required
the user to enter a complicated command such as:

\begin{bVerbatim}
purge volume action=truncate storage=File pool=Default
\end{bVerbatim}

The above command is now simplified to be:

\begin{bVerbatim}
truncate storage=File pool=Default
\end{bVerbatim}

\section{\baculaenterprise{} 6.4.x}

The following features were added during the 6.4.x life cycle.

\subsection{SAP Plugin}

The \baculaenterprise{} SAP Plugin is designed to implement the official SAP
Backint interface to simplify the backup and restore procedure through your
traditional SAP database tools. See SAP-Backint whitepaper for more
information.

\subsection{Oracle SBT Plugin}

By default, the Oracle backup Manager, \acs{RMAN}, sends all backups to an operating
system specific directory on disk. You can also configure \acs{RMAN} to make backups
to media such as tape using the SBT module. \bacula{} will act as Media Manager, and
the data will be transfered directly from \acs{RMAN} to \bacula{}. See Oracle Plugin
whitepaper for more information.

\subsection{\mysql{} Plugin}

The \mysql{} plugin is designed to simplify the backup and restore of your \mysql{}
database, the backup administrator doesn't need to know about the internals of \mysql{}
backup techniques or how to write complex scripts.  This plugin will
automatically backup essential information such as configurations and user
definitions.  The \mysql{} plugin supports both dump (with support for Incremental
backup) and binary backup techniques.  See the \mysql{} Plugin whitepaper for more
information.

\section{\baculaenterprise{} 6.4.0}
\subsection{Deduplication Optimized Volumes}
This version of \bacula{} includes a new alternative (or additional)
volume format that optimizes the placement of files so
that an underlying deduplicating filesystem such as \acs{ZFS}
can optimally deduplicate the backup data that is written
by \bacula{}. These are called Deduplication Optimized Volumes
or Aligned Volumes for short. The details of how to use this
feature and its considerations are in the \baculasystems{}
Deduplication Optimized Volumes whitepaper.

\subsection{Migration/Copy/VirtualFull Performance Enhancements}
The \bacula{} Storage daemon now permits multiple jobs to simultaneously read
from the same disk volume which gives substantial performance enhancements when
running Migration, Copy, or VirtualFull jobs that read disk volumes.  Our
testing shows that when running multiple simultaneous jobs, the jobs can
finish up to ten times faster with this version of \bacula{}.  This is
built-in to the Storage daemon, so it happens automatically and
transparently.

\subsection{VirtualFull Backup Consolidation Enhancements}
By default \bacula{} selects jobs automatically for a VirtualFull backup.
However, you may want to create the virtual backup based on a
particular backup (point in time) that exists.

For example, if you have the following backup Jobs in your catalog:
\begin{bVerbatim}
+-------+---------+-------+----------+----------+-----------+
| JobId | Name    | Level | JobFiles | JobBytes | JobStatus |
+-------+---------+-------+----------+----------+-----------+
| 1     | Vbackup | F     | 1754     | 50118554 | T         |
| 2     | Vbackup | I     | 1        | 4        | T         |
| 3     | Vbackup | I     | 1        | 4        | T         |
| 4     | Vbackup | D     | 2        | 8        | T         |
| 5     | Vbackup | I     | 1        | 6        | T         |
| 6     | Vbackup | I     | 10       | 60       | T         |
| 7     | Vbackup | I     | 11       | 65       | T         |
| 8     | Save    | F     | 1758     | 50118564 | T         |
+-------+---------+-------+----------+----------+-----------+
\end{bVerbatim}

and you want to consolidate only the first 3 jobs and create a
virtual backup equivalent to Job 1 + Job 2 + Job 3, you will use
\texttt{jobid=3} in the \bcommandname{run} command, then \bacula{} will select the
previous Full backup, the previous Differential (if any) and all subsequent
Incremental jobs.

\begin{bVerbatim}
run job=Vbackup jobid=3 level=VirtualFull
\end{bVerbatim}

If you want to consolidate a specific job list, you must specify the exact
list of jobs to merge in the \bcommandname{run} command line.  For example, to consolidate
the last Differential and all subsequent Incrementals, you will use
\texttt{jobid=4,5,6,7} or \texttt{jobid=4-7} on the \bcommandname{run} command line. Because one
of the Jobs in the list is a Differential backup, \bacula{} will set the new job
level to Differential. If the list is composed of only Incremental jobs,
the new job will have its level set to Incremental.

\begin{bVerbatim}
run job=Vbackup jobid=4-7 level=VirtualFull
\end{bVerbatim}

When using this feature, \bacula{} will automatically discard jobs that are
not related to the current Job.  For example, specifying
\texttt{jobid=7,8}, \bacula{} will discard JobId 8 because it is not
part of the same backup Job.

We do not recommend it, but if you really want to consolidate jobs that have
different names (so probably different clients, filesets, etc...), you must
use \texttt{alljobid=} keyword instead of \texttt{jobid=}.

\begin{bVerbatim}
run job=Vbackup alljobid=1-3,6-8 level=VirtualFull
\end{bVerbatim}

\subsection{New Prune ``Expired'' Volume Command}
In \baculaenterprise{} 6.4, it is now possible to prune all volumes
(from a pool, or globally) that are ``expired''.  This option can be
scheduled after or before the backup of the catalog and can be
combined with the \texttt{Truncate On Purge} option.  The \bcommandname{prune expired volme} command may
be used instead of the \btool{manual\_prune.pl} script.

\begin{bVerbatim}
* prune expired volume

* prune expired volume pool=FullPool
\end{bVerbatim}

To schedule this option automatically, it can be added to the Catalog backup job
definition.

\begin{bVerbatim}
Job {
   Name = CatalogBackup
   ...
   RunScript {
     Console = "prune expired volume yes"
     RunsWhen = Before
   }
}
\end{bVerbatim}

\section{\baculaenterprise{} 6.2.3}

\subsection{New Job Edit Codes \%P \%C}
In various places such as RunScripts, you have now access to \%P to get the
current \bacula{} process ID (PID) and \%C to know if the current job is a
cloned job.

\section{\baculaenterprise{} 6.2.0}

\subsection{\bweb{} \bacula{} Configuration GUI}
In \baculaenterprise{} version 6.2, the \bweb{} Management Suite integrates a
\bacula{} configuration GUI module which is designed to help you create and
modify the \bacula{} configuration files such as \bfilename{bacula-dir.conf},
\bfilename{bacula-sd.conf}, \bfilename{bacula-fd.conf} and \bfilename{bconsole.conf}.

The \bweb{} Management Suite offers a number of Wizards which support the
Administrator in his daily work. The wizards provide a step by step set of
required actions that graphically guide the Administrator to perform quick
and easy creation and modification of configuration files.

\bweb{} also provides diagnostic tools that enable the Administrator to check
that the Catalog Database is well configured, and that \bweb{} is installed
properly.

The new Online help mode displays automatic help text suggestions when the
user searches data types.

\bimageH{bweb-config-screen}{Configuration with \bweb{} Management Suite}{fig:BwebBconfigScreen}
\smallskip

This project was funded by \baculasystems{} and is available with the \bee{}.

\subsection{Performance Improvements}
\baculaenterprise{} 6.2 has a number of new performance improvements:

\begin{bitemize}
\item An improved way of storing \bacula{} Resources (as defined in
the .conf files). This new handling permits much faster loading or
reloading of the conf files, and permits larger numbers of resources.

\item Improved performance when inserting large numbers of files in
the DB catalog by breaking the insertion into smaller chunks, thus
allowing better sharing when running multiple simultaneous jobs.

\item Performance enhancements in \acs{BVFS} concerning eliminating 
duplicate path records.

\item Performance improvement when getting Pool records.

\item Pruning performance enhancements.
\end{bitemize}

\subsection{Enhanced Status and Error Messages}
We have enhanced the Storage daemon status output to be more
readable. This is important when there are a large number of
devices. In addition to formatting changes, it also includes more
details on which devices are reading and writing.

A number of error messages have been enhanced to have more specific
data on what went wrong.

If a file changes size while being backed up the old and new size
are reported.

\subsection{WinBMR 3}
The Windows \ac{BMR} plugin enables you to do safe, reliable
Disaster Recovery with \bee{} on Windows and allows you to
get critical systems up and running again quickly.  The \enterpriseedition{}
Windows \acs{BMR} is a toolkit that allows the Administrator to perform the restore
of a complete operating system to the same or similar hardware without actually
going through the operating system's installation procedure.
\smallskip

The WinBMR 3 version is a major rewrite of the product that support all x86
Windows versions and technologies. Especially \acs{UEFI} and secure boot systems. The
WinBMR 3 File Daemon plugin is now part of the plugins included with the \bacula{}
File Daemon package.  The rescue CD or USB key is available separately.

\subsection{Miscellaneous New Features}
\begin{bitemize}
\item Allow unlimited line lengths in .conf files (previously limited
to 2000 characters).

\item Allow \bfilename{/dev/null} in ChangerCommand to indicated a Virtual Autochanger.

\item Add a \texttt{{-}{-}fileprune} option to the \btool{manual\_prune.pl} script.

\item Add a \texttt{-m} option to \btool{make\_catalog\_backup.pl} to do maintenance
on the catalog.

\item Safer code that cleans up the working directory when starting
the daemons. It limits what files can be deleted, hence enhances 
security.

\item Added a new \bcommandname{.ls} command in bconsole to permit browsing a client's
filesystem.

\item Fixed a number of bugs, includes some obscure seg faults, and a
race condition that occurred infrequently when running Copy, Migration,
or Virtual Full backups.

\item Included a new vSphere library version, which will hopefully
fix some of the more obscure bugs.

\item Upgraded to a newer version of Qt4 for \bat{}. All indications
are that this will improve \bat{}'s stability on Windows machines.

\item The Windows installers now detect and refuse to install on
an \acs{OS} that does not match the 32/64 bit value of the installer.
\end{bitemize}


\section{\baculaenterprise{} 6.0.6}

\subsection{Incremental Accelerator Plugin for NetApp}

The Incremental Accelerator for NetApp Plugin is designed to simplify the
backup and restore procedure of your NetApp \acs{NAS} hosting a huge number of files.
\smallskip{}

When using the NetApp \acs{HFC} Plugin, \baculaenterprise{} will query the
NetApp device to get the list of all files modified since the last backup
instead of having to walk through the entire filesystem. Once \bacula{} have the
list of all files to back's up, it will use a standard network share (such as
\acs{NFS} or \acs{CIFS}) to access files.
\smallskip

This project was funded by \baculasystems{} and is available with the \bee{}.

\subsection{\postgresql{} Plugin}

The \postgresql{} plugin is designed to simplify the backup and restore procedure
of your \postgresql{} cluster, the backup administrator doesn't need to learn about
internals of \postgresql{} backup techniques or write complex scripts. The plugin
will automatically take care for you to backup essential information such as
configuration, users definition or tablespaces. The \postgresql{} plugin supports
both dump and \ac{PITR} backup techniques.
\smallskip

This project was funded by \baculasystems{} and is available with the \bee{}.

\subsection{Maximum Reload Requests}

The new Director directive \texttt{Maximum Reload Requests} permits to
configure the number of reload requests that can be done while jobs are
running.

\begin{bVerbatim}
Director {
  Name = localhost-dir
  Maximum Reload Requests = 64
  ...
}
\end{bVerbatim}

\subsection{FD Storage Address}

When the Director is behind a \acs{NAT}, in a \acs{WAN} area, to connect to
% the FileDaemon or
the StorageDaemon, the Director uses an ``external'' ip address,
and the FileDaemon should use an ``internal'' IP address to contact the
StorageDaemon.

The normal way to handle this situation is to use a canonical name such as
``storage-server'' that will be resolved on the Director side as the \acs{WAN}
address and on the Client side as the \acs{LAN} address. This is now possible to
configure this parameter using the new directive \bdirectivename{FDStorageAddress} in
the Storage or Client resource.


\bimageH{BackupOverWan1}{Backup Over WAN}{figbs6:fdstorageaddress}
%  \label{blb:fig:fdstorageaddress}

\begin{bVerbatim}
Storage {
     Name = storage1
     Address = 65.1.1.1
     FD Storage Address = 10.0.0.1
     SD Port = 9103
     ...
}
\end{bVerbatim}

% # or in the Client resouce
%

\begin{bVerbatim}
 Client {
      Name = client1
      Address = 65.1.1.2
      FD Storage Address = 10.0.0.1
      FD Port = 9102
      ...
 }
\end{bVerbatim}

Note that using the Client \bdirectivename{FDStorageAddress} directive will not allow
to use multiple Storage Daemon, all Backup or Restore requests will be sent to
the specified \bdirectivename{FDStorageAddress}.

\subsection{Maximum Concurrent Read Jobs}
This is a new directive that can be used in the \bfilename{bacula-dir.conf} file
in the Storage resource.  The main purpose is to limit the number
of concurrent Copy, Migration, and VirtualFull jobs so that
they don't monopolize all the Storage drives causing a deadlock situation
where all the drives are allocated for reading but none remain for
writing.  This deadlock situation can occur when running multiple
simultaneous Copy, Migration, and VirtualFull jobs.
\smallskip

The default value is set to \bdefaultvalue{0} (zero), which means there is no
limit on the number of read jobs.  Note, limiting the read jobs
does not apply to Restore jobs, which are normally started by
hand.  A reasonable value for this directive is one half the number
of drives that the Storage resource has rounded down.  Doing so,
will leave the same number of drives for writing and will generally
avoid over committing drives and a deadlock.


\section{\baculaenterprise{} 6.0.4}

\subsection{VMWare vSphere VADP Plugin}

The \baculaenterprise{} vSphere plugin provides virtual
machine bare metal recovery, while the backup at the guest level simplify data
protection of critical applications.

The plugin integrates the VMware's \ac{CBT} technology to
ensure only blocks that have changed since the initial Full, and/or the last
Incremental or Differential Backup are sent to the current Incremental or
Differential backup stream to give you more efficient backups and reduced
network load.

\subsection{Oracle \acs{RMAN} Plugin}

The \baculaenterprise{} Oracle Plugin is designed to simplify the backup and
restore procedure of your Oracle Database instance, the backup administrator
don't need to learn about internals of Oracle backup techniques or write
complex scripts.  The \baculaenterprise{} Oracle plugin supports both dump and
\ac{PITR} with \acs{RMAN} backup techniques.


\section{\baculaenterprise{} 6.0.2}

To make \bacula{} function properly with multiple Autochanger definitions, in
the Director's configuration, you must adapt your \bfilename{bacula-dir.conf}
\textbf{Storage} directives.
\smallskip

Each autochanger that you have defined in an \textbf{Autochanger} 
resource in the Storage daemon's \bfilename{bacula-sd.conf} file,
must have a corresponding \textbf{Autochanger} resource defined
in the Director's \bfilename{bacula-dir.conf} file.  Normally you will
already have a \textbf{Storage} resource that points to the
Storage daemon's \textbf{Autochanger} resource.  Thus you need
only to change the name of the \textbf{Storage} resource to
\textbf{Autochanger}.  In addition the \textbf{Autochanger = yes}
directive is not needed in the Director's \textbf{Autochanger}
resource, since the resource name is \textbf{Autochanger}, the
Director already knows that it represents an autochanger.
\smallskip

In addition to the above change (\textbf{Storage} to \textbf{Autochanger}),
you must modify any additional \textbf{Storage} resources that correspond
to devices that are part of the \textbf{Autochanger} device.
Instead of the previous \textbf{Autochanger = yes} directive they
should be modified to be \textbf{Autochanger = xxx} where you
replace the \textbf{xxx} with the name of the Autochanger.
\smallskip

For example, in the \bfilename{bacula-dir.conf} file:

\begin{bVerbatim}
Autochanger {             # New resource
  Name = Changer-1
  Address = cibou.company.com
  SDPort = 9103
  Password = "xxxxxxxxxx"
  Device = LTO-Changer-1
  Media Type = LTO-4
  Maximum Concurrent Jobs = 50
}

Storage {
  Name = Changer-1-Drive0
  Address = cibou.company.com
  SDPort = 9103
  Password = "xxxxxxxxxx"
  Device = LTO4_1_Drive0
  Media Type = LTO-4
  Maximum Concurrent Jobs = 5
  Autochanger = Changer-1  # New directive
}

Storage {
  Name = Changer-1-Drive1
  Address = cibou.company.com
  SDPort = 9103
  Password = "xxxxxxxxxx"
  Device = LTO4_1_Drive1
  Media Type = LTO-4
  Maximum Concurrent Jobs = 5
  Autochanger = Changer-1  # New directive
}

...
\end{bVerbatim}

Note that Storage resources \textbf{Changer-1-Drive0} and 
\textbf{Changer-1-Drive1} are not required since they make
up part of an autochanger, and normally, Jobs refer only
to the Autochanger resource.
However, by referring to those
Storage definitions in a Job, you will use only
the indicated drive.  This is not normally what
you want to do, but it is very useful and often used
for reserving a drive for restores.  See the Storage daemon
example .conf below and the use of \textbf{AutoSelect = no}.

So, in summary, the changes are:
\begin{bitemize}
\item Change \textbf{Storage} to \textbf{Autochanger} in the LTO4 resource.
\item Remove the \textbf{Autochanger = yes} from the \textbf{Autochanger}
LTO4 resource.
\item Change the \textbf{Autochanger = yes} in each of the \textbf{Storage}
device that belong to the \textbf{Autochanger} to point to the
\textbf{Autochanger} resource with for the example above the
directive \textbf{Autochanger = LTO4}.
\end{bitemize}

\section{\baculaenterprise{} 6.0.0}

\subsection{Incomplete Jobs}
During a backup, if the Storage daemon experiences disconnection
with the File daemon during backup (normally a comm line problem
or possibly an FD failure), under conditions that the SD determines
to be safe it will make the failed job as Incomplete rather than
failed.  This is done only if there is sufficient valid backup
data that was written to the Volume. The advantage of an Incomplete
job is that it can be restarted by the new bconsole \bcommandname{restart}
command from the point where it left off rather than from the
beginning of the jobs as is the case with a cancel.

\subsection{The \bcommandname{stop} Command}
\bacula{} has been enhanced to provide a \bcommandname{stop} command,
very similar to the \bcommandname{cancel} command with the main difference
that the Job that is stopped is marked as Incomplete so that
it can be restarted later by the \bcommandname{restart} command where
it left off (see below).  The \bcommandname{stop} command with no
arguments, will like the cancel command, prompt you with the
list of running jobs allowing you to select one, which might
look like the following:

\begin{bVerbatim}
*stop
Select Job:
     1: JobId=3 Job=Incremental.2012-03-26_12.04.26_07
     2: JobId=4 Job=Incremental.2012-03-26_12.04.30_08
     3: JobId=5 Job=Incremental.2012-03-26_12.04.36_09
Choose Job to stop (1-3): 2
2001 Job "Incremental.2012-03-26_12.04.30_08" marked to be stopped.
3000 JobId=4 Job="Incremental.2012-03-26_12.04.30_08" marked to be stopped.
\end{bVerbatim}

\subsection{The \bcommandname{restart} Command}
The new \bcommandname{restart} command allows console users to restart
a canceled, failed, or incomplete Job.  For canceled and failed
Jobs, the Job will restart from the beginning.  For incomplete
Jobs the Job will restart at the point that it was stopped either
by a stop command or by some recoverable failure.
\smallskip

If you enter the \bcommandname{restart} command in bconsole, you will get the
following prompts:

\begin{bVerbatim}
*restart
You have the following choices:
     1: Incomplete
     2: Canceled
     3: Failed
     4: All
Select termination code:  (1-4):
\end{bVerbatim}

If you select the \textbf{All} option, you may see something like:

\begin{bVerbatim}
Select termination code:  (1-4): 4
+-------+-------------+---------------------+------+-------+----------+-----------+-----------+
| jobid | name        | starttime           | type | level | jobfiles | jobbytes  | jobstatus |
+-------+-------------+---------------------+------+-------+----------+-----------+-----------+
|     1 | Incremental | 2012-03-26 12:15:21 | B    | F     |        0 |         0 | A         |
|     2 | Incremental | 2012-03-26 12:18:14 | B    | F     |      350 | 4,013,397 | I         |
|     3 | Incremental | 2012-03-26 12:18:30 | B    | F     |        0 |         0 | A         |
|     4 | Incremental | 2012-03-26 12:18:38 | B    | F     |      331 | 3,548,058 | I         |
+-------+-------------+---------------------+------+-------+----------+-----------+-----------+
Enter the JobId list to select:
\end{bVerbatim}

Then you may enter one or more JobIds to be restarted, which may
take the form of a list of JobIds separated by commas, and/or JobId
ranges such as \textbf{1-4}, which indicates you want to restart JobIds
1 through 4, inclusive.

\subsection{Support for Exchange Incremental Backups}
The \baculaenterprise{} version 6.0 \acs{VSS} plugin now supports
Full and Incremental backups for Exchange.  We strongly
recommend that you do not attempt to run Differential jobs with
Exchange as it is likely to produce a situation where restores
will no longer select the correct jobs, and thus the
Windows Exchange \acs{VSS} writer will fail when applying log files.
There is a \baculasystems{} Enterprise white paper that provides
the details of backup and restore of Exchange 2010 with the
\bacula{} \acs{VSS} plugin.
\smallskip

Restores can be done while Exchange is running, but you
must first unmount (dismount in Microsoft terms) any database
you wish to restore and explicitly mark them to permit a
restore operation (see the white paper for details).
\smallskip

This project was funded by \baculasystems{} and is available with the \bee{}.

\subsection{Support for \mssql{} Block Level Backups}
With the addition of block level backup support to the
\baculaenterprise{} \acs{VSS} \mssql{} component, you can now do
Differential backups in addition to Full backups.
Differential backups use Microsoft's partial block backup
(a block differencing or deduplication that we call Delta).
This partial block backup permits backing up only those
blocks that have changed.  Database restores can be made while
the \mssql{} server is running, but any databases selected for
restore will be automatically taken offline by the \mssql{}
server during the restore process.

Incremental backups for \mssql{} are not support by
Microsoft. We strongly recommend that you not perform Incremental
backups with \mssql{} as they will probably produce a situation
where restore will no longer work correctly.
\smallskip

 We are currently working on producing a white paper that will give more
details of backup and restore with \mssql{}. One point to note is that during
a restore, you will normally not want to restore the \textbf{master} database.
You must exclude it from the backup selections that you have made or the
restore will fail.
\smallskip

It is possible to restore the \textbf{master} database, but you must
first shutdown the \mssql{} server, then you must perform special
recovery commands.  Please see Microsoft documentation on how
to restore the master database.
\smallskip

This project was funded by \baculasystems{} and is available with the \bee{}.


\subsection{Job Bandwidth Limitation}

The new \textbf{Job Bandwidth Limitation} directive may be added to the File
daemon's and/or Director's configuration to limit the bandwidth used by a
Job on a Client.  It can be set in the File daemon's conf file for all Jobs
run in that File daemon, or it can be set for each Job in the Director's
conf file. The speed is always specified in bytes per second.

For example:
\begin{bVerbatim}
FileDaemon {
  Name = localhost-fd
  Working Directory = /some/path
  Pid Directory = /some/path
  ...
  Maximum Bandwidth Per Job = 5Mb/s
}
\end{bVerbatim}

The above example would cause any jobs running with the FileDaemon to not
exceed 5 megabytes per second of throughput when sending data to the
Storage Daemon. Note, the speed is always specified in \bhighlight{bytes per second}
(not in bits per second), and the case (upper/lower) of the specification
characters is ignored (i.e. 1MB/s = 1Mb/s).

You may specify the following speed parameter modifiers:
   k/s (1,024 bytes per second), kb/s (1,000 bytes per second),
   m/s (1,048,576 bytes per second), or mb/s (1,000,000 bytes per second).

For example:
\begin{bVerbatim}
Job {
  Name = locahost-data
  FileSet = FS_localhost
  Accurate = yes
  ...
  Maximum Bandwidth = 5Mb/s
  ...
}
\end{bVerbatim}

The above example would cause Job \texttt{localhost-data} to not exceed 5MB/s
of throughput when sending data from the File daemon to the Storage daemon.

A new console command \bcommandname{setbandwidth} permits to set dynamically the
maximum throughput of a running Job or for future jobs of a Client.

\begin{bVerbatim}
* setbandwidth limit=1000 jobid=10
\end{bVerbatim}

Number of bytes can be expressed using modifiers mentioned above (k/s, kb/s, m/s or mb/s).
\medskip

This project was funded by \baculasystems{} and is available in \bee{}.

\subsection{Incremental/Differential Block Level Difference Backup}

The new \texttt{delta} Plugin is able to compute and apply signature-based file
differences. It can be used to backup only changes in a big binary file like
Outlook PST, VirtualBox/VMware images or database files.

It supports both Incremental and Differential backups and stores signatures
database in the File Daemon working directory. This plugin is available on all
platform including Windows 32 and 64bit.

Accurate option should be turned on in the Job resource.
\begin{bVerbatim}
Job {
 Accurate = yes
 FileSet = DeltaFS
 ...
}

FileSet {
 Name = DeltaFS
 ...
 Include {
   # Specify one file
   Plugin = "delta:/home/eric/.VirtualBox/HardDisks/lenny-i386.vdi"
 }
}

FileSet {
 Name = DeltaFS-Include
 ...
 Include {
   Options {
      Compression = GZIP1
      Signature = MD5
      Plugin = delta
   }
   # Use the Options{} filtering and options
   File = /home/user/.VirtualBox
 }
}
\end{bVerbatim}

Please contact \baculasystems{} support to get Delta Plugin specific
documentation.
\medskip

This project was funded by \baculasystems{} and is available with the \bee{}.

\subsection{SAN Shared Tape Storage Plugin}

The problem with backing up multiple servers at the same time to the
same tape library (or autoloader) is that if both servers access the
same tape drive same time, you will very likely get data corruption.
This is where the \baculasystems{} shared tape storage plugin comes into play.  The
plugin ensures that only one server at a time can connect to each device
(tape drive) by using the SPC-3 SCSI reservation protocol.  Please contact
\baculasystems{} support to get SAN Shared Storage Plugin specific
documentation.
\medskip

This project was funded by \baculasystems{} and is available with \bee{}.

\subsection{Advanced Autochanger Usage}

The new \texttt{Shared Storage} Director's directive is a \baculaenterprise{}
feature that allows you to share volumes between different Storage
resources. This directive should be used \textbf{only} if all
\texttt{Media Type} are correctly set across all Devices.

The \texttt{Shared Storage} directive should be used when using the SAN
Shared Storage plugin or when accessing from the Director Storage resources
directly to Devices of an Autochanger.

When sharing volumes between different Storage resources, you will
need also to use the \btool{reset-storageid} script before using the
\bcommandname{update slots} command. This script can be scheduled once a day in
an Admin job.

\begin{bVerbatim}
 $ /opt/bacula/scripts/reset-storageid MediaType StorageName
 $ bconsole
 * update slots storage=StorageName drive=0
\end{bVerbatim}

Please contact \baculasystems{} support to get help on this advanced
configuration.
\medskip

This project was funded by \baculasystems{} and is available with \bee{}.
\medskip

\textbf{The reset-storageid procedure is no longer required when using
the appropriate Autochanger configuration in the Director configuration
side.}

\subsection{Enhancement of the \acs{NDMP} Plugin}

The previous \acs{NDMP} Plugin 4.0 was fully supporting only the NetApp hardware, the
new \acs{NDMP} Plugin should now be able to support all \acs{NAS} vendors with the
\texttt{volume\_format} plugin command option.

On some \acs{NDMP} devices such as Celera or Blueray, the administrator can use arbitrary
volume structure name, ex:

\begin{bVerbatim}
/dev/volume_home
/rootvolume/volume_tmp
/VG/volume_var
\end{bVerbatim}

The \acs{NDMP} plugin should be aware of the structure organization in order to
detect if the administrator wants to restore in a new volume
(\texttt{where=/dev/vol\_tmp}) or inside a subdirectory of the targeted volume
(\texttt{where=/tmp}).

\begin{bVerbatim}
FileSet {
 Name = NDMPFS
 ...
 Include {
   Plugin = "ndmp:host=nasbox user=root pass=root file=/dev/vol1 volume_format=/dev/"
 }
}
\end{bVerbatim}

Please contact \baculasystems{} support to get \acs{NDMP} Plugin specific
documentation.
\medskip

This project was funded by \baculasystems{} and is available with the \bee{}

\subsection{Always Backup a File}

When the Accurate mode is turned on, you can decide to always backup a file
by using then new \textbf{A} Accurate option in your FileSet. For example:

\begin{bVerbatim}
Job {
   Name = ...
   FileSet = FS_Example
   Accurate = yes
   ...
}

FileSet {
 Name = FS_Example
 Include {
   Options {
     Accurate = A
   }
   File = /file
   File = /file2
 }
 ...
}
\end{bVerbatim}

This project was funded by \baculasystems{} based on an idea of James Harper and
is available with the \bee{}.

\subsection{Setting Accurate Mode at Runtime}

You are now able to specify the Accurate mode on the \bcommandname{run} command and
in the Schedule resource.

\begin{bVerbatim}
* run accurate=yes job=Test
\end{bVerbatim}

\begin{bVerbatim}
Schedule {
  Name = WeeklyCycle
  Run = Full 1st sun at 23:05
  Run = Differential accurate=yes 2nd-5th sun at 23:05
  Run = Incremental  accurate=no  mon-sat at 23:05
}
\end{bVerbatim}

It can allow you to save memory and and CPU resources on the catalog server in
some cases.
\medskip

These advanced tuning options are available with the \bee{}.

% Common with community
\subsection{Additions to RunScript variables}
You can have access to JobBytes, JobFiles and Director name using \%b, \%F and \%D
in your runscript command. The Client address is now available through \%h.

\begin{bVerbatim}
RunAfterJob = "/bin/echo Job=%j JobBytes=%b JobFiles=%F ClientAddress=%h Dir=%D"
\end{bVerbatim}

\subsection{\acs{LZO} Compression}

\acs{LZO} compression was added in the Unix File Daemon. From the user point of view,
it works like the \acs{GZIP} compression (just replace \textbf{compression=GZIP} with
\textbf{compression=LZO}).

For example:
\begin{bVerbatim}
Include {
   Options { compression=LZO }
   File = /home
   File = /data
}
\end{bVerbatim}

\acs{LZO} provides much faster compression and decompression speed but lower
compression ratio than \acs{GZIP}. It is a good option when you backup to disk. For
tape, the built-in compression may be a better option.

\acs{LZO} is a good alternative for \acs{GZIP}1 when you don't want to slow down your
backup. On a modern CPU it should be able to run almost as fast as:

\begin{bitemize}
\item your client can read data from disk. Unless you have very fast disks like
  \acs{SSD} or large/fast \acs{RAID} array.
\item the data transfers between the file daemon and the storage daemon even on
  a 1Gb/s link.
\end{bitemize}

Note that bacula only use one compression level \acs{LZO}1X-1.
\medskip

The code for this feature was contributed by Laurent Papier.

\subsection{New Tray Monitor}

Since the old integrated Windows tray monitor doesn't work with
recent Windows versions, we have written a new Qt Tray Monitor that is available
for both Linux and Windows.  In addition to all the previous features,
this new version allows you to run Backups from
the tray monitor menu.

\bimageH{tray-monitor}{New tray monitor}{figbs6:traymonitor}

\bimageH{tray-monitor1}{Run a Job through the new tray monitor}{figbs6:traymonitor1}



To be able to run a job from the tray monitor, you need to
allow specific commands in the Director monitor console:
\begin{bVerbatim}
Console {
    Name = win2003-mon
    Password = "xxx"
    CommandACL = status, .clients, .jobs, .pools, .storage, .filesets, .messages, run
    ClientACL = *all*               # you can restrict to a specific host
    CatalogACL = *all*
    JobACL = *all*
    StorageACL = *all*
    ScheduleACL = *all*
    PoolACL = *all*
    FileSetACL = *all*
    WhereACL = *all*
}
\end{bVerbatim}
\medskip

This project was funded by \baculasystems{} and is available with \bee{} and \bce{}.

\subsection{Purge Migration Job}

The new \bdirectivename{Purge Migration Job} directive may be added to the Migration
Job definition in the \bdirectorname{}'s configuration file. When it is enabled
the Job that was migrated during a migration will be purged at
the end of the migration job.

For example:
\begin{bVerbatim}
Job {
  Name = "migrate-job"
  Type = Migrate
  Level = Full
  Client = localhost-fd
  FileSet = "Full Set"
  Messages = Standard
  Storage = DiskChanger
  Pool = Default
  Selection Type = Job
  Selection Pattern = ".*Save"
...
  Purge Migration Job = yes
}
\end{bVerbatim}
\medskip

This project was submitted by Dunlap Blake; testing and documentation was funded
by \baculasystems{}.

\subsection{Changes in the Pruning Algorithm}

We rewrote the job pruning algorithm in this version. Previously, in some
users reported that the pruning process at the end of jobs was very long.
It should not be longer the case. Now, \bacula{} won't prune automatically a
Job if this particular Job is needed to restore data. Example:

\begin{bVerbatim}
JobId: 1  Level: Full
JobId: 2  Level: Incremental
JobId: 3  Level: Incremental
JobId: 4  Level: Differential
.. Other incrementals up to now
\end{bVerbatim}

In this example, if the Job Retention defined in the Pool or in the Client
resource causes that Jobs with Jobid in 1,2,3,4 can be pruned, \bacula{} will
detect that JobId 1 and 4 are essential to restore data at the current state
and will prune only JobId 2 and 3.

\texttt{Important}, this change affect only the automatic pruning step
after a Job and the \bcommandname{prune jobs} \bconsole{} command. If a volume
expires after the \bdirectivename{VolumeRetention} period, important jobs can be
pruned.

\subsection{Ability to Verify any specified Job}
You now have the ability to tell \bacula{} which Job should verify instead of
automatically verify just the last one.

This feature can be used with VolumeToCatalog, DiskToCatalog and Catalog level.

To verify a given job, just specify the Job jobid in argument when starting the
job.
\begin{bVerbatim}
*run job=VerifyVolume jobid=1 level=VolumeToCatalog
Run Verify job
JobName:     VerifyVolume
Level:       VolumeToCatalog
Client:      127.0.0.1-fd
FileSet:     Full Set
Pool:        Default (From Job resource)
Storage:     File (From Job resource)
Verify Job:  VerifyVol.2010-09-08_14.17.17_03
Verify List: /tmp/regress/working/VerifyVol.bsr
When:        2010-09-08 14:17:31
Priority:    10
OK to run? (yes/mod/no):
\end{bVerbatim}
\medskip

This project was funded by \baculasystems{} and is available with \bacula{} \enterpriseedition{} and \communityedition{}.
