FOSS Solution For Network Configuration Backups

In the networks that I run I typically try and follow the FCAPS model. The (C)onfiguration part of that is often overlooked. I have used paid and free products, and they all work fine, but I wanted something that was simple, version controlled, and fully open source. This solution uses Cisco's Embedded Event Manager (EEM) to ship configurations to the backup server, Git for version control, and Gitlist for browsing your network configurations.

The End Result

After this is deployed, all of your Cisco IOS devices will automatically backup their configurations after write mem or copy run start are issued from the command line. At midnight, git will commit all of the changes for that day. Then you will be able to browse version-controlled, network configurations via the Gitlist web interface. Here are some screenshots of what it will look like.

Configuration Repositories List

Configuration Diff

EEM Configuration Backup

Set Up The Configuration Backup Server

To set up a server to house all of the network configurations that have been backed up, we will be using Centos 6.5. For the transport medium to ship the configurations to the server we will be using SFTP. Since SFTP is a subsystem of OpenSSH, setup is super simple. Just a vanilla install of Centos will already have SSH running. All we need to do is create a specific user account for the configuration backup process and install and configure a Git repository to store and version the configurations.

Do all of the following as root by using sudo -s or su - if you are not in sudoers.

Create a directory to store everything:

mkdir /var/data

Change permissions on the directory so everyone can read and execute (list the contents):

chmod 755 /var/data

Add the user configbackup and set home directory to /var/data/configbackup:

useradd -d /var/data/configbackup configbackup

Change the permissions on the new user folder. This is needed to permit Gitlist to view the Git repository.

chmod 755 /var/data/configbackup

Create a password for the user account:

passwd configbackup

Change user to configbackup:

su - configbackup

Make a directory for all of your configuration repositories:

mkdir git_repos

Make a directory specific for your network configurations:

mkdir git_repos/config_repo

exit

Set Up Git for Version Control

Install Git version control:

yum install git -y

Change user to configbackup:

su - configbackup

Set our Git username and email address, so when doing commits, it will show the correct user information.

git config --global user.name "configbackup"

git config --global user.email netops@example.com

Initialize the Git repository:

cd git_repos/config_repo

git init

Create a test file:

touch test

Add all files in the directory for tracking by Git:

git add .

Perform the initial Git commit:

git commit -m"initial commit"

Verify the first commit:

git log

Create a cronjob to perform a nightly commit of any changes:

vim /var/data/configbackup/nightly-git-commit.sh

Insert the following text into the file by pressing i, then paste the text. To save the file, first press the escape key, and then :wq and the enter key to write the file and quit.

#!/bin/bash
cd /var/data/configbackup/git_repos/config_repo
/bin/date -u +"%Y-%m-%dT%H:%M:%SZ" > datetimestamp
/usr/bin/git add . --all
/usr/bin/git commit -am"automatic nightly commit"

Make the script executable:

chmod 755 /var/data/configbackup/nightly-git-commit.sh

Install the cronjob for the user configbackup:

crontab -e

Insert the following text into the file by pressing i, then paste the text. To save the file, first press the escape key, and then :wq and the enter key to write the file and quit.

0 0 * * * /var/data/configbackup/nightly-git-commit.sh > /dev/null 2>&1

Set Up Gitlist for Viewing Configurations

Gitlist is used for viewing a Git repo in a web browser. This makes it very easy to traverse your network configuration backups.

Change from configbackup user back to root:

exit

Install Apache web server and PHP:

yum install httpd php -y

Download Gitlist:

wget https://s3.amazonaws.com/gitlist/gitlist-0.5.0.tar.gz

Untar the file:

tar zxf gitlist-0.5.0.tar.gz

Copy the gitlist directory to Apache's root directory. (Copying the files to keep from having to correct the SELinux labels)

cp -R gitlist /var/www/html/

Rename the example configuration to config.ini:

mv /var/www/html/gitlist/config.ini-example /var/www/html/gitlist/config.ini

Tell Gitlist where your repo directory is located:

vim /var/www/html/gitlist/config.ini

Insert the following text into the file by pressing i, then paste the text. To save the file, first press the escape key, and then :wq and the enter key to write the file and quit.

repositories[] = '/var/data/configbackup/git_repos/' ;

Modify the .htaccess file to rewrite the base URL to /gitlist/. This is necessary if you host Gitlist somewhere other than the root (i.e. - http://host.domain.com/gitlist).

vim /var/www/html/gitlist/.htaccess

Insert the following text into the file by pressing i, then paste the text. To save the file, first press the escape key, and then :wq and the enter key to write the file and quit.

RewriteBase /gitlist/

Create a cache directory for Gitlist:

mkdir /var/www/html/gitlist/cache

Change the directory to read/write for all:

chmod 777 /var/www/html/gitlist/cache

Modify Apache configuration to allow the Gitlist .htaccess file to be executed. This is done by changing the AllowOveride directive to All.

vim /etc/httpd/conf/httpd.conf

Insert the following text into the file by pressing i, then paste the text. To save the file, first press the escape key, and then :wq and the enter key to write the file and quit.

<Directory "/var/www/html">
    ...
    AllowOverride All
    ...
</Directory>

Create the username and password to protect Gitlist:

htpasswd -c /var/www/html/gitlist/.htpasswd admin

Modify the Gitlist .htaccess file protect Gitlist with HTTP Basic authentication:

sudo vim /var/www/html/gitlist/.htaccess

Insert the following text into the file by pressing i, then paste the text. To save the file, first press the escape key, and then :wq and the enter key to write the file and quit.

AuthUserFile /var/www/html/gitlist/.htpasswd
AuthType Basic
AuthName "Restricted"
Require valid-user

Enable SSL for Apache so we don't send HTTP basic authentication in the clear:

Install the SSL module for Apache:

yum install mod_ssl -y

If you want to install your own certificate, then you need to modify the /etc/httpd/conf.d/ssl.conf file and point Apache to your certificate. I will not go into the details of this. I will just use the self-signed certificate that is installed when mod_ssl is installed.

Enable Apache to automatically start:

chkconfig httpd on

Start the Apache web server:

service httpd start

Modify iptables to permit inbound TCP/443, and block inbound TCP/80 (the default ruleset already does the latter):

iptables -I INPUT 5 -m state --state NEW -m tcp -p tcp --dport 443 -j ACCEPT

Save the iptables configuration so it is persistent:

iptables-save > /etc/sysconfig/iptables

SELinux is a security measure that enforces mandatory access control (MAC) on Linux. Sometimes this will not permit processes to function properly if the labels are not set up correctly. I would highly suggest not disabling SELinux, instead, learn how to use it and fix whatever issues you may come across. That being said, if you don't want to mess around with it you can set it to permissive by modifying /etc/selinux/config and rebooting the server.

Verify everything is functioning:

browse to https://host.domain.com/gitlist

Configuration Backup Using EEM on Cisco IOS Devices

In order to make your switches and routers ship their configurations to the server, you need to install an EEM script. This script listens for write mem or copy run start and when either of these commands is run at the cli, the script will execute and copy the startup-config to the remote server via secure copy protocol (SCP).

Install on all Cisco IOS devices you wish to back up:

First, we need to silence all of the prompts that IOS presents us when we use the copy command. If this is not done, your EEM script will hang indefinitely. Then install the EEM script.

!Silence file prompts so the EEM script does not hang waiting on user input.
file prompt quiet
!Install the EEM script...
event manager applet copyConfigurationToServer
 event cli pattern "(wr.* mem.*)|(copy run.* start.*)" sync no skip no occurs 1 period 5
 action 1 info type routername
 action 2 cli command "enable"
 action 3 cli command "copy startup-config scp://configbackup:password@192.168.17.131/git_repos/config_repo/$_info_routername"
 action 4 syslog msg "EEM Script copyConfigurationToServer writing configurations to remote server"