Blog

A few weeks back I was tasked with turning up a new OpenStack region. This region couldn’t share anything with existing regions because the plan was to test pre-release versions of OpenStack there, and if we shared something like glance then we would either have to endanger glance for all regions during testing, or not test glance. However, our users already have a favorite set of images uploaded to glance, and I really wanted to make it as easy as possible for them to use the new region — I wanted all of their images to magically just appear there. What I needed was some form of glance replication.

I’d sat in on the glance replication session at the Folsom OpenStack Design Summit. The NeCTAR use case at the bottom is exactly what I wanted, so its reassuring that other people wanted something like that too. However, no one was working on this feature. So I wrote it. In fact, because of the code review process I wrote it twice, but let’s not dwell on that too much.

So, as of change id I7dabbd6671ec75a0052db58312054f611707bdcf there is a very simple replicator script in glance/bin. Its not perfect, and I expect it will need to be extended a bunch, but its a start at least and I’m using it in production now so I am relatively confident its not totally wrong.

The replicator supports the following commands at the moment:

livecopy

glance-replicator livecopy fromserver:port toserver:port

    Load the contents of one glance instance into another.

    fromserver:port: the location of the master glance instance.
    toserver:port:   the location of the slave glance instance.

This is the main meat of the replicator. Take a copy of the fromserver, and dump it onto the toserver. Only images visible to the user running the replicator will be copied if you’re using Keystone. Only images active on fromserver are copied across. The copy is done “on-the-wire”, so there are no large temporary files on the machine running the replicator to clean up.

dump

glance-replicator dump server:port path

    Dump the contents of a glance instance to local disk.

    server:port: the location of the glance instance.
    path:        a directory on disk to contain the data.

Do the same thing as livecopy, but dump the contents of the glance server to a directory on disk. This includes meta data and image data, and this directory is probably going to be quite large so be prepared.

load

glance-replicator load server:port path

    Load the contents of a local directory into glance.

    server:port: the location of the glance instance.
    path:        a directory on disk containing the data.

Load a directory created by the dump command into a glance server. dump / load was originally written because I had two glance servers who couldn’t talk to each other over the network for policy reasons. However, I could dump the data and move it to the destination network out of band. If you had a very large glance installation and were bringing up a new region at the end of a slow link, then this might be something you’d be interested in.

compare

glance-replicator compare fromserver:port toserver:port

    Compare the contents of fromserver with those of toserver.

    fromserver:port: the location of the master glance instance.
    toserver:port:   the location of the slave glance instance.

What would a livecopy do? The compare command will show you the differences between the two servers, so its a bit like a dry run of the replication.

size

glance-replicator size 

    Determine the size of a glance instance if dumped to disk.

    server:port: the location of the glance instance.

The size command will tell you how much disk is going to be used by image data in either a dump or a livecopy. It doesn’t however know about redundancy costs with things like swift, so it just gives you the raw number of bytes that would be written to the destination.

The glance replicator is very new code, so I wouldn’t be too surprised if there are bugs out there or obvious features that are lacking. For example, there is no support for SSL at the moment. Let me know if you have any comments or encounter problems using the replicator.