What is a Libcloud driver ?

Most users uses Libcloud only for make request against a compute service. In fact the library covers several IaaS components:

• Authentication
• Compute service
• Object storage and CDN
• DNS
• Load balancers

A driver is a Python class which after instanciation will allow you to handle one the service listed above (except authentication).

So, what's a compute driver ?

As written before, the driver is the API used for handle a cloud service. In compute example, basic compute drivers should implement the following public methods:

• list_nodes
• create_node
• destroy_node
• list_images
• list_sizes

Whatever driver you use these methods are available and sometimes with custom parameters prefixed by ex_. Example, some driver may allow to filter images by family, method's definition could be:

def list_images(self, ex_family=None):
    ...

Same for custom method, they are prefixed with ex_. Example, a clone function could be:

def ex_clone_node(self, node, name):
    ....

Easy, I create a lot of methods with requests ?

No, Libcloud integrate every components of Cloud ecosystem and it contains a catalog of classes for describe all. A basic cloud as describe before will need the following objects:

• libcloud.compute.base.NodeDriver: Base driver class, must be subclassed for create a compute Driver. It gathers all methods for interact with a compute.
• libcloud.compute.base.NodeImage: Define an OS image used to boot from
• libcloud.compute.base.NodeSize: Define size/flavor with common attributes as vCPU, memory or boot disk size
• libcloud.compute.base.Node: Define VM/instance with common specification, size, image and more
• libcloud.common.base.Connection: Define how to interact with a service

Except NodeDriver which must be subclassed, you can use this classes directly or create children if they don't match with your need or if you feel you will reuse this.

A little bit more about Connection ?

Connection is defined as attribute connectionCls of a NodeDriver. During NodeDriver instantiation, it is instantiated and set as connection attribute. Driver will use it for every connection to its provider. The most important part of this object is Connection.request, an HTTP client method which does:

1. Add parameters which should be used in every request, through method Connection.add_default_params
2. Add headers which should be used in every request, throught method Connection.add_default_headers
3. Control cache for GET requests
4. Make HTTP/HTTPS request
5. Handle response with desired format set in Connection.responseCls

The one who wanted to use requests in driver wil be served, here's the method's signature:

def request(self, action, params=None, data=None, headers=None, method='GET', raw=False):

Nothing more simple !

I couldn't write about Connection without drop some words about LoggingConnection. In Libcloud behavior, if the environment variable LIBCLOUD_DEBUG is defined, all requests will be print in curl command format, into the file set by variable. I personaly use as following:

$ LIBCLOUD_DEBUG=/dev/stderr
$ python mylibcloud_script.py

or more simple:

$ LIBCLOUD_DEBUG=/dev/stderr python mylibcloud_script.py

The output will be like below:

# -------- begin 140581194640080 request ----------
curl -i -X GET -H 'Host: api.runabove.com' -H 'X-LC-Request-ID: 140581194640080' -H 'Accept-Encoding: gzip,deflate' -H 'User-Agent: libcloud/0.18.0 ' --compress https://api.runabove.com:443//1.0/auth/time
# -------- begin 140581194640080:140581086770888 response ----------
HTTP/1.1 200 OK
Content-Length: 10
X-Ra-Queryid: CA.ws-2.55cb5668.27574.1561
Server: Apache/2.2.20 (Unix) mod_ssl/2.2.20 OpenSSL/0.9.8o mod-xslt/1.3.9
Cache-Control: no-cache
Date: Wed, 12 Aug 2015 14:21:28 GMT
Access-Control-Allow-Origin: *
Content-Type: application/json; charset=utf-8

1439389288
# -------- end 140581194640080:140581086770888 response ----------

Ok, do you have a fast last thing, I've got some work ? 

I wrote above about Node, NodeImage and NodeSize. Driver is also in charge to convert data returned by connection (raw, JSON, XML or more) into one of theses objects. I'm doing it quickly, a driver has private method for this job and the below example will be the most adapted:

An cloud's API answer the following JSON when we list nodes:

[
{'name': 'node1', 'id': '06', 'ip': '1.2.3.4', 'state': 'running', 'description': 'Trolololo'},
{'name': 'node2', 'id': '02', 'ip': '1.2.3.5', 'state': 'stopped', 'description': ''}
]

Conversion from JSON into a Node is made with _to_node, see thefollowing code:

from libcloud.compute.base import NodeDriver, Node
from libcloud.compute.types import NodeState


class MyDriver(NodeDriver):
    ...

    # Mapping between cloud and libcloud states
    NODE_STATE_MAP = {
        'running': NodeState.RUNNING,
        'stopped': NodeState.STOPPED
    }

    def list_nodes(self):
        # Make request and get response
        response = self.connection.request('/nodes')
        # Get list of dict from JSON reponse
        objs = response.object
        # Convert to Node objects
        nodes = self._to_nodes(response)
        return nodes

    def _to_node(self, obj):
        # Gather needed informations
        _id = obj['id']
        name = obj['name']
        state = self.NODE_STATE_MAP[obj['status']]
        public_ips = [obj['ip']]
        # Put non-libcloud info in extra dict
        extra = {'description': obj['description']}
        # Return a Node object
        return Node(id=_id, name=name, state=state, public_ips=public_ips,
                    extra=extra, driver=self)

    def _to_nodes(self, objs):
        # Fastly launch self._to_node with multiple node
        # In some drivers, it can make more
        return [self._to_node(obj) for obj in objs]

Simple like Bonjour, you write a method _to_node which format response into Node object and you use it when necessary. Same thing for other components: NodeSize should have a _to_size and _to_sizes and NodeImage, _to_image and _to_images.

Reference and link

• Libcloud GitHub repo