APITutorial
In this tutorial you will learn how to use the API. This API offers every functionality of the editor.
We will use the command-line client to access the API. All these functions can also be used in a script if you want to create automated scripts.
This tutorial assumes that you understand how the ToMaTo editor works. If you do not know ToMaTo, you should start with the web tutorials. It further assumes basic knowledge about python. You should be familiar with a Linux terminal.
First, clone the ToMaTo repository using Git in a temporary directory: git clone git@github.com:GLab/ToMaTo.git
Then, copy the cli directory inside this repository to a place you want it to be.
In order to connect, we have to start the command-line client with the needed parameters.
$> ./tomato.py -h master.tomato-lab.org -p 8000 -s -U john.doe
This creates a connection to the host master.tomato-lab.org on port 8000, using SSL, as the user john.doe. You must enter your own username here.
When prompted for the password
Password:
just enter it.
You can use help() in order to show a list of all available API commands. You can use help(command_name) in order to get more information (description, usage, etc.) about a command.
We can create a new topology by using the command topology_create():
>>> topology_create()
{'elements': [], 'connections': [], 'attrs': {'name': 'Topology #102', 'site': None}, 'timeout': 1420467725.234108, 'usage': None, 'id': 102, 'permissions': {'john.doe@': 'owner'}}
>>> top_id = _['id']
>>>
The command returns the info of the newly created topology. This is always a python dictionary. You need to remember the ID, which is 102 in this case. the second command saves this ID to a variable called top_id.
The topology contains additional information, like the timeout (when you have to renew the topology), permissions, a list of all elements and connections, etc.
If you want to see more information about the topology, you can always use the topology_info(102) method:
>>> topology_info(top_id)
{'elements': [], 'connections': [], 'attrs': {'name': 'Topology #102', 'site': None}, 'timeout': 1420467725.23411, 'usage': {'usage': {'traffic': 0.0, 'cputime': 0.0, 'diskspace': 0.0, 'memory': 0.0}, 'type': '5minutes', 'begin': 1420464000.0, 'measurements': 0, 'end': 1420464300.0}, 'id': 102, 'permissions': {'john.doe@': 'owner'}}
>>>
The difference to the return value of the first call was that this includes some usage statistics, which are automatically created by the backend.
We can change the name of this topology using the topology_modify command:
>>> topology_modify(top_id,{'name': 'My first try on using the API'})
{'elements': [], 'connections': [], 'attrs': {'name': 'My first try on using the API', 'site': None}, 'timeout': 1420467725.23411, 'usage': {'usage': {'traffic': 0.0, 'cputime': 0.0, 'diskspace': 0.0, 'memory': 0.0}, 'type': '5minutes', 'begin': 1420464300.0, 'measurements': 0, 'end': 1420464600.0}, 'id': 102, 'permissions': {'john.doe@': 'owner'}}
>>>
To create an element, we use the element_create(102,'openvz') command. This creates an OpenVZ on the topology 102:
>>> element_create(top_id,'openvz')
{'parent': None, 'info_sync_date': 0, 'state': 'created', 'id': 351, 'connection': None, 'attrs': {'profile': 'normal', 'name': 'openvz351', 'host_info': {'fileserver_port': None, 'problems': None, 'site': None, 'address': None}, 'hostname': 'openvz351', 'site': None, 'host': None, 'template': 'debian-7.0_x86'}, 'debug': {'host_connections': [], 'host_elements': []}, 'type': 'openvz', 'children': [], 'topology': 102}
>>> dev_id = _['id']
>>>
This element has the ID 351 (you can see this in the return value of the command), and is now visible in the topology info:
>>> topology_info(top_id)
{'elements': [351], 'connections': [], 'attrs': {'name': 'My first try on using the API', 'site': None}, 'timeout': 1420467725.23411, 'usage': {'usage': {'traffic': 0.0, 'cputime': 0.0, 'diskspace': 0.0, 'memory': 0.0}, 'type': '5minutes', 'begin': 1420464600.0, 'measurements': 0, 'end': 1420464900.0}, 'id': 102, 'permissions': {'john.doe@': 'owner'}}
>>>
The topology info contains a list of all elements, and a list of all connections.
We now want to change the template of our element. To do this, we first need to get a list of all available templates.
Templates are, like profiles, external networks, and external network interfaces, resources. This means, that there is no template_list command. Instead, you can use resource_list('template'), which lists all resources of type template.
However, this command creates a huge list of resources, including all the details about everything. We are only interested in the names of all linux templates for OpenVZ devices. Thus, we make use of the fact that we are using a python shell.
the API call returns a list of dictionaries. We can iterate through this list, and printing the names and descriptions of all templates of the subtype linux and tech openvz, using python syntax:
>>> for t in resource_list('template'):
... if t['attrs']['subtype'] == 'linux' and t['attrs']['tech'] == 'openvz':
... print t['attrs']['name'], '|', t['attrs']['description']
...
debian-6.0_x86 | Debian 6.0 (x86)
debian-7.0_x86 | Debian 7.0 (x86)
debian-6.0_x86_64 | Debian 6.0 (x86_64)
>>>
To change the template of the previously created element, we have to modify the template value of the element:
>>> element_modify(dev_id,{'template':'debian-6.0_x86'})
{'parent': None, 'info_sync_date': 0, 'state': 'created', 'id': 351, 'connection': None, 'attrs': {'profile': 'normal', 'name': 'openvz351', 'host_info': {'fileserver_port': None, 'problems': None, 'site': None, 'address': None}, 'hostname': 'openvz351', 'site': None, 'host': None, 'template': 'debian-6.0_x86'}, 'debug': {'host_connections': [], 'host_elements': []}, 'type': 'openvz', 'children': [], 'topology': 102}
>>>
You can try to change different settings if you want to. Some values must be references to other objects (like template), some can be freely changed (like name), and some are read-only (like state).
As a next step, we want to connect our element to the Internet. In the editor, this would mean to place an Internet element in the workspace, and then connect it to our OpenVZ device. So first, let's create a new internet device. To do this, we have to create an external_network element first, which we then set to be of the kind internet.
>>> element_create(top_id,"external_network")
{'parent': None, 'state': 'created', 'id': 352, 'connection': None, 'attrs': {'restricted': False, 'samenet': False, 'name': 'external_network352'}, 'debug': {'host_connections': [], 'host_elements': []}, 'type': 'external_network', 'children': [], 'topology': 102}
>>> inet_id = _['id']
>>> element_modify(inet_id,{'kind':'internet'})
{'parent': None, 'state': 'created', 'id': 352, 'connection': None, 'attrs': {'restricted': False, 'kind': 'internet', 'samenet': False, 'name': 'external_network352'}, 'debug': {'host_connections': [], 'host_elements': []}, 'type': 'external_network', 'children': [], 'topology': 102}
>>>
Similar to the list of templates, you can get all available types of external network by querying resource_list('network')
Connecting these two first requires us to create interfaces on the devices.
So let's create an openvz_interface for the OpenVZ device, and an external_network_endpoint for the external network:
>>> element_create(top_id,'openvz_interface',parent=dev_id)
{'parent': 351, 'state': 'created', 'id': 353, 'connection': None, 'attrs': {'name': 'eth0'}, 'debug': {'host_connections': [], 'host_elements': []}, 'type': 'openvz_interface', 'children': [], 'topology': 102}
>>> element_create(top_id,'external_network_endpoint',parent=inet_id)
{'parent': 352, 'state': 'created', 'id': 354, 'connection': None, 'attrs': {'name': 'external_network_endpoint354'}, 'debug': {'host_connections': [], 'host_elements': []}, 'type': 'external_network_endpoint', 'children': [], 'topology': 102}
>>>
Now, we can connect the interfaces:
>>> connection_create(dev_id,inet_id)
{'elements': [353, 354], 'state': 'created', 'attrs': {'host_info': {'fileserver_port': None, 'problems': None, 'site': None, 'address': None}, 'bandwidth_to': 10000, 'host': None, 'bandwidth_from': 10000, 'emulation': True}, 'debug': {'host_connections': [], 'host_elements': []}, 'type': 'fixed_bridge', 'id': 70}
>>>
This is a summary of what we have achieved so far
- created a topology (ID 102, top_id)
- created an OpenVZ device (ID 351, dev_id)
- created an external network (ID 352, inet_id), and network interfaces on both this network and the OpenVZ device
- created a connection (ID 70)
As a next step, we want to start our topology.
Before we start, we may want to renew our topology (To prevent topology timeout). To do this, use the renew action:
>>> topology_action(top_id,'renew',{'timeout':3*60*60*24})
>>>
Now, we can either start each device and connection individually, or let the backend handle this by calling an action on the topology.
To control the topology, you can use the topology_action(id,action) command.
However, in this tutorial, we want to run the commands individually. You can run actions on each element or connection using the command element_action(id,action), e.g. element_action(id,'prepare'), or connection_action(id,action) respectively.
In this case, you only need to control the root elements (i.e., the OpenVZ device and the external network). All child elements and connections automatically have the same state as their parents; the connection is only started when both sides are started.
There is no need to prepare the external network; it can be started directly.
To prepare our OpenVZ device, run
>>> element_action(dev_id,'prepare')
>>>
You can see that the interface (ID 353) has also been prepared by fetching its element_info.
Now, we can start both elements:
>>> element_action(dev_id, 'start')
>>> element_action(inet_id,'start')
>>>
First, let's import json so we can get a better readable output:
>>> import json
>>>
You can now use print json.dumps(res, indent=2) (where res is the result of a command) after a command to get a better structured result.
You will now see that the element_info of the element has some more attributes:
>>> el_inf = element_info(dev_id)
{'parent': None, 'info_sync_date': 1420729239.524069, 'state': 'started', 'id': 351, 'connection': None, 'attrs': {'profile': 'normal', 'vncpassword': 'e91a11fc29cab36b30d19583432c7084', 'rextfv_supported': True, 'vncport': 6650, 'diskspace': 10240, 'host_info': {'fileserver_port': 8888, 'problems': [], 'site': 'ukl', 'address': '131.246.112.50'}, 'websocket_pid': 278232, 'ram': 512, 'vmid': 1621, 'site': None, 'websocket_port': 6222, 'vncpid': 278190, 'host': 'ukl.50', 'template': 'debian-6.0_x86', 'rextfv_run_status': {'readable': False}, 'rextfv_max_size': None, 'last_sync': 1420729239.524069, 'hostname': 'openvz351', 'name': 'openvz351'}, 'debug': {'host_connections': [], 'host_elements': [['ukl.50', 300]]}, 'type': 'openvz', 'children': [353], 'topology': 102}
>>> print json.dumps(el_inf, indent=2)
{'parent': None,
'info_sync_date': 1420729239.524069,
'state': 'started',
'id': 351,
'connection': None,
'attrs': {'profile': 'normal',
'vncpassword':
'e91a11fc29cab36b30d19583432c7084',
'rextfv_supported': True,
'vncport': 6650,
'diskspace': 10240,
'host_info': {'fileserver_port': 8888,
'problems': [],
'site': 'ukl',
'address': '131.246.112.50'},
'websocket_pid': 278232,
'ram': 512,
'vmid': 1621,
'site': None,
'websocket_port': 6222,
'vncpid': 278190,
'host': 'ukl.50',
'template': 'debian-6.0_x86',
'rextfv_run_status': {'readable': False},
'rextfv_max_size': None,
'last_sync': 1420729239.524069,
'hostname': 'openvz351',
'name': 'openvz351'},
'debug': {'host_connections': [],
'host_elements': [['ukl.50', 300]]},
'type': 'openvz',
'children': [353],
'topology': 102}
>>>
For accessing the device, we are interested in the VNC info. For this, we need: the host's address (attrs/host_info/address), the VNC port (attrs/vncport), and the VNC password (attrs/vncpassword). You can get this via the command:
>>> print "vnc://"+el_inf["attrs"]["host_info"]["address"]+":"+el_inf["attrs"]["host_info"]["port"]
or, including the password:
>>> print "vnc://:"+el_inf["attrs"]["vncpassword"]+"@"+el_inf["attrs"]["host_info"]["address"]+":"+el_inf["attrs"]["vncport"]
you can use this information (host, port, password) to access your device with any VNC client.
This section is about uploading or downloading disk images or executable archives.
The tomato library offers upload and download functions which help us performing uploads and downloads. You have to import them to the cli client first:
>>> from .lib import upload,download
>>>
First, you need some information about the element's host which you can get from the element info. attrs/host_info/address attrs/host_info/fileserver_port
In order to download a disk image, you need a so-called grant. You can get it via an action. Since our device is currently started, you have to stop it first.
>>> element_action(dev_id,"stop")
>>> element_action(dev_id,"download_grant")
'a83d2277fcac3c1e7c7d849f0eaecb92'
>>>
You can then combine the host information and the grant to get the download URL: http://[address]:[port]/[grant]/download. The download function from the library takes the URL and a target filename as arguments. So in our case, we run:
>>> download('http://131.246.112.50:8888/a83d2277fcac3c1e7c7d849f0eaecb92/download','/home/t-gerhard/dl.tar.gz')
>>>
The upload process is similar: You need a grant to upload, and then upload the file via HTTP. After this, you have to tell the system that the upload is completed and it shall use the upload:
>>> element_action(dev_id,"upload_grant")
'd471f771cae79ecd72bb41fb17bade7c'
>>> upload('http://131.246.112.50:8888/d471f771cae79ecd72bb41fb17bade7c/upload','/home/t-gerhard/dl.tar.gz')
>>> element_action(dev_id,"upload_use")
>>>
In this example, the previously downloaded file was uploaded. Of course, you could have chosen another disk image as well.
To upload or download executable archives, you can use the actions rextfv_upload_grant, rextfv_upload_use, or rextfv_download_grant similarly. The status of execution can be accessed in the element info.
After you have finished your experiment, you should shut down your topology. To do this, you can simply run
>>> topology_action(top_id,"destroy")
To completely remove the topology, run
topology_remove(top_id)
If you want to find out more (e.g., how to create a switch), you can inspect the elements in the editor. In the Options tab, you can enable Show IDs and Debug. The first one will show you element and connection IDs in the respective right-click menus. The Latter will grant you the Debug entry in all right-click menus, which you can use to view element_info or connection_info directly. Use this to examine what you want to do, and then re-create this in the command-line client.
You can also use the API documentation.