I’m working in a SUSE Cloud 5 environment consisting of 4 nodes, one admin node (VM, SLES11), one control node (physical, SLES11) and two compute nodes (both physical, one with SLES11, one with SLES12). The OpenStack version is Juno.
This article describes how to assign specific fixed ip addresses to your Cloud instances with nova boot command even though this feature is not supported.
Background
In my current tests I have set up a practice environment for different users. They are supposed to use identical network settings and images, but each user in his or her own project. When launching an instance, the default cloud behavior is to assign a random ip address from the network you selected. To demonstrate that, I created 6 VMs simultaneously (server2-XXXXX), look at the assigned ip addresses:
root@d0c-c4-7a-06-72-96:~ # nova list +--------------------------------------+----------------------------------------------+---------+------------+-------------+-------------------+ | ID | Name | Status | Task State | Power State | Networks | +--------------------------------------+----------------------------------------------+---------+------------+-------------+-------------------+ | 16f95c29-d89d-4029-b63c-e7702f8f8bf7 | server2-16f95c29-d89d-4029-b63c-e7702f8f8bf7 | ACTIVE | - | Running | vnet1=172.17.2.6 | | 30d5ac57-159c-4f27-97fc-2727fd40630b | server2-30d5ac57-159c-4f27-97fc-2727fd40630b | ACTIVE | - | Running | vnet1=172.17.2.24 | | 5f2b6081-3647-47fa-86bc-43c7d79e0ad3 | server2-5f2b6081-3647-47fa-86bc-43c7d79e0ad3 | ACTIVE | - | Running | vnet1=172.17.2.26 | | 85b06201-a8c8-4158-96b8-3304a1096f9c | server2-85b06201-a8c8-4158-96b8-3304a1096f9c | ACTIVE | - | Running | vnet1=172.17.2.5 | | aa33ad31-0fe6-44e3-8920-aaad074fb17b | server2-aa33ad31-0fe6-44e3-8920-aaad074fb17b | ACTIVE | - | Running | vnet1=172.17.2.7 | | f2170206-a779-449a-b7cf-95b9b997194c | server2-f2170206-a779-449a-b7cf-95b9b997194c | ACTIVE | - | Running | vnet1=172.17.2.25 | +--------------------------------------+----------------------------------------------+---------+------------+-------------+-------------------+
Possible Problem
But what if I wanted to assign a specific ip address to a new VM?
This is impossible if you are using the Horizon dashboard. You would have to switch to the CLI and use the nova boot command, for example:
nova boot --flavor 1 --image <IMAGE-ID> --nic net-id=<NET-ID>,v4-fixed-ip=172.17.2.21 server3
Depending on your software environment, this can lead to following error in nova-compute.log (on both compute nodes):
2015-09-15 10:57:24.785 2344 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s) 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager Traceback (most recent call last): 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1646, in _allocate_network_async 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager dhcp_options=dhcp_options) 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 443, in allocate_for_instance 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager self._delete_ports(neutron, instance, created_port_ids) 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line 82, in __exit__ 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager six.reraise(self.type_, self.value, self.tb) 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 423, in allocate_for_instance 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager security_group_ids, available_macs, dhcp_opts) 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 214, in _create_port 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager port_id = port_client.create_port(port_req_body)['port']['id'] 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/__init__.py", line 84, in wrapper 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager ret = obj(*args, **kwargs) 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 98, in with_params 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager ret = self.function(instance, *args, **kwargs) 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 322, in create_port 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager return self.post(self.ports_path, body=body) 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/__init__.py", line 84, in wrapper 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager ret = obj(*args, **kwargs) 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 1325, in post 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager headers=headers, params=params) 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/__init__.py", line 84, in wrapper 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager ret = obj(*args, **kwargs) 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 1236, in do_request 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager body = self.serialize(body) 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/__init__.py", line 84, in wrapper 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager ret = obj(*args, **kwargs) 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 1266, in serialize 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager self.get_attr_metadata()).serialize(data, self.content_type()) 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/neutronclient/common/serializer.py", line 390, in serialize 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager return self._get_serialize_handler(content_type).serialize(data) 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/neutronclient/common/serializer.py", line 54, in serialize 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager return self.dispatch(data, action=action) 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/neutronclient/common/serializer.py", line 44, in dispatch 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager return action_method(*args, **kwargs) 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/neutronclient/common/serializer.py", line 66, in default 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager return jsonutils.dumps(data, default=sanitizer) 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/neutronclient/openstack/common/jsonutils.py", line 168, in dumps 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager return json.dumps(value, default=default, **kwargs) 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager File "/usr/lib64/python2.7/json/__init__.py", line 250, in dumps 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager sort_keys=sort_keys, **kw).encode(obj) 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager File "/usr/lib64/python2.7/json/encoder.py", line 207, in encode 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager chunks = self.iterencode(o, _one_shot=True) 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager File "/usr/lib64/python2.7/json/encoder.py", line 270, in iterencode 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager return _iterencode(o, 0) 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/neutronclient/common/serializer.py", line 65, in sanitizer 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager return six.text_type(obj, 'utf8') 2015-09-15 10:57:24.785 2344 TRACE nova.compute.manager TypeError: coercing to Unicode: need string or buffer, IPAddress found
As I already wrote at the beginning, this cloud setup runs with OpenStack Juno.
Unfortunately, the v4-fixed-ip option is not supported in Juno, read the comment here. It says
Not really a bug, moreover a lack of feature, but it is documented: “Networking does not currently support the v4-fixed-ip parameter of the –nic option for the nova command”
Workaround
It also offers a workaround with neutron and nova:
If you want to specify an ip address you have to use:
$ neutron port-create –fixed-ip subnet_id=SUBNET_ID,ip_address=IP_ADDRESS NET_ID
$ nova boot –image IMAGE –flavor FLAVOR –nic port-id=PORT_ID VM_NAME
In practice, this means you have to complete the following steps to assign a specific fixed ip address to your instance. First I list all networks with their corresponding subnets to get their IDs:
root@d0c-c4-7a-06-72-96:~ # neutron net-list +--------------------------------------+----------+-------------------------------------------------------+ | id | name | subnets | +--------------------------------------+----------+-------------------------------------------------------+ | c0d4b1ba-e6e4-424e-afc2-2e6df086ba6f | floating | 4aa1b32c-6d92-4388-885b-899251d763e9 192.168.126.0/24 | | e423e6c2-54e1-4e41-99bb-231427cc7c8f | fixed | 817193c3-ade4-48ec-a063-d4635dc03f05 192.168.123.0/24 | | 64b1717f-6fbc-4c8c-92e4-dcc37b95e082 | nde_127 | ea12ab1a-2c19-4572-b2be-a67b03eee812 192.168.127.0/24 | | f470a175-7711-45e3-b162-b0c51919adda | vnet1 | 6e50d1a0-b045-4800-b904-c5e7c9d48b6d 172.17.2.0/24 | | e152412d-61d8-4eef-b523-19ec4bc641cd | vnet1 | 1c4b50e5-aaa7-45d8-ad01-07c697951f6c 172.17.2.0/24 | | 8d4d1f70-5d03-4b80-901f-e4587c02fe1e | vnet1 | e56f0893-79a9-4eac-ac75-7f7ec25ba03a 172.17.2.0/24 | +--------------------------------------+----------+-------------------------------------------------------+
To identify the correct net-id you could look in Horizon dashboard into your project details. As you can see in this example, neutron lists all networks, even if you have sourced the credentials for a specific project. The advantage of using neutron net-list compared to nova net-list is that you don’t have to execute another command to list the subnets:
root@d0c-c4-7a-06-72-96:~ # nova net-list +--------------------------------------+----------+------+ | ID | Label | CIDR | +--------------------------------------+----------+------+ | c0d4b1ba-e6e4-424e-afc2-2e6df086ba6f | floating | None | | e423e6c2-54e1-4e41-99bb-231427cc7c8f | fixed | None | | 64b1717f-6fbc-4c8c-92e4-dcc37b95e082 | nde_127 | None | | f470a175-7711-45e3-b162-b0c51919adda | vnet1 | None | | e152412d-61d8-4eef-b523-19ec4bc641cd | vnet1 | None | | 8d4d1f70-5d03-4b80-901f-e4587c02fe1e | vnet1 | None | +--------------------------------------+----------+------+
Now that I have my net-id and my subnet-id, I need to create a port:
root@d0c-c4-7a-06-72-96:~ # neutron port-create --fixed-ip subnet_id=e56f0893-79a9-4eac-ac75-7f7ec25ba03a,ip_address=172.17.2.21 8d4d1f70-5d03-4b80-901f-e4587c02fe1e
Created a new port:
+-----------------------+------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+------------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:host_id | |
| binding:profile | {} |
| binding:vif_details | {} |
| binding:vif_type | unbound |
| binding:vnic_type | normal |
| device_id | |
| device_owner | |
| fixed_ips | {"subnet_id": "e56f0893-79a9-4eac-ac75-7f7ec25ba03a", "ip_address": "172.17.2.21"} |
| id | d81c9ee5-5fc2-46b6-a04e-bedff222240b |
| mac_address | fa:16:3e:8e:68:b2 |
| name | |
| network_id | 8d4d1f70-5d03-4b80-901f-e4587c02fe1e |
| security_groups | 6ad13e65-bc05-4a0c-8d13-3fe0c3018423 |
| status | DOWN |
| tenant_id | c32d23772eff4a49a2662103f63fd411 |
+-----------------------+------------------------------------------------------------------------------------+
Finally, I can boot my instance and assign the desired ip address to it, using the port-id I just created:
root@d0c-c4-7a-06-72-96:~ # nova boot --flavor 1 --image 63b4996d-c8e9-44e4-850b-75c064a1d189 --nic port-id=d81c9ee5-5fc2-46b6-a04e-bedff222240b server3
+--------------------------------------+-----------------------------------------------+
| Property | Value |
+--------------------------------------+-----------------------------------------------+
| created | 2015-09-15T14:21:57Z |
| flavor | m1.tiny (1) |
| hostId | |
| id | a826b44e-a3df-4ed2-b8f5-a2cb25d1ded1 |
| image | cirros (63b4996d-c8e9-44e4-850b-75c064a1d189) |
| key_name | - |
| metadata | {} |
| name | server3 |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| security_groups | default |
| status | BUILD |
| tenant_id | c32d23772eff4a49a2662103f63fd411 |
| updated | 2015-09-15T14:21:58Z |
| user_id | 85f54aca144449bcb44420d8f710f303 |
+--------------------------------------+-----------------------------------------------+
The instance is running and has a correctly assigned ip address:
root@d0c-c4-7a-06-72-96:~ # nova show server3
+--------------------------------------+----------------------------------------------------------+
| Property | Value |
+--------------------------------------+----------------------------------------------------------+
| created | 2015-09-15T14:21:57Z |
| flavor | m1.tiny (1) |
| hostId | 5f32f887ac847e2a3a8a799b3d7aa40d0a40066cf58ac1a8540dd342 |
| id | a826b44e-a3df-4ed2-b8f5-a2cb25d1ded1 |
| image | cirros (63b4996d-c8e9-44e4-850b-75c064a1d189) |
| key_name | - |
| metadata | {} |
| name | server3 |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| security_groups | default |
| status | ACTIVE |
| tenant_id | c32d23772eff4a49a2662103f63fd411 |
| updated | 2015-09-15T14:22:03Z |
| user_id | 85f54aca144449bcb44420d8f710f303 |
| vnet1 network | 172.17.2.21 |
+--------------------------------------+----------------------------------------------------------+
You’ll have to admit, this workaround is not very elegant, although it works perfectly fine.
For OpenStack’s Kilo release there is a fix available, read the details, it was released on February 5, 2015, the review is here.
A fix for Juno release is also available, see here, last update is from March 10, 2015.
This fix seems not to be part of the current SUSE Cloud release.
If you want to use that option in nova boot command anyway, you can fix it yourself!
But please be aware that you could lose any Support claims if you decide to change the sources. If you are aware, keep reading ;-)
Solution
There are several ways to achieve that, I tested two of them. Both require only minor changes in the file /usr/lib64/python2.6/site-packages/nova/network/neutronv2/api.py on a SLES11 compute node or in the file /usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py on a SLES12 compute node.
The first method I tested was to import jsonutils and convert the requested ip address into a readable format for the function allocate_for_instance in api.py. The differences between the modified and the original file:
root@d0c-c4-7a-06-72-c6:/usr/lib64/python2.6/site-packages/nova/network/neutronv2 # diff -c2 api.py api.py.dist
*** api.py Tue Sep 15 14:29:00 2015
--- api.py.dist Wed Feb 11 13:19:25 2015
***************
*** 21,25 ****
from neutronclient.common import exceptions as neutron_client_exc
from oslo.config import cfg
! from oslo.serialization import jsonutils
from nova.api.openstack import extensions
from nova.compute import flavors
--- 21,25 ----
from neutronclient.common import exceptions as neutron_client_exc
from oslo.config import cfg
!
from nova.api.openstack import extensions
from nova.compute import flavors
***************
*** 418,425 ****
ports_in_requested_order.append(port['id'])
else:
- address = jsonutils.to_primitive(request.address)
created_port = self._create_port(
port_client, instance, request.network_id,
! port_req_body, address,
security_group_ids, available_macs, dhcp_opts)
created_port_ids.append(created_port)
--- 418,424 ----
ports_in_requested_order.append(port['id'])
else:
created_port = self._create_port(
port_client, instance, request.network_id,
! port_req_body, request.address,
security_group_ids, available_macs, dhcp_opts)
created_port_ids.append(created_port)
See the details here. After restarting nova-compute service you can assign your desired ip address to a new instance:
root@d0c-c4-7a-06-72-96:~ # nova boot --flavor 1 --image 63b4996d-c8e9-44e4-850b-75c064a1d189 --availability-zone host_sles11 --nic net-id=8d4d1f70-5d03-4b80-901f-e4587c02fe1e,v4-fixed-ip=172.17.2.21 server3
+--------------------------------------+-----------------------------------------------+
| Property | Value |
+--------------------------------------+-----------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | - |
| OS-EXT-SRV-ATTR:hypervisor_hostname | - |
| OS-EXT-SRV-ATTR:instance_name | instance-00000145 |
| OS-EXT-STS:power_state | 0 |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | - |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| adminPass | Zb3c99ZsFTp5 |
| config_drive | |
| created | 2015-09-16T10:25:40Z |
| flavor | m1.tiny (1) |
| hostId | |
| id | 55dca376-9c79-4976-9597-dba72ca25683 |
| image | cirros (63b4996d-c8e9-44e4-850b-75c064a1d189) |
| key_name | - |
| metadata | {} |
| name | server3 |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| security_groups | default |
| status | BUILD |
| tenant_id | c32d23772eff4a49a2662103f63fd411 |
| updated | 2015-09-16T10:25:40Z |
| user_id | 85f54aca144449bcb44420d8f710f303 |
+--------------------------------------+-----------------------------------------------+
root@d0c-c4-7a-06-72-96:~ # nova list
+--------------------------------------+---------+---------+------------+-------------+-------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+---------+---------+------------+-------------+-------------------+
| 0969b03a-e206-4501-a344-4cf542d89673 | server1 | SHUTOFF | - | Shutdown | vnet1=172.17.2.2 |
| 55dca376-9c79-4976-9597-dba72ca25683 | server3 | ACTIVE | - | Running | vnet1=172.17.2.21 |
+--------------------------------------+---------+---------+------------+-------------+-------------------+
Another option is to convert the ip address directly when requesting it during port creation, this way you need to change only one single line:
root@d0c-c4-7a-06-71-f0:/usr/lib/python2.7/site-packages/nova/network/neutronv2 # diff -c2 api.py api.py.dist
*** api.py Tue Sep 15 14:51:58 2015
--- api.py.dist Wed Feb 11 13:19:25 2015
***************
*** 198,202 ****
try:
if fixed_ip:
! port_req_body['port']['fixed_ips'] = [{'ip_address': str(fixed_ip)}]
port_req_body['port']['network_id'] = network_id
port_req_body['port']['admin_state_up'] = True
--- 198,202 ----
try:
if fixed_ip:
! port_req_body['port']['fixed_ips'] = [{'ip_address': fixed_ip}]
port_req_body['port']['network_id'] = network_id
port_req_body['port']['admin_state_up'] = True
Restart the nova-compute service and launch a new instance:
root@d0c-c4-7a-06-72-96:~ # nova boot --flavor 1 --image 63b4996d-c8e9-44e4-850b-75c064a1d189 --availability-zone host_sles12 --nic net-id=8d4d1f70-5d03-4b80-901f-e4587c02fe1e,v4-fixed-ip=172.17.2.22 server4
+--------------------------------------+-----------------------------------------------+
| Property | Value |
+--------------------------------------+-----------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | - |
| OS-EXT-SRV-ATTR:hypervisor_hostname | - |
| OS-EXT-SRV-ATTR:instance_name | instance-00000146 |
| OS-EXT-STS:power_state | 0 |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | - |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| adminPass | EbpR4Z55V5Ms |
| config_drive | |
| created | 2015-09-16T10:31:14Z |
| flavor | m1.tiny (1) |
| hostId | |
| id | 0229c13d-c95d-448b-96aa-88abc3a855dc |
| image | cirros (63b4996d-c8e9-44e4-850b-75c064a1d189) |
| key_name | - |
| metadata | {} |
| name | server4 |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| security_groups | default |
| status | BUILD |
| tenant_id | c32d23772eff4a49a2662103f63fd411 |
| updated | 2015-09-16T10:31:14Z |
| user_id | 85f54aca144449bcb44420d8f710f303 |
+--------------------------------------+-----------------------------------------------+
root@d0c-c4-7a-06-72-96:~ # nova list
+--------------------------------------+---------+---------+------------+-------------+-------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+---------+---------+------------+-------------+-------------------+
| 0969b03a-e206-4501-a344-4cf542d89673 | server1 | SHUTOFF | - | Shutdown | vnet1=172.17.2.2 |
| 55dca376-9c79-4976-9597-dba72ca25683 | server3 | ACTIVE | - | Running | vnet1=172.17.2.21 |
| 0229c13d-c95d-448b-96aa-88abc3a855dc | server4 | ACTIVE | - | Running | vnet1=172.17.2.22 |
+--------------------------------------+---------+---------+------------+-------------+-------------------+
Both ways lead to the same result, you can launch instances with a specific ip address with nova boot without the workaround through neutron. This fix will probably be available in the official SUSE Cloud channels, too, but that’s just an assumption.
Personally, I would find it even better if you could assign a specific ip to your instance in Horizon dashboard, but it doesn’t seem to be designed that way.