Compute driver that uses BOTO API to utilize EC2.
Go to file
OpenDev Sysadmins 8672472c50 OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:51:59 +00:00
img Adding image to update the Readme 2015-02-06 11:09:44 +05:30
tempest Ed & Cameron | Add accounts.yaml and update readme 2014-11-21 15:43:33 -06:00
tests Ed & Cameron | Copy from_port and to_port as strings in EC2RuleTransformer, refactoring transformers 2014-11-06 11:31:51 -06:00
.gitignore kashyap | spawn now specifies availability zone, added test for detach volume 2014-10-24 15:32:09 +05:30
.gitreview OpenDev Migration Patch 2019-04-19 19:51:59 +00:00
README.md Venu | Updated OpenStack summit 2015 video 2015-07-10 15:01:24 +05:30
Vagrantfile Venu | amended readme 2015-03-13 15:19:35 +05:30
__init__.py Kashyap | Corrected import to be just ec2 2014-08-12 10:13:45 +00:00
cloud_burst_filter.py Venu & kashyap | added the filter used for cloud bursting 2014-11-12 15:32:00 +05:30
credentials.py Venu | Updated License 2014-11-28 17:44:50 +05:30
ec2_group_transformer.py Venu | Updated License 2014-11-28 17:44:50 +05:30
ec2_rule_service.py Venu | Updated License 2014-11-28 17:44:50 +05:30
ec2_rule_transformer.py Venu | Updated License 2014-11-28 17:44:50 +05:30
ec2driver.py Venu | Spice console 2015-03-18 20:21:09 +05:30
ec2driver_config.py Venu | Updated License 2014-11-28 17:44:50 +05:30
ec2driver_standard_config.py Removed placement and added subnet_id to run_instances. Added 2014-12-15 17:46:54 +05:30
ec2driver_test_config.py Venu | Updated License 2014-11-28 17:44:50 +05:30
group.py Venu | Updated License 2014-11-28 17:44:50 +05:30
group_rule_refresher.py Venu | Updated License 2014-11-28 17:44:50 +05:30
instance_rule_refresher.py Venu | Updated License 2014-11-28 17:44:50 +05:30
openstack_group_service.py Venu | Updated License 2014-11-28 17:44:50 +05:30
openstack_group_transformer.py Venu | Updated License 2014-11-28 17:44:50 +05:30
openstack_rule_service.py Venu | Updated License 2014-11-28 17:44:50 +05:30
openstack_rule_transformer.py Venu | Updated License 2014-11-28 17:44:50 +05:30
rule.py Venu | Updated License 2014-11-28 17:44:50 +05:30
rule_comparator.py Venu | Updated License 2014-11-28 17:44:50 +05:30

README.md

ThoughtWorks OpenStack to EC2 Driver

Thank you for your interest in this project. This is a ThoughtWorks internal R&D project to make Hybrid cloud real using OpenStack! Using this enterprises can enjoy the benefits of the private cloud without being limited by it. In the event of all the Private cloud resources being utilised to their maximum capacity, the traffic can be managed by bursting to the public cloud for extra capacity.

For now we are focusing on being able to burst to Amazon EC2.

Whether you are using the OpenStack native Dashboard—Horizon Or Command Line Interface (CLI) Or OpenStack APIs Or Orchestrating using Heat this Driver will provide the level of abstraction that is required to treat Amazon EC2 or any public cloud service provider as just another hypervisor with unlimited resources.

Getting Started

  • OpenStack Icehouse/Juno installed
  • Python 2.7 and above
  • Amazon Web Service (AWS) SDK for Python -- Boto 2.34

Quick Setup

  1. $ cd <openstack_root_dir>/nova/nova/virt/

  2. $ git clone https://github.com/ThoughtWorksInc/OpenStack-EC2-Driver.git ec2

  3. $ vim /etc/nova/nova.conf # add the following options in the respective sections

     [DEFAULT]
     compute_driver=ec2.EC2Driver
    
     [conductor]
     use_local=True
    
     [ec2driver]
     ec2_secret_access_key = <your_aws_secret_access_key>
     ec2_access_key_id = <your_aws_access_key_id>
    
  4. ec2driver_standard_config.py can be edited to configure the default AMI, AWS region and endpoints.

  5. Restart the nova-compute service. You are now all set cloud burst!

To see it in action, watch this video

ScreenShot

We were honoured to present this at the OpenStack summit 2015, you can watch the video here.

Multi-node hybrid cloud

This will enable a hybrid cloud infrastucture using this driver on one of the multiple compute-hosts. Follow the Quick setup to install the EC2 Driver on a (psuedo) compute-host in it's own availability zone.

  1. $ cd <openstack_root_dir>/nova/nova/virt/

  2. $ cp ./cloud_burst_filter.py ../scheduler/filters/

  3. $ vim /etc/nova/nova.conf # add the following options in the respective sections

     [DEFAULT]
     cloud_burst = # Switch to enable could bursting
     cloud_burst_availability_zone = # The availability zone of only compute hosts with the public cloud driver
    
     scheduler_driver = nova.scheduler.filter_scheduler.FilterScheduler
     scheduler_available_filters = nova.scheduler.filters.all_filters
     scheduler_default_filters = RetryFilter, AvailabilityZoneFilter, RamFilter, ComputeFilter, ComputeCapabilitiesFilter, ImagePropertiesFilter, ServerGroupAntiAffinityFilter, ServerGroupAffinityFilter, CloudBurstFilter
    
  4. Restart nova-api, nova-compute and nova-scheduler services for the filter to take effect.

What's supported!

  • Launch
  • Reboot
  • Terminate
  • Resize
  • Config drive / User Data
  • Pause/Unpause*
  • Suspend/Resume*
  • Attach/Detach/Swap Volume
  • Snapshot
  • Security Groups
  • Nova Diagnostics

###Some more to be added!

  • Spice, VNC and RDP Console
  • Serial Console Output
  • iSCSI
  • Service Control

#For Contributors

###Instructions for Developer Environment setup

  1. Install git, Virtualbox and Vagrant.
  2. $ git clone https://github.com/ThoughtWorksInc/OpenStack-EC2-Driver.git ec2
  3. $ cd ec2/ && vagrant up This will download the development environment from Vagrant clound and setup devstack.
  4. $ vagrant ssh
  5. Edit nova.conf and add the ec2 configuration options, refer to step 3 in Quick setup guide.
  6. Restart nova-compute
  • ~/devstack/rejoin-stack.sh
  • go to the nova-cpu screen (ctrl+a, 7)
  • restart the process with ctrl+c, press up, and then enter
  • go to nova-api (screen 6), and repeat

The driver should now be loaded. The contents of the repository is mapped to /opt/stack/nova/nova/virt/ec2/, and you can edit it directly from your host computer with an IDE of your choice.

###Running Tests

  1. Moto can be used to mock the EC2 server. To install moto, run sudo pip install moto.
  2. To optionally use Moto, run source /opt/stack/nova/nova/virt/ec2/tests/setup_moto.sh.
  3. ~/devstack/rejoin-stack.sh
  4. cd /opt/stack/nova/nova/virt/ec2/tests
  5. Use nosetests -s test_ec2driver.py
  6. To stop Moto, run source /opt/stack/nova/nova/virt/ec2/tests/shutdown_moto.sh.

###Using tempest/tempest.conf to run tempest tests

  1. Clone the tempest repo from https://github.com/openstack/tempest
  2. ln -s ~/nova/nova/virt/ec2/tempest/tempest.conf <path to tempest repo>/etc/tempest.conf
  3. ln -s ~/nova/nova/virt/ec2/tempest/accounts.yaml <path to tempest repo>/etc/accounts.yaml

* In Amazons EC2 there is no concept of suspend and resume on instances. Therefore, we simply stop EC2 instances when suspended and start the instances when resumed, we do the same on pause and un-pause.

###License

Copyright (c) 2015 ThoughtWorks
All Rights Reserved.

Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at

  http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.