Atmosphere

Atmosphere is a hybrid computational cloud management framework which sits atop computational cloud sites and provides a consistent interface for creation and management of cloud-based services and applications. It provides a set of APIs as well as an embeddable administrative GUI through which computational clouds can be managed. Atmosphere also provides separate tools for developers of services and for their end users.

Atmosphere introduces an abstraction above cloud-based virtual machines by referring to services as Appliance types and their instances (VMs) as Appliances:

• An Appliance Type represents an image of a cloud-based service, along with ancillary metadata and all other types of information required to expose the service to end users. From the end user's perspective, an Appliance Type is just a service template which can be launched (instantiated) on demand.
• An Appliance represents a running instance of an Appliance Type. An Appliance is always spawned in the context of a specific user account, however Appliances have a many-to-many relation with virtual machines. For services which are declared as "shared", a single Virtual Machine may map to many Appliances, thus enabling many users to share a single VM even though each user sees their own "virtual service". For services which are declared as "scalable", the opposite happens - a user may launch an Appliance which Atmosphere will then link to many Virtual Machines, redirecting incoming requests to specific VMs in accordance with an internal optimization strategy.

The above abstraction enables the platform to conserve hardware resources while also providing scale-out capabilities for more demanding applications.

Atmosphere also resolves a host of other issues involved in operating computational clouds. It is capable of automatically creating redirections to services exposed by Appliances in private IP spaces, through the use of two types of proxies: a NAT-based port proxy for arbitrary interfaces, and a Nginx-based HTTP proxy for web interfaces (SOAP, REST, web applications etc.) A billing system is provided to keep track of financial resource usage and a set of monitoring workers are in place to validate the accessibility of service interfaces. A template migration mechanism is also provided whereby Appliance Types stored on one participating cloud site may be automatically propagated to other cloud sites, with on-the-fly image conversion where necessary.

The principal use of Atmosphere is in aggregating multiple computational cloud sites into a coherent infrastructure which the users perceive as a single, shared resource space.

Requirements

Supported Operating Systems

• Ubuntu 12.04
• Ubuntu 14.04

It may be possible to install Atmosphere on other operating systems. The above list only includes operating systems which have already been used for Atmosphere deployment in production mode. What is more, some commands used in the installation manual are Debian-specific (e.g. apt-get). If your OS uses a different package management system, you will need to modify these commands appropriately (e.g. by calling yum if you are using CentOS).

Ruby versions

Atmosphere requires Ruby (MRI) >= 2.1.

Hardware requirements

CPU

2 cores is the recommended minimum number of cores.

Memory

6GB is the recommended minimum memory size.

Note: The 25 workers of Sidekiq will show up as separate processes in your process overview (such as top or htop) but they share the same RAM allocation since Sidekiq is a multithreaded application.

Storage

The following components must reside in your attached storage:

• Atmosphere codebase (30MB)
• Atmosphere dependencies - gems (1GB)
• Application logs (3GB)
• Atmosphere database (Note: the volume of the database depends on how many compute sites will be integrated and how many Appliance Types and Appliances will be registered. 100MB should be enough for standard deployments.)

Redis and Sidekiq

Redis manages the background task queue. Storage requirements for Redis are minimal (on the order of 10 MB). Sidekiq processes background jobs using a multithreaded process. This process starts along with the entire Rails stack (1GB+) but it may grow over time, depending of the number of compute sites integrated with atmosphere and the number of instance-bound HTTP endpoints which need to be monitored. On a heavily loaded server the Sidekiq process may allocate 2GB+ of memory.

Getting started

Atmosphere works with Rails 4.1 onwards. You can add it to your Gemfile with:

gem 'atmosphere', github: 'dice-cyfronet/atmosphere', branch: 'master'

Run the bundle command to install it.

After you install Atmosphere and add it to your Gemfile, you need to run the generator:

rails generate atmosphere:install

The generator will install an initializer which describes ALL of Atmosphere's configuration options. It is imperative that you take a look at it. When you are done, you are can start your rails apps with Atmosphere mounted.

If you need to extend Atmosphere then you can install extensions by executing:

rails generate atmosphere:extensions

We use concerns to add additional, deployment specific logic into atmosphere. If you need additional extension points please create new issue.

New version

We are using continous delivery methodology. All features are implemented in branches, when feature is ready and tested then it is merged into master. As a conclusion it should be safe to use master in your project.

To upgrade to latest master run following commands:

# update atmosphere gem
bundle update atmosphere

# apply new migrations
rake db:migrate

Using atmosphere factories

Atmosphere factories are exposed into dependent projects in test environment. To use them you need to define following dependency in your Gemfile:

gem 'ffaker', '~>2.0.0'

Contributing

1. Fork the project
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create new pull request
6. When feature is ready add "ready for review" label and mention atmosphere development team (e.g. "/cc @dice-cyfronet/atmo-dev-team please take a look")