Integrate Savanna and Swift

Registered by Nadya Privalova

The Swift service is a standard object storage in OpenStack environment, analog of Amazon S3. As a rule it is deployed on bare metal machines. It is natural to expect Hadoop on OpenStack to process data stored there. There are a couple of enhancements on the way which can help there.

First, a FileSystem implementation for Swift: https://issues.apache.org/jira/browse/HADOOP-8545. With that thing in place, Hadoop jobs can work with Swift as naturally as with HDFS.

On the Swift side, we have the change request: https://review.openstack.org/#/c/21015/ (available in Grizzly). It implements the ability to list endpoints for an object, account or container, to make it possible to integrate swift with software that relies on data locality information to avoid network overhead.

We'd like to introduce this functionality to Savanna and provide users detailed HOW-TO documentation.

Blueprint information

Status:
Complete
Approver:
Sergey Lukjanov
Priority:
High
Drafter:
Nadya Privalova
Direction:
Approved
Assignee:
Nadya Privalova
Definition:
Approved
Series goal:
Accepted for 0.2
Implementation:
Implemented
Milestone target:
milestone icon 0.2a1
Started by
Nadya Privalova
Completed by
Nadya Privalova

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/savanna-swift-integration,n,z

Addressed by: https://review.openstack.org/32432
    Helper for Swift integration was added.

Addressed by: https://review.openstack.org/31532
    Implementation of Vanilla Plugin

Gerrit topic: https://review.openstack.org/#q,topic:bp/vanilla-plugin-configs-improvements,n,z

Addressed by: https://review.openstack.org/33755
    Applied Swift Integration in Vanilla Plugin:

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.