Tuesday, November 11, 2014

Useful Swift presentations from OpenStack Summit 2015

Building Applications with Swift: The Swift Developer On-Ramp
http://www.youtube.com/watch?v=rAGjrqiuoC8&list=UUQ74G2gKXdpwZkXEsclzcrA

Extensibility in Swift
http://www.youtube.com/watch?v=7DmwFvf504k&list=UUQ74G2gKXdpwZkXEsclzcrA

Swift 101: Technology and Architecture for Beginners
http://www.youtube.com/watch?v=6wh1QD7nQxo&list=UUQ74G2gKXdpwZkXEsclzcrA

Sponsor Demo Theater - Swiftstack: Swift Storage Policies with SwiftStack - Live Demo
http://www.youtube.com/watch?v=rnvDf9CLKXg&list=UUQ74G2gKXdpwZkXEsclzcrA

Vagrant Up a Stand-alone, Multi-Node OpenStack Swift Cluster
http://www.youtube.com/watch?v=8qkO14n31vw&list=UUQ74G2gKXdpwZkXEsclzcrA

Docker Meets Swift: A Broadcaster's Experience
http://www.youtube.com/watch?v=naB4RzISELA&list=UUQ74G2gKXdpwZkXEsclzcrA

The Perfect Match: Apache Spark Meets Swift
http://www.youtube.com/watch?v=7tqMT0arV2k&list=UUQ74G2gKXdpwZkXEsclzcrA

Using OpenStack Swift for Extreme Data Durability
http://www.youtube.com/watch?v=5EnujUjuegw&list=UUQ74G2gKXdpwZkXEsclzcrA

Taking the Mystery out of Erasure Codes: A Swift Implementation
http://www.youtube.com/watch?v=igDANtL1jRU&list=UUQ74G2gKXdpwZkXEsclzcrA

Monday, November 10, 2014

Ring Ring Ring!!!

In OpenStack Swift there are 3 Rings, an Account ring, a Container ring, and an Object ring.

What are these Rings and what are they made of?
The rings are modified consistent hash rings that hold partitions that are on the physical disks.

So what are partitions?
Think of partitions as directories that hold the actual data.

And how may partitions are there on the a disk drive?
Swift documentation recommends a minimum of 100 partitions per drive. We have seen customers use up to 1000 partitions per drive.

What are these Rings used for?
The ring is used to help us find the information and data that we are looking for when do a GET. During a PUT, the ring tells us where(physical disk) the data will be stored.

What do the data structures in the Ring tell us?
Actually there are two internal data structures. The first one tells us where the three replications of a partition are stored eg. Partition 0 is on device 7, 12, and 1. While partition 2 is on device 1, 8 and 10


The second data structure tells us where and how to find these devices. Eg. Device 1 is in region 2, zone 1, with the following IP address and port number.