Monday, October 27, 2014

OpenStack Swift container creation error.

I was following the https://github.com/swiftstack/kinetic-swift/wiki/Deployment page to deploy Swift using Kinetic drives.

After creating the account, container, and object rings I started Swift using “swift-init start main”. The Swift processes came up fine, and I was also able to do “swift stat”.

Next, when I tried to create a container called foobar, I got this error message.

user@vm:~$ sudo swift -U test:tester -K testing -A http://localhost:8080/auth/v1.0 post foobar
Container PUT failed: http://localhost:8080/v1/AUTH_test/foobar 404 Not Found  [first 60 chars of response] <html><h1>Not Found</h1><p>The resource could not be found.<

Looking at /var/log/syslog showed that there was an issue with creating the accounts directory in /swift/sdv due to permission issues.

user@vm:~$ tail -f /var/log/syslog
Oct 21 15:22:34 vm account-server: ERROR __call__ error with PUT /sdv/802/AUTH_test : #012Traceback (most recent call last):#012  File "/home/user/git/kinetic-swift/swift/swift/account/server.py", line 274, in __call__#012    res = method(req)#012  File "/home/user/git/kinetic-swift/swift/swift/common/utils.py", line 2422, in wrapped#012    return func(*a, **kw)#012  File "/home/user/git/kinetic-swift/swift/swift/common/utils.py", line 1023, in _timing_stats#012    resp = func(ctrl, *args, **kwargs)#012  File "/home/user/git/kinetic-swift/swift/swift/account/server.py", line 142, in PUT#012    broker.initialize(timestamp.internal)#012  File "/home/user/git/kinetic-swift/swift/swift/common/db.py", line 232, in initialize#012    mkdirs(self.db_dir)#012  File "/home/user/git/kinetic-swift/swift/swift/common/utils.py", line 740, in mkdirs#012    os.makedirs(path)#012  File "/usr/lib/python2.7/os.py", line 150, in makedirs#012    makedirs(head, mode)#012  File "/usr/lib/python2.7/os.py", line 150, in makedirs#012    makedirs(head, mode)#012  File "/usr/lib/python2.7/os.py", line 150, in makedirs#012    makedirs(head, mode)#012  File "/usr/lib/python2.7/os.py", line 157, in makedirs#012    mkdir(name, mode)#012OSError: [Errno 13] Permission denied: '/swift/sdv/accounts' (txn: txc2152db00f0a4ee5ba0b8-005446dcaa)
Oct 21 15:22:34 vm account-server: 127.0.0.1 - - [21/Oct/2014:22:22:34 +0000] "PUT /sdv/802/AUTH_test" 500 1132 "-" "txc2152db00f0a4ee5ba0b8-005446dcaa" "-" 0.0143 "-"
Oct 21 15:22:34 vm proxy-server: Container GET returning 503 for (503,) (txn: txc2152db00f0a4ee5ba0b8-005446dcaa) (client_ip: 127.0.0.1)
Oct 21 15:22:34 vm proxy-server: Could not autocreate account '/AUTH_test' (txn: txc2152db00f0a4ee5ba0b8-005446dcaa) (client_ip: 127.0.0.1)
Oct 21 15:22:34 vm proxy-server: 127.0.0.1 127.0.0.1 21/Oct/2014/22/22/34 PUT /v1/AUTH_test/foobar HTTP/1.0 404 - python-swiftclient-1.8.0.7.g775a24b AUTH_tkceea92749... - 70 - txc2152db00f0a4ee5ba0b8-005446dcaa - 0.0204 - - 1413930154.009783983 1413930154.030232906  


user@vm:/swift$ ls -l total 4 d-wxrw-r-t 2 root root 4096 Oct 17 16:04

Fix:
user@vm:/swift$ sudo chmod o+rw /swift/sdv


Friday, October 10, 2014

Theoretical Maximum throughput of various ports

When designing a balanced systems these numbers can come pretty handy, particularly the Ethernet port maximum theoretical throughputs and FC Port maximum theoretical throughputs.

If you are deploying a Swift cluster, the throughput you deliver to your clients is very dependent on the number of outward facing NICs on Porxy server. Note that if you Proxy has 2 x 10 Gbps NICs, one outward-facing and one cluster-facing, then the theoretical maximum throughput that you can deliver from that proxy to the clients is 1.25 GB/s irrespective of how many drives you have on the backend. The way to scale would be to add additional NIC's to the proxy server, or adding another Proxy server to the cluster.

In the case of deploying Relational Databases like Oracle, if you are interested in more throughout, then just adding spindles won't work. What you need is a balanced system, meaning if you are looking to get 6 GB/s then make sure you don't bottleneck on the controller. If you need 6 GB/s then you would need to have 3 x 16 Gbps FC ports, 6 x 8 Gbps FC ports, or 12 x 4 Gbps FC ports on your storage controller.


SpeedMax. Throughput
Wireless 802.11b11 Mbps1.375 MB/s
Wireless 802.11g54 Mbps6.75MB/s
Wireless 802.11n300 Mbps37.5 MB/s
Ethernet port 1 Gbps
125 MB/s

10 Gbps
1.25 GB/s
FC Port4 Gbps500 MB/s

8 Gbps1 GB/s

16 Gbps2 GB/s

Monday, October 6, 2014

Swift deployments classified by size

Here is a slide on the different types of Swift deployments classified small, small/medium, medium/large, and large deployment.

This slide is from the “Optimizing Hardware for OpenStack Object Storage
http://www.youtube.com/watch?v=9R2BmQslBzk