読者です 読者をやめる 読者になる 読者になる

脳汁portal

アメリカ在住(だった)新米エンジニアがその日学んだIT知識を書き綴るブログ

LeoFSのv1.2.8をCentOS6.5にsingle hostとしてinstallしてみた

LeoFS

RIT(Rakuten Institute of Technology)が作成したオブジェクトストレージ

leo-project.net
2015/05/13現在でv1.2.8が最新

Install

  • LeoFSにはstorage・Manager・Gatewayと3種類のサーバがあり、通常はStabilityの観点からもそれぞれ別のサーバに立てた方がよいが、今回は検証用として全て1サーバ(single host)で作成する

Procedure

"パッケージダウンロード"
$ cd ~
$ export VERSION=1.2.8   ### change by yourself
$ echo ${VERSION}
$ wget http://leo-project.net/leofs/packages/rpm/x86_64/leofs-${VERSION}-1.el6.x86_64.rpm
$ sudo rpm -ivh leofs-${VERSION}-1.el6.x86_64.rpm
$ ls -l /usr/local/leofs/
$ sudo chown -R {your user}:${your user} /usr/local/leofs/${VERSION}

$ sudo vi /etc/hosts
## Replace {BUCKET_NAME} with the name of the bucket ##
##今回はleobucket.localhostにする
==============================================================
127.0.0.1 localhost {BUCKET_NAME}.localhost
==============================================================

$ cd /usr/local/leofs/${VERSION}
$ leo_manager_0/bin/leo_manager start
$ leo_manager_1/bin/leo_manager start
$ leo_storage/bin/leo_storage start

$ ./leofs-adm start
Generating RING...
Generated RING
OK 100% - storage_0@127.0.0.1
OK

$ leo_gateway/bin/leo_gateway start

Status Check

$ ./leofs-adm status
 [System Confiuration]
-----------------------------------+----------
 Item                              | Value
-----------------------------------+----------
 Basic/Consistency level
-----------------------------------+----------
                    system version | 1.2.8  # versionチェック
                        cluster Id | leofs_1
                             DC Id | dc_1
                    Total replicas | 1
          number of successes of R | 1
          number of successes of W | 1
          number of successes of D | 1
 number of rack-awareness replicas | 0
                         ring size | 2^128
-----------------------------------+----------
 Multi DC replication settings
-----------------------------------+----------
        max number of joinable DCs | 2
           number of replicas a DC | 1
-----------------------------------+----------
 Manager RING hash
-----------------------------------+----------
                 current ring-hash | 433fe365
                previous ring-hash | 433fe365
-----------------------------------+----------
 [State of Node(s)]
-------+--------------------------+--------------+----------------+----------------+----------------------------
 type  |           node           |    state     |  current ring  |   prev ring    |          updated at
-------+--------------------------+--------------+----------------+----------------+----------------------------
  S    | storage_0@127.0.0.1      | running      | 433fe365       | 433fe365       | 2015-05-08 05:37:22 +0000 # ステータスチェック
  G    | gateway_0@127.0.0.1      | running      | 433fe365       | 433fe365       | 2015-05-08 05:37:55 +0000 # ステータスチェック
-------+--------------------------+--------------+----------------+----------------+----------------------------

Try to Use LeoFS

"ユーザー作成"
$ ./leofs-adm create-user leo
  access-key-id: ***********************
  secret-access-key: ************************************
###↑後で使うのでメモっておく

"Ruby install"
# yum install zlib-devel openssl-devel sqlite-devel
# curl -sSL https://rvm.io/mpapis.asc | sudo gpg2 --import -
# curl -L https://get.rvm.io | bash -s stable
 
# source /etc/profile.d/rvm.sh
# rvm -v
 
" Ruby installation "
# rvm requirements
# rvm list known
# rvm install 2.1.2
# rvm use 2.1.2 --default
# rvm list
# ruby -v
# gem list
  
# gpasswd -a leo leo
# chown -R leo:leo /usr/local/rvm/gems
  
$ gem install aws-sdk -v "~>1" # 公式のドキュメントがversion1系を使用しているため、1系をinstall
$ gem list | grep aws

LeoFSにはS3コマンドやDragon Diskなどからアクセスできるが、今回は選択肢の一つであるRubyのgemのaws-sdkを用いてスクリプトを作る

$ cd ~
### テスト用の画像ファイルを用意する(今回はtest.png)

" テストスクリプトの作成 "
$ vi connect.rb
#ここから=======================================================================================================================
## This code supports "aws-sdk v1.9.5"
require "aws-sdk"

puts "CONNECT======================="
Endpoint = "localhost"
Port = 8080
# set your s3 key
AccessKeyId = "***************************" # ユーザ作成した際に表示されたもの
SecretAccessKey = "*********************************" # ユーザ作成した際に表示されたもの
class LeoFSHandler < AWS::Core::Http::NetHttpHandler
  def handle(request, response)
    request.port = ::Port
    super
  end
end
SP = AWS::Core::CredentialProviders::StaticProvider.new(
{
    :access_key_id     => AccessKeyId,
    :secret_access_key => SecretAccessKey
})
AWS.config(
  access_key_id: AccessKeyId,
  secret_access_key: SecretAccessKey,
  s3_endpoint: Endpoint,
  http_handler: LeoFSHandler.new,
  credential_provider: SP,
  s3_force_path_style: true,
  use_ssl: false
)
s3 = AWS::S3.new
puts s3.class #確認
puts s3 #確認


puts "MAKE BUCKET======================="
# create bucket
s3.buckets.create("photo")
# get bucket
bucket = s3.buckets["photo"]

 
puts "PUT DATA==========================="
# create a new object
object = bucket.objects.create("image", "value")
# show objects in the bucket
bucket.objects.with_prefix("").each do |obj|
  p obj
end
# retrieve an object
object = bucket.objects["image"]
# insert an object
object.write(
  file: "./test.png", # LeoFSに保存したい画像ファイルを指定
  content_type: "png/image"
)
puts object.class # 確認
puts object # 確認
 

puts "GET DATA============================"
# get bucket
bucket = s3.buckets["photo"]
# retrieve an object
object = bucket.objects["image"]
image = object.read
puts image.class # 確認
#puts image
File.write('validate.png', image) # getした画像ファイルを別名で保存

 
puts "GET Meta Data==================="
bucket = s3.buckets["photo"]
object = bucket.objects["image"]
metadata = object.head
p metadata.to_hash # 確認

#ここまで======================================================================================================================
 

$ ruby connect.rb
CONNECT=======================
AWS::S3
#<AWS::S3:0x000000024c74b0>
PUT DATA=======================
<AWS::S3::S3Object:photo/image>
AWS::S3::S3Object
#<AWS::S3::S3Object:0x00000002db3998>
GET DATA============================
String
GET Meta Data===================
{:meta=>{}, :restore_in_progress=>false, :content_type=>"application/octet-stream", :etag=>"\"ec560ee18a1b78ec4617e54ad0f311e7\"", :last_modified=>2015-05-08 06:21:46 +0000, :content_length=>22879}
 
 
$ ls -la
total 52
-rw-rw-r-- 1 leo leo  1662 May  8 06:35 connect.rb
-rw-rw-r-- 1 leo leo 22879 Dec 12 02:57 test.png
-rw-rw-r-- 1 leo leo 22879 May  8 06:34 validate.png
 
$ diff test.png validate.png
###差分なし
 
### 実際にファイルを開いてみて、画像に問題がないか確認する


Reference

http://leo-project.net/leofs/docs/getting_started/getting_started_1.html