HBase operation example code analysis in Python

黄舟

Release： 2017-05-07 11:02:51

Original

1760 people have browsed it

Installing HBase

HBase is a distributed column storage system built on HDFS, mainly used for massive structured data storage. Here, our goal is just to provide a basic environment for Python to access HBase, so we download the binary package directly and install it on a single machine. After downloading, unzip it, modify the configuration file, and then start HBase directly. The system version used is ubuntu14.04.

Download

wget https://mirrors.tuna.tsinghua.edu.cn/apache/hbase/1.2.4/hbase-1.2.4-bin.tar.gz tar zxvf hbase-1.2.4-bin.tar.gz

Copy after login

Configuration

Modify hbase-env.sh and set JAVA_HOME.

export JAVA_HOME=/usr/lib/jvm/java-8-oracle

Copy after login

Modify hbase-site.xml and set the root directory for storing data.

<configuration>
    <property>
        <name>hbase.rootdir</name>
        <value>file:///home/mi/work/hbase/data</value>
    </property></configuration>

Copy after login

Start

bin/start-hbase.sh  # 启动bin/hbase shell  # 进入hbase交互shell

Copy after login

Install Thrift

After installing HBase, you need to install Thrift, because when calling HBase in other languages, you need to connect through Thrift.

Installing Thrift dependencies

sudo apt-get install automake bison flex g++ git libboost1.55 libevent-dev libssl-dev libtool make pkg-config

Copy after login

PS: libboost1.55-all-dev, there was a problem with the installation on my ubuntu14.04, so I installed libboost1.55.

Compile and install

Download the source code, unzip it and compile and install. Thrift download address

tar zxf thrift-0.10.0.tar.gzcd thrift-0.10.0/./configure 
--with-cpp --with-boost --with-python --without-csharp --with-java 
--without-erlang --without-perl --with-php --without-php_extension --without-ruby --without-haskell  
--without-gomake  # 编译耗时较长sudo make install

Copy after login

Start the Thrift service of HBase

bin/hbase-daemon.sh start thrift

Copy after login

Check the system process

~/work/hbase/hbase-1.2.4/conf$ jps3009 ThriftServer4184 HMaster5932 Jps733 Main

Copy after login

You can see that the ThriftServer has been started successfully, and then we can use multiple languages through Thrift comes to access HBase.

Python operation HBase

The following uses Python as an example to demonstrate how to access HBase.

Install dependency packages

sudo pip install thriftsudo pip install hbase-thrift

Copy after login

Demo program

from thrift import Thriftfrom thrift.transport 
import TSocketfrom thrift.transport import TTransportfrom thrift.protocol 
import TBinaryProtocolfrom hbase import Hbasefrom hbase.ttypes import *

transport = TSocket.TSocket(&#39;localhost&#39;, 9090)

transport = TTransport.TBufferedTransport(transport)
protocol = TBinaryProtocol.TBinaryProtocol(transport)

client = Hbase.Client(protocol)
transport.open()

contents = ColumnDescriptor(name=&#39;cf:&#39;, maxVersions=1)# client.deleteTable(&#39;test&#39;)client.createTable(&#39;test&#39;, [contents])print client.getTableNames()# insert datatransport.open()

row = &#39;row-key1&#39;mutations = [Mutation(column="cf:a", value="1")]
client.mutateRow(&#39;test&#39;, row, mutations)
# get one rowtableName = &#39;test&#39;rowKey = &#39;row-key1&#39;result = client.getRow(tableName, rowKey)
print resultfor r in result:    
print &#39;the row is &#39;, r.row    
print &#39;the values is &#39;, r.columns.get(&#39;cf:a&#39;).value

Copy after login

Execution results:

[&#39;test&#39;]
[TRowResult(columns={&#39;cf:a&#39;: TCell(timestamp=1488617173254, value=&#39;1&#39;)}, row=&#39;row-key1&#39;)]
the row is  row-key1
the values is  1

Copy after login

The above is the detailed content of HBase operation example code analysis in Python. For more information, please follow other related articles on the PHP Chinese website!