#Hadoop# #hadoop# #大資料#
1 概述HUE是一個開源的Apache Hadoop UI系統,早期由Cloudera開發,後來貢獻給開源社群。它是基於Python Web框架Django實現的。
1.1 功能、作用透過使用Hue我們可以透過瀏覽器方式操縱Hadoop叢集、HDFS、HBasse、Hive等元件。例如put、get、執行MapReduce Job等等。
2 環境設定2.1 直接從docker啟動docker pull sf2gis/hadoop:hue01
docker run -p 8122:22 -p 8188:8088 -p 50170:50070 -p 50175:50075 -p 8120:8020 -p 19888:19888 -p 60110:60010 -p 60130:60030 -p 8888:8888 -it --name hue01 --privileged -h hadoop-senior01.test.com --network=mynet --ip 172.18.0.2 sf2gis/hadoop:hue01
注意:需要HDFS支援。
埠:8888:web端
2.2 下載安裝下載的為原始碼,需要編譯後執行
下載:http://archive.cloudera.com/cdh5/cdh/5/hue-3.9.0-cdh5.9.0.tar.gz
安裝依賴包:
yum -y install ant asciidoc cyrus-sasl-devel cyrus-sasl-gssapi gcc gcc-c++ krb5-devel libtidy libxml2-devel libxslt-devel openldap-devel python-devel sqlite-devel openssl-devel mysql-devel gmp-devel
編譯:make apps
啟動驗證:build/env/bin/supervisor
開啟:ip:8888驗證網頁是否開啟。
3 應用3.1 連線hadoopfs_defaultfs=hdfs://localhost:8020
hadoop要啟用webhdfs
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
core-site.xml啟用對hue的支援
<property>
<name>hadoop.proxyuser.hue.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.hue.groups</name>
<value>*</value>
</property>
3.2 連線mysql:設定mysql連線屬性。[[[mysql]]]
# Name to show in the UI.
nice_name="My SQL DB"
# For MySQL and PostgreSQL, name is the name of the database.
# For Oracle, Name is instance of the Oracle server. For express edition
# this is 'xe' by default.
name=test
# Database backend to use. This can be:
# 1. mysql
# 2. postgresql
# 3. oracle
engine=mysql
# IP or hostname of the database to connect to.
host=192.168.71.2
# Port the database server is listening to. Defaults are:
# 1. MySQL: 3306
# 2. PostgreSQL: 5432
# 3. Oracle Express Edition: 1521
port=3306
# Username to authenticate with when connecting to the database.
user=root
# Password matching the username to authenticate with when
# connecting to the database.
password=root
# Database options to send to the server when connecting.
# https://docs.djangoproject.com/en/1.4/ref/databases/
## options={}