1.ambari介绍及安装

1. 什么是ambari？

Apache Ambari是一种基于Web的工具，支持Apache Hadoop集群的供应、管理和监控。Ambari已支持大多数Hadoop组件，包括HDFS、MapReduce、Hive、Pig、 Hbase、Zookeper、Sqoop和Hcatalog等。

Apache Ambari 支持HDFS、MapReduce、Hive、Pig、Hbase、Zookeper、Sqoop和Hcatalog等的集中管理。也是5个顶级hadoop管理工具之一。Ambari能够安装安全的（基于Kerberos）Hadoop集群，以此实现了对Hadoop 安全的支持，提供了基于角色的用户认证、授权和审计功能，并为用户管理集成了LDAP和Active Directory。

2. 为什么用ambari？

Cloudera有免费版和企业版，企业版只有60天试用期，不过基本上免费版也满足生产环境的使用，监控工具为中文，不过Cloudera的代码不开源，只能通过产生本身的BUG修改补丁升级等，不支持hadoop2.7.x。所以选择Ambari部署hadoop而不是CDH，是因为CDH最新版本只支持Hadoop2.6.X，Ambari支持Hadoop2.7.3。 100%开源的产品，HDP除了常见的项目外还包含了Ambari，一款开源的安装和管理系统。HCatalog，一个元数据管理系统。

3. ambari各版本兼容hdp版本对照

ambari及hdp版本：https://supportmatrix.hortonworks.com/ 选择ambari版本2.6.2.2，选择对应的HDP版本是2.6.5

4. hdp2.6.5与各组件对照

hdp各组件版本：https://docs.cloudera.com/HDPDocuments/HDP2/HDP-2.6.5/bk_release-notes/content/comp_versions.html

5. 安装ambari

下载ambari：https://docs.cloudera.com/HDPDocuments/Ambari-2.6.2.2/bk_ambari-installation/content/ambari_repositories.html wget http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.2.2/ambari-2.6.2.2-centos7.tar.gz
下载HDP及HDP-UTILS： https://docs.cloudera.com/HDPDocuments/Ambari-2.6.2.2/bk_ambari-installation/content/hdp_26_repositories.html wget http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/HDP-2.6.5.0-centos7-rpm.tar.gz wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz
执行预CDH安装脚本：advance-cdh.sh
安装jdk
修改umask值（所有机器）： sudo sh -c “echo umask 0022 » /etc/profile”
永久设置(所有机器) vi /etc/security/limits.conf

# End of file
添加
* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072

重新登录，ulimit -a 7. 启动httpd，将上面下载的三个包放到/var/www/html/ambari目录下（主服务器）

cd /var/www/html
mkdir ambari
#上传资源到此目录
ls
#ambari-2.6.2.2-centos7.tar.gz HDP-UTILS-1.1.0.22-centos7.tar.gz HDP-2.6.5.0-centos7-rpm.tar.gz
tar zxvf ambari-2.6.2.2-centos7.tar.gz
tar zxvf HDP-2.6.5.0-centos7-rpm.tar.gz
mkdir HDP-UTILS
tar zxvf HDP-UTILS-1.1.0.22-centos7.tar.gz -C HDP-UTILS
rm -rf ./*.gz
ls
#ambari  HDP  HDP-UTILS

访问:http://cdh-01/ambari/ 验证是否能够访问

制作本地源

安装yum源制作工具 yum install yum-utils createrepo yum-plugin-priorities -y cd /var/www/html/ambari createrepo ./
制作本地源 vim ambari/centos7/2.6.2.2-1/ambari.repo

############修改前#################
#VERSION_NUMBER=2.6.2.2-1
[ambari-2.6.2.2]
name=ambari Version - ambari-2.6.2.2
baseurl=http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.2.2
gpgcheck=1
gpgkey=http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.2.2/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
#############修改后#################
VERSION_NUMBER=2.6.2.2-1
[ambari-2.6.2.2]
name=ambari Version - ambari-2.6.2.2
baseurl=http://cdh-01/ambari/ambari/centos7/2.6.2.2-1/
gpgcheck=1
gpgkey=http://cdh-01/ambari/ambari/centos7/2.6.2.2-1/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

cp ambari/centos7/2.6.2.2-1/ambari.repo /etc/yum.repos.d/

############修改前#################
#VERSION_NUMBER=2.6.5.0-292
[HDP-2.6.5.0]
name=HDP Version - HDP-2.6.5.0
baseurl=http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0
gpgcheck=1
gpgkey=http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

[HDP-UTILS-1.1.0.22]
name=HDP-UTILS Version - HDP-UTILS-1.1.0.22
baseurl=http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7
gpgcheck=1
gpgkey=http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
############修改后#################
#VERSION_NUMBER=2.6.5.0-292
[HDP-2.6.5.0]
name=HDP Version - HDP-2.6.5.0
baseurl=http://cdh-01/ambari/HDP/centos7/2.6.5.0-292/
gpgcheck=1
gpgkey=http://cdh-01/ambari/HDP/centos7/2.6.5.0-292/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

[HDP-UTILS-1.1.0.22]
name=HDP-UTILS Version - HDP-UTILS-1.1.0.22
baseurl=http://cdh-01/ambari/HDP-UTILS/HDP-UTILS/centos7/1.1.0.22/
gpgcheck=1
gpgkey=http://cdh-01/ambari/HDP-UTILS/HDP-UTILS/centos7/1.1.0.22/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

cp HDP/centos7/2.6.5.0-292/hdp.repo /etc/yum.repos.d/ yum clean all yum makecache yum repolist

源标识源名称状态 HDP-2.6.5.0 HDP Version - HDP-2.6.5.0 238 HDP-UTILS-1.1.0.22 HDP-UTILS Version - HDP-UTILS-1.1.0.22 16 ambari-2.6.2.2 ambari Version - ambari-2.6.2.2 12

拷贝源文件到子节点

cd /etc/yum.repos.d
scp ambari.repo hdp.repo cdh-02:`pwd`
scp ambari.repo hdp.repo cdh-03:`pwd`

安装ambari

安装ambari yum install ambari-server -y
登录mysql，执行下面语句

CREATE DATABASE ambari;  
use ambari;  
CREATE USER 'ambari'@'%' IDENTIFIED BY 'ambarizk123';  
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%';  
CREATE USER 'ambari'@'localhost' IDENTIFIED BY 'ambarizk123';  
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'localhost';  
CREATE USER 'ambari'@'master' IDENTIFIED BY 'ambarizk123';  
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'master';  
FLUSH PRIVILEGES;  
source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql  
show tables;  
use mysql;  
select Host,User from user where User='ambari';  
CREATE DATABASE hive;  
use hive;  
CREATE USER 'hive'@'%' IDENTIFIED BY 'hive';  
GRANT ALL PRIVILEGES ON *.* TO 'hive'@'%';  
CREATE USER 'hive'@'localhost' IDENTIFIED BY 'hive';  
GRANT ALL PRIVILEGES ON *.* TO 'hive'@'localhost';  
CREATE USER 'hive'@'master' IDENTIFIED BY 'hive';  
GRANT ALL PRIVILEGES ON *.* TO 'hive'@'master';  
FLUSH PRIVILEGES;  
CREATE DATABASE oozie;  
use oozie;  
CREATE USER 'oozie'@'%' IDENTIFIED BY 'oozie';  
GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'%';  
CREATE USER 'oozie'@'localhost' IDENTIFIED BY 'oozie';  
GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'localhost';  
CREATE USER 'oozie'@'master' IDENTIFIED BY 'oozie';  
GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'master';  
FLUSH PRIVILEGES;

上传mysql连接驱动，并复制到目标文件夹 mkdir /usr/share/java cp ~/mysql-connector-java-5.1.36.jar /usr/share/java/mysql-connector-java.jar cp /usr/share/java/mysql-connector-java.jar /var/lib/ambari-server/resources/mysql-jdbc-driver.jar
初始化设置ambari-server并启动 [root@cdh-01 ~]# ambari-server setup

Using python  /usr/bin/python
Setup ambari-server
（1）检查防火墙是否关闭
Checking SELinux...
SELinux status is 'disabled'
（2）提示是否自定义设置。输入：y
Customize user account for ambari-server daemon [y/n] (n)? y
（3）如果直接回车就是默认选择root用户
如果输入已经创建的用户就会显示：ambari
Enter user account for ambari-server daemon (root):ambari
Adjusting ambari-server permissions and ownership...
Checking firewall status...
（4）设置JDK。输入：3
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[3] Custom JDK
==============================================================================
Enter choice (1): 3
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
如果上面选择3自定义JDK,则需要设置JAVA_HOME。输入：/usr/java/jdk1.8.0_161
Path to JAVA_HOME: /usr/local/jdk1.8.0_171
Validating JDK on Ambari Server...done.
Checking GPL software agreement...
GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
（5）数据库配置。选择：y
Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? y
Completing setup...
Configuring database...
选择：y
Enter advanced database configuration [y/n] (n)? y
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
（6）选择数据库类型。输入：3
Enter choice (1): 3
（7）设置数据库的具体配置信息，根据实际情况输入，如果和括号内相同，则可以直接回车。如果想重命名，就输入。
Hostname (localhost):
Port (3306):
Database name (ambari):
Username (ambari):
Enter Database Password (bigdata):
Re-enter password:
Configuring ambari database...
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
（8）将Ambari数据库脚本导入到数据库
Proceed with configuring remote database connection properties [y/n] (y)? y
Extracting system views...
ambari-admin-2.6.2.2.1.jar
...........
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.

启动ambari ambari-server start

安装安装配置部署HDP集群

1. 访问页面：http://cdh-01:8080/

登录界面，默认管理员账户登录，账户：admin 密码：admin

2. 安装向导

配置集群的名字为hadoop
选择版本并修改为本地源地址 http://cdh-01/ambari/HDP/centos7/2.6.5.0-292/ http://cdh-01/ambari/HDP-UTILS/HDP-UTILS/centos7/1.1.0.22/

3. 安装配置填写需要安装的主机名；选择要安装的服务器和上传上面教程所拷贝出来的秘钥文件/root/.ssh/id_rsa 确认 4. 安装ambari的agent，同时检查系统问题等待检查，状态变化：Installing->Registering->Success，然后做一些与检查检查结果查看，没有错误，接着下一步 5. 选择要安装的服务，默认会全装，此次安装只选择hdfs，yarn，zookeeper 6. 会自动选择增加ambari监控 7. 调整组件分配 8. 选择建议nodemanager、regionserver、client全都勾选上，第一个datanode根据主机的磁盘容量选择。 9. 根据提示修改一些错误，注意hive和oozie的数据库配置问题。如果最开始初始化ambari-server的时候使用的是默认配置，这里就最好使用new的方式，如果使用的是New MySQL Database ，这需要选择Existing MySQL / MariaDB Database ，然后测试一下是否能连接上添加grafana及Activity Explorer的密码为admin，admin 10. 接着下一步安装 11. 自动安装 12. 部署完成 13.

报错

mysql
页面安装hdp的时候，注册失败

NetUtil.py:96 - EOF occurred in violation of protocol (_ssl.c:579) NetUtil.py:97 - SSLError: Failed to connect. Please check openssl library versions.

解决办法：在ambari-agent的配置文件ambari-agent.ini 在［security］增加一项 sed -i '/ssl_ver/a\force_https_protocol=PROTOCOL_TLSv1_2' /etc/ambari-agent/conf/ambari-agent.ini 增加好之后重试