CDH Hadoop集群虛擬機本地配置搭建筆記_part2_配置yum local repo
CDH安裝就不同客戶環境提供多種安裝方式
1.是透過Cloudera官方網站，來直接進行完全自動化安裝。
(目前這條路已不通，除非付費跟註冊有會員訂閱)
CDH官網沒提供免費早期CDH5,CDH6(Cloudera Manager也無法免費用)下載的因應策略
2.使用者的集群環境可能屬於公司內網，不可以訪問外網。
則須在集群內部來建立一個軟體repo，再來進行自動化集群部署。
一般大部分私人企業內部網路管制十分嚴謹，任何軟體安裝都會先掃毒過後才允許放行安裝到主機上，這邊就模擬情境假設這些抓好的軟體包都已成熟在使用於企業內主機環境。
企業內部自己去搭建yum local repo方式。
這邊會將CDH yum原始軟體repo存放在host1節點。
這邊先在節點來FTP上傳(透過WinSCP)上傳到Linux VM host1這台環境中。
Step1.將httpd相關rpm從本地上傳到VM host1上。
透過本地安裝方式來把httpd裝好
yum -y localinstall ./*.rpm
完成安裝後就可以來啟動httpd服務
systemctl start httpd
systemctl status httpd
systemctl enable httpd
http://192.168.80.81/
Step3.上傳createrepo指令、和其他Linux常用的相關rpm包(包含CDH的RPM)
先回到/root/目錄 刪除剛已安裝好的rpm
上傳createrepo的rpm額外單獨先安裝
在/root/
運行 yum -y localinstall ./*.rpm
額外創建目錄/var/www/html/sam666
將檔案透過WinSCP上傳
/var/www/html/sam666 會存放其他節點可能也會共用的一些工具安裝包
jdk的rpm (比較大要等一陣子)
oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm
lrzsz-0.12.20-36.el7.x86_64.rpm
時間同步指令
autogen-libopts-5.18-5.el7.x86_64.rpm
ntp-4.2.6p5-28.el7.centos.x86_64.rpm
ntpdate-4.2.6p5-28.el7.centos.x86_64.rpm
查看文件(夾)結構
tree-1.6.0-10.el7.x86_64.rpm
mysql的 rpm包
mysql-community-client-5.7.26-1.el7.x86_64.rpm
mysql-community-common-5.7.26-1.el7.x86_64.rpm
mysql-community-libs-5.7.26-1.el7.x86_64.rpm
mysql-community-libs-compat-5.7.26-1.el7.x86_64.rpm
mysql-community-server-5.7.26-1.el7.x86_64.rpm
跟mysql對應perl依賴相關的rpm包
net-tools-2.0-0.24.20131004git.el7.x86_64.rpm
perl-5.16.3-294.el7_6.x86_64.rpm
perl-Carp-1.26-244.el7.noarch.rpm
perl-constant-1.27-2.el7.noarch.rpm
perl-Encode-2.51-7.el7.x86_64.rpm
perl-Exporter-5.68-3.el7.noarch.rpm
perl-File-Path-2.09-2.el7.noarch.rpm
perl-File-Temp-0.23.01-3.el7.noarch.rpm
perl-Filter-1.49-3.el7.x86_64.rpm
perl-Getopt-Long-2.40-3.el7.noarch.rpm
perl-HTTP-Tiny-0.033-3.el7.noarch.rpm
perl-libs-5.16.3-294.el7_6.x86_64.rpm
perl-macros-5.16.3-294.el7_6.x86_64.rpm
perl-parent-0.225-244.el7.noarch.rpm
perl-PathTools-3.40-5.el7.x86_64.rpm
perl-Pod-Escapes-1.04-294.el7_6.noarch.rpm
perl-podlators-2.5.1-3.el7.noarch.rpm
perl-Pod-Perldoc-3.20-4.el7.noarch.rpm
perl-Pod-Simple-3.28-4.el7.noarch.rpm
perl-Pod-Usage-1.63-3.el7.noarch.rpm
perl-Scalar-List-Utils-1.27-248.el7.x86_64.rpm
perl-Socket-2.010-4.el7.x86_64.rpm
perl-Storable-2.45-3.el7.x86_64.rpm
perl-Text-ParseWords-3.29-4.el7.noarch.rpm
perl-threads-1.87-4.el7.x86_64.rpm
perl-threads-shared-1.43-6.el7.x86_64.rpm
perl-Time-HiRes-1.9725-3.el7.x86_64.rpm
perl-Time-Local-1.2300-2.el7.noarch.rpm
再來就是CDH
Cloudera Manager依賴rpm (比較大要等一陣子)
apr-1.4.8-3.el7_4.1.x86_64.rpm
apr-util-1.5.2-6.el7.x86_64.rpm
at-3.1.13-24.el7.x86_64.rpm
avahi-libs-0.6.31-19.el7.x86_64.rpm
bc-1.06.95-13.el7.x86_64.rpm
bind-libs-9.9.4-74.el7_6.2.x86_64.rpm
bind-libs-lite-9.9.4-74.el7_6.2.x86_64.rpm
bind-license-9.9.4-74.el7_6.2.noarch.rpm
bind-utils-9.9.4-74.el7_6.2.x86_64.rpm
cups-client-1.6.3-35.el7.x86_64.rpm
cups-libs-1.6.3-35.el7.x86_64.rpm
cyrus-sasl-gssapi-2.1.26-23.el7.x86_64.rpm
cyrus-sasl-plain-2.1.26-23.el7.x86_64.rpm
ed-1.9-4.el7.x86_64.rpm
fuse-2.9.2-11.el7.x86_64.rpm
fuse-libs-2.9.2-11.el7.x86_64.rpm
httpd-2.4.6-89.el7.centos.1.x86_64.rpm
httpd-tools-2.4.6-89.el7.centos.1.x86_64.rpm
keyutils-libs-devel-1.5.8-3.el7.x86_64.rpm
krb5-devel-1.15.1-37.el7_6.x86_64.rpm
krb5-libs-1.15.1-37.el7_6.x86_64.rpm
libcom_err-devel-1.42.9-13.el7.x86_64.rpm
libkadm5-1.15.1-37.el7_6.x86_64.rpm
libselinux-devel-2.5-14.1.el7.x86_64.rpm
libsepol-devel-2.5-10.el7.x86_64.rpm
libtirpc-0.2.4-0.15.el7.x86_64.rpm
libverto-devel-0.2.5-4.el7.x86_64.rpm
libxslt-1.1.28-5.el7.x86_64.rpm
m4-1.4.16-10.el7.x86_64.rpm
mailcap-2.1.41-2.el7.noarch.rpm
mailx-12.5-19.el7.x86_64.rpm
mod_ssl-2.4.6-89.el7.centos.1.x86_64.rpm
MySQL-python-1.2.5-1.el7.x86_64.rpm
openssl-1.0.2k-16.el7_6.1.x86_64.rpm
openssl-devel-1.0.2k-16.el7_6.1.x86_64.rpm
openssl-libs-1.0.2k-16.el7_6.1.x86_64.rpm
patch-2.7.1-10.el7_5.x86_64.rpm
pcre-devel-8.32-17.el7.x86_64.rpm
perl-5.16.3-294.el7_6.x86_64.rpm
perl-Carp-1.26-244.el7.noarch.rpm
perl-constant-1.27-2.el7.noarch.rpm
perl-Encode-2.51-7.el7.x86_64.rpm
perl-Exporter-5.68-3.el7.noarch.rpm
perl-File-Path-2.09-2.el7.noarch.rpm
perl-File-Temp-0.23.01-3.el7.noarch.rpm
perl-Filter-1.49-3.el7.x86_64.rpm
perl-Getopt-Long-2.40-3.el7.noarch.rpm
perl-HTTP-Tiny-0.033-3.el7.noarch.rpm
perl-libs-5.16.3-294.el7_6.x86_64.rpm
perl-macros-5.16.3-294.el7_6.x86_64.rpm
perl-parent-0.225-244.el7.noarch.rpm
perl-PathTools-3.40-5.el7.x86_64.rpm
perl-Pod-Escapes-1.04-294.el7_6.noarch.rpm
perl-podlators-2.5.1-3.el7.noarch.rpm
perl-Pod-Perldoc-3.20-4.el7.noarch.rpm
perl-Pod-Simple-3.28-4.el7.noarch.rpm
perl-Pod-Usage-1.63-3.el7.noarch.rpm
perl-Scalar-List-Utils-1.27-248.el7.x86_64.rpm
perl-Socket-2.010-4.el7.x86_64.rpm
perl-Storable-2.45-3.el7.x86_64.rpm
perl-Text-ParseWords-3.29-4.el7.noarch.rpm
perl-threads-1.87-4.el7.x86_64.rpm
perl-threads-shared-1.43-6.el7.x86_64.rpm
perl-Time-HiRes-1.9725-3.el7.x86_64.rpm
perl-Time-Local-1.2300-2.el7.noarch.rpm
postgresql-libs-9.2.24-1.el7_5.x86_64.rpm
psmisc-22.20-15.el7.x86_64.rpm
python-psycopg2-2.5.1-3.el7.x86_64.rpm
redhat-lsb-core-4.1-27.el7.centos.1.x86_64.rpm
redhat-lsb-submod-security-4.1-27.el7.centos.1.x86_64.rpm
rpcbind-0.2.0-47.el7.x86_64.rpm
spax-1.5.2-13.el7.x86_64.rpm
time-1.7-45.el7.x86_64.rpm
zlib-devel-1.2.7-18.el7.x86_64.rpm
cloudera-manager-daemon的rpm (比較大要等一陣子)
cloudera-manager-daemons-6.3.0-1281944.el7.x86_64.rpm
cloudera-manager-agent-6.3.0-1281944.el7.x86_64.rpm
cloudera-manager-server-6.3.0-1281944.el7.x86_64.rpm
剛上傳的所有檔案都可訪問到目錄資料
Step4.產生yum local repo
於/var/www/html/sam666 該目錄下運行指令產生yum local repo
運行 createrepo .
以便於在此目錄下建立Yum Local Repo
Step5.替換三台節點各自本來的yum源檔案 *.repo
通常是在/etc/yum.repos.d
rename .repo .repo.bak *
把本來yum repo重命名讓其失效
Step6.在host1節點上/etc/yum.repos.d目錄中創建sam666.repo
vi sam666.repo
增加如下內容
[sam666]
name=sam666
baseurl=http://host1/sam666/
gpgcheck = 0
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Step7.把上一步在host1建好的sam666.repo檔案發送到另外兩台節點host2,host3
scp /etc/yum.repos.d/sam666.repo host2:/etc/yum.repos.d/sam666.repo
scp /etc/yum.repos.d/sam666.repo host3:/etc/yum.repos.d/sam666.repo
去host2,host3分別確認repo有無傳送成功
清除yum緩存
yum clean all
查看um倉庫清單
yum repolist
重建yum緩存
yum makecache
yum makecache
嘗試在各節點依序先安裝通用工具，已驗證是否本地yum repo製作成功。
yum install -y net-tools tree lrzsz
留言
張貼留言