greenplum安装部署 - openEuler 20.03 SP3

一、安装说明

1.1 环境说明

1、首先确定部署的环境，确定下服务器的端口，一般默认是22的端口

2、操作系统：openEuler 20.03 SP3

3、 数据库：greenplum-db-6.13.0-rhel7-x86_64.rpm

1.2 集群介绍

1、本次采用的是3台机器组成的集群：1台机器是master节点，2台机器是segment节点。

2、集群的ip和hostname如下（都是虚拟机环境）：

IP	主机名	说明
192.168.100.100	gp-master	master节点
192.168.100.101	gp-sdw1	segment节点
192.168.100.102	gp-sdw2	segment节点

二、安装环境准备

2.1关闭防火墙（所有节点）

1、配置的时候，要保证所有的机器的网络都是通的，并且每台机器的防火墙都是关闭的，避免存在网络不通的情况。

具体操作：

# 停止防火墙
$ systemctl stop firewalld
# 关闭防火墙
$ systemctl disable firewalld
# 查看防火墙状态
$ systemctl status firewalld

2.2关闭SELINUX（所有节点）

1、修改/etc/selinux/config文件（在某些系统中，可能是/etc/sysconfig/selinux文件），将SELINUX=enforcing修改为SELINUX=disabled。更新配置之后要重启服务器生效或者执行：setenforce 0，使其立即生效。

2、具体操作：（每个节点直接执行如下命令）

$ sed -i s/SELINUX=enforcing/SELINUX=disabled/g /etc/selinux/config

注：执行完也可以直接进入/etc/selinux/config的文件查看是否修改完全

2.3修改内核配置参数 (所有节点)

1、修改/etc/sysctl.conf文件，其中的配置文件是官方配置的。

具体操作：

$ vi /etc/sysctl.conf
# 计算：echo $(expr $(getconf _PHYS_PAGES) / 2 \* $(getconf PAGE_SIZE))
# kernel.shmmax = kernel.shmall * PAGE_SIZE
kernel.shmmax = 15000000000
kernel.shmmni = 4096
# kernel.shmall = _PHYS_PAGES / 2
# 计算：echo $(expr $(getconf _PHYS_PAGES) / 2)
kernel.shmall = 8000000000
kernel.sem = 500 1024000 200 4096
kernel.sysrq = 1
kernel.core_uses_pid = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.msgmni = 2048
net.ipv4.tcp_syncookies = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.conf.all.arp_filter = 1
net.ipv4.ip_local_port_range = 10000 65535
net.core.netdev_max_backlog = 10000
net.core.rmem_max = 2097152
net.core.wmem_max = 2097152
vm.overcommit_memory = 2
vm.overcommit_ratio = 95
vm.swappiness = 10
vm.dirty_expire_centisecs = 500
vm.dirty_writeback_centisecs = 100
vm.dirty_background_ratio = 3
vm.dirty_ratio=10

net.ipv4.tcp_tw_recycle：4.10内核之后就废弃了

2、执行完上述操作后，执行sysctl -p 使配置生效。

$ sysctl -p

2.4配置/etc/hosts (所有节点)

1、主要是为之后 Greenplum 能够在各个节点之间相互通信做准备。

$ vi /etc/hosts
192.168.100.100	gp-master
192.168.100.101	gp-sdw1
192.168.100.102	gp-sdw2

2.5配置资源限制参数/etc/security/limits.conf(所有节点)

$ vi /etc/security/limits.conf
* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072

2.6安装依赖

如果登录时发现报错：-bash: bc：未找到命令，建议补充一下bc

$ yum install -y bc

安装GP依赖

$ yum install -y apr apr-util e2fsprogs-devel keyutils-libs-devel krb5-devel libselinux-devel libsepol-devel libverto-devel net-tools pcre2-devel rsync

额外补充依赖

# 安装 libnsl
$ yum install -y libnsl

# 可能用到tar，如果没有也装一下
 $ yum install -y tar

# 针对高版本libreadline.so处理
## 确定好已有的高版本libreadline.so的名称，创建链接到/usr/lib64/libreadline.so.{6,7}
ll /usr/lib64|grep libreadline.so
ln -s /usr/lib64/libreadline.so.8.0 /usr/lib64/libreadline.so.6 
ln -s /usr/lib64/libreadline.so.8.0 /usr/lib64/libreadline.so.7

# 下载安装：
$ wget https://pkgs.sysadmins.ws/el8/extras/x86_64/compat-openssl10-1.0.2o-3.el8.x86_64.rpm
$ yum install -y compat-openssl10-1.0.2o-3.el8.x86_64.rpm

2.7将所有服务器重启（所有节点)

$ reboot

三、GreenPlum的数据库安装

3.1添加组和用户（所有节点机器 root 用户操作）

1、创建 gpadmin 用户及用户组，将其作为安装 greenplum 的操作系统用户。

2、如果已经存在，先删除，

$ groupdel gpadmin
$ userdel gpadmin

3、创建新的用户和用户组

$ groupadd gpadmin
$ useradd -r -m -g gpadmin gpadmin
$ echo "gpadmin:password" | chpasswd

3.2集群相互之间ssh免密（所有节点机器 gpadmin 用户操作）

1、具体操作：（操作为master节点）

$ su - gpadmin
$ mkdir -p /home/gpadmin/conf
$ cd /home/gpadmin/
$ ssh-keygen -t rsa （生成密钥对）
$ ssh-copy-id gp-master
$ ssh-copy-id gp-sdw1
$ ssh-copy-id gp-sdw2

注：每个节点都需要这样操作。

验证如下：在gpadmin用户下，ssh smdw 输入后，无需密码直接登录到了smdw服务器。

$ su - gpadmin
$ ssh gp-master

问题注意点：（如果是10002端口/其他端口 这个地方一定需要改）

解决办法：

修改/etc/services文件，将ssh 对应的端口改为10022/对应端口 即可

3.3 ssh权限互通设置（Master节点）

1、在/home/gpadmin/conf的路径下生成 hostlist，seg_hosts文件：

$ cd /home/gpadmin/conf

$ vi hostlist
gp-master
gp-sdw1
gp-sdw2

$ vi seg_hosts
gp-sdw1
gp-sdw2

3.4 Greenplum集群安装（所有节点）

1、首先是将greenplum的安装包导入到服务器上（可以新建一个文件夹）；

2、具体操作：

# root用户
su root
# 这个安装包的所在位置
cd /root/gpadmin
# 查看安装包，确认位置
ls
# 这个是官方的安装步骤，是不需要再去下载依赖的
yum install -y ./greenplum-db-6.13.0-rhel7-x86_64.rpm
# 安装完之后，这个地方是将这个文件在gpadmin用户赋予权限
chown -R gpadmin:gpadmin /usr/local/greenplum*

注：所有节点都需要安装

3.5使用 gpssh-exkeys 打通所有服务器 (Master节点)

1、使用 gpssh-exkeys 将所有机器的通道打开，这样在每台机器之间跳转，就不需要输入密码。

具体操作：

$ su - gpadmin
$ source /usr/local/greenplum-db-6.13.0/greenplum_path.sh
$ postgres -V
$ gpssh-exkeys -f /home/gpadmin/conf/hostlist

执行样例

$ source /usr/local/greenplum-db-6.13.0/greenplum_path.sh
# 这里一定要出现类似于以下内容才行，如果有缺少依赖一定要处理，再继续往后走
$ postgres -V
postgres (Greenplum Database) 9.4.24

$ gpssh-exkeys -f /home/gpadmin/conf/hostlist
[STEP 1 of 5] create local ID and authorize on local host
  ... /home/gpadmin/.ssh/id_rsa file exists ... key generation skipped

[STEP 2 of 5] keyscan all hosts and update known_hosts file

[STEP 3 of 5] retrieving credentials from remote hosts
  ... send to gp-sdw1
  ... send to gp-sdw2

[STEP 4 of 5] determine common authentication file content

[STEP 5 of 5] copy authentication files to all remote hosts
  ... finished key exchange with gp-sdw1
  ... finished key exchange with gp-sdw2

[INFO] completed successfully

3.6环境变量配置.bashrc和GPHOME（所有节点机器 gpadmin 用户操作）

1、配置环境变量.bashrc

$ su - gpadmin
$ vi /home/gpadmin/.bashrc
source /usr/local/greenplum-db/greenplum_path.sh
export MASTER_DATA_DIRECTORY=/data/gpdata/master/gpseg-1
export GPPORT=5432
export PGDATABASE=analysis

$ source /home/gpadmin/.bashrc

2、配置环境变量GPHOME，首先进到文件中直接修改

$ vi /usr/local/greenplum-db/greenplum_path.sh
添加的路径：GPHOME=/usr/local/greenplum-db
# 修改export GPHOME即可
export GPHOME=/usr/local/greenplum-db

3.7创建数据文件夹（Master节点）

1、创建各个节点的数据文件夹，该步骤之后 hostlist 文件包含机器下都会创建 data目录，data目录下都会创建master、primary、mirror文件夹。

具体操作：

# root
$ mkdir /data/gpdata
chown -R gpadmin:gpadmin /data/gpdata

# gpadmin
$ su - gpadmin
$ gpssh -f /home/gpadmin/conf/hostlist
cd /data/gpdata
mkdir master
mkdir primary
mkdir mirror
exit

执行样例

$ gpssh -f /home/gpadmin/conf/hostlist
=> cd /data/gpdata
[gp-master]
[  gp-sdw2]
[  gp-sdw1]
=> mkdir master
[gp-master]
[  gp-sdw2]
[  gp-sdw1]
=> mkdir primary
[gp-master]
[  gp-sdw2]
[  gp-sdw1]
=> mkdir mirror
[gp-master]
[  gp-sdw2]
[  gp-sdw1]
=> exit

如果前面创建了目录的话，这里可能会报错提示File exists，忽略即可

3.8连通性检查（主节点 gpadmin 用户操作 ）

1、检查各个节点的连通性 具体操作：

$ gpcheckperf -f /home/gpadmin/conf/hostlist -r N -d /tmp

执行样例

$ gpcheckperf -f /home/gpadmin/conf/hostlist -r N -d /tmp
/usr/local/greenplum-db-6.13.0/bin/gpcheckperf -f /home/gpadmin/conf/hostlist -r N -d /tmp

-------------------
--  NETPERF TEST
-------------------

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.
NOTICE: -t is deprecated, and has no effect
NOTICE: -f is deprecated, and has no effect
NOTICE: -t is deprecated, and has no effect
NOTICE: -f is deprecated, and has no effect

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.
NOTICE: -t is deprecated, and has no effect
NOTICE: -f is deprecated, and has no effect
NOTICE: -t is deprecated, and has no effect
NOTICE: -f is deprecated, and has no effect

====================
==  RESULT 2024-10-02T15:06:47.594302
====================
Netperf bisection bandwidth test
gp-master -> gp-sdw1 = 249.020000
gp-sdw2 -> gp-master = 150.110000
gp-sdw1 -> gp-master = 140.670000
gp-master -> gp-sdw2 = 259.100000

Summary:
sum = 798.90 MB/sec
min = 140.67 MB/sec
max = 259.10 MB/sec
avg = 199.72 MB/sec
median = 249.02 MB/sec

[Warning] connection between gp-sdw2 and gp-master is no good
[Warning] connection between gp-sdw1 and gp-master is no good

3.9初始化配置文件编辑修改（主节点 gpadmin 用户操作 ）

1、这里修改初始化文件，首先拷贝一个文件gpinitsystem_config，在修改。

具体操作：

$ mkdir /home/gpadmin/gpconfigs
$ cp /usr/local/greenplum-db/docs/cli_help/gpconfigs/gpinitsystem_config /home/gpadmin/gpconfigs/gpinitsystem_config （这个部分是一个命令，比较长）

vi /home/gpadmin/gpconfigs/gpinitsystem_config
# 这里先搜索再修改：
# ------------------------以下为配置内容------------------------
# 该项配置设置主节点数据存储位置，括号里边有几个,代表每台主机创建几个postgresql数据库实例，即segment的实例数，示例是2个。
declare -a DATA_DIRECTORY=(/data/gpdata/primary /data/gpdata/primary)
# 该项配置设置主节点机器名字
MASTER_HOSTNAME=gp-master
# 该项配置设置主节点数据存储位置
MASTER_DIRECTORY=/data/gpdata/master
# 该项配置设置是备节点数据存储位置,规则同DATA_DIRECTORY，括号里边数量和DATA_DIRECTORY保持一致。
# greenplum数据分主节点和备节点，主节点挂掉时候备节点数据会启用。
# declare -a MIRROR_DATA_DIRECTORY=(/home/gpadmin/data/primary /home/gpadmin/data/primary)
# 该项配置设置默认数据库名字，和环境变量数据库名字保持一致，不然会失败。
DATABASE_NAME=analysis

2、在/home/gpadmin/gpconfigs新增一个配置文件hostfile_gpinitsystem

$ cd /home/gpadmin/gpconfigs
$ vi hostfile_gpinitsystem
gp-sdw1
gp-sdw2

3.10初始化数据库（主节点 gpadmin 用户操作 ）

# 注意在gpadmin根目录执行
$ cd
$ gpinitsystem -c gpconfigs/gpinitsystem_config -h gpconfigs/hostfile_gpinitsystem
# 备用：gpinitsystem -c /home/gpadmin/gpconfigs/gpinitsystem_config -h /home/gpadmin/gpconfigs/hostfile_gpinitsystem

执行样例

$ cd
$ gpinitsystem -c gpconfigs/gpinitsystem_config -h gpconfigs/hostfile_gpinitsystem
20241002:15:09:08:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Checking configuration parameters, please wait...
20241002:15:09:08:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Reading Greenplum configuration file gpconfigs/gpinitsystem_config
20241002:15:09:08:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Locale has not been set in gpconfigs/gpinitsystem_config, will set to default value
20241002:15:09:09:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Locale set to en_US.utf8
20241002:15:09:09:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-MASTER_MAX_CONNECT not set, will set to default value 250
20241002:15:09:09:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Checking configuration parameters, Completed
20241002:15:09:09:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Commencing multi-home checks, please wait...
.
Authorized users only. All activities may be monitored and reported.
.
Authorized users only. All activities may be monitored and reported.

20241002:15:09:09:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Configuring build for standard array
20241002:15:09:10:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Commencing multi-home checks, Completed
20241002:15:09:10:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Building primary segment instance array, please wait...
.
Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.
.
Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

20241002:15:09:11:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Checking Master host
20241002:15:09:11:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Checking new segment hosts, please wait...

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.
.
Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.
.
Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Checking new segment hosts, Completed
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Greenplum Database Creation Parameters
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:---------------------------------------
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Master Configuration
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:---------------------------------------
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Master instance name       = Greenplum Data Platform
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Master hostname            = gp-master
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Master port                = 5432
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Master instance dir        = /data/gpdata/master/gpseg-1
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Master LOCALE              = en_US.utf8
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Greenplum segment prefix   = gpseg
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Master Database            = analysis
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Master connections         = 250
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Master buffers             = 128000kB
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Segment connections        = 750
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Segment buffers            = 128000kB
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Checkpoint segments        = 8
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Encoding                   = UNICODE
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Postgres param file        = Off
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Initdb to be used          = /usr/local/greenplum-db/bin/initdb
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-GP_LIBRARY_PATH is         = /usr/local/greenplum-db/lib
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-HEAP_CHECKSUM is           = on
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-HBA_HOSTNAMES is           = 0
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Ulimit check               = Passed
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Array host connect type    = Single hostname per node
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Master IP address [1]      = ::1
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Master IP address [2]      = 192.168.100.100
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Master IP address [3]      = 192.168.31.100
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Master IP address [4]      = fe80::55b2:bf24:33bd:96ad
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Master IP address [5]      = fe80::b027:498a:53a2:b183
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Standby Master             = Not Configured
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Number of primary segments = 1
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Total Database segments    = 2
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Trusted shell              = ssh
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Number segment hosts       = 2
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Mirroring config           = OFF
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:----------------------------------------
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Greenplum Primary Segment Configuration
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:----------------------------------------
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-gp-sdw1         6000    gp-sdw1         /data/gpdata/primary/gpseg0     2
20241002:15:09:16:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-gp-sdw2         6000    gp-sdw2         /data/gpdata/primary/gpseg1     3

Continue with Greenplum creation Yy|Nn (default=N):
> Y
20241002:15:09:19:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Building the Master instance database, please wait...
20241002:15:09:26:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Starting the Master in admin mode
20241002:15:09:28:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Commencing parallel build of primary segment instances
20241002:15:09:28:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Spawning parallel processes    batch [1], please wait...
..
20241002:15:09:28:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Waiting for parallel processes batch [1], please wait...
..........
Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.
.
Authorized users only. All activities may be monitored and reported.
.
Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.
..
Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.
.
Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.
.
Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.
.
Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.
.
Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.
.
Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.
.
Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.

Authorized users only. All activities may be monitored and reported.
.
20241002:15:09:49:070322 gpinitsystem:gp-master:gpadmin-[INFO]:------------------------------------------------
20241002:15:09:49:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Parallel process exit status
20241002:15:09:49:070322 gpinitsystem:gp-master:gpadmin-[INFO]:------------------------------------------------
20241002:15:09:49:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Total processes marked as completed           = 2
20241002:15:09:49:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Total processes marked as killed              = 0
20241002:15:09:49:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Total processes marked as failed              = 0
20241002:15:09:49:070322 gpinitsystem:gp-master:gpadmin-[INFO]:------------------------------------------------
20241002:15:09:49:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Deleting distributed backout files
20241002:15:09:49:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Removing back out file
20241002:15:09:49:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-No errors generated from parallel processes
20241002:15:09:49:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Restarting the Greenplum instance in production mode
20241002:15:09:49:077052 gpstop:gp-master:gpadmin-[INFO]:-Starting gpstop with args: -a -l /home/gpadmin/gpAdminLogs -m -d /data/gpdata/master/gpseg-1
20241002:15:09:49:077052 gpstop:gp-master:gpadmin-[INFO]:-Gathering information and validating the environment...
20241002:15:09:49:077052 gpstop:gp-master:gpadmin-[INFO]:-Obtaining Greenplum Master catalog information
20241002:15:09:49:077052 gpstop:gp-master:gpadmin-[INFO]:-Obtaining Segment details from master...
20241002:15:09:49:077052 gpstop:gp-master:gpadmin-[INFO]:-Greenplum Version: 'postgres (Greenplum Database) 6.13.0 build commit:4f1adf8e247a9685c19ea02bcaddfdc200937ecd Open Source'
20241002:15:09:49:077052 gpstop:gp-master:gpadmin-[INFO]:-Commencing Master instance shutdown with mode='smart'
20241002:15:09:49:077052 gpstop:gp-master:gpadmin-[INFO]:-Master segment instance directory=/data/gpdata/master/gpseg-1
20241002:15:09:49:077052 gpstop:gp-master:gpadmin-[INFO]:-Stopping master segment and waiting for user connections to finish ...
server shutting down
20241002:15:09:50:077052 gpstop:gp-master:gpadmin-[INFO]:-Attempting forceful termination of any leftover master process
20241002:15:09:51:077052 gpstop:gp-master:gpadmin-[INFO]:-Terminating processes for segment /data/gpdata/master/gpseg-1
20241002:15:09:51:077052 gpstop:gp-master:gpadmin-[ERROR]:-Failed to kill processes for segment /data/gpdata/master/gpseg-1: ([Errno 3] No such process)
20241002:15:09:51:077137 gpstart:gp-master:gpadmin-[INFO]:-Starting gpstart with args: -a -l /home/gpadmin/gpAdminLogs -d /data/gpdata/master/gpseg-1
20241002:15:09:51:077137 gpstart:gp-master:gpadmin-[INFO]:-Gathering information and validating the environment...
20241002:15:09:51:077137 gpstart:gp-master:gpadmin-[INFO]:-Greenplum Binary Version: 'postgres (Greenplum Database) 6.13.0 build commit:4f1adf8e247a9685c19ea02bcaddfdc200937ecd Open Source'
20241002:15:09:51:077137 gpstart:gp-master:gpadmin-[INFO]:-Greenplum Catalog Version: '301908232'
20241002:15:09:51:077137 gpstart:gp-master:gpadmin-[INFO]:-Starting Master instance in admin mode
20241002:15:09:51:077137 gpstart:gp-master:gpadmin-[INFO]:-Obtaining Greenplum Master catalog information
20241002:15:09:51:077137 gpstart:gp-master:gpadmin-[INFO]:-Obtaining Segment details from master...
20241002:15:09:51:077137 gpstart:gp-master:gpadmin-[INFO]:-Setting new master era
20241002:15:09:51:077137 gpstart:gp-master:gpadmin-[INFO]:-Master Started...
20241002:15:09:51:077137 gpstart:gp-master:gpadmin-[INFO]:-Shutting down master
20241002:15:09:52:077137 gpstart:gp-master:gpadmin-[INFO]:-Commencing parallel segment instance startup, please wait...
.
20241002:15:09:53:077137 gpstart:gp-master:gpadmin-[INFO]:-Process results...
20241002:15:09:53:077137 gpstart:gp-master:gpadmin-[INFO]:-----------------------------------------------------
20241002:15:09:53:077137 gpstart:gp-master:gpadmin-[INFO]:-   Successful segment starts                                            = 2
20241002:15:09:53:077137 gpstart:gp-master:gpadmin-[INFO]:-   Failed segment starts                                                = 0
20241002:15:09:53:077137 gpstart:gp-master:gpadmin-[INFO]:-   Skipped segment starts (segments are marked down in configuration)   = 0
20241002:15:09:53:077137 gpstart:gp-master:gpadmin-[INFO]:-----------------------------------------------------
20241002:15:09:53:077137 gpstart:gp-master:gpadmin-[INFO]:-Successfully started 2 of 2 segment instances 
20241002:15:09:53:077137 gpstart:gp-master:gpadmin-[INFO]:-----------------------------------------------------
20241002:15:09:53:077137 gpstart:gp-master:gpadmin-[INFO]:-Starting Master instance gp-master directory /data/gpdata/master/gpseg-1 
20241002:15:09:54:077137 gpstart:gp-master:gpadmin-[INFO]:-Command pg_ctl reports Master gp-master instance active
20241002:15:09:54:077137 gpstart:gp-master:gpadmin-[INFO]:-Connecting to dbname='template1' connect_timeout=15
20241002:15:09:54:077137 gpstart:gp-master:gpadmin-[INFO]:-No standby master configured.  skipping...
20241002:15:09:54:077137 gpstart:gp-master:gpadmin-[INFO]:-Database successfully started
20241002:15:09:54:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Completed restart of Greenplum instance in production mode
20241002:15:09:55:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Scanning utility log file for any warning messages
20241002:15:09:55:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Log file scan check passed
20241002:15:09:55:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Greenplum Database instance successfully created
20241002:15:09:55:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-------------------------------------------------------
20241002:15:09:55:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-To complete the environment configuration, please 
20241002:15:09:55:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-update gpadmin .bashrc file with the following
20241002:15:09:55:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-1. Ensure that the greenplum_path.sh file is sourced
20241002:15:09:55:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-2. Add "export MASTER_DATA_DIRECTORY=/data/gpdata/master/gpseg-1"
20241002:15:09:55:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-   to access the Greenplum scripts for this instance:
20241002:15:09:55:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-   or, use -d /data/gpdata/master/gpseg-1 option for the Greenplum scripts
20241002:15:09:55:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-   Example gpstate -d /data/gpdata/master/gpseg-1
20241002:15:09:55:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Script log file = /home/gpadmin/gpAdminLogs/gpinitsystem_20241002.log
20241002:15:09:55:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-To remove instance, run gpdeletesystem utility
20241002:15:09:55:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-To initialize a Standby Master Segment for this Greenplum instance
20241002:15:09:55:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Review options for gpinitstandby
20241002:15:09:55:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-------------------------------------------------------
20241002:15:09:55:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-The Master /data/gpdata/master/gpseg-1/pg_hba.conf post gpinitsystem
20241002:15:09:55:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-has been configured to allow all hosts within this new
20241002:15:09:55:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-array to intercommunicate. Any hosts external to this
20241002:15:09:55:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-new array must be explicitly added to this file
20241002:15:09:55:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-Refer to the Greenplum Admin support guide which is
20241002:15:09:55:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-located in the /usr/local/greenplum-db/docs directory
20241002:15:09:55:070322 gpinitsystem:gp-master:gpadmin-[INFO]:-------------------------------------------------------

3.11设置远程登录（主节点 gpadmin 用户操作 ）

如需要Navicat能够链接，需要配置如下：

$ echo "host all gpadmin 0.0.0.0/0 trust" >> /data/gpdata/master/gpseg-1/pg_hba.conf
$ gpstop -u （重新加载数据库配置）

在服务器上，验证相关端口是否运行

$ ps -ef|grep postgres

四、Greenplum数据库验证

4.1登录验证（主节点 gpadmin 用户操作 ）

1、登录验证

$ psql -d postgres

2、navicat登录验证

主机: master的IP地址
端口: 5432【默认，如修改使用修改后的】
初始数据库: postgres
用户名: gpadmin
密码: gpadmin

如果密码不正确可以尝试

# 在master登录控制台
$ psql -d postgres
# 然后执行
ALTER USER gpadmin with password '新密码';

3、查看集群状态验证，在主节点上操作命令

具体操作：

$ gpstate

执行样例

$ gpstate
20241002:15:11:48:090336 gpstate:gp-master:gpadmin-[INFO]:-Starting gpstate with args: 
20241002:15:11:48:090336 gpstate:gp-master:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.13.0 build commit:4f1adf8e247a9685c19ea02bcaddfdc200937ecd Open Source'
20241002:15:11:48:090336 gpstate:gp-master:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.24 (Greenplum Database 6.13.0 build commit:4f1adf8e247a9685c19ea02bcaddfdc200937ecd Open Source) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on Dec 18 2020 22:31:16'
20241002:15:11:48:090336 gpstate:gp-master:gpadmin-[INFO]:-Obtaining Segment details from master...
20241002:15:11:48:090336 gpstate:gp-master:gpadmin-[INFO]:-Gathering data from segments...
20241002:15:11:49:090336 gpstate:gp-master:gpadmin-[INFO]:-Greenplum instance status summary
20241002:15:11:49:090336 gpstate:gp-master:gpadmin-[INFO]:-----------------------------------------------------
20241002:15:11:49:090336 gpstate:gp-master:gpadmin-[INFO]:-   Master instance                                = Active
20241002:15:11:49:090336 gpstate:gp-master:gpadmin-[INFO]:-   Master standby                                 = No master standby configured
20241002:15:11:49:090336 gpstate:gp-master:gpadmin-[INFO]:-   Total segment instance count from metadata     = 2
20241002:15:11:49:090336 gpstate:gp-master:gpadmin-[INFO]:-----------------------------------------------------
20241002:15:11:49:090336 gpstate:gp-master:gpadmin-[INFO]:-   Primary Segment Status
20241002:15:11:49:090336 gpstate:gp-master:gpadmin-[INFO]:-----------------------------------------------------
20241002:15:11:49:090336 gpstate:gp-master:gpadmin-[INFO]:-   Total primary segments                         = 2
20241002:15:11:49:090336 gpstate:gp-master:gpadmin-[INFO]:-   Total primary segment valid (at master)        = 2
20241002:15:11:49:090336 gpstate:gp-master:gpadmin-[INFO]:-   Total primary segment failures (at master)     = 0
20241002:15:11:49:090336 gpstate:gp-master:gpadmin-[INFO]:-   Total number of postmaster.pid files missing   = 0
20241002:15:11:49:090336 gpstate:gp-master:gpadmin-[INFO]:-   Total number of postmaster.pid files found     = 2
20241002:15:11:49:090336 gpstate:gp-master:gpadmin-[INFO]:-   Total number of postmaster.pid PIDs missing    = 0
20241002:15:11:49:090336 gpstate:gp-master:gpadmin-[INFO]:-   Total number of postmaster.pid PIDs found      = 2
20241002:15:11:49:090336 gpstate:gp-master:gpadmin-[INFO]:-   Total number of /tmp lock files missing        = 0
20241002:15:11:49:090336 gpstate:gp-master:gpadmin-[INFO]:-   Total number of /tmp lock files found          = 2
20241002:15:11:49:090336 gpstate:gp-master:gpadmin-[INFO]:-   Total number postmaster processes missing      = 0
20241002:15:11:49:090336 gpstate:gp-master:gpadmin-[INFO]:-   Total number postmaster processes found        = 2
20241002:15:11:49:090336 gpstate:gp-master:gpadmin-[INFO]:-----------------------------------------------------
20241002:15:11:49:090336 gpstate:gp-master:gpadmin-[INFO]:-   Mirror Segment Status
20241002:15:11:49:090336 gpstate:gp-master:gpadmin-[INFO]:-----------------------------------------------------
20241002:15:11:49:090336 gpstate:gp-master:gpadmin-[INFO]:-   Mirrors not configured on this array
20241002:15:11:49:090336 gpstate:gp-master:gpadmin-[INFO]:-----------------------------------------------------

五、Greenplum常用命令

在一个Greenplum数据库DBMS中，数据库服务器实例（Master和所有的Segment）在系统中所有的主机上一起被启动或者停止，它们以这样一种方式作为一个统一的DBMS工作。

由于一个Greenplum数据库系统分布在很多机器上，启动和停止一个Greenplum数据库系统的过程与普通PostgreSQL DBMS不同。

分别使用gpstart和gpstop工具来启动和停止Greenplum数据库。这些工具位于Greenplum数据库的Master主机的$GPHOME/bin目录中。

重要： 不要发出kill命令来结束任何Postgres进程。而是使用数据库命令pg_cancel_backend()。

发出kill -9或者kill -11可能会导致数据库损坏并且妨碍对根本原因的分析。

有关gpstart和gpstop的信息请见Greenplum数据库工具指南。

启动Greenplum数据库

通过在Master实例上运行gpstart可以启动一个初始化好的Greenplum数据库系统。

可以使用gpstart工具来启动一个已经由gpinitsystem工具初始化好但已经被gpstop工具停止的Greenplum数据库系统。gpstart通过启动Greenplum数据库集群上所有的Postgres数据库实例来启动Greenplum数据库。gpstart会精心安排这一过程并且以并行的方式执行它。

在Master主机上运行gpstart启动Greenplum数据库：

$ gpstart

重启Greenplum数据库

停止Greenplum数据库系统然后重新启动它。

带-r选项的gpstop工具可以停止Greenplum数据库，并且在关闭完成后重新启动Greenplum数据库。

要重启Greenplum数据库，在Master主机上输入下面的命令：

$ gpstop -r

仅重新载入配置文件更改

重新载入对Greenplum数据库配置文件的更改而不中断系统。

gpstop工具可以在不中断服务的前提下重新载入对pg_hba.conf配置文件和Master上postgresql.conf、pg_hba.conf文件中运行时参数的更改。活动会话将会在它们重新连接到数据库时使用这些更新。很多服务器配置参数需要完全重启系统（gpstop -r）才能激活。有关服务器配置参数的信息请见Greenplum数据库参考指南。

使用gpstop工具重新载入配置文件更改而不关闭系统：

$ gpstop -u

以维护模式启动Master

只启动Master来执行维护或者管理任务而不影响Segment上的数据。

例如，可以用维护模式连接到一个只在Master实例上的数据库并且编辑系统目录设置。更多有关系统目录表的信息请见Greenplum数据库参考指南。

1. 使用-m模式运行gpstart：

   $ gpstart -m

2. 以维护模式连接到Master进行目录维护。例如：

   $ PGOPTIONS='-c gp_session_role=utility' psql postgres

3. 在完成管理任务后，停止处于维护模式的额Master。然后以生产模式重启它。

   $ gpstop -mr

   警告：

   对维护模式连接的不当使用可能会导致不一致的系统状态。只有技术支持才应该执行这一操作。

停止Greenplum数据库

gpstop工具可以停止或者重启Greenplum数据库系统，它总是运行在Master主机上。当被激活时，gpstop会停止系统中所有的postgres进程，包括Master和所有的Segment实例。gpstop工具使用一种默认的最多64个并行工作者线程的方式来关闭构成整个Greenplum数据库集群的Postgres实例。系统在关闭之前会等待所有的活动事务完成。要立即停止Greenplum数据库，可以使用快速模式。

• 要停止Greenplum数据库：

  $ gpstop

• 要以快速模式停止Greenplum数据库：

  $ gpstop -M fast

  默认情况下，如果有任何客户端连接存在，就不允许关闭Greenplum数据库。使用-M fast选项可以在关闭前回滚所有正在进行中的事务并且中断所有连接。

六、问题

libssl.so.10 问题

$ postgres -V
postgres: error while loading shared libraries: libssl.so.10: cannot open shared object file: No such file or directory

解决办法：

# 对标EL7、EL8、EL9相关的compat-openssl10安装包，下载安装：
$ wget https://pkgs.sysadmins.ws/el8/extras/x86_64/compat-openssl10-1.0.2o-3.el8.x86_64.rpm
$ yum install compat-openssl10-1.0.2o-3.el8.x86_64.rpm

libnsl.so.1 问题

$ postgres -V
postgres: error while loading shared libraries: libnsl.so.1: cannot open shared object file: No such file or directory

解决办法

$ yum install -y libnsl

libreadline.so.6 问题

$ psql -d postgres
psql: error while loading shared libraries: libreadline.so.6: cannot open shared object file: No such file or directory

原因：系统存在高版本的libreadline.so

解决方式

# 注意看libreadline.so的文件名哈
$ ls /usr/lib64 | greo libreadline.so
$ ln -s /usr/lib64/libreadline.so.8.0 /usr/lib64/libreadline.so.6 
$ ln -s /usr/lib64/libreadline.so.8.0 /usr/lib64/libreadline.so.7

单节点命令行登录问题

$ psql -d postgres
psql: FATAL:  System was started in master-only utility mode - only utility mode connections are allowed

处理方式

PGOPTIONS='-c gp_session_role=utility' psql -d postgres

按说高一点的版本应该没有这个问题，但遇到这种情况可能客户端只能从gp源码上下功夫了，参考：Greenplum5单机部署连接报错

Checking Master host 错误

20241001:22:33:31:871237 gpinitsystem:greenplum:gpadmin-[INFO]:-Checking Master host
20241001:22:33:31:871237 gpinitsystem:greenplum:gpadmin-[WARN]:-Have lock file /tmp/.s.PGSQL.5432.lock but no process running on port 5432
20241001:22:33:31:871237 gpinitsystem:greenplum:gpadmin-[FATAL]:-Found indication of postmaster process on port 5432 on Master host Script Exiting!

20241001:23:27:51:175321 gpinitsystem:greenplum:gpadmin-[INFO]:-Checking Master host
20241001:23:27:51:175321 gpinitsystem:greenplum:gpadmin-[FATAL]:-Found indication of postmaster process on port 5432 on Master host Script Exiting!

遇到这种情况，重新初始化，启动啥的都报错

解决办法

$ rm -f /tmp/.s.PGSQL.5432 rm -f /tmp/.s.PGSQL.5432.lock

远程链接报错

FATAL: no pg_hba.conf entry for host "xxx.xxx.xxx.xxx", user "username", database "database", sSL off

处理方式

$ echo "host all gpadmin 0.0.0.0/0 trust" >> /data/gpdata/master/gpseg-1/pg_hba.conf
$ gpstop -u （重新加载数据库配置）

在服务器上，验证相关端口是否运行

$ ps -ef|grep postgres

初始化报错1

20241001:23:23:01:124807 gpinitsystem:greenplum:gpadmin-[INFO]:-Building the Master instance database, please wait...
20241001:23:23:06:124807 gpinitsystem:greenplum:gpadmin-[INFO]:-Starting the Master in admin mode
20241001:23:23:06:124807 gpinitsystem:greenplum:gpadmin-[FATAL]:-Failed to complete obtain psql count Master gp_segment_configuration  Script Exiting!

方法一：

# echo "RemoveIPC=no" >> /etc/systemd/logind.conf;
systemctl restart systemd-logind.service;

解决方式二：

# 重启【有很大可能是由于前面调整了配置内容未重启造成的】
$ reboot

初始化报错2

Authorized users only. All activities may be monitored and reported.
20241001:23:25:31:144875 gpinitsystem:greenplum:gpadmin-[INFO]:-Can resolve gp-master to this host
20241001:23:25:31:144875 gpinitsystem:greenplum:gpadmin-[FATAL]:-Master host data directory /data/gpdata/master/gpseg-1 already exists Script Exiting!

主要由于上面 # 初始化报错1 初始化时已经产生了部分文件

解决方式：

清除/data/gpdata/master下的内容

$ rm -rf /data/gpdata/master/*
# 查看是否有进程未关闭
ps -ef|grep postgres|grep -v grep| awk '{print $2}'

初始化报错3

20241001:23:29:20:192639 gpinitsystem:greenplum:gpadmin-[INFO]:-Building the Master instance database, please wait...
20241001:23:29:25:192639 gpinitsystem:greenplum:gpadmin-[INFO]:-Starting the Master in admin mode
20241001:23:29:26:192639 gpinitsystem:greenplum:gpadmin-[FATAL]:-Failed to complete obtain psql count Master gp_segment_configuration  Script Exiting!

由于无法获取 gp_segment_configuration 表中的 psql 计数，脚本退出，可能由于前面未正确初始化完毕造成的

解决方式一：

# echo "RemoveIPC=no" >> /etc/systemd/logind.conf;
systemctl restart systemd-logind.service;

解决方式二：

# 重启【有很大可能是由于前面调整了配置内容未重启造成的】
$ reboot

关于初始化报错说明

绝大多数情况报错都给出了回滚命令bash /home/gpadmin/gpAdminLogs/backout_gpinitsystem_gpadmin_xxxxxxx_xxxxxxx如下，可以执行命令尝试回滚

Run command bash /home/gpadmin/gpAdminLogs/backout_gpinitsystem_gpadmin_20241001_232256 on master to remove these changes