database/sqream
SQream Database Upgrade (v2022.1.2.1 to v2022.1.4)
난왕
2023. 4. 11. 14:50
반응형
@ 테스트 환경
CPU : Intel® Core™ i7-11600H 2.9GHz
RAM : 32GB
HDD : 512GB SSD(M.2, NVMe)
GPU : NVIDIA RTX 3050 Ti GDDR6 4GB
OS : CentOS Linux release 7.9.2009 (Core)
DB : v2022.1.4
1. 임의 디렉터리에 업로드한 SQream Database 설치파일의 압축을 해제합니다.
[sqream@nt761xda /home/sqream/product/03_sqream]$ cd
[sqream@nt761xda /home/sqream]$ cd /home/sqream/product/03_sqream
[sqream@nt761xda /home/sqream/product/03_sqream]$ ls -rlt
합계 2241584
-rw-r--r-- 1 sqream sqream 496 11월 2 12:33 license.enc
-rw-r--r-- 1 sqream sqream 765168220 11월 2 12:41 sqream-db-v2022.1.4.cpu-generic.pascal.cuda-10.1.el7.centos.x86_64.tar.gz
-rw-r--r-- 1 sqream sqream 765103999 11월 2 13:18 sqream-db-v2022.1.2.cpu-generic.pascal.cuda-10.1.el7.centos.x86_64.tar.gz
-rw-r--r-- 1 root root 765101315 11월 2 14:25 sqream-db-v2022.1.2.1.cpu-generic.pascal.cuda-10.1.el7.centos.x86_64.tar.gz
[sqream@nt761xda /home/sqream/product/03_sqream]$ gzip -d sqream-db-v2022.1.4.cpu-generic.pascal.cuda-10.1.el7.centos.x86_64.tar.gz
[sqream@nt761xda /home/sqream/product/03_sqream]$ ls -rlt
합계 3638448
4 -rw-r--r-- 1 sqream sqream 496 11월 2 12:33 license.enc
2144100 -rw-r--r-- 1 sqream sqream 2195558400 11월 2 12:41 sqream-db-v2022.1.4.cpu-generic.pascal.cuda-10.1.el7.centos.x86_64.tar
747172 -rw-r--r-- 1 sqream sqream 765103999 11월 2 13:18 sqream-db-v2022.1.2.cpu-generic.pascal.cuda-10.1.el7.centos.x86_64.tar.gz
747172 -rw-r--r-- 1 root root 765101315 11월 2 14:25 sqream-db-v2022.1.2.1.cpu-generic.pascal.cuda-10.1.el7.centos.x86_64.tar.gz
[sqream@nt761xda /home/sqream/product/03_sqream]$ tar xvf sqream-db-v2022.1.4.cpu-generic.pascal.cuda-10.1.el7.centos.x86_64.tar
......
[sqream@nt761xda /home/sqream/product/03_sqream]$ ls -rlt
합계 3638448
-rw-r--r-- 1 sqream sqream 496 11월 2 12:33 license.enc
-rw-r--r-- 1 sqream sqream 2195558400 11월 2 12:41 sqream-db-v2022.1.4.cpu-generic.pascal.cuda-10.1.el7.centos.x86_64.tar
-rw-r--r-- 1 sqream sqream 765103999 11월 2 13:18 sqream-db-v2022.1.2.cpu-generic.pascal.cuda-10.1.el7.centos.x86_64.tar.gz
-rw-r--r-- 1 root root 765101315 11월 2 14:25 sqream-db-v2022.1.2.1.cpu-generic.pascal.cuda-10.1.el7.centos.x86_64.tar.gz
drwxrwxr-x 10 sqream sqream 145 11월 2 14:39 sqream
2. 압축을 해제한 SQream Database의 디렉터리의 위치 및 이름을 변경합니다.
[sqream@nt761xda /home/sqream/product/03_sqream]$ mv sqream /home/sqream/SQREAM/sqream-db-v2022.1.4
[sqream@nt761xda /home/sqream/product/03_sqream]$ ls -rl /home/sqream/SQREAM
합계 4
drwxrwxr-x 3 sqream sqream 152 11월 2 12:31 sqream_script
drwxrwxr-x 2 sqream sqream 25 10월 27 18:37 sqream_license
drwxrwxr-x 3 sqream sqream 17 10월 27 19:53 sqream_config
drwxrwxr-x 3 sqream sqream 22 10월 27 18:54 sqream_cluster
drwxrwxr-x 10 sqream sqream 145 11월 2 14:39 sqream-db-v2022.1.4
drwxrwxr-x 10 sqream sqream 145 11월 2 13:20 sqream-db-v2022.1.2.1
drwxrwxr-x 10 sqream sqream 145 10월 27 18:31 sqream-db-v2022.1.2
drwxrwxr-x 7 sqream sqream 242 11월 1 22:16 sqream-dashboard
drwxrwxr-x 5 sqream sqream 102 10월 30 23:54 sqream-admin-5.4.4
drwxrwxr-x 5 sqream sqream 102 11월 2 09:11 sqream-admin
lrwxrwxrwx 1 sqream sqream 21 11월 2 13:43 sqream -> sqream-db-v2022.1.2.1
drwxrwxr-x 5 sqream sqream 80 11월 1 21:32 prometheus
-rw-rw-r-- 1 sqream sqream 1701 11월 2 14:01 meta_traceback
drwxrwxr-x 3 sqream sqream 23 10월 29 22:36 logs
drwxrwxr-x 3 sqream sqream 72 11월 2 13:48 127.0.0.1
3. SQream Database에 접속하여 현재 버전과 Delete Predicates를 확인합니다.
[sqream@nt761xda /home/sqream/product/03_sqream]$ sqream sql --database=master --username=sqream --password=sqream --host=127.0.0.1 --port=3108 --clustered --trim-varchars
Interactive client mode
To quit, use ^D or \q.
master=> SELECT show_version();
v2022.1.2.1
1 row
time: 0.187722s
master=> SELECT * FROM sqream_catalog.delete_predicates;
0 rows
time: 0.246957s
4. Upgrade 전, 현재 SQream Database의 metadata_server를 백업합니다.
master=> SELECT backup_metadata('/home/sqream/SQREAM/sqream_cluster/sqreamdb/leveldb_2022.1.2.1_20221102_old');
executed
time: 0.048300s
master=> \q
[sqream@nt761xda /home/sqream/product/03_sqream]$
5. SQream Database를 중지합니다.
[sqream@nt761xda /home/sqream/product/03_sqream]$ killall metadata_server
6. SQream Studio가 기동 되어 있다면 SQream Studio 관련 프로세스도 모두 종료시킵니다.
[sqream@nt761xda /home/sqream/product/03_sqream]$ ps -ef|grep sqream
sqream 1282 1 0 06:23 ? 00:03:16 /usr/local/bin/prometheus --config.file=/home/sqream/SQREAM/prometheus/prometheus.yml --storage.tsdb.path=/home/sqream/SQREAM/prometheus/lib/prometheus --web.console.templates=/home/sqream/SQREAM/prometheus/consoles --web.console.libraries=/home/sqream/SQREAM/prometheus/console_libraries
sqream 1284 1 0 06:23 ? 00:02:39 /usr/bin/node_exporter
sqream 1287 1 0 06:23 ? 00:00:32 /usr/bin/nvidia_exporter
root 15042 15000 0 12:30 pts/3 00:00:00 su - sqream
sqream 15043 15042 0 12:30 pts/3 00:00:00 -bash
root 15401 15110 0 12:32 pts/4 00:00:00 su - sqream
sqream 15402 15401 0 12:32 pts/4 00:00:00 -bash
root 17391 17196 0 13:25 pts/0 00:00:00 su - sqream
sqream 17392 17391 0 13:25 pts/0 00:00:00 -bash
sqream 21363 1 0 14:18 pts/3 00:00:00 npm
sqream 21386 21363 0 14:18 pts/3 00:00:00 npm
sqream 21400 1 0 14:18 pts/3 00:00:00 npm
sqream 21414 21386 0 14:18 pts/3 00:00:01 sqream-studio
sqream 21425 21400 0 14:18 pts/3 00:00:01 node index.js
sqream 21974 15402 0 14:39 pts/4 00:00:00 ps -ef
sqream 21975 15402 0 14:39 pts/4 00:00:00 grep --color=auto sqream
[sqream@nt761xda /home/sqream/SQREAM/sqream_script]$ kill -9 21363 21386 21400 21414 21425
7. SQream Database를 Upgrade(Product Link) 합니다.
[sqream@nt761xda /home/sqream/product/03_sqream]$ cd /home/sqream/SQREAM
[sqream@nt761xda /home/sqream/SQREAM]$ ls -rl
합계 4
drwxrwxr-x 3 sqream sqream 152 11월 2 12:31 sqream_script
drwxrwxr-x 2 sqream sqream 25 10월 27 18:37 sqream_license
drwxrwxr-x 3 sqream sqream 17 10월 27 19:53 sqream_config
drwxrwxr-x 3 sqream sqream 22 10월 27 18:54 sqream_cluster
drwxrwxr-x 10 sqream sqream 145 11월 2 14:39 sqream-db-v2022.1.4
drwxrwxr-x 10 sqream sqream 145 11월 2 13:20 sqream-db-v2022.1.2.1
drwxrwxr-x 10 sqream sqream 145 10월 27 18:31 sqream-db-v2022.1.2
drwxrwxr-x 7 sqream sqream 242 11월 1 22:16 sqream-dashboard
drwxrwxr-x 5 sqream sqream 102 10월 30 23:54 sqream-admin-5.4.4
drwxrwxr-x 5 sqream sqream 102 11월 2 09:11 sqream-admin
lrwxrwxrwx 1 sqream sqream 21 11월 2 13:43 sqream -> sqream-db-v2022.1.2.1
drwxrwxr-x 5 sqream sqream 80 11월 1 21:32 prometheus
-rw-rw-r-- 1 sqream sqream 1701 11월 2 14:01 meta_traceback
drwxrwxr-x 3 sqream sqream 23 10월 29 22:36 logs
drwxrwxr-x 3 sqream sqream 72 11월 2 13:48 127.0.0.1
[sqream@nt761xda /home/sqream/SQREAM]$ ln -sTf sqream-db-v2022.1.4 sqream
[sqream@nt761xda /home/sqream/SQREAM]$ ls -rl
합계 4
drwxrwxr-x 3 sqream sqream 152 11월 2 12:31 sqream_script
drwxrwxr-x 2 sqream sqream 25 10월 27 18:37 sqream_license
drwxrwxr-x 3 sqream sqream 17 10월 27 19:53 sqream_config
drwxrwxr-x 3 sqream sqream 22 10월 27 18:54 sqream_cluster
drwxrwxr-x 10 sqream sqream 145 11월 2 14:39 sqream-db-v2022.1.4
drwxrwxr-x 10 sqream sqream 145 11월 2 13:20 sqream-db-v2022.1.2.1
drwxrwxr-x 10 sqream sqream 145 10월 27 18:31 sqream-db-v2022.1.2
drwxrwxr-x 7 sqream sqream 242 11월 1 22:16 sqream-dashboard
drwxrwxr-x 5 sqream sqream 102 10월 30 23:54 sqream-admin-5.4.4
drwxrwxr-x 5 sqream sqream 102 11월 2 09:11 sqream-admin
lrwxrwxrwx 1 sqream sqream 19 11월 2 14:51 sqream -> sqream-db-v2022.1.4
drwxrwxr-x 5 sqream sqream 80 11월 1 21:32 prometheus
-rw-rw-r-- 1 sqream sqream 1701 11월 2 14:01 meta_traceback
drwxrwxr-x 3 sqream sqream 23 10월 29 22:36 logs
drwxrwxr-x 3 sqream sqream 72 11월 2 13:48 127.0.0.1
8. Upgrade Storage를 수행합니다.
[sqream@nt761xda /home/sqream/SQREAM]$ upgrade_storage /home/sqream/SQREAM/sqream_cluster/sqreamdb
get_leveldb_version path{/home/sqream/SQREAM/sqream_cluster/sqreamdb/leveldb}
current storage version 41
upgrade_v42
upgrade_storage to 42
line{373} update value key{rl0}
line{373} update value key{rl1}
upgrade_storage to 42 - Done
upgrade_storage to 42
upgrade_storage to 42 - Done
validate_leveldb
log4cxx: No appender could be found for logger (file_utils).
log4cxx: Please initialize the log4cxx system properly.
storage has been upgraded successfully to version 42• Upgrade Storage 명령어는 다음과 같습니다. upgrade_storage <cluster path>
9. SQream Database를 기동 합니다.
[sqream@nt761xda /home/sqream/SQREAM]$ nohup /home/sqream/SQREAM/sqream/bin/metadata_server 127.0.0.1 3105 >> /home/sqream/SQREAM/logs/127.0.0.1/127.0.0.1_metadata_server.log 2>&1 &
[1] 26097
[sqream@nt761xda /home/sqream/SQREAM]$ nohup /home/sqream/SQREAM/sqream/bin/server_picker 127.0.0.1 3105 3108 >> /home/sqream/SQREAM/logs/127.0.0.1/127.0.0.1_server_picker.log 2>&1 &
[2] 26147
[sqream@nt761xda /home/sqream/SQREAM]$ nohup /home/sqream/SQREAM/sqream/bin/sqreamd -config /home/sqream/SQREAM/sqream_config/4sq/sqream0_config.json >> /home/sqream/SQREAM/logs/127.0.0.1/127.0.0.1_sqream0.log 2>&1 &
[3] 26200
[sqream@nt761xda /home/sqream/SQREAM]$ nohup /home/sqream/SQREAM/sqream/bin/sqreamd -config /home/sqream/SQREAM/sqream_config/4sq/sqream1_config.json >> /home/sqream/SQREAM/logs/127.0.0.1/127.0.0.1_sqream1.log 2>&1 &
[4] 26204
[sqream@nt761xda /home/sqream/SQREAM]$ nohup /home/sqream/SQREAM/sqream/bin/sqreamd -config /home/sqream/SQREAM/sqream_config/4sq/sqream2_config.json >> /home/sqream/SQREAM/logs/127.0.0.1/127.0.0.1_sqream2.log 2>&1 &
[5] 26283
[sqream@nt761xda /home/sqream/SQREAM]$ nohup /home/sqream/SQREAM/sqream/bin/sqreamd -config /home/sqream/SQREAM/sqream_config/4sq/sqream3_config.json >> /home/sqream/SQREAM/logs/127.0.0.1/127.0.0.1_sqream3.log 2>&1 &
[6] 26360
10. SQream Database에 접속해서 현재 Database의 버전을 확인합니다.
[sqream@nt761xda /home/sqream/SQREAM]$ sqream sql --database=master --username=sqream --password=sqream --host=127.0.0.1 --port=3108 --clustered --trim-varchars
Interactive client mode
To quit, use ^D or \q.
master=> select show_version();
v2022.1.4
1 row
time: 0.224598s
11. 현재 상태의 metadata_server를 백업합니다.
master=> SELECT backup_metadata('/home/sqream/SQREAM/sqream_cluster/sqreamdb/leveldb_2022.1.4_20221102_new');
executed
time: 0.154616s
master=> \q
[sqream@nt761xda /home/sqream/SQREAM]$• metadata_server를 백업받는 이유는, 백업받은 metadata_server로 덮어 씌움으로써 metadata_server를 정리하기 위함입니다.
12. SQream Database를 중지합니다.
[sqream@nt761xda /home/sqream/SQREAM]$ killall metadata_server
13. 현재 버전의 metadata_server는 이름 변경 후, 바로 전에 받은 metadata_server의 백업본으로 현재 metadata_server를 덮어쓰기 합니다.
[sqream@nt761xda /home/sqream/SQREAM]$ cd /home/sqream/SQREAM/sqream_cluster/sqreamdb
sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ ls -rl
합계 15
drwxrwxr-x 3 sqream sqream 23 11월 2 15:10 temp
drwxrwxr-x 7 sqream sqream 142 11월 1 09:37 logs
drwxrwxr-x 2 sqream sqream 177 11월 2 15:57 leveldb_2022.1.4_20221102_new
drwxrwxr-x 2 sqream sqream 177 11월 2 13:37 leveldb_2022.1.2_20221102
drwxrwxr-x 2 sqream sqream 177 11월 2 14:49 leveldb_2022.1.2.1_20221102_old
drwxrwxr-x 2 sqream sqream 4096 11월 2 15:14 leveldb
drwxrwxr-x 3 sqream sqream 20 10월 27 18:54 databases
[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ mv leveldb leveldb_2022.1.2.1_20221102
[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ mv leveldb_2022.1.4_20221102_new leveldb
[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ ls -rl
합계 12
drwxrwxr-x 3 sqream sqream 23 11월 2 15:10 temp
drwxrwxr-x 7 sqream sqream 142 11월 1 09:37 logs
drwxrwxr-x 2 sqream sqream 177 11월 2 13:37 leveldb_2022.1.2_20221102
drwxrwxr-x 2 sqream sqream 177 11월 2 14:49 leveldb_2022.1.2.1_20221102
drwxr-xr-x 2 sqream sqream 4096 11월 2 13:56 leveldb_2022.1.2.1_20221102_old
drwxrwxr-x 2 sqream sqream 177 11월 2 15:57 leveldb
drwxrwxr-x 3 sqream sqream 20 10월 27 18:54 databases
14. SQream Database를 기동 합니다.
[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ nohup /home/sqream/SQREAM/sqream/bin/metadata_server 127.0.0.1 3105 >> /home/sqream/SQREAM/logs/127.0.0.1/127.0.0.1_metadata_server.log 2>&1 &
[1] 9975
[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ nohup /home/sqream/SQREAM/sqream/bin/server_picker 127.0.0.1 3105 3108 >> /home/sqream/SQREAM/logs/127.0.0.1/127.0.0.1_server_picker.log 2>&1 &
[2] 10060
[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ nohup /home/sqream/SQREAM/sqream/bin/sqreamd -config /home/sqream/SQREAM/sqream_config/4sq/sqream0_config.json >> /home/sqream/SQREAM/logs/127.0.0.1/127.0.0.1_sqream0.log 2>&1 &
[3] 10138
[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ nohup /home/sqream/SQREAM/sqream/bin/sqreamd -config /home/sqream/SQREAM/sqream_config/4sq/sqream1_config.json >> /home/sqream/SQREAM/logs/127.0.0.1/127.0.0.1_sqream1.log 2>&1 &
[4] 10247
[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ nohup /home/sqream/SQREAM/sqream/bin/sqreamd -config /home/sqream/SQREAM/sqream_config/4sq/sqream2_config.json >> /home/sqream/SQREAM/logs/127.0.0.1/127.0.0.1_sqream2.log 2>&1 &
[5] 10271
[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ nohup /home/sqream/SQREAM/sqream/bin/sqreamd -config /home/sqream/SQREAM/sqream_config/4sq/sqream3_config.json >> /home/sqream/SQREAM/logs/127.0.0.1/127.0.0.1_sqream3.log 2>&1 &
[6] 10368
15. SQream Database에 접속해서 현재 Database의 버전을 확인합니다.
[sqream@nt761xda /home/sqream/SQREAM]$ sqream sql --database=master --username=sqream --password=sqream --host=127.0.0.1 --port=3108 --clustered --trim-varchars
Interactive client mode
To quit, use ^D or \q.
master=> select show_version();
v2022.1.4
1 row
time: 0.237170s
master=> \q
[sqream@nt761xda /home/sqream/SQREAM]$
16. 백업받은 metadata_server 파일이 필요 없다면 삭제 하도록 합니다.
[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ ls -rl
합계 12
drwxrwxr-x 3 sqream sqream 23 11월 2 15:10 temp
drwxrwxr-x 7 sqream sqream 142 11월 1 09:37 logs
drwxrwxr-x 2 sqream sqream 177 11월 2 13:37 leveldb_2022.1.2_20221102
drwxrwxr-x 2 sqream sqream 177 11월 2 14:49 leveldb_2022.1.2.1_20221102
drwxr-xr-x 2 sqream sqream 4096 11월 2 13:56 leveldb_2022.1.2.1_20221102_old
drwxrwxr-x 2 sqream sqream 177 11월 2 15:57 leveldb
drwxrwxr-x 3 sqream sqream 20 10월 27 18:54 databases
[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ rm -rf leveldb_2022.1.2.1_20221102_old
[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ ls -rl
합계 12
drwxrwxr-x 3 sqream sqream 23 11월 2 15:10 temp
drwxrwxr-x 7 sqream sqream 142 11월 1 09:37 logs
drwxrwxr-x 2 sqream sqream 177 11월 2 13:37 leveldb_2022.1.2_20221102
drwxr-xr-x 2 sqream sqream 4096 11월 2 13:56 leveldb_2022.1.2.1_20221102
drwxrwxr-x 2 sqream sqream 227 11월 2 16:05 leveldb
drwxrwxr-x 3 sqream sqream 20 10월 27 18:54 databases반응형