SQream Database Upgrade (v2022.1.4 to v2022.1.5.3)

@ 테스트 환경
CPU : Intel® Core™ i7-11600H 2.9GHz
RAM : 32GB
HDD : 512GB SSD(M.2, NVMe)
GPU : NVIDIA RTX 3050 Ti GDDR6 4GB
OS : CentOS Linux release 7.9.2009 (Core)
DB : v2022.1.5.3

 1. 임의 디렉터리에 업로드한 SQream Database 설치파일의 압축을 해제합니다.

[sqream@nt761xda /home/sqream]$ cd product/03_sqream/

[sqream@nt761xda /home/sqream/product/03_sqream]$ ls -rlt *.gz
-rw-r--r--. 1 sqream sqream 765101315 11월 12 18:05 sqream-db-v2022.1.2.1.cpu-generic.pascal.cuda-10.1.el7.centos.x86_64.tar.gz
-rw-r--r--. 1 sqream sqream 765103999 11월 12 18:17 sqream-db-v2022.1.2.cpu-generic.pascal.cuda-10.1.el7.centos.x86_64.tar.gz
-rw-r--r--. 1 sqream sqream 765168220  2월 17 13:56 sqream-db-v2022.1.4.cpu-generic.pascal.cuda-10.1.el7.centos.x86_64.tar.gz
-rw-r--r--. 1 sqream sqream 765952837  4월  6 11:24 sqream-db-v2022.1.5.3.cpu-generic.pascal.cuda-10.1.el7.centos.x86_64.tar.gz

[sqream@nt761xda /home/sqream/product/03_sqream]$ gzip -d sqream-db-v2022.1.5.3.cpu-generic.pascal.cuda-10.1.el7.centos.x86_64.tar.gz

[sqream@nt761xda /home/sqream/product/03_sqream]$ ls -rlt *.tar
-rw-r--r--. 1 sqream sqream 2196633600  4월  6 11:24 sqream-db-v2022.1.5.3.cpu-generic.pascal.cuda-10.1.el7.centos.x86_64.tar
 
[sqream@nt761xda /home/sqream/product/03_sqream]$ tar xvf sqream-db-v2022.1.5.3.cpu-generic.pascal.cuda-10.1.el7.centos.x86_64.tar
......

[sqream@nt761xda /home/sqream/product/03_sqream]$ ls -rlt
합계 4386732
-rw-r--r--.  1 sqream sqream  765101315 11월 12 18:05 sqream-db-v2022.1.2.1.cpu-generic.pascal.cuda-10.1.el7.centos.x86_64.tar.gz
-rw-r--r--.  1 sqream sqream  765103999 11월 12 18:17 sqream-db-v2022.1.2.cpu-generic.pascal.cuda-10.1.el7.centos.x86_64.tar.gz
-rw-r--r--.  1 sqream sqream  765168220  2월 17 13:56 sqream-db-v2022.1.4.cpu-generic.pascal.cuda-10.1.el7.centos.x86_64.tar.gz
-rw-r--r--.  1 sqream sqream 2196633600  4월  6 11:24 sqream-db-v2022.1.5.3.cpu-generic.pascal.cuda-10.1.el7.centos.x86_64.tar
drwxrwxr-x. 10 sqream sqream        145  4월 11 15:16 sqream

 
2. 압축을 해제한 SQream Database의 디렉터리의 위치 및 이름을 변경합니다.

[sqream@nt761xda /home/sqream/product/03_sqream]$ mv sqream /home/sqream/SQREAM/sqream-db-v2022.1.5.3
[sqream@nt761xda /home/sqream/product/03_sqream]$ ls -rlt /home/sqream/SQREAM
합계 0
drwxrwxr-x.  2 sqream sqream  25 11월 12 22:11 sqream_license
drwxrwxr-x. 10 sqream sqream 145 11월 12 22:13 sqream-db-v2022.1.2
drwxrwxr-x.  3 sqream sqream  22 11월 12 22:14 sqream_cluster
drwxrwxr-x.  3 sqream sqream  17 11월 12 22:31 sqream_config
drwxrwxr-x.  5 sqream sqream 102 11월 12 23:18 sqream-studio-5.4.4
drwxrwxr-x.  3 sqream sqream  47 11월 12 23:21 logs
drwxrwxr-x. 10 sqream sqream 145 11월 15 10:36 sqream-db-v2022.1.2.1
drwxrwxr-x. 10 sqream sqream 145 11월 15 10:46 sqream-db-v2022.1.4
lrwxrwxrwx.  1 sqream sqream  19 11월 15 10:48 sqream -> sqream-db-v2022.1.4
drwxrwxr-x.  5 sqream sqream 102 11월 15 11:00 sqream-admin
drwxrwxr-x.  5 sqream sqream  80 11월 16 14:41 prometheus
drwxrwxr-x.  7 sqream sqream 242 11월 16 15:03 sqream-dashboard
drwxrwxr-x.  2 sqream sqream 160  4월  6 09:44 sqream_script
drwxrwxr-x. 10 sqream sqream 145  4월 11 15:16 sqream-db-v2022.1.5.3

 
3. SQream Database에 접속하여 현재 버전과 Delete Predicates를 확인합니다.

[sqream@nt761xda /home/sqream/product/03_sqream]$ sqream sql --database=master --username=sqream --password=sqream --host=127.0.0.1 --port=3108 --clustered --trim-varchars
Interactive client mode
To quit, use ^D or \q.

master=> SELECT show_version();
v2022.1.4
1 row
time: 0.055925s

master=> SELECT * FROM sqream_catalog.delete_predicates;
0 rows
time: 0.071056s

4. Upgrade 전, 현재 SQream Database의 metadata_server를 백업합니다.

master=> SELECT backup_metadata('/home/sqream/SQREAM/sqream_cluster/sqreamdb/leveldb_2022.1.4_20230411_old');
executed
time: 0.060955s

master=> \q

5. SQream Database를 중지합니다.

[sqream@nt761xda /home/sqream/product/03_sqream]$ killall metadata_server

 

6. SQream Studio가 기동 되어 있다면 SQream Studio 관련 프로세스도 모두 종료시킵니다.

[sqream@nt761xda /home/sqream/product/03_sqream]$ ps -ef|grep sqream
sqream    1305     1  0 07:35 ?        00:02:01 /usr/local/bin/prometheus --config.file=/home/sqream/SQREAM/prometheus/prometheus.yml --storage.tsdb.path=/home/sqream/SQREAM/prometheus/lib/prometheus --web.console.templates=/home/sqream/SQREAM/prometheus/consoles --web.console.libraries=/home/sqream/SQREAM/prometheus/console_libraries
sqream    1306     1  1 07:35 ?        00:05:00 /usr/bin/node_exporter
sqream    1307     1  0 07:35 ?        00:01:08 /usr/bin/nvidia_exporter
root     15140 15088  0 11:04 pts/1    00:00:00 su - sqream
sqream   15141 15140  0 11:04 pts/1    00:00:00 -bash
sqream   16190 15141  0 15:28 pts/1    00:00:00 ps -ef
sqream   16191 15141  0 15:28 pts/1    00:00:00 grep --color=auto sqream

7. SQream Database를 Upgrade(Product Link) 합니다.

[sqream@nt761xda /home/sqream/product/03_sqream]$ cd /home/sqream/SQREAM

[sqream@nt761xda /home/sqream/SQREAM]$ ls -rl
합계 0
drwxrwxr-x.  2 sqream sqream 160  4월  6 09:44 sqream_script
drwxrwxr-x.  2 sqream sqream  25 11월 12 22:11 sqream_license
drwxrwxr-x.  3 sqream sqream  17 11월 12 22:31 sqream_config
drwxrwxr-x.  3 sqream sqream  22 11월 12 22:14 sqream_cluster
drwxrwxr-x.  5 sqream sqream 102 11월 12 23:18 sqream-studio-5.4.4
drwxrwxr-x. 10 sqream sqream 145  4월 11 15:16 sqream-db-v2022.1.5.3
drwxrwxr-x. 10 sqream sqream 145 11월 15 10:46 sqream-db-v2022.1.4
drwxrwxr-x. 10 sqream sqream 145 11월 15 10:36 sqream-db-v2022.1.2.1
drwxrwxr-x. 10 sqream sqream 145 11월 12 22:13 sqream-db-v2022.1.2
drwxrwxr-x.  7 sqream sqream 242 11월 16 15:03 sqream-dashboard
drwxrwxr-x.  5 sqream sqream 102 11월 15 11:00 sqream-admin
lrwxrwxrwx.  1 sqream sqream  19 11월 15 10:48 sqream -> sqream-db-v2022.1.4
drwxrwxr-x.  5 sqream sqream  80 11월 16 14:41 prometheus
drwxrwxr-x.  3 sqream sqream  47 11월 12 23:21 logs

[sqream@nt761xda /home/sqream/SQREAM]$ ln -sTf sqream-db-v2022.1.5.3 sqream

[sqream@nt761xda /home/sqream/SQREAM]$ ls -rl
합계 0
drwxrwxr-x.  2 sqream sqream 160  4월  6 09:44 sqream_script
drwxrwxr-x.  2 sqream sqream  25 11월 12 22:11 sqream_license
drwxrwxr-x.  3 sqream sqream  17 11월 12 22:31 sqream_config
drwxrwxr-x.  3 sqream sqream  22 11월 12 22:14 sqream_cluster
drwxrwxr-x.  5 sqream sqream 102 11월 12 23:18 sqream-studio-5.4.4
drwxrwxr-x. 10 sqream sqream 145  4월 11 15:16 sqream-db-v2022.1.5.3
drwxrwxr-x. 10 sqream sqream 145 11월 15 10:46 sqream-db-v2022.1.4
drwxrwxr-x. 10 sqream sqream 145 11월 15 10:36 sqream-db-v2022.1.2.1
drwxrwxr-x. 10 sqream sqream 145 11월 12 22:13 sqream-db-v2022.1.2
drwxrwxr-x.  7 sqream sqream 242 11월 16 15:03 sqream-dashboard
drwxrwxr-x.  5 sqream sqream 102 11월 15 11:00 sqream-admin
lrwxrwxrwx.  1 sqream sqream  21  4월 11 15:29 sqream -> sqream-db-v2022.1.5.3
drwxrwxr-x.  5 sqream sqream  80 11월 16 14:41 prometheus

8. Upgrade Storage를 수행합니다.

[sqream@nt761xda /home/sqream/SQREAM]$ upgrade_storage /home/sqream/SQREAM/sqream_cluster/sqreamdb
get_leveldb_version path{/home/sqream/SQREAM/sqream_cluster/sqreamdb/leveldb}
current storage version 42
no need to upgrade

• Upgrade Storage 명령어는 다음과 같습니다. upgrade_storage <cluster path>

9. SQream Database를 기동 합니다.

[sqream@nt761xda /home/sqream/SQREAM]$ nohup /home/sqream/SQREAM/sqream/bin/metadata_server 127.0.0.1 3105 >> /home/sqream/SQREAM/logs/127.0.0.1/127.0.0.1_metadata_server.log 2>&1 &
[1] 26097

[sqream@nt761xda /home/sqream/SQREAM]$ nohup /home/sqream/SQREAM/sqream/bin/server_picker 127.0.0.1 3105 3108 >> /home/sqream/SQREAM/logs/127.0.0.1/127.0.0.1_server_picker.log 2>&1 &
[2] 26147

[sqream@nt761xda /home/sqream/SQREAM]$ nohup /home/sqream/SQREAM/sqream/bin/sqreamd -config /home/sqream/SQREAM/sqream_config/4sq/sqream0_config.json >> /home/sqream/SQREAM/logs/127.0.0.1/127.0.0.1_sqream0.log 2>&1 &
[3] 26200

[sqream@nt761xda /home/sqream/SQREAM]$ nohup /home/sqream/SQREAM/sqream/bin/sqreamd -config /home/sqream/SQREAM/sqream_config/4sq/sqream1_config.json >> /home/sqream/SQREAM/logs/127.0.0.1/127.0.0.1_sqream1.log 2>&1 &
[4] 26204

[sqream@nt761xda /home/sqream/SQREAM]$ nohup /home/sqream/SQREAM/sqream/bin/sqreamd -config /home/sqream/SQREAM/sqream_config/4sq/sqream2_config.json >> /home/sqream/SQREAM/logs/127.0.0.1/127.0.0.1_sqream2.log 2>&1 &
[5] 26283

[sqream@nt761xda /home/sqream/SQREAM]$ nohup /home/sqream/SQREAM/sqream/bin/sqreamd -config /home/sqream/SQREAM/sqream_config/4sq/sqream3_config.json >> /home/sqream/SQREAM/logs/127.0.0.1/127.0.0.1_sqream3.log 2>&1 &
[6] 26360

10. SQream Database에 접속해서 현재 Database의 버전을 확인합니다.

[sqream@nt761xda /home/sqream/SQREAM]$ sqream sql --database=master --username=sqream --password=sqream --host=127.0.0.1 --port=3108 --clustered --trim-varchars
Interactive client mode
To quit, use ^D or \q.

master=> SELECT show_version();
v2022.1.5.3
1 row
time: 0.088109s

11. 현재 상태의 metadata_server를 백업합니다.

master=> SELECT backup_metadata('/home/sqream/SQREAM/sqream_cluster/sqreamdb/leveldb_2022.1.5.3_20230411_new');
executed
time: 0.090575s

master=> \q

[sqream@nt761xda /home/sqream/SQREAM]$

• metadata_server를 백업받는 이유는, 백업받은 metadata_server로 덮어 씌움으로써 metadata_server를 정리하기 위함입니다.

12. SQream Database를 중지합니다.

[sqream@nt761xda /home/sqream/SQREAM]$ killall metadata_server

13. 현재 버전의 metadata_server는 이름 변경 후, 바로 전에 받은 metadata_server의 백업본으로 현재 metadata_server를 덮어쓰기 합니다.

[sqream@nt761xda /home/sqream/SQREAM]$ cd /home/sqream/SQREAM/sqream_cluster/sqreamdb

[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ ls -rl
합계 16
drwxrwxr-x. 4 sqream sqream   42 11월 15 10:48 temp
drwxrwxr-x. 5 sqream sqream  102 11월 14 14:59 logs
drwxrwxr-x. 2 sqream sqream  177  4월 11 15:36 leveldb_2022.1.5.3_20230411_new
drwxrwxr-x. 2 sqream sqream  177  4월 11 15:19 leveldb_2022.1.4_20230411_old
drwxrwxr-x. 2 sqream sqream 4096 11월 15 10:51 leveldb_2022.1.4_20221115
drwxrwxr-x. 2 sqream sqream  177 11월 15 10:37 leveldb_2022.1.2_20221115
drwxr-xr-x. 2 sqream sqream 4096 11월 15 10:40 leveldb_2022.1.2.1_20221115
drwxrwxr-x. 2 sqream sqream 4096  4월 11 15:35 leveldb
drwxrwxr-x. 5 sqream sqream   48 11월 16 14:59 databases

[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ mv leveldb leveldb_2022.1.5.3_20230411

[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ mv leveldb_2022.1.5.3_20230411_new leveldb

[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ ls -rl
합계 16
drwxrwxr-x. 4 sqream sqream   42 11월 15 10:48 temp
drwxrwxr-x. 5 sqream sqream  102 11월 14 14:59 logs
drwxrwxr-x. 2 sqream sqream 4096  4월 11 15:35 leveldb_2022.1.5.3_20230411
drwxrwxr-x. 2 sqream sqream  177  4월 11 15:19 leveldb_2022.1.4_20230411_old
drwxrwxr-x. 2 sqream sqream 4096 11월 15 10:51 leveldb_2022.1.4_20221115
drwxrwxr-x. 2 sqream sqream  177 11월 15 10:37 leveldb_2022.1.2_20221115
drwxr-xr-x. 2 sqream sqream 4096 11월 15 10:40 leveldb_2022.1.2.1_20221115
drwxrwxr-x. 2 sqream sqream  177  4월 11 15:36 leveldb
drwxrwxr-x. 5 sqream sqream   48 11월 16 14:59 databases

14. SQream Database를 기동 합니다.

[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ nohup /home/sqream/SQREAM/sqream/bin/metadata_server 127.0.0.1 3105 >> /home/sqream/SQREAM/logs/127.0.0.1/127.0.0.1_metadata_server.log 2>&1 &
[1] 9975

[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ nohup /home/sqream/SQREAM/sqream/bin/server_picker 127.0.0.1 3105 3108 >> /home/sqream/SQREAM/logs/127.0.0.1/127.0.0.1_server_picker.log 2>&1 &
[2] 10060

[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ nohup /home/sqream/SQREAM/sqream/bin/sqreamd -config /home/sqream/SQREAM/sqream_config/4sq/sqream0_config.json >> /home/sqream/SQREAM/logs/127.0.0.1/127.0.0.1_sqream0.log 2>&1 &
[3] 10138

[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ nohup /home/sqream/SQREAM/sqream/bin/sqreamd -config /home/sqream/SQREAM/sqream_config/4sq/sqream1_config.json >> /home/sqream/SQREAM/logs/127.0.0.1/127.0.0.1_sqream1.log 2>&1 &
[4] 10247

[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ nohup /home/sqream/SQREAM/sqream/bin/sqreamd -config /home/sqream/SQREAM/sqream_config/4sq/sqream2_config.json >> /home/sqream/SQREAM/logs/127.0.0.1/127.0.0.1_sqream2.log 2>&1 &
[5] 10271

[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ nohup /home/sqream/SQREAM/sqream/bin/sqreamd -config /home/sqream/SQREAM/sqream_config/4sq/sqream3_config.json >> /home/sqream/SQREAM/logs/127.0.0.1/127.0.0.1_sqream3.log 2>&1 &
[6] 10368

15. SQream Database에 접속해서 현재 Database의 버전을 확인합니다.

[sqream@nt761xda /home/sqream]$ sqream sql --database=master --username=sqream --password=sqream --host=127.0.0.1 --port=3108 --clustered --trim-varchars
Interactive client mode
To quit, use ^D or \q.

master=> SELECT show_version();
v2022.1.5.3
1 row
time: 0.066449s

master=> \q

[sqream@nt761xda /home/sqream/SQREAM]$

16. 백업받은 metadata_server 파일이 필요 없다면 삭제 하도록 합니다.

[sqream@nt761xda /home/sqream]$ cd /home/sqream/SQREAM/sqream_cluster/sqreamdb

[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ ls -rl
합계 16
drwxrwxr-x. 4 sqream sqream   42 11월 15 10:48 temp
drwxrwxr-x. 5 sqream sqream  102 11월 14 14:59 logs
drwxrwxr-x. 2 sqream sqream 4096  4월 11 15:35 leveldb_2022.1.5.3_20230411
drwxrwxr-x. 2 sqream sqream  177  4월 11 15:19 leveldb_2022.1.4_20230411_old
drwxrwxr-x. 2 sqream sqream 4096 11월 15 10:51 leveldb_2022.1.4_20221115
drwxrwxr-x. 2 sqream sqream  177 11월 15 10:37 leveldb_2022.1.2_20221115
drwxr-xr-x. 2 sqream sqream 4096 11월 15 10:40 leveldb_2022.1.2.1_20221115
drwxrwxr-x. 2 sqream sqream  227  4월 11 16:13 leveldb
drwxrwxr-x. 5 sqream sqream   48 11월 16 14:59 databases

[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ rm -rf leveldb_2022.1.4_20221115

[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ mv leveldb_2022.1.4_20230411_old leveldb_2022.1.4_20230411

[sqream@nt761xda /home/sqream/SQREAM/sqream_cluster/sqreamdb]$ ls -rl
합계 12
drwxrwxr-x. 4 sqream sqream   42 11월 15 10:48 temp
drwxrwxr-x. 5 sqream sqream  102 11월 14 14:59 logs
drwxrwxr-x. 2 sqream sqream 4096  4월 11 15:35 leveldb_2022.1.5.3_20230411
drwxrwxr-x. 2 sqream sqream  177  4월 11 15:19 leveldb_2022.1.4_20230411
drwxrwxr-x. 2 sqream sqream  177 11월 15 10:37 leveldb_2022.1.2_20221115
drwxr-xr-x. 2 sqream sqream 4096 11월 15 10:40 leveldb_2022.1.2.1_20221115
drwxrwxr-x. 2 sqream sqream  227  4월 11 16:13 leveldb
drwxrwxr-x. 5 sqream sqream   48 11월 16 14:59 databases