Ubuntu16安装ClickHouse
第1步:检查是否支持SSE4.2
因为向量化执行需要用到这一项特性
如果不支持SSE指令集,则不能直接使用预编译安装包,
需要通过源码编译特定的版本进行安装
grep -q sse4_2 /proc/cpuinfo && echo "SSE 4.2 supported" || echo "SSE 4.2 not supported"
第2步:为apt添加clickhouse仓库
网上好多都说使用Yandex的官方仓库,
我一开始的时候也是用的Yandex的官方仓库,但是速度特慢,20KB/s左右下载完要1个多小时
于是换了清华源,链接在这里
https://mirrors.cnnic.cn/help/clickhouse/
当然你也可以选用阿里云源,163源,(一般阿里云源比清华源用得人更多,但是不知道为什么clickhouse相关的很少)
vim /etc/apt/sources.list
添加:
deb https://mirrors.tuna.tsinghua.edu.cn/clickhouse/deb/stable/ main/
或者:
deb http://repo.yandex.ru/clickhouse/deb/stable/ main/
第3步:添加依赖并更新apt数据库
apt install dirmngr
apt-key adv --keyserver keyserver.ubuntu.com --recv E0C56BD4
apt update
第4步:安装clickhouse服务端及客户端
apt-get install clickhouse-client clickhouse-server
第5步:启动clickhouse
先修改配置文件,可以远程访问
# 文件可能没有写入权限,先添加写入权限
sudo chmod 600 /etc/clickhouse-server/config.xml
vim /etc/clickhouse-server/config.xml
# config.xml文件添加以下内容或者直接找到把注释去除即可
<listen_host>0.0.0.0</listen_host>
# 启动命令
systemctl start clickhouse-server
# 查看clickhouse是否启动成功
systemctl status clickhouse-server
# 添加开机启动
systemctl enable clickhouse-server
安装DBeaver
DBeaver很好安装的,不需要破解
点击下面可以直接下载DBeaver
https://dbeaver.io/files/dbeaver-ce-latest-x86_64-setup.exe
你也可以直接去他们官网下载:
https://dbeaver.io/download/
DBeaver连接clickhouse
数据导出:
clickhouse-client -h 127.0.0.1 -u default --password --database="default" --query="select * from default.dy_user_info FORMAT CSV" > ts-20210611.csv
数据导入:
clickhouse-client -h 127.0.0.1 -u default --password --database="default" --query="insert into default.dy_user_info FORMAT CSV" < ./ts-20210611.csv
安装完成后需要配置日志表query_log和query_thread_log的生命周期,要不然这两个表会非常大
ALTER TABLE system.query_log MODIFY TTL event_date + toIntervalDay(3);
ALTER TABLE system.query_thread_log MODIFY TTL event_date + toIntervalDay(3);
ALTER TABLE system.asynchronous_metric_log MODIFY TTL event_date + toIntervalDay(60);
ALTER TABLE system.trace_log MODIFY TTL event_date + toIntervalDay(60);
ALTER TABLE system.metric_log MODIFY TTL event_date + toIntervalDay(60);
清理日志表(用工具执行SQL表过大导致执行超时,SQL实际在后台服务运行,等待即可):
alter table system.query_thread_log drop partition '202201';
alter table system.query_log drop partition '202201';
如果日志表大于50G需要做以下配置:
cd /var/lib/clickhouse/flags/
sudo touch force_drop_table && sudo chmod 666 force_drop_table
查询统计表数据大小:
select
database,
table,
formatReadableSize(size) as size,
formatReadableSize(bytes_on_disk) as bytes_on_disk,
formatReadableSize(data_uncompressed_bytes) as data_uncompressed_bytes,
formatReadableSize(data_compressed_bytes) as data_compressed_bytes,
compress_rate,
rows,
days,
formatReadableSize(avgDaySize) as avgDaySize
from
(
select
database,
table,
sum(bytes) as size,
sum(rows) as rows,
min(min_date) as min_date,
max(max_date) as max_date,
sum(bytes_on_disk) as bytes_on_disk,
sum(data_uncompressed_bytes) as data_uncompressed_bytes,
sum(data_compressed_bytes) as data_compressed_bytes,
(data_compressed_bytes / data_uncompressed_bytes) * 100 as compress_rate,
max_date - min_date as days,
size / (max_date - min_date) as avgDaySize
from system.parts
where active
group by
database,
table
order by data_compressed_bytes desc
);