TiDB中TiKV性能参数调整过程

环境信息：

$ ansible --version

ansible 2.6.3

config file = /home/tidb/tidb-ansible/ansible.cfg

configured module search path = [u'/home/tidb/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']

ansible python module location = /usr/lib/python2.7/site-packages/ansible

executable location = /bin/ansible

python version = 2.7.5 (default, Jul 13 2018, 13:06:57) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]

# ./tikv-server --version

TiKV

Release Version: 2.0.6

Git Commit Hash: 57c83dc4ebc93d38d77dc8f7d66db224760766cc

Git Commit Branch: release-2.0

UTC Build Time: 2018-08-03 11:28:38

Rust Version: 1.27.0-nightly (48fa6f963 2018-04-05)

# ./tidb-server -V

Release Version: v2.0.6

Git Commit Hash: b13bc08462a584a085f377625a7bab0cc0351570

Git Commit Branch: release-2.0

UTC Build Time: 2018-08-06 10:57:46

GoVersion: go version go1.10.2 linux/amd64

TiKV Min Version: 2.0.0-rc.4.1

# ./pd-server -V

Release Version: v2.0.5

Git Commit Hash: b64716707b7279a4ae822be767085ff17b5f3fea

Git Branch: release-2.0

UTC Build Time: 2018-07-06 10:27:51

1.在配置文件中修改：

$ pwd

/home/tidb/tidb-ansible

$ vim conf/tikv.yml

需要修改的参数如下：

[server]

grpc-concurrency = 4 -->8

grpc-raft-conn-num = 10 -->24

[storage]

scheduler-concurrency = 2048000

scheduler-worker-pool-size = 8 -->16

[coprocessor]

region-max-size = "144MB" -->384MB

region-split-size = "96MB" -->256MB

[raftstore]

region-split-check-diff = "6MB" -->32MB

sync-log = true

[rocksdb]

max-background-jobs = 6 -->32

max-open-files = 40960 -->65535

[rocksdb.defaultcf]

block-cache-size = "48331MB" -->80G

[rocksdb.writecf]

block-cache-size = "28998MB" -->30G

[raftdb.defaultcf]

block-cache-size = "2GB"

此外参数需要调整为32MB。

target-file-size-base = "8MB" -->32MB

2.启动：使用tidb的身份执行滚动升级。

$ pwd

/home/tidb/tidb-ansible

$ ansible-playbook rolling_update.yml --tags=tikv

3.验证：

...

TASK [tikv : debug] *******************************************************

ok: [10.19.75.102] => {

"tikv_conf": {

"coprocessor": {

"region-max-size": "384MB",

"region-split-size": "256MB"

},

"global": {},

"import": {},

"metric": {

"address": "10.19.85.149:9091",

"interval": "15s",

"job": "tikv"

},

"pd": {},

"raftdb": {

"defaultcf": {

"target-file-size-base": "32MB"

},

"max-open-files": 65535

},

"raftstore": {

"raftdb-path": "",

"region-split-check-diff": "32MB",

"sync-log": true

},

"readpool": {

"coprocessor": {},

"storage": {}

},

"rocksdb": {

"defaultcf": {

"block-cache-size": "80GB",

"target-file-size-base": "32MB"

},

"lockcf": {

"target-file-size-base": "32MB"

},

"max-background-jobs": 32,

"max-open-files": 65535,

"wal-dir": "",

"writecf": {

"block-cache-size": "30GB",

"target-file-size-base": "32MB"

}

},

"security": {

"ca-path": "",

"cert-path": "",

"key-path": ""

},

"server": {

"grpc-concurrency": 8,

"grpc-raft-conn-num": 24,

"labels": {}

},

"storage": {

"scheduler-concurrency": 2048000,

"scheduler-worker-pool-size": 16

}

}

}

...

--修改成功的提示信息：

PLAY RECAP *****************************************************************************************

10.19.130.141 : ok=55 changed=4 unreachable=0 failed=0

10.19.148.186 : ok=55 changed=4 unreachable=0 failed=0

10.19.15.103 : ok=10 changed=0 unreachable=0 failed=0

10.19.189.221 : ok=10 changed=0 unreachable=0 failed=0

10.19.75.102 : ok=55 changed=4 unreachable=0 failed=0

10.19.85.149 : ok=11 changed=0 unreachable=0 failed=0

localhost : ok=1 changed=0 unreachable=0 failed=0

Congrats! All goes well. :-)

登录待tikv节点查看配置,随机查看修改过的参数是否生效。

# pwd

/data/deploy/conf

# cat tikv.toml | grep -i target

target-file-size-base = "32MB"

target-file-size-base = "32MB"

target-file-size-base = "32MB"

target-file-size-base = "32MB"

对于既有的sst文件大小还是按照之前默认的8M存储的。

对于新加入的数据sst文件的大小则按照新设置的32MB存储。

-rw-r--r-- 1 tidb tidb 9.6M Aug 23 13:41 797214.sst

-rw-r--r-- 1 tidb tidb 9.5M Aug 23 13:41 797215.sst

-rw-r--r-- 1 tidb tidb 2.8M Aug 23 13:41 797216.sst

-rw-rw-r-- 1 tidb tidb 584M Aug 23 14:09 LOG.old.1535004595111057

-rw-r--r-- 1 tidb tidb 16 Aug 23 14:10 CURRENT

-rw-r--r-- 1 tidb tidb 12K Aug 23 14:10 OPTIONS-797235

-rw-r--r-- 1 tidb tidb 36M Aug 23 14:17 797527.sst

-rw-r--r-- 1 tidb tidb 41M Aug 23 14:17 797528.sst

-rw-r--r-- 1 tidb tidb 38M Aug 23 14:18 797529.sst

-rw-r--r-- 1 tidb tidb 37M Aug 23 14:18 797530.sst

-rw-r--r-- 1 tidb tidb 35M Aug 23 14:18 797531.sst

-rw-r--r-- 1 tidb tidb 35M Aug 23 14:18 797532.sst