记一次PMM容器被误删除的恢复

昨天PMM Data Container和PMM Server Container 都不小心被误删了, 还好是使用的docker rm container_id删的, 没有加-v. 本人docker渣渣, 后来找运维同时帮忙终于恢复了
首先找到宿主机volumes路径

[root@node1 volumes]# pwd
/var/lib/docker/volumes
[root@node1 volumes]# ll
total 32
drwxr-xr-x 3 root root 26 Aug 3 2018 4430beeb61fbc26802dc1b31d9fe3a02772482d8ae31bd50110957f16256f02e
drwxr-xr-x 3 root root 26 Jul 19 18:29 47ce712fd5e32abcf60e9650f349ac1304e0c20d062bd54724123f6cb700a79e
drwxr-xr-x 3 root root 26 Aug 3 2018 6ecf1c674f9f8e6f3dda7f7da352b70860125147b4489c6a8fe00fa59892436f
drwxr-xr-x 3 root root 26 Aug 3 2018 83db182943136c354bf95c0b4d07a8272017fd1698557d27b97dae49c82a8e76
drwxr-xr-x 3 root root 26 Jul 19 18:29 c9e3a964fd76dcad88ac992933ca97ac920c41239f9807f1ca91f939bd71ea03
drwxr-xr-x 3 root root 26 Jul 19 18:29 e92a37881c9b378bc4b00094b3f7e38ace3ae3655c33677a79da1be5652a318a
drwxr-xr-x 3 root root 26 Aug 3 2018 fcddf90d2cb64035c8c69165f69c5d7a780ac580c991f02af801bc65fa2ca609
-rw------- 1 root root 65536 Jul 19 18:29 metadata.db
drwxr-xr-x 3 root root 26 Jul 19 18:26 sentry-data
drwxr-xr-x 3 root root 26 Jul 19 18:26 sentry-postgres

找到对应的目录

prometheus
[root@node1 volumes]# ls /var/lib/docker/volumes/6ecf1c674f9f8e6f3dda7f7da352b70860125147b4489c6a8fe00fa59892436f/_data
00 06 0c 12 18 1e 24 2a 30 36 3c 42 48 4e 54 5a 60 66 6c 72 78 7e 84 8a 90 96 9c a2 a8 ae b2 b8 be c4 ca d0 d6 dc e1 e7 ed f3 f9 ff
01 07 0d 13 19 1f 25 2b 31 37 3d 43 49 4f 55 5b 61 67 6d 73 79 7f 85 8b 91 97 9d a3 a9 af b3 b9 bf c5 cb d1 d7 dd e2 e8 ee f4 fa heads.db
02 08 0e 14 1a 20 26 2c 32 38 3e 44 4a 50 56 5c 62 68 6e 74 7a 80 86 8c 92 98 9e a4 aa archived_fingerprint_to_metric b4 ba c0 c6 cc d2 d8 de e3 e9 ef f5 fb labelname_to_labelvalues
03 09 0f 15 1b 21 27 2d 33 39 3f 45 4b 51 57 5d 63 69 6f 75 7b 81 87 8d 93 99 9f a5 ab archived_fingerprint_to_timerange b5 bb c1 c7 cd d3 d9 df e4 ea f0 f6 fc labelpair_to_fingerprints
04 0a 10 16 1c 22 28 2e 34 3a 40 46 4c 52 58 5e 64 6a 70 76 7c 82 88 8e 94 9a a0 a6 ac b0 b6 bc c2 c8 ce d4 da DIRTY e5 eb f1 f7 fd mappings.db
05 0b 11 17 1d 23 29 2f 35 3b 41 47 4d 53 59 5f 65 6b 71 77 7d 83 89 8f 95 9b a1 a7 ad b1 b7 bd c3 c9 cf d5 db e0 e6 ec f2 f8 fe VERSION

consul
[root@node1 volumes]# ls /var/lib/docker/volumes/fcddf90d2cb64035c8c69165f69c5d7a780ac580c991f02af801bc65fa2ca609/_data
checkpoint-signature node-id raft serf

mysql
[root@node1 volumes]# ls /var/lib/docker/volumes/4430beeb61fbc26802dc1b31d9fe3a02772482d8ae31bd50110957f16256f02e/_data
ibdata1 ib_logfile0 ib_logfile1 mysql mysql.sock orchestrator performance_schema pmm pmm@002dmanaged RPM_UPGRADE_HISTORY RPM_UPGRADE_MARKER-LAST test

grafana
[root@node1 volumes]# ls /var/lib/docker/volumes/83db182943136c354bf95c0b4d07a8272017fd1698557d27b97dae49c82a8e76/_data
grafana.db PERCONA_DASHBOARDS_VERSION plugins png sessions

重新创建data container

docker create --name pmm-data2 \
-v /var/lib/docker/volumes/6ecf1c674f9f8e6f3dda7f7da352b70860125147b4489c6a8fe00fa59892436f/_data:/opt/prometheus/data \
-v /var/lib/docker/volumes/fcddf90d2cb64035c8c69165f69c5d7a780ac580c991f02af801bc65fa2ca609/_data:/opt/consul-data \
-v /var/lib/docker/volumes/4430beeb61fbc26802dc1b31d9fe3a02772482d8ae31bd50110957f16256f02e/_data:/var/lib/mysql \
-v /var/lib/docker/volumes/83db182943136c354bf95c0b4d07a8272017fd1698557d27b97dae49c82a8e76/_data:/var/lib/grafana \
percona/pmm-server:latest /bin/ture

重新创建server

docker run -d \
  -p 8080:80 \
  --volumes-from pmm-data2 \
  --name pmm-server \
  -e SERVER_USER=username \
  -e SERVER_PASSWORD=password \
  -e METRICS_RETENTION=720h \
  -e METRICS_RESOLUTION=5s \
  -e DISABLE_TELEMETRY=true \
  --restart always \
  percona/pmm-server:latest

之后PMM页面就可以访问了, 不过如果你做了一些额外配置, 比如我再prometheus.yml中加了canal监控, 那就需要重新添加了.