2020-12-30T06:50:31.785 INFO:root:teuthology version: 1.0.0-8e49dda4 2020-12-30T06:50:31.787 INFO:teuthology.results:subset: '954/9999' 2020-12-30T06:50:31.787 INFO:teuthology.results:seed: '2477' 2020-12-30T11:02:12.819 INFO:root:teuthology version: 1.0.0-8e49dda4 2020-12-30T11:02:12.952 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-12-30T22:39:40.711 INFO:teuthology.results:Tests finished! gathering results... 2020-12-30T22:48:41.243 INFO:scrape:Found 446 jobs 2020-12-30T22:48:55.550 WARNING:scrape:Misunderstood line: 2020-12-30T22:48:55.682 WARNING:scrape:Misunderstood line: 2020-12-30T22:50:08.599 WARNING:scrape:Misunderstood line: 2020-12-30T22:50:09.516 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7f52be8860d6] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x2712f0) [0x7f52be8862f0] 2020-12-30T22:50:09.517 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-12-30T08:08:14.285 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-12-30T08:08:14.693 INFO:teuthology.orchestra.run.smithi141.stdout: 2020-12-30T22:50:09.518 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:09:17.093 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi141.front.sepia.ceph.com , 2020-12-30T08:09:17.094 DEBUG:teuthology.orchestra.run.smithi141:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 , 2020-12-30T08:09:17.216 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi141.front.sepia.ceph.com 2020-12-30T22:50:09.519 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:10:29.166 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog , 2020-12-30T08:10:29.465 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... , 2020-12-30T08:10:29.467 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2020-12-30T22:50:09.645 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7f52be8860d6] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x2712f0) [0x7f52be8862f0] 2020-12-30T22:50:09.647 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-12-30T08:08:14.285 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-12-30T08:08:14.693 INFO:teuthology.orchestra.run.smithi141.stdout: 2020-12-30T22:50:09.647 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:09:17.093 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi141.front.sepia.ceph.com , 2020-12-30T08:09:17.094 DEBUG:teuthology.orchestra.run.smithi141:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 , 2020-12-30T08:09:17.216 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi141.front.sepia.ceph.com 2020-12-30T22:50:09.648 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:10:29.166 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog , 2020-12-30T08:10:29.465 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... , 2020-12-30T08:10:29.467 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2020-12-30T22:50:09.775 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7f52be8860d6] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x2712f0) [0x7f52be8862f0] 2020-12-30T22:50:09.777 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-12-30T08:08:14.285 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-12-30T08:08:14.693 INFO:teuthology.orchestra.run.smithi141.stdout: 2020-12-30T22:50:09.778 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:09:17.093 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi141.front.sepia.ceph.com , 2020-12-30T08:09:17.094 DEBUG:teuthology.orchestra.run.smithi141:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 , 2020-12-30T08:09:17.216 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi141.front.sepia.ceph.com 2020-12-30T22:50:09.778 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:10:29.166 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog , 2020-12-30T08:10:29.465 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... , 2020-12-30T08:10:29.467 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2020-12-30T22:50:09.908 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7f52be8860d6] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x2712f0) [0x7f52be8862f0] 2020-12-30T22:50:09.909 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-12-30T08:08:14.285 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-12-30T08:08:14.693 INFO:teuthology.orchestra.run.smithi141.stdout: 2020-12-30T22:50:09.910 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:09:17.093 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi141.front.sepia.ceph.com , 2020-12-30T08:09:17.094 DEBUG:teuthology.orchestra.run.smithi141:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 , 2020-12-30T08:09:17.216 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi141.front.sepia.ceph.com 2020-12-30T22:50:09.911 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:10:29.166 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog , 2020-12-30T08:10:29.465 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... , 2020-12-30T08:10:29.467 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2020-12-30T22:50:10.057 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7f52be8860d6] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x2712f0) [0x7f52be8862f0] 2020-12-30T22:50:10.059 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-12-30T08:08:14.285 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-12-30T08:08:14.693 INFO:teuthology.orchestra.run.smithi141.stdout: 2020-12-30T22:50:10.060 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:09:17.093 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi141.front.sepia.ceph.com , 2020-12-30T08:09:17.094 DEBUG:teuthology.orchestra.run.smithi141:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 , 2020-12-30T08:09:17.216 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi141.front.sepia.ceph.com 2020-12-30T22:50:10.061 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:10:29.166 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog , 2020-12-30T08:10:29.465 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... , 2020-12-30T08:10:29.467 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2020-12-30T22:50:10.190 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7f52be8860d6] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x2712f0) [0x7f52be8862f0] 2020-12-30T22:50:10.191 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-12-30T08:08:14.285 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-12-30T08:08:14.693 INFO:teuthology.orchestra.run.smithi141.stdout: 2020-12-30T22:50:10.192 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:09:17.093 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi141.front.sepia.ceph.com , 2020-12-30T08:09:17.094 DEBUG:teuthology.orchestra.run.smithi141:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 , 2020-12-30T08:09:17.216 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi141.front.sepia.ceph.com 2020-12-30T22:50:10.193 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:10:29.166 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog , 2020-12-30T08:10:29.465 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... , 2020-12-30T08:10:29.467 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2020-12-30T22:50:10.320 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7f52be8860d6] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x2712f0) [0x7f52be8862f0] 2020-12-30T22:50:10.322 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-12-30T08:08:14.285 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-12-30T08:08:14.693 INFO:teuthology.orchestra.run.smithi141.stdout: 2020-12-30T22:50:10.323 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:09:17.093 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi141.front.sepia.ceph.com , 2020-12-30T08:09:17.094 DEBUG:teuthology.orchestra.run.smithi141:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 , 2020-12-30T08:09:17.216 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi141.front.sepia.ceph.com 2020-12-30T22:50:10.323 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:10:29.166 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog , 2020-12-30T08:10:29.465 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... , 2020-12-30T08:10:29.467 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2020-12-30T22:50:10.457 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7f52be8860d6] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x2712f0) [0x7f52be8862f0] 2020-12-30T22:50:10.459 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-12-30T08:08:14.285 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-12-30T08:08:14.693 INFO:teuthology.orchestra.run.smithi141.stdout: 2020-12-30T22:50:10.460 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:09:17.093 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi141.front.sepia.ceph.com , 2020-12-30T08:09:17.094 DEBUG:teuthology.orchestra.run.smithi141:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 , 2020-12-30T08:09:17.216 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi141.front.sepia.ceph.com 2020-12-30T22:50:10.460 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:10:29.166 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog , 2020-12-30T08:10:29.465 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... , 2020-12-30T08:10:29.467 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2020-12-30T22:50:10.588 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7f52be8860d6] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x2712f0) [0x7f52be8862f0] 2020-12-30T22:50:10.590 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-12-30T08:08:14.285 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2020-12-30T08:08:14.693 INFO:teuthology.orchestra.run.smithi141.stdout: 2020-12-30T22:50:10.591 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:09:17.093 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi141.front.sepia.ceph.com , 2020-12-30T08:09:17.094 DEBUG:teuthology.orchestra.run.smithi141:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 , 2020-12-30T08:09:17.216 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi141.front.sepia.ceph.com 2020-12-30T22:50:10.591 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:10:29.166 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog , 2020-12-30T08:10:29.465 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... , 2020-12-30T08:10:29.467 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2020-12-30T23:00:01.637 WARNING:scrape:Misunderstood line: 2020-12-30T23:00:03.204 WARNING:scrape:Misunderstood line: 2020-12-30T23:00:04.247 WARNING:scrape:Misunderstood line: 2020-12-30T23:00:08.527 WARNING:scrape:Misunderstood line: 2020-12-30T23:00:21.661 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f3c76f94816] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x5567855d5fe6] 2020-12-30T23:00:21.663 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:01.661 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument , 2020-12-30T08:52:01.661 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. , 2020-12-30T08:52:01.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-12-30T23:00:21.664 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:34.781+0000 7fc090962700 -1 osd.7 126 handle_osd_map injecting map crc failure , 2020-12-30T08:52:35.200 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 , set osd.1 primary-affinity to 1 (8655362) 2020-12-30T23:00:21.666 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:46.335 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 , instructed osd(s) 2 to deep-scrub , 2020-12-30T08:52:46.648 INFO:tasks.ceph:Scrubbing osd.3 2020-12-30T23:00:21.667 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 479, in wait , proc.wait() , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 161, in wait 2020-12-30T23:00:21.667 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:53:36.921 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/crash to /home/teuthworker/archive/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747514/remote/smithi079/crash , 2020-12-30T08:53:36.923 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar cz -f - -C /var/lib/ceph/crash -- . , 2020-12-30T08:53:36.949 INFO:tasks.ceph:Compressing logs... 2020-12-30T23:00:21.668 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:54:04.114 DEBUG:teuthology.misc:Transferring archived files from smithi098:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747514/remote/smithi098 , 2020-12-30T08:54:04.114 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . , 2020-12-30T08:54:20.886 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2020-12-30T23:00:21.751 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f3c76f94816] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x5567855d5fe6] 2020-12-30T23:00:21.752 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:01.661 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument , 2020-12-30T08:52:01.661 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. , 2020-12-30T08:52:01.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-12-30T23:00:21.754 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:34.781+0000 7fc090962700 -1 osd.7 126 handle_osd_map injecting map crc failure , 2020-12-30T08:52:35.200 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 , set osd.1 primary-affinity to 1 (8655362) 2020-12-30T23:00:21.755 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:46.335 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 , instructed osd(s) 2 to deep-scrub , 2020-12-30T08:52:46.648 INFO:tasks.ceph:Scrubbing osd.3 2020-12-30T23:00:21.756 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 479, in wait , proc.wait() , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 161, in wait 2020-12-30T23:00:21.757 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:53:36.921 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/crash to /home/teuthworker/archive/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747514/remote/smithi079/crash , 2020-12-30T08:53:36.923 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar cz -f - -C /var/lib/ceph/crash -- . , 2020-12-30T08:53:36.949 INFO:tasks.ceph:Compressing logs... 2020-12-30T23:00:21.757 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:54:04.114 DEBUG:teuthology.misc:Transferring archived files from smithi098:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747514/remote/smithi098 , 2020-12-30T08:54:04.114 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . , 2020-12-30T08:54:20.886 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2020-12-30T23:00:21.836 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f3c76f94816] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x5567855d5fe6] 2020-12-30T23:00:21.837 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:01.661 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument , 2020-12-30T08:52:01.661 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. , 2020-12-30T08:52:01.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-12-30T23:00:21.838 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:34.781+0000 7fc090962700 -1 osd.7 126 handle_osd_map injecting map crc failure , 2020-12-30T08:52:35.200 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 , set osd.1 primary-affinity to 1 (8655362) 2020-12-30T23:00:21.840 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:46.335 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 , instructed osd(s) 2 to deep-scrub , 2020-12-30T08:52:46.648 INFO:tasks.ceph:Scrubbing osd.3 2020-12-30T23:00:21.840 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 479, in wait , proc.wait() , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 161, in wait 2020-12-30T23:00:21.841 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:53:36.921 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/crash to /home/teuthworker/archive/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747514/remote/smithi079/crash , 2020-12-30T08:53:36.923 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar cz -f - -C /var/lib/ceph/crash -- . , 2020-12-30T08:53:36.949 INFO:tasks.ceph:Compressing logs... 2020-12-30T23:00:21.842 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:54:04.114 DEBUG:teuthology.misc:Transferring archived files from smithi098:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747514/remote/smithi098 , 2020-12-30T08:54:04.114 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . , 2020-12-30T08:54:20.886 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2020-12-30T23:00:21.919 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f3c76f94816] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x5567855d5fe6] 2020-12-30T23:00:21.920 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:01.661 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument , 2020-12-30T08:52:01.661 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. , 2020-12-30T08:52:01.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-12-30T23:00:21.922 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:34.781+0000 7fc090962700 -1 osd.7 126 handle_osd_map injecting map crc failure , 2020-12-30T08:52:35.200 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 , set osd.1 primary-affinity to 1 (8655362) 2020-12-30T23:00:21.923 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:46.335 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 , instructed osd(s) 2 to deep-scrub , 2020-12-30T08:52:46.648 INFO:tasks.ceph:Scrubbing osd.3 2020-12-30T23:00:21.924 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 479, in wait , proc.wait() , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 161, in wait 2020-12-30T23:00:21.925 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:53:36.921 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/crash to /home/teuthworker/archive/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747514/remote/smithi079/crash , 2020-12-30T08:53:36.923 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar cz -f - -C /var/lib/ceph/crash -- . , 2020-12-30T08:53:36.949 INFO:tasks.ceph:Compressing logs... 2020-12-30T23:00:21.925 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:54:04.114 DEBUG:teuthology.misc:Transferring archived files from smithi098:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747514/remote/smithi098 , 2020-12-30T08:54:04.114 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . , 2020-12-30T08:54:20.886 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2020-12-30T23:00:22.020 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f3c76f94816] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x5567855d5fe6] 2020-12-30T23:00:22.021 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:01.661 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument , 2020-12-30T08:52:01.661 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. , 2020-12-30T08:52:01.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-12-30T23:00:22.023 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:34.781+0000 7fc090962700 -1 osd.7 126 handle_osd_map injecting map crc failure , 2020-12-30T08:52:35.200 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 , set osd.1 primary-affinity to 1 (8655362) 2020-12-30T23:00:22.024 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:46.335 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 , instructed osd(s) 2 to deep-scrub , 2020-12-30T08:52:46.648 INFO:tasks.ceph:Scrubbing osd.3 2020-12-30T23:00:22.025 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 479, in wait , proc.wait() , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 161, in wait 2020-12-30T23:00:22.026 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:53:36.921 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/crash to /home/teuthworker/archive/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747514/remote/smithi079/crash , 2020-12-30T08:53:36.923 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar cz -f - -C /var/lib/ceph/crash -- . , 2020-12-30T08:53:36.949 INFO:tasks.ceph:Compressing logs... 2020-12-30T23:00:22.026 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:54:04.114 DEBUG:teuthology.misc:Transferring archived files from smithi098:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747514/remote/smithi098 , 2020-12-30T08:54:04.114 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . , 2020-12-30T08:54:20.886 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2020-12-30T23:00:22.109 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f3c76f94816] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x5567855d5fe6] 2020-12-30T23:00:22.111 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:01.661 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument , 2020-12-30T08:52:01.661 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. , 2020-12-30T08:52:01.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-12-30T23:00:22.112 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:34.781+0000 7fc090962700 -1 osd.7 126 handle_osd_map injecting map crc failure , 2020-12-30T08:52:35.200 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 , set osd.1 primary-affinity to 1 (8655362) 2020-12-30T23:00:22.114 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:46.335 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 , instructed osd(s) 2 to deep-scrub , 2020-12-30T08:52:46.648 INFO:tasks.ceph:Scrubbing osd.3 2020-12-30T23:00:22.114 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 479, in wait , proc.wait() , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 161, in wait 2020-12-30T23:00:22.115 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:53:36.921 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/crash to /home/teuthworker/archive/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747514/remote/smithi079/crash , 2020-12-30T08:53:36.923 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar cz -f - -C /var/lib/ceph/crash -- . , 2020-12-30T08:53:36.949 INFO:tasks.ceph:Compressing logs... 2020-12-30T23:00:22.116 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:54:04.114 DEBUG:teuthology.misc:Transferring archived files from smithi098:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747514/remote/smithi098 , 2020-12-30T08:54:04.114 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . , 2020-12-30T08:54:20.886 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2020-12-30T23:00:22.198 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f3c76f94816] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x5567855d5fe6] 2020-12-30T23:00:22.199 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:01.661 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument , 2020-12-30T08:52:01.661 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. , 2020-12-30T08:52:01.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-12-30T23:00:22.201 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:34.781+0000 7fc090962700 -1 osd.7 126 handle_osd_map injecting map crc failure , 2020-12-30T08:52:35.200 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 , set osd.1 primary-affinity to 1 (8655362) 2020-12-30T23:00:22.202 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:46.335 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 , instructed osd(s) 2 to deep-scrub , 2020-12-30T08:52:46.648 INFO:tasks.ceph:Scrubbing osd.3 2020-12-30T23:00:22.203 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 479, in wait , proc.wait() , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 161, in wait 2020-12-30T23:00:22.203 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:53:36.921 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/crash to /home/teuthworker/archive/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747514/remote/smithi079/crash , 2020-12-30T08:53:36.923 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar cz -f - -C /var/lib/ceph/crash -- . , 2020-12-30T08:53:36.949 INFO:tasks.ceph:Compressing logs... 2020-12-30T23:00:22.204 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:54:04.114 DEBUG:teuthology.misc:Transferring archived files from smithi098:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747514/remote/smithi098 , 2020-12-30T08:54:04.114 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . , 2020-12-30T08:54:20.886 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2020-12-30T23:00:22.285 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f3c76f94816] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x5567855d5fe6] 2020-12-30T23:00:22.287 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:01.661 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument , 2020-12-30T08:52:01.661 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. , 2020-12-30T08:52:01.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-12-30T23:00:22.288 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:34.781+0000 7fc090962700 -1 osd.7 126 handle_osd_map injecting map crc failure , 2020-12-30T08:52:35.200 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 , set osd.1 primary-affinity to 1 (8655362) 2020-12-30T23:00:22.290 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:46.335 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 , instructed osd(s) 2 to deep-scrub , 2020-12-30T08:52:46.648 INFO:tasks.ceph:Scrubbing osd.3 2020-12-30T23:00:22.290 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 479, in wait , proc.wait() , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 161, in wait 2020-12-30T23:00:22.291 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:53:36.921 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/crash to /home/teuthworker/archive/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747514/remote/smithi079/crash , 2020-12-30T08:53:36.923 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar cz -f - -C /var/lib/ceph/crash -- . , 2020-12-30T08:53:36.949 INFO:tasks.ceph:Compressing logs... 2020-12-30T23:00:22.291 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:54:04.114 DEBUG:teuthology.misc:Transferring archived files from smithi098:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747514/remote/smithi098 , 2020-12-30T08:54:04.114 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . , 2020-12-30T08:54:20.886 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2020-12-30T23:00:22.368 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f3c76f94816] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x5567855d5fe6] 2020-12-30T23:00:22.369 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:01.661 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument , 2020-12-30T08:52:01.661 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. , 2020-12-30T08:52:01.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-12-30T23:00:22.371 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:34.781+0000 7fc090962700 -1 osd.7 126 handle_osd_map injecting map crc failure , 2020-12-30T08:52:35.200 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 , set osd.1 primary-affinity to 1 (8655362) 2020-12-30T23:00:22.372 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:52:46.335 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 , instructed osd(s) 2 to deep-scrub , 2020-12-30T08:52:46.648 INFO:tasks.ceph:Scrubbing osd.3 2020-12-30T23:00:22.373 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 479, in wait , proc.wait() , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 161, in wait 2020-12-30T23:00:22.373 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:53:36.921 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/crash to /home/teuthworker/archive/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747514/remote/smithi079/crash , 2020-12-30T08:53:36.923 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar cz -f - -C /var/lib/ceph/crash -- . , 2020-12-30T08:53:36.949 INFO:tasks.ceph:Compressing logs... 2020-12-30T23:00:22.374 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:54:04.114 DEBUG:teuthology.misc:Transferring archived files from smithi098:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747514/remote/smithi098 , 2020-12-30T08:54:04.114 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . , 2020-12-30T08:54:20.886 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2020-12-30T23:00:23.429 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f69dd512816] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x55d303af9fe6] 2020-12-30T23:00:23.430 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.563908+0000", , 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.00019472900000000001 , 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.431 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.187571661, , 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "type_data": { , 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "delayed", 2020-12-30T23:00:23.431 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.107 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", , 2020-12-30T08:57:52.108 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.759798+0000", , 2020-12-30T08:57:52.108 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 5.1649e-05 2020-12-30T23:00:23.432 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "initiated", , 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.611817+0000", 2020-12-30T23:00:23.432 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "started", , 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.867908+0000", 2020-12-30T23:00:23.433 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "sub_op_commit_rec", , 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.879692+0000", 2020-12-30T23:00:23.434 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.189 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.190 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.190 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.434 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", 2020-12-30T23:00:23.435 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", 2020-12-30T23:00:23.435 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "reached pg", , 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: "events": [ , 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.436 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.272 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.002481568 , 2020-12-30T08:57:52.272 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.273 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.436 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.292 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 8.7658000000000007e-05 , 2020-12-30T08:57:52.293 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.293 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.437 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.87972017899999999 , 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2020-12-30T23:00:23.437 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "type_data": { , 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "started", , 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "events": [ 2020-12-30T23:00:23.438 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:28.437546+0000", , 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 6.2249999999999997e-06 , 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.439 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 600, , 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: "ops": [ , 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.439 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", , 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.755828+0000", 2020-12-30T23:00:23.440 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "header_read", 2020-12-30T23:00:23.440 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: } 2020-12-30T23:00:23.441 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.820 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4.3782000000000002e-05 , 2020-12-30T08:57:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.442 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:10.493947+0000", , 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4294967295.9999809 , 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.442 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.310989+0000", , 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.012873951999999999 , 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: } 2020-12-30T23:00:23.443 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "queued_for_pg", , 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:17.586655+0000", , 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4.7491000000000003e-05 2020-12-30T23:00:23.443 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", , 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:33.372282+0000", 2020-12-30T23:00:23.444 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:44.022360+0000", , 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "age": 8.7134268660000007, , 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.346765198, 2020-12-30T23:00:23.445 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "started", 2020-12-30T23:00:23.645 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.963 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.0046062250000000002 , 2020-12-30T08:57:52.963 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.964 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.646 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.983 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.984 INFO:teuthology.orchestra.run.smithi169.stdout:} , 2020-12-30T08:57:52.985 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-12-30T23:00:23.646 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", 2020-12-30T23:00:23.647 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.403 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0 , 2020-12-30T08:57:53.404 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.404 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.648 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: "description": "osd_op(client.4598.0:1164 4.0 4:249a18ca:::smithi04340772-260:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e34)", , 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:04.096815+0000", 2020-12-30T23:00:23.648 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:09.599534+0000", , 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.036992197999999997 , 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.649 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.470 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", , 2020-12-30T08:57:53.470 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.144872+0000", , 2020-12-30T08:57:53.471 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 1.0566e-05 2020-12-30T23:00:23.649 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "initiated", , 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.156798+0000", 2020-12-30T23:00:23.650 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.513 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:53.513 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:53.514 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.651 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.535 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.535 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.536 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "queued_for_pg", 2020-12-30T23:00:23.651 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4294967295.9999795 , 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.652 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: "description": "osd_op(client.4598.0:2898 4.0 4:1f210125:::smithi04340772-284:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e38)", , 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:17.505116+0000", 2020-12-30T23:00:23.652 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.600 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:22.235621+0000", , 2020-12-30T08:57:53.600 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.035873975000000002 , 2020-12-30T08:57:53.601 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.653 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3320 , 2020-12-30T08:57:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.027 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2020-12-30T23:00:23.653 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:23.886767+0000", , 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8839999999999997e-06 , 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:23.654 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:32.913785+0000", , 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049845687999999999 , 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:23.655 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", , 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:27.332932+0000", , 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2020-12-30T23:00:23.655 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", , 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:27.333311+0000", , 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1931999999999999e-05 2020-12-30T23:00:23.656 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", , 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:28.423645+0000", 2020-12-30T23:00:23.656 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for readable", , 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:28.440591+0000", 2020-12-30T23:00:23.657 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { , 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4598", , 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "v1:172.21.15.43:0/972417042", 2020-12-30T23:00:23.658 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2020-12-30T23:00:23.658 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 , 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2020-12-30T23:00:23.659 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { , 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", , 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2020-12-30T23:00:23.659 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.249599+0000", , 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4746000000000001e-05 , 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:23.660 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.636403+0000", , 2020-12-30T08:57:54.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0044951799999999997 , 2020-12-30T08:57:54.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:23.661 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.655240+0000", , 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047171491000000003 , 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2020-12-30T23:00:23.662 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:58:15.685 INFO:tasks.ceph.osd.0:Stopped , 2020-12-30T08:58:15.685 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 , osd.0 is already down. 2020-12-30T23:00:23.663 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:59:26.097 INFO:teuthology.orchestra.run.smithi043.stdout:mon.a: {} , 2020-12-30T08:59:26.113 INFO:teuthology.orchestra.run.smithi043.stdout:mon.b: {} , 2020-12-30T08:59:26.127 INFO:teuthology.orchestra.run.smithi043.stdout:mon.c: {} 2020-12-30T23:00:23.665 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:59:48.212 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2020-12-30T08:59:47.527120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1407295001,"num_objects":477,"num_object_clones":0,"num_object_copies":1431,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":449,"num_whiteouts":3,"num_read":3286,"num_read_kb":2359369,"num_write":3461,"num_write_kb":1206573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":773,"num_bytes_recovered":1169499301,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":887,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":44,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2967,"ondisk_log_size":2967,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2702316,"kb_used_data":2595628,"kb_used_omap":11,"kb_used_meta":105780,"kb_avail":752272404,"statfs":{"total":773094113280,"available":770326941696,"internally_reserved":0,"allocated":2657923072,"data_stored":2643000988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11269,"internal_metadata":108319739},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.529700"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":"111","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252322+0000","last_change":"2020-12-30T08:59:26.805818+0000","last_active":"2020-12-30T08:59:40.252322+0000","last_peered":"2020-12-30T08:59:40.252322+0000","last_clean":"2020-12-30T08:59:40.252322+0000","last_became_active":"2020-12-30T08:59:26.805280+0000","last_became_peered":"2020-12-30T08:59:26.805280+0000","last_unstale":"2020-12-30T08:59:40.252322+0000","last_undegraded":"2020-12-30T08:59:40.252322+0000","last_fullsized":"2020-12-30T08:59:40.252322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:04.206849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:04.206849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"57'44","reported_seq":"156","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252340+0000","last_change":"2020-12-30T08:59:09.166665+0000","last_active":"2020-12-30T08:59:40.252340+0000","last_peered":"2020-12-30T08:59:40.252340+0000","last_clean":"2020-12-30T08:59:40.252340+0000","last_became_active":"2020-12-30T08:56:39.905257+0000","last_became_peered":"2020-12-30T08:56:39.905257+0000","last_unstale":"2020-12-30T08:59:40.252340+0000","last_undegraded":"2020-12-30T08:59:40.252340+0000","last_fullsized":"2020-12-30T08:59:40.252340+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'44","last_scrub_stamp":"2020-12-30T08:59:09.166549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:09.166549+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63684286,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":42,"num_read_kb":51083,"num_write":29,"num_write_kb":4590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":319,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":25,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.578479+0000","last_change":"2020-12-30T08:58:46.630261+0000","last_active":"2020-12-30T08:59:40.578479+0000","last_peered":"2020-12-30T08:59:40.578479+0000","last_clean":"2020-12-30T08:59:40.578479+0000","last_became_active":"2020-12-30T08:56:17.453297+0000","last_became_peered":"2020-12-30T08:56:17.453297+0000","last_unstale":"2020-12-30T08:59:40.578479+0000","last_undegraded":"2020-12-30T08:59:40.578479+0000","last_fullsized":"2020-12-30T08:59:40.578479+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:46.630148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:46.630148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"57'794","reported_seq":"1885","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.596274+0000","last_change":"2020-12-30T08:59:31.962315+0000","last_active":"2020-12-30T08:59:41.596274+0000","last_peered":"2020-12-30T08:59:41.596274+0000","last_clean":"2020-12-30T08:59:41.596274+0000","last_became_active":"2020-12-30T08:59:31.961183+0000","last_became_peered":"2020-12-30T08:59:31.961183+0000","last_unstale":"2020-12-30T08:59:41.596274+0000","last_undegraded":"2020-12-30T08:59:41.596274+0000","last_fullsized":"2020-12-30T08:59:41.596274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'794","last_scrub_stamp":"2020-12-30T08:59:09.405174+0000","last_deep_scrub":"54'709","last_deep_scrub_stamp":"2020-12-30T08:57:41.784481+0000","last_clean_scrub_stamp":"2020-12-30T08:59:09.405174+0000","log_size":794,"ondisk_log_size":794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":324600875,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":956,"num_read_kb":765811,"num_write":895,"num_write_kb":309669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":381323268,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"125","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.432965+0000","last_change":"2020-12-30T08:59:26.807827+0000","last_active":"2020-12-30T08:59:40.432965+0000","last_peered":"2020-12-30T08:59:40.432965+0000","last_clean":"2020-12-30T08:59:40.432965+0000","last_became_active":"2020-12-30T08:59:26.807377+0000","last_became_peered":"2020-12-30T08:59:26.807377+0000","last_unstale":"2020-12-30T08:59:40.432965+0000","last_undegraded":"2020-12-30T08:59:40.432965+0000","last_fullsized":"2020-12-30T08:59:40.432965+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.818206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.818206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"57'598","reported_seq":"1017","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252337+0000","last_change":"2020-12-30T08:59:08.196312+0000","last_active":"2020-12-30T08:59:40.252337+0000","last_peered":"2020-12-30T08:59:40.252337+0000","last_clean":"2020-12-30T08:59:40.252337+0000","last_became_active":"2020-12-30T08:56:35.883250+0000","last_became_peered":"2020-12-30T08:56:35.883250+0000","last_unstale":"2020-12-30T08:59:40.252337+0000","last_undegraded":"2020-12-30T08:59:40.252337+0000","last_fullsized":"2020-12-30T08:59:40.252337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'598","last_scrub_stamp":"2020-12-30T08:59:08.196199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:34.853614+0000","last_clean_scrub_stamp":"2020-12-30T08:59:08.196199+0000","log_size":598,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":299758950,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":689,"num_read_kb":446464,"num_write":755,"num_write_kb":264543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.578494+0000","last_change":"2020-12-30T08:58:53.577047+0000","last_active":"2020-12-30T08:59:40.578494+0000","last_peered":"2020-12-30T08:59:40.578494+0000","last_clean":"2020-12-30T08:59:40.578494+0000","last_became_active":"2020-12-30T08:56:18.363833+0000","last_became_peered":"2020-12-30T08:56:18.363833+0000","last_unstale":"2020-12-30T08:59:40.578494+0000","last_undegraded":"2020-12-30T08:59:40.578494+0000","last_fullsized":"2020-12-30T08:59:40.578494+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:53.576887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:53.576887+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":"145","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001415+0000","last_change":"2020-12-30T08:59:29.950684+0000","last_active":"2020-12-30T08:59:41.001415+0000","last_peered":"2020-12-30T08:59:41.001415+0000","last_clean":"2020-12-30T08:59:41.001415+0000","last_became_active":"2020-12-30T08:59:29.950219+0000","last_became_peered":"2020-12-30T08:59:29.950219+0000","last_unstale":"2020-12-30T08:59:41.001415+0000","last_undegraded":"2020-12-30T08:59:41.001415+0000","last_fullsized":"2020-12-30T08:59:41.001415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":4,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'2","last_scrub_stamp":"2020-12-30T08:58:43.023209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:05.570594+0000","last_clean_scrub_stamp":"2020-12-30T08:58:43.023209+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"57'737","reported_seq":"1556","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.577945+0000","last_change":"2020-12-30T08:59:27.273198+0000","last_active":"2020-12-30T08:59:40.577945+0000","last_peered":"2020-12-30T08:59:40.577945+0000","last_clean":"2020-12-30T08:59:40.577945+0000","last_became_active":"2020-12-30T08:57:34.885212+0000","last_became_peered":"2020-12-30T08:57:34.885212+0000","last_unstale":"2020-12-30T08:59:40.577945+0000","last_undegraded":"2020-12-30T08:59:40.577945+0000","last_fullsized":"2020-12-30T08:59:40.577945+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'737","last_scrub_stamp":"2020-12-30T08:59:27.273101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:34.853614+0000","last_clean_scrub_stamp":"2020-12-30T08:59:27.273101+0000","log_size":737,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":370415644,"num_objects":124,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":0,"num_read":829,"num_read_kb":576628,"num_write":921,"num_write_kb":353530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":210025962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"112","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001472+0000","last_change":"2020-12-30T08:59:26.807466+0000","last_active":"2020-12-30T08:59:41.001472+0000","last_peered":"2020-12-30T08:59:41.001472+0000","last_clean":"2020-12-30T08:59:41.001472+0000","last_became_active":"2020-12-30T08:59:26.806602+0000","last_became_peered":"2020-12-30T08:59:26.806602+0000","last_unstale":"2020-12-30T08:59:41.001472+0000","last_undegraded":"2020-12-30T08:59:41.001472+0000","last_fullsized":"2020-12-30T08:59:41.001472+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:06.756813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:06.756813+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"76'751","reported_seq":"1682","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.000117+0000","last_change":"2020-12-30T08:59:30.952423+0000","last_active":"2020-12-30T08:59:41.000117+0000","last_peered":"2020-12-30T08:59:41.000117+0000","last_clean":"2020-12-30T08:59:41.000117+0000","last_became_active":"2020-12-30T08:59:30.951003+0000","last_became_peered":"2020-12-30T08:59:30.951003+0000","last_unstale":"2020-12-30T08:59:41.000117+0000","last_undegraded":"2020-12-30T08:59:41.000117+0000","last_fullsized":"2020-12-30T08:59:41.000117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'751","last_scrub_stamp":"2020-12-30T08:59:17.342114+0000","last_deep_scrub":"76'751","last_deep_scrub_stamp":"2020-12-30T08:59:17.342114+0000","last_clean_scrub_stamp":"2020-12-30T08:59:17.342114+0000","log_size":751,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":288680016,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":742,"num_read_kb":486335,"num_write":828,"num_write_kb":269542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":495679005,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"57'31","reported_seq":"176","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.000230+0000","last_change":"2020-12-30T08:59:30.953648+0000","last_active":"2020-12-30T08:59:41.000230+0000","last_peered":"2020-12-30T08:59:41.000230+0000","last_clean":"2020-12-30T08:59:41.000230+0000","last_became_active":"2020-12-30T08:59:30.952821+0000","last_became_peered":"2020-12-30T08:59:30.952821+0000","last_unstale":"2020-12-30T08:59:41.000230+0000","last_undegraded":"2020-12-30T08:59:41.000230+0000","last_fullsized":"2020-12-30T08:59:41.000230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'31","last_scrub_stamp":"2020-12-30T08:59:18.654169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:18.654169+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41801084,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":14,"num_read_kb":4410,"num_write":24,"num_write_kb":3616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":82471066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":293,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":14,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"115","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001437+0000","last_change":"2020-12-30T08:59:26.807130+0000","last_active":"2020-12-30T08:59:41.001437+0000","last_peered":"2020-12-30T08:59:41.001437+0000","last_clean":"2020-12-30T08:59:41.001437+0000","last_became_active":"2020-12-30T08:59:26.806409+0000","last_became_peered":"2020-12-30T08:59:26.806409+0000","last_unstale":"2020-12-30T08:59:41.001437+0000","last_undegraded":"2020-12-30T08:59:41.001437+0000","last_fullsized":"2020-12-30T08:59:41.001437+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.095702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.095702+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252325+0000","last_change":"2020-12-30T08:58:49.511387+0000","last_active":"2020-12-30T08:59:40.252325+0000","last_peered":"2020-12-30T08:59:40.252325+0000","last_clean":"2020-12-30T08:59:40.252325+0000","last_became_active":"2020-12-30T08:56:18.101527+0000","last_became_peered":"2020-12-30T08:56:18.101527+0000","last_unstale":"2020-12-30T08:59:40.252325+0000","last_undegraded":"2020-12-30T08:59:40.252325+0000","last_fullsized":"2020-12-30T08:59:40.252325+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:49.511270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:49.511270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"99","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.577925+0000","last_change":"2020-12-30T08:59:26.808227+0000","last_active":"2020-12-30T08:59:40.577925+0000","last_peered":"2020-12-30T08:59:40.577925+0000","last_clean":"2020-12-30T08:59:40.577925+0000","last_became_active":"2020-12-30T08:59:26.807467+0000","last_became_peered":"2020-12-30T08:59:26.807467+0000","last_unstale":"2020-12-30T08:59:40.577925+0000","last_undegraded":"2020-12-30T08:59:40.577925+0000","last_fullsized":"2020-12-30T08:59:40.577925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.951459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.951459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"92","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.310721+0000","last_change":"2020-12-30T08:58:34.893621+0000","last_active":"2020-12-30T08:59:40.310721+0000","last_peered":"2020-12-30T08:59:40.310721+0000","last_clean":"2020-12-30T08:59:40.310721+0000","last_became_active":"2020-12-30T08:56:17.453289+0000","last_became_peered":"2020-12-30T08:56:17.453289+0000","last_unstale":"2020-12-30T08:59:40.310721+0000","last_undegraded":"2020-12-30T08:59:40.310721+0000","last_fullsized":"2020-12-30T08:59:40.310721+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:34.893489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:34.893489+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"57'10","reported_seq":"102","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.596156+0000","last_change":"2020-12-30T08:58:50.189729+0000","last_active":"2020-12-30T08:59:41.596156+0000","last_peered":"2020-12-30T08:59:41.596156+0000","last_clean":"2020-12-30T08:59:41.596156+0000","last_became_active":"2020-12-30T08:56:40.286191+0000","last_became_peered":"2020-12-30T08:56:40.286191+0000","last_unstale":"2020-12-30T08:59:41.596156+0000","last_undegraded":"2020-12-30T08:59:41.596156+0000","last_fullsized":"2020-12-30T08:59:41.596156+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'10","last_scrub_stamp":"2020-12-30T08:58:50.189625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:58:50.189625+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18354146,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":28638,"num_write":7,"num_write_kb":1074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":275,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":5,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":4,"stat_sum":{"num_bytes":123839516,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":3,"num_read":70,"num_read_kb":84131,"num_write":60,"num_write_kb":9280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":82471066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":887,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":44,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":363405312,"data_stored":363060657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":12,"acting":12,"num_store_stats":7},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":1283455485,"num_objects":428,"num_object_clones":0,"num_object_copies":1284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":428,"num_whiteouts":0,"num_read":3216,"num_read_kb":2275238,"num_write":3399,"num_write_kb":1197284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":739,"num_bytes_recovered":1087028235,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2289242112,"data_stored":2277676566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2880,"ondisk_log_size":2880,"up":12,"acting":12,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10297,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46828,"kb_used_data":40940,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":94325012,"statfs":{"total":96636764160,"available":96588812288,"internally_reserved":0,"allocated":41922560,"data_stored":41513062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:57:16 2020","interfaces":[{"interface":"back","average":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":81.242000000000004,"5min":81.242000000000004,"15min":81.242000000000004},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1.0409999999999999}]},{"osd":1,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":19.870000000000001,"15min":28.206},"min":{"1min":0.83199999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4970000000000001,"5min":501.495,"15min":501.495},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":26.629000000000001,"15min":25.437000000000001},"min":{"1min":0.73399999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.5720000000000001,"5min":958.63099999999997,"15min":958.63099999999997},"last":1.2589999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":59.027999999999999,"5min":12.720000000000001,"15min":5.008},"min":{"1min":0.85799999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":927.80799999999999,"5min":927.80799999999999,"15min":927.80799999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":21.974,"5min":12.337999999999999,"15min":4.8849999999999998},"min":{"1min":0.98799999999999999,"5min":0.753,"15min":0.753},"max":{"1min":333.58300000000003,"5min":597.27800000000002,"15min":597.27800000000002},"last":1.093}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.202,"15min":1.216},"min":{"1min":0.81399999999999995,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.45,"5min":1.474,"15min":1.474},"last":1.2310000000000001},{"interface":"front","average":{"1min":8.3640000000000008,"5min":2.5859999999999999,"15min":1.623},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":123.247,"5min":123.247,"15min":123.247},"last":0.84299999999999997}]},{"osd":4,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":26.962,"15min":38.468000000000004},"min":{"1min":0.81499999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5609999999999999,"5min":747.37,"15min":747.37},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":1.052,"15min":1.0920000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.486,"5min":1.623,"15min":1.623},"last":1.1080000000000001}]},{"osd":5,"last update":"Wed Dec 30 08:59:24 2020","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1379999999999999,"15min":1.125},"min":{"1min":0.76500000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.6160000000000001,"5min":1.7,"15min":1.7},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.1879999999999999,"15min":1.2390000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5469999999999999,"5min":2.198,"15min":2.198},"last":1.198}]},{"osd":6,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":8.0190000000000001,"5min":27.074000000000002,"15min":37.076000000000001},"min":{"1min":0.77300000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":106.654,"5min":861.60799999999995,"15min":861.60799999999995},"last":1.177},{"interface":"front","average":{"1min":52.908999999999999,"5min":11.640000000000001,"15min":4.7830000000000004},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":778.01900000000001,"5min":778.01900000000001,"15min":778.01900000000001},"last":1.3280000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617820,"kb_used_data":597728,"kb_used_omap":0,"kb_used_meta":19968,"kb_avail":93754020,"statfs":{"total":96636764160,"available":96004116480,"internally_reserved":0,"allocated":612073472,"data_stored":608728491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20447232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:15 2020","interfaces":[{"interface":"back","average":{"1min":63.369,"5min":56.469000000000001,"15min":55.319000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":999.76300000000003,"5min":999.76300000000003,"15min":999.76300000000003},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.04,"15min":1.028},"min":{"1min":0.746,"5min":0.62,"15min":0.62},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":1.3069999999999999}]},{"osd":1,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.01,"15min":0.97699999999999998},"min":{"1min":0.79400000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.3480000000000001,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.875},{"interface":"front","average":{"1min":0.96799999999999997,"5min":36.709000000000003,"15min":52.585000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.3009999999999999,"5min":953.40800000000002,"15min":953.40800000000002},"last":1.1899999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":46.109999999999999,"5min":10.127000000000001,"15min":4.109},"min":{"1min":0.98899999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":714.93299999999999,"5min":714.93299999999999,"15min":714.93299999999999},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.206,"15min":1.1990000000000001},"min":{"1min":1.0309999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.4870000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.6060000000000001}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.151,"5min":1.139,"15min":1.137},"min":{"1min":0.94599999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.3919999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.274}]},{"osd":4,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.96199999999999997,"15min":0.97899999999999998},"min":{"1min":0.63600000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.393,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0740000000000001,"15min":1.0860000000000001},"min":{"1min":0.81299999999999994,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":1.375}]},{"osd":5,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.1020000000000001,"15min":1.123},"min":{"1min":0.77300000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.286,"5min":1.637,"15min":1.637},"last":1.3540000000000001},{"interface":"front","average":{"1min":33.957000000000001,"5min":47.049999999999997,"15min":60.192999999999998},"min":{"1min":0.81299999999999994,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":691.76199999999994,"5min":985.03399999999999,"15min":985.03399999999999},"last":1.3080000000000001}]},{"osd":7,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":21.876000000000001,"5min":5.3460000000000001,"15min":2.5979999999999999},"min":{"1min":0.86099999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":435.39499999999998,"5min":435.39499999999998,"15min":435.39499999999998},"last":1.331},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2330000000000001,"15min":1.2490000000000001},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.5740000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":247340,"kb_used_data":233900,"kb_used_omap":0,"kb_used_meta":13312,"kb_avail":94124500,"statfs":{"total":96636764160,"available":96383488000,"internally_reserved":0,"allocated":239513600,"data_stored":238202894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13631488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:13 2020","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.0569999999999999,"15min":1.0880000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2869999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77500000000000002,"15min":0.77600000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.74399999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":27.254000000000001,"15min":39.072000000000003},"min":{"1min":0.47199999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84799999999999998,"5min":882.37400000000002,"15min":882.37400000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.90400000000000003,"15min":0.92200000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.298,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed Dec 30 08:59:31 2020","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.95299999999999996,"15min":0.95699999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.29,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.86899999999999999,"15min":0.88200000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.302,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.80500000000000005}]},{"osd":4,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1339999999999999,"15min":1.1040000000000001},"min":{"1min":0.92200000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.3899999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.1599999999999999,"15min":1.169},"min":{"1min":0.73299999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.46,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.84699999999999998}]},{"osd":5,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2649999999999999,"15min":1.2829999999999999},"min":{"1min":0.89100000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.4159999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2390000000000001,"15min":1.2470000000000001},"min":{"1min":1.002,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.383,"5min":1.96,"15min":1.96},"last":1.0229999999999999}]},{"osd":6,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2390000000000001,"15min":1.2749999999999999},"min":{"1min":0.82299999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4259999999999999,"5min":4.4829999999999997,"15min":4.4829999999999997},"last":1.149},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.242,"15min":1.272},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.4159999999999999,"5min":1.734,"15min":1.734},"last":1.0069999999999999}]},{"osd":7,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":19.978999999999999,"15min":28.309000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.486,"5min":623.82100000000003,"15min":623.82100000000003},"last":828.46799999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.2250000000000001,"15min":1.2350000000000001},"min":{"1min":0.91700000000000004,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.3500000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.98699999999999999}]}]},{"osd":0,"up_from":75,"seq":322122547208,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410660,"kb_used_data":392612,"kb_used_omap":10,"kb_used_meta":17909,"kb_avail":93961180,"statfs":{"total":96636764160,"available":96216248320,"internally_reserved":0,"allocated":402034688,"data_stored":399673841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10297,"internal_metadata":18339783},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.649}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.679}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":179560,"kb_used_data":172264,"kb_used_omap":0,"kb_used_meta":7168,"kb_avail":94192280,"statfs":{"total":96636764160,"available":96452894720,"internally_reserved":0,"allocated":176398336,"data_stored":175142793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7340032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:15 2020","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.81999999999999995,"15min":0.81399999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.999,"5min":33.204000000000001,"15min":38.572000000000003},"min":{"1min":0.63500000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.8049999999999999,"5min":767.24300000000005,"15min":767.24300000000005},"last":0.67100000000000004}]},{"osd":1,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":30.774000000000001,"5min":7.2549999999999999,"15min":3.4510000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":754.12800000000004,"5min":754.12800000000004,"15min":754.12800000000004},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.91400000000000003,"15min":0.91100000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4430000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.80300000000000005}]},{"osd":3,"last update":"Wed Dec 30 08:59:29 2020","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.0629999999999999,"15min":1.0900000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.476,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.747,"15min":0.747},"min":{"1min":0.55500000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.89100000000000001}]},{"osd":4,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":7.5339999999999998,"15min":3.2320000000000002},"min":{"1min":0.67200000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6200000000000001,"5min":742.45799999999997,"15min":742.45799999999997},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.2110000000000001,"15min":1.2070000000000001},"min":{"1min":0.81699999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5449999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.1970000000000001}]},{"osd":5,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":2.4820000000000002,"15min":1.631},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.464,"5min":152.709,"15min":152.709},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.1919999999999999,"15min":1.214},"min":{"1min":0.80800000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.4039999999999999,"5min":2.181,"15min":2.181},"last":0.96899999999999997}]},{"osd":6,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":4.1260000000000003,"5min":29.439,"15min":41.308},"min":{"1min":0.753,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":73.947000000000003,"5min":874.04999999999995,"15min":874.04999999999995},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.1439999999999999,"15min":1.1819999999999999},"min":{"1min":0.73399999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.508,"5min":4.0439999999999996,"15min":4.0439999999999996},"last":1.0700000000000001}]},{"osd":7,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":20.821999999999999,"5min":21.945,"15min":26.760999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":489.90300000000002,"5min":502.05900000000003,"15min":502.05900000000003},"last":1.214},{"interface":"front","average":{"1min":1.3220000000000001,"5min":23.478000000000002,"15min":33.302999999999997},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":2.0270000000000001,"5min":502.10599999999999,"15min":502.10599999999999},"last":1.2290000000000001}]}]},{"osd":3,"up_from":41,"seq":176093659166,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400584,"kb_used_data":387072,"kb_used_omap":0,"kb_used_meta":13503,"kb_avail":93971256,"statfs":{"total":96636764160,"available":96226566144,"internally_reserved":0,"allocated":396361728,"data_stored":394019686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":972,"internal_metadata":13827124},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":1,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.93000000000000005,"15min":0.94999999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.242,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.84599999999999997,"15min":0.86799999999999999},"min":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90100000000000002,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.85499999999999998}]},{"osd":2,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.83699999999999997,"15min":0.85299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1319999999999999,"5min":1.581,"15min":1.581},"last":0.628},{"interface":"front","average":{"1min":26.884,"5min":6.1639999999999997,"15min":2.71},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":469.85500000000002,"5min":469.85500000000002,"15min":469.85500000000002},"last":0.82099999999999995}]},{"osd":4,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.0760000000000001,"15min":1.0840000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.254,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.23,"15min":1.254},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.4079999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.90800000000000003}]},{"osd":5,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":25.748000000000001,"15min":29.870000000000001},"min":{"1min":0.81799999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.2010000000000001,"5min":524.471,"15min":524.471},"last":0.88},{"interface":"front","average":{"1min":47.161999999999999,"5min":10.489000000000001,"15min":4.3769999999999998},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":830.05200000000002,"5min":830.05200000000002,"15min":830.05200000000002},"last":0.94699999999999995}]},{"osd":6,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.254,"15min":1.258},"min":{"1min":1.046,"5min":1.046,"15min":1.046},"max":{"1min":1.5960000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.181},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.266,"15min":1.268},"min":{"1min":1.071,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.6259999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.204}]},{"osd":7,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.298,"15min":1.3220000000000001},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.3009999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.089},{"interface":"front","average":{"1min":1.159,"5min":1.264,"15min":1.282},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.1599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":444756,"kb_used_data":430292,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":93927084,"statfs":{"total":96636764160,"available":96181334016,"internally_reserved":0,"allocated":440619008,"data_stored":438222994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:57:13 2020","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.298},{"interface":"front","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.3360000000000001}]},{"osd":1,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.119,"15min":1.1479999999999999},"min":{"1min":0.755,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.4039999999999999,"5min":1.661,"15min":1.661},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.022,"15min":1.012},"min":{"1min":0.78600000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4019999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.361}]},{"osd":2,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.161,"15min":1.1870000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.498,"5min":1.776,"15min":1.776},"last":1.282},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0940000000000001,"15min":1.1040000000000001},"min":{"1min":0.84499999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.446,"5min":1.478,"15min":1.478},"last":1.446}]},{"osd":3,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1000000000000001,"15min":1.0920000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.2050000000000001,"15min":1.2210000000000001},"min":{"1min":0.91400000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.222}]},{"osd":5,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.96399999999999997,"15min":0.94599999999999995},"min":{"1min":0.81399999999999995,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3440000000000001,"5min":1.45,"15min":1.45},"last":0.995},{"interface":"front","average":{"1min":1.014,"5min":0.98999999999999999,"15min":0.94899999999999995},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.226,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.26}]},{"osd":6,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.1160000000000001,"15min":1.153},"min":{"1min":0.73499999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.4950000000000001,"5min":1.71,"15min":1.71},"last":1.4199999999999999},{"interface":"front","average":{"1min":18.32,"5min":4.508,"15min":2.214},"min":{"1min":0.90100000000000002,"5min":0.629,"15min":0.629},"max":{"1min":276.41800000000001,"5min":276.41800000000001,"15min":276.41800000000001},"last":1.0489999999999999}]},{"osd":7,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":3.4380000000000002,"15min":1.8959999999999999},"min":{"1min":1.022,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.385,"5min":181.79499999999999,"15min":181.79499999999999},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":5.6840000000000002,"15min":2.7799999999999998},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":1.4510000000000001,"5min":353.72500000000002,"15min":353.72500000000002},"last":1.476}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":354768,"kb_used_data":340820,"kb_used_omap":0,"kb_used_meta":13824,"kb_avail":94017072,"statfs":{"total":96636764160,"available":96273481728,"internally_reserved":0,"allocated":348999680,"data_stored":347497227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14155776},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:12 2020","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.1439999999999999,"15min":1.1599999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.4239999999999999,"5min":1.526,"15min":1.526},"last":1},{"interface":"front","average":{"1min":42.222000000000001,"5min":9.3520000000000003,"15min":3.8740000000000001},"min":{"1min":0.83799999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":821.10900000000004,"5min":821.10900000000004,"15min":821.10900000000004},"last":0.89200000000000002}]},{"osd":1,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0009999999999999,"15min":0.98499999999999999},"min":{"1min":0.79100000000000004,"5min":0.627,"15min":0.627},"max":{"1min":1.2769999999999999,"5min":1.379,"15min":1.379},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":5.5030000000000001,"15min":2.5710000000000002},"min":{"1min":0.84099999999999997,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.3300000000000001,"5min":440.44999999999999,"15min":440.44999999999999},"last":0.91800000000000004}]},{"osd":2,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":7.0289999999999999,"15min":2.9950000000000001},"min":{"1min":0.752,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.294,"5min":605.79899999999998,"15min":605.79899999999998},"last":0.94599999999999995},{"interface":"front","average":{"1min":23.163,"5min":5.524,"15min":2.5800000000000001},"min":{"1min":0.88200000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":501.64600000000002,"5min":501.64600000000002,"15min":501.64600000000002},"last":1.329}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1419999999999999,"15min":1.1339999999999999},"min":{"1min":0.90600000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.411,"5min":1.494,"15min":1.494},"last":1.282},{"interface":"front","average":{"1min":1.159,"5min":17.323,"15min":20.016999999999999},"min":{"1min":0.92700000000000005,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.393,"5min":404.60300000000001,"15min":404.60300000000001},"last":1.0800000000000001}]},{"osd":4,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":21.57,"15min":22.916},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.4079999999999999,"5min":704.28399999999999,"15min":704.28399999999999},"last":1.113},{"interface":"front","average":{"1min":0.92100000000000004,"5min":46.520000000000003,"15min":66.030000000000001},"min":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.0920000000000001,"5min":861.90599999999995,"15min":861.90599999999995},"last":1.0529999999999999}]},{"osd":6,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0700000000000001,"15min":1.0660000000000001},"min":{"1min":0.66300000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.4299999999999999,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.085,"5min":1.1020000000000001,"15min":1.121},"min":{"1min":0.77700000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4410000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.145}]},{"osd":7,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.284,"15min":1.3919999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3899999999999999,"5min":8.3629999999999995,"15min":8.3629999999999995},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":11.603999999999999,"15min":12.991},"min":{"1min":0.79200000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.6000000000000001,"5min":291.65300000000002,"15min":291.65300000000002},"last":1.3089999999999999}]}]}]}} , 2020-12-30T08:59:48.214 INFO:tasks.ceph.ceph_manager.ceph:recovered! , 2020-12-30T08:59:48.215 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2020-12-30T23:00:23.667 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 33, in nested , yield vars , File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-12-30-1123/qa/tasks/ceph.py", line 1876, in task 2020-12-30T23:00:23.667 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T09:00:18.496 DEBUG:teuthology.orchestra.run.smithi169:> mktemp , 2020-12-30T09:00:18.513 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.C79Qha9RXd , 2020-12-30T09:00:18.514 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.C79Qha9RXd 2020-12-30T23:00:23.668 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T09:00:35.626 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- , 2020-12-30T09:00:35.644 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- , 2020-12-30T09:00:35.706 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2020-12-30T23:00:23.721 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f69dd512816] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x55d303af9fe6] 2020-12-30T23:00:23.722 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.563908+0000", , 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.00019472900000000001 , 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.723 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.187571661, , 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "type_data": { , 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "delayed", 2020-12-30T23:00:23.724 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.107 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", , 2020-12-30T08:57:52.108 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.759798+0000", , 2020-12-30T08:57:52.108 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 5.1649e-05 2020-12-30T23:00:23.724 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "initiated", , 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.611817+0000", 2020-12-30T23:00:23.725 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "started", , 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.867908+0000", 2020-12-30T23:00:23.725 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "sub_op_commit_rec", , 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.879692+0000", 2020-12-30T23:00:23.726 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.189 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.190 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.190 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.727 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", 2020-12-30T23:00:23.727 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", 2020-12-30T23:00:23.728 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "reached pg", , 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: "events": [ , 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.728 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.272 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.002481568 , 2020-12-30T08:57:52.272 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.273 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.729 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.292 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 8.7658000000000007e-05 , 2020-12-30T08:57:52.293 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.293 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.730 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.87972017899999999 , 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2020-12-30T23:00:23.730 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "type_data": { , 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "started", , 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "events": [ 2020-12-30T23:00:23.731 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:28.437546+0000", , 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 6.2249999999999997e-06 , 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.731 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 600, , 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: "ops": [ , 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.732 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", , 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.755828+0000", 2020-12-30T23:00:23.732 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "header_read", 2020-12-30T23:00:23.733 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: } 2020-12-30T23:00:23.734 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.820 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4.3782000000000002e-05 , 2020-12-30T08:57:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.734 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:10.493947+0000", , 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4294967295.9999809 , 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.735 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.310989+0000", , 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.012873951999999999 , 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: } 2020-12-30T23:00:23.735 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "queued_for_pg", , 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:17.586655+0000", , 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4.7491000000000003e-05 2020-12-30T23:00:23.736 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", , 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:33.372282+0000", 2020-12-30T23:00:23.737 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:44.022360+0000", , 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "age": 8.7134268660000007, , 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.346765198, 2020-12-30T23:00:23.737 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "started", 2020-12-30T23:00:23.738 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.963 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.0046062250000000002 , 2020-12-30T08:57:52.963 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.964 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.738 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.983 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.984 INFO:teuthology.orchestra.run.smithi169.stdout:} , 2020-12-30T08:57:52.985 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-12-30T23:00:23.739 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", 2020-12-30T23:00:23.740 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.403 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0 , 2020-12-30T08:57:53.404 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.404 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.740 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: "description": "osd_op(client.4598.0:1164 4.0 4:249a18ca:::smithi04340772-260:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e34)", , 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:04.096815+0000", 2020-12-30T23:00:23.741 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:09.599534+0000", , 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.036992197999999997 , 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.741 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.470 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", , 2020-12-30T08:57:53.470 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.144872+0000", , 2020-12-30T08:57:53.471 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 1.0566e-05 2020-12-30T23:00:23.742 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "initiated", , 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.156798+0000", 2020-12-30T23:00:23.742 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.513 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:53.513 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:53.514 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.743 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.535 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.535 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.536 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "queued_for_pg", 2020-12-30T23:00:23.744 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4294967295.9999795 , 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.744 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: "description": "osd_op(client.4598.0:2898 4.0 4:1f210125:::smithi04340772-284:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e38)", , 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:17.505116+0000", 2020-12-30T23:00:23.745 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.600 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:22.235621+0000", , 2020-12-30T08:57:53.600 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.035873975000000002 , 2020-12-30T08:57:53.601 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.745 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3320 , 2020-12-30T08:57:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.027 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2020-12-30T23:00:23.746 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:23.886767+0000", , 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8839999999999997e-06 , 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:23.746 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:32.913785+0000", , 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049845687999999999 , 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:23.747 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", , 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:27.332932+0000", , 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2020-12-30T23:00:23.748 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", , 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:27.333311+0000", , 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1931999999999999e-05 2020-12-30T23:00:23.748 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", , 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:28.423645+0000", 2020-12-30T23:00:23.749 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for readable", , 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:28.440591+0000", 2020-12-30T23:00:23.749 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { , 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4598", , 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "v1:172.21.15.43:0/972417042", 2020-12-30T23:00:23.750 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2020-12-30T23:00:23.750 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 , 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2020-12-30T23:00:23.751 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { , 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", , 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2020-12-30T23:00:23.752 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.249599+0000", , 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4746000000000001e-05 , 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:23.752 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.636403+0000", , 2020-12-30T08:57:54.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0044951799999999997 , 2020-12-30T08:57:54.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:23.753 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.655240+0000", , 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047171491000000003 , 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2020-12-30T23:00:23.754 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:58:15.685 INFO:tasks.ceph.osd.0:Stopped , 2020-12-30T08:58:15.685 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 , osd.0 is already down. 2020-12-30T23:00:23.755 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:59:26.097 INFO:teuthology.orchestra.run.smithi043.stdout:mon.a: {} , 2020-12-30T08:59:26.113 INFO:teuthology.orchestra.run.smithi043.stdout:mon.b: {} , 2020-12-30T08:59:26.127 INFO:teuthology.orchestra.run.smithi043.stdout:mon.c: {} 2020-12-30T23:00:23.757 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:59:48.212 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2020-12-30T08:59:47.527120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1407295001,"num_objects":477,"num_object_clones":0,"num_object_copies":1431,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":449,"num_whiteouts":3,"num_read":3286,"num_read_kb":2359369,"num_write":3461,"num_write_kb":1206573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":773,"num_bytes_recovered":1169499301,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":887,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":44,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2967,"ondisk_log_size":2967,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2702316,"kb_used_data":2595628,"kb_used_omap":11,"kb_used_meta":105780,"kb_avail":752272404,"statfs":{"total":773094113280,"available":770326941696,"internally_reserved":0,"allocated":2657923072,"data_stored":2643000988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11269,"internal_metadata":108319739},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.529700"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":"111","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252322+0000","last_change":"2020-12-30T08:59:26.805818+0000","last_active":"2020-12-30T08:59:40.252322+0000","last_peered":"2020-12-30T08:59:40.252322+0000","last_clean":"2020-12-30T08:59:40.252322+0000","last_became_active":"2020-12-30T08:59:26.805280+0000","last_became_peered":"2020-12-30T08:59:26.805280+0000","last_unstale":"2020-12-30T08:59:40.252322+0000","last_undegraded":"2020-12-30T08:59:40.252322+0000","last_fullsized":"2020-12-30T08:59:40.252322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:04.206849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:04.206849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"57'44","reported_seq":"156","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252340+0000","last_change":"2020-12-30T08:59:09.166665+0000","last_active":"2020-12-30T08:59:40.252340+0000","last_peered":"2020-12-30T08:59:40.252340+0000","last_clean":"2020-12-30T08:59:40.252340+0000","last_became_active":"2020-12-30T08:56:39.905257+0000","last_became_peered":"2020-12-30T08:56:39.905257+0000","last_unstale":"2020-12-30T08:59:40.252340+0000","last_undegraded":"2020-12-30T08:59:40.252340+0000","last_fullsized":"2020-12-30T08:59:40.252340+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'44","last_scrub_stamp":"2020-12-30T08:59:09.166549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:09.166549+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63684286,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":42,"num_read_kb":51083,"num_write":29,"num_write_kb":4590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":319,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":25,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.578479+0000","last_change":"2020-12-30T08:58:46.630261+0000","last_active":"2020-12-30T08:59:40.578479+0000","last_peered":"2020-12-30T08:59:40.578479+0000","last_clean":"2020-12-30T08:59:40.578479+0000","last_became_active":"2020-12-30T08:56:17.453297+0000","last_became_peered":"2020-12-30T08:56:17.453297+0000","last_unstale":"2020-12-30T08:59:40.578479+0000","last_undegraded":"2020-12-30T08:59:40.578479+0000","last_fullsized":"2020-12-30T08:59:40.578479+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:46.630148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:46.630148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"57'794","reported_seq":"1885","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.596274+0000","last_change":"2020-12-30T08:59:31.962315+0000","last_active":"2020-12-30T08:59:41.596274+0000","last_peered":"2020-12-30T08:59:41.596274+0000","last_clean":"2020-12-30T08:59:41.596274+0000","last_became_active":"2020-12-30T08:59:31.961183+0000","last_became_peered":"2020-12-30T08:59:31.961183+0000","last_unstale":"2020-12-30T08:59:41.596274+0000","last_undegraded":"2020-12-30T08:59:41.596274+0000","last_fullsized":"2020-12-30T08:59:41.596274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'794","last_scrub_stamp":"2020-12-30T08:59:09.405174+0000","last_deep_scrub":"54'709","last_deep_scrub_stamp":"2020-12-30T08:57:41.784481+0000","last_clean_scrub_stamp":"2020-12-30T08:59:09.405174+0000","log_size":794,"ondisk_log_size":794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":324600875,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":956,"num_read_kb":765811,"num_write":895,"num_write_kb":309669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":381323268,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"125","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.432965+0000","last_change":"2020-12-30T08:59:26.807827+0000","last_active":"2020-12-30T08:59:40.432965+0000","last_peered":"2020-12-30T08:59:40.432965+0000","last_clean":"2020-12-30T08:59:40.432965+0000","last_became_active":"2020-12-30T08:59:26.807377+0000","last_became_peered":"2020-12-30T08:59:26.807377+0000","last_unstale":"2020-12-30T08:59:40.432965+0000","last_undegraded":"2020-12-30T08:59:40.432965+0000","last_fullsized":"2020-12-30T08:59:40.432965+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.818206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.818206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"57'598","reported_seq":"1017","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252337+0000","last_change":"2020-12-30T08:59:08.196312+0000","last_active":"2020-12-30T08:59:40.252337+0000","last_peered":"2020-12-30T08:59:40.252337+0000","last_clean":"2020-12-30T08:59:40.252337+0000","last_became_active":"2020-12-30T08:56:35.883250+0000","last_became_peered":"2020-12-30T08:56:35.883250+0000","last_unstale":"2020-12-30T08:59:40.252337+0000","last_undegraded":"2020-12-30T08:59:40.252337+0000","last_fullsized":"2020-12-30T08:59:40.252337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'598","last_scrub_stamp":"2020-12-30T08:59:08.196199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:34.853614+0000","last_clean_scrub_stamp":"2020-12-30T08:59:08.196199+0000","log_size":598,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":299758950,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":689,"num_read_kb":446464,"num_write":755,"num_write_kb":264543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.578494+0000","last_change":"2020-12-30T08:58:53.577047+0000","last_active":"2020-12-30T08:59:40.578494+0000","last_peered":"2020-12-30T08:59:40.578494+0000","last_clean":"2020-12-30T08:59:40.578494+0000","last_became_active":"2020-12-30T08:56:18.363833+0000","last_became_peered":"2020-12-30T08:56:18.363833+0000","last_unstale":"2020-12-30T08:59:40.578494+0000","last_undegraded":"2020-12-30T08:59:40.578494+0000","last_fullsized":"2020-12-30T08:59:40.578494+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:53.576887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:53.576887+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":"145","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001415+0000","last_change":"2020-12-30T08:59:29.950684+0000","last_active":"2020-12-30T08:59:41.001415+0000","last_peered":"2020-12-30T08:59:41.001415+0000","last_clean":"2020-12-30T08:59:41.001415+0000","last_became_active":"2020-12-30T08:59:29.950219+0000","last_became_peered":"2020-12-30T08:59:29.950219+0000","last_unstale":"2020-12-30T08:59:41.001415+0000","last_undegraded":"2020-12-30T08:59:41.001415+0000","last_fullsized":"2020-12-30T08:59:41.001415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":4,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'2","last_scrub_stamp":"2020-12-30T08:58:43.023209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:05.570594+0000","last_clean_scrub_stamp":"2020-12-30T08:58:43.023209+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"57'737","reported_seq":"1556","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.577945+0000","last_change":"2020-12-30T08:59:27.273198+0000","last_active":"2020-12-30T08:59:40.577945+0000","last_peered":"2020-12-30T08:59:40.577945+0000","last_clean":"2020-12-30T08:59:40.577945+0000","last_became_active":"2020-12-30T08:57:34.885212+0000","last_became_peered":"2020-12-30T08:57:34.885212+0000","last_unstale":"2020-12-30T08:59:40.577945+0000","last_undegraded":"2020-12-30T08:59:40.577945+0000","last_fullsized":"2020-12-30T08:59:40.577945+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'737","last_scrub_stamp":"2020-12-30T08:59:27.273101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:34.853614+0000","last_clean_scrub_stamp":"2020-12-30T08:59:27.273101+0000","log_size":737,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":370415644,"num_objects":124,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":0,"num_read":829,"num_read_kb":576628,"num_write":921,"num_write_kb":353530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":210025962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"112","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001472+0000","last_change":"2020-12-30T08:59:26.807466+0000","last_active":"2020-12-30T08:59:41.001472+0000","last_peered":"2020-12-30T08:59:41.001472+0000","last_clean":"2020-12-30T08:59:41.001472+0000","last_became_active":"2020-12-30T08:59:26.806602+0000","last_became_peered":"2020-12-30T08:59:26.806602+0000","last_unstale":"2020-12-30T08:59:41.001472+0000","last_undegraded":"2020-12-30T08:59:41.001472+0000","last_fullsized":"2020-12-30T08:59:41.001472+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:06.756813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:06.756813+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"76'751","reported_seq":"1682","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.000117+0000","last_change":"2020-12-30T08:59:30.952423+0000","last_active":"2020-12-30T08:59:41.000117+0000","last_peered":"2020-12-30T08:59:41.000117+0000","last_clean":"2020-12-30T08:59:41.000117+0000","last_became_active":"2020-12-30T08:59:30.951003+0000","last_became_peered":"2020-12-30T08:59:30.951003+0000","last_unstale":"2020-12-30T08:59:41.000117+0000","last_undegraded":"2020-12-30T08:59:41.000117+0000","last_fullsized":"2020-12-30T08:59:41.000117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'751","last_scrub_stamp":"2020-12-30T08:59:17.342114+0000","last_deep_scrub":"76'751","last_deep_scrub_stamp":"2020-12-30T08:59:17.342114+0000","last_clean_scrub_stamp":"2020-12-30T08:59:17.342114+0000","log_size":751,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":288680016,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":742,"num_read_kb":486335,"num_write":828,"num_write_kb":269542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":495679005,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"57'31","reported_seq":"176","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.000230+0000","last_change":"2020-12-30T08:59:30.953648+0000","last_active":"2020-12-30T08:59:41.000230+0000","last_peered":"2020-12-30T08:59:41.000230+0000","last_clean":"2020-12-30T08:59:41.000230+0000","last_became_active":"2020-12-30T08:59:30.952821+0000","last_became_peered":"2020-12-30T08:59:30.952821+0000","last_unstale":"2020-12-30T08:59:41.000230+0000","last_undegraded":"2020-12-30T08:59:41.000230+0000","last_fullsized":"2020-12-30T08:59:41.000230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'31","last_scrub_stamp":"2020-12-30T08:59:18.654169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:18.654169+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41801084,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":14,"num_read_kb":4410,"num_write":24,"num_write_kb":3616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":82471066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":293,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":14,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"115","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001437+0000","last_change":"2020-12-30T08:59:26.807130+0000","last_active":"2020-12-30T08:59:41.001437+0000","last_peered":"2020-12-30T08:59:41.001437+0000","last_clean":"2020-12-30T08:59:41.001437+0000","last_became_active":"2020-12-30T08:59:26.806409+0000","last_became_peered":"2020-12-30T08:59:26.806409+0000","last_unstale":"2020-12-30T08:59:41.001437+0000","last_undegraded":"2020-12-30T08:59:41.001437+0000","last_fullsized":"2020-12-30T08:59:41.001437+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.095702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.095702+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252325+0000","last_change":"2020-12-30T08:58:49.511387+0000","last_active":"2020-12-30T08:59:40.252325+0000","last_peered":"2020-12-30T08:59:40.252325+0000","last_clean":"2020-12-30T08:59:40.252325+0000","last_became_active":"2020-12-30T08:56:18.101527+0000","last_became_peered":"2020-12-30T08:56:18.101527+0000","last_unstale":"2020-12-30T08:59:40.252325+0000","last_undegraded":"2020-12-30T08:59:40.252325+0000","last_fullsized":"2020-12-30T08:59:40.252325+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:49.511270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:49.511270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"99","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.577925+0000","last_change":"2020-12-30T08:59:26.808227+0000","last_active":"2020-12-30T08:59:40.577925+0000","last_peered":"2020-12-30T08:59:40.577925+0000","last_clean":"2020-12-30T08:59:40.577925+0000","last_became_active":"2020-12-30T08:59:26.807467+0000","last_became_peered":"2020-12-30T08:59:26.807467+0000","last_unstale":"2020-12-30T08:59:40.577925+0000","last_undegraded":"2020-12-30T08:59:40.577925+0000","last_fullsized":"2020-12-30T08:59:40.577925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.951459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.951459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"92","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.310721+0000","last_change":"2020-12-30T08:58:34.893621+0000","last_active":"2020-12-30T08:59:40.310721+0000","last_peered":"2020-12-30T08:59:40.310721+0000","last_clean":"2020-12-30T08:59:40.310721+0000","last_became_active":"2020-12-30T08:56:17.453289+0000","last_became_peered":"2020-12-30T08:56:17.453289+0000","last_unstale":"2020-12-30T08:59:40.310721+0000","last_undegraded":"2020-12-30T08:59:40.310721+0000","last_fullsized":"2020-12-30T08:59:40.310721+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:34.893489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:34.893489+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"57'10","reported_seq":"102","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.596156+0000","last_change":"2020-12-30T08:58:50.189729+0000","last_active":"2020-12-30T08:59:41.596156+0000","last_peered":"2020-12-30T08:59:41.596156+0000","last_clean":"2020-12-30T08:59:41.596156+0000","last_became_active":"2020-12-30T08:56:40.286191+0000","last_became_peered":"2020-12-30T08:56:40.286191+0000","last_unstale":"2020-12-30T08:59:41.596156+0000","last_undegraded":"2020-12-30T08:59:41.596156+0000","last_fullsized":"2020-12-30T08:59:41.596156+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'10","last_scrub_stamp":"2020-12-30T08:58:50.189625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:58:50.189625+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18354146,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":28638,"num_write":7,"num_write_kb":1074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":275,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":5,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":4,"stat_sum":{"num_bytes":123839516,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":3,"num_read":70,"num_read_kb":84131,"num_write":60,"num_write_kb":9280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":82471066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":887,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":44,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":363405312,"data_stored":363060657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":12,"acting":12,"num_store_stats":7},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":1283455485,"num_objects":428,"num_object_clones":0,"num_object_copies":1284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":428,"num_whiteouts":0,"num_read":3216,"num_read_kb":2275238,"num_write":3399,"num_write_kb":1197284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":739,"num_bytes_recovered":1087028235,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2289242112,"data_stored":2277676566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2880,"ondisk_log_size":2880,"up":12,"acting":12,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10297,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46828,"kb_used_data":40940,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":94325012,"statfs":{"total":96636764160,"available":96588812288,"internally_reserved":0,"allocated":41922560,"data_stored":41513062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:57:16 2020","interfaces":[{"interface":"back","average":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":81.242000000000004,"5min":81.242000000000004,"15min":81.242000000000004},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1.0409999999999999}]},{"osd":1,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":19.870000000000001,"15min":28.206},"min":{"1min":0.83199999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4970000000000001,"5min":501.495,"15min":501.495},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":26.629000000000001,"15min":25.437000000000001},"min":{"1min":0.73399999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.5720000000000001,"5min":958.63099999999997,"15min":958.63099999999997},"last":1.2589999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":59.027999999999999,"5min":12.720000000000001,"15min":5.008},"min":{"1min":0.85799999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":927.80799999999999,"5min":927.80799999999999,"15min":927.80799999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":21.974,"5min":12.337999999999999,"15min":4.8849999999999998},"min":{"1min":0.98799999999999999,"5min":0.753,"15min":0.753},"max":{"1min":333.58300000000003,"5min":597.27800000000002,"15min":597.27800000000002},"last":1.093}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.202,"15min":1.216},"min":{"1min":0.81399999999999995,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.45,"5min":1.474,"15min":1.474},"last":1.2310000000000001},{"interface":"front","average":{"1min":8.3640000000000008,"5min":2.5859999999999999,"15min":1.623},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":123.247,"5min":123.247,"15min":123.247},"last":0.84299999999999997}]},{"osd":4,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":26.962,"15min":38.468000000000004},"min":{"1min":0.81499999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5609999999999999,"5min":747.37,"15min":747.37},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":1.052,"15min":1.0920000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.486,"5min":1.623,"15min":1.623},"last":1.1080000000000001}]},{"osd":5,"last update":"Wed Dec 30 08:59:24 2020","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1379999999999999,"15min":1.125},"min":{"1min":0.76500000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.6160000000000001,"5min":1.7,"15min":1.7},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.1879999999999999,"15min":1.2390000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5469999999999999,"5min":2.198,"15min":2.198},"last":1.198}]},{"osd":6,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":8.0190000000000001,"5min":27.074000000000002,"15min":37.076000000000001},"min":{"1min":0.77300000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":106.654,"5min":861.60799999999995,"15min":861.60799999999995},"last":1.177},{"interface":"front","average":{"1min":52.908999999999999,"5min":11.640000000000001,"15min":4.7830000000000004},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":778.01900000000001,"5min":778.01900000000001,"15min":778.01900000000001},"last":1.3280000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617820,"kb_used_data":597728,"kb_used_omap":0,"kb_used_meta":19968,"kb_avail":93754020,"statfs":{"total":96636764160,"available":96004116480,"internally_reserved":0,"allocated":612073472,"data_stored":608728491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20447232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:15 2020","interfaces":[{"interface":"back","average":{"1min":63.369,"5min":56.469000000000001,"15min":55.319000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":999.76300000000003,"5min":999.76300000000003,"15min":999.76300000000003},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.04,"15min":1.028},"min":{"1min":0.746,"5min":0.62,"15min":0.62},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":1.3069999999999999}]},{"osd":1,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.01,"15min":0.97699999999999998},"min":{"1min":0.79400000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.3480000000000001,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.875},{"interface":"front","average":{"1min":0.96799999999999997,"5min":36.709000000000003,"15min":52.585000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.3009999999999999,"5min":953.40800000000002,"15min":953.40800000000002},"last":1.1899999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":46.109999999999999,"5min":10.127000000000001,"15min":4.109},"min":{"1min":0.98899999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":714.93299999999999,"5min":714.93299999999999,"15min":714.93299999999999},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.206,"15min":1.1990000000000001},"min":{"1min":1.0309999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.4870000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.6060000000000001}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.151,"5min":1.139,"15min":1.137},"min":{"1min":0.94599999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.3919999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.274}]},{"osd":4,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.96199999999999997,"15min":0.97899999999999998},"min":{"1min":0.63600000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.393,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0740000000000001,"15min":1.0860000000000001},"min":{"1min":0.81299999999999994,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":1.375}]},{"osd":5,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.1020000000000001,"15min":1.123},"min":{"1min":0.77300000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.286,"5min":1.637,"15min":1.637},"last":1.3540000000000001},{"interface":"front","average":{"1min":33.957000000000001,"5min":47.049999999999997,"15min":60.192999999999998},"min":{"1min":0.81299999999999994,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":691.76199999999994,"5min":985.03399999999999,"15min":985.03399999999999},"last":1.3080000000000001}]},{"osd":7,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":21.876000000000001,"5min":5.3460000000000001,"15min":2.5979999999999999},"min":{"1min":0.86099999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":435.39499999999998,"5min":435.39499999999998,"15min":435.39499999999998},"last":1.331},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2330000000000001,"15min":1.2490000000000001},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.5740000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":247340,"kb_used_data":233900,"kb_used_omap":0,"kb_used_meta":13312,"kb_avail":94124500,"statfs":{"total":96636764160,"available":96383488000,"internally_reserved":0,"allocated":239513600,"data_stored":238202894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13631488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:13 2020","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.0569999999999999,"15min":1.0880000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2869999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77500000000000002,"15min":0.77600000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.74399999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":27.254000000000001,"15min":39.072000000000003},"min":{"1min":0.47199999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84799999999999998,"5min":882.37400000000002,"15min":882.37400000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.90400000000000003,"15min":0.92200000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.298,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed Dec 30 08:59:31 2020","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.95299999999999996,"15min":0.95699999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.29,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.86899999999999999,"15min":0.88200000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.302,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.80500000000000005}]},{"osd":4,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1339999999999999,"15min":1.1040000000000001},"min":{"1min":0.92200000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.3899999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.1599999999999999,"15min":1.169},"min":{"1min":0.73299999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.46,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.84699999999999998}]},{"osd":5,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2649999999999999,"15min":1.2829999999999999},"min":{"1min":0.89100000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.4159999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2390000000000001,"15min":1.2470000000000001},"min":{"1min":1.002,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.383,"5min":1.96,"15min":1.96},"last":1.0229999999999999}]},{"osd":6,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2390000000000001,"15min":1.2749999999999999},"min":{"1min":0.82299999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4259999999999999,"5min":4.4829999999999997,"15min":4.4829999999999997},"last":1.149},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.242,"15min":1.272},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.4159999999999999,"5min":1.734,"15min":1.734},"last":1.0069999999999999}]},{"osd":7,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":19.978999999999999,"15min":28.309000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.486,"5min":623.82100000000003,"15min":623.82100000000003},"last":828.46799999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.2250000000000001,"15min":1.2350000000000001},"min":{"1min":0.91700000000000004,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.3500000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.98699999999999999}]}]},{"osd":0,"up_from":75,"seq":322122547208,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410660,"kb_used_data":392612,"kb_used_omap":10,"kb_used_meta":17909,"kb_avail":93961180,"statfs":{"total":96636764160,"available":96216248320,"internally_reserved":0,"allocated":402034688,"data_stored":399673841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10297,"internal_metadata":18339783},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.649}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.679}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":179560,"kb_used_data":172264,"kb_used_omap":0,"kb_used_meta":7168,"kb_avail":94192280,"statfs":{"total":96636764160,"available":96452894720,"internally_reserved":0,"allocated":176398336,"data_stored":175142793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7340032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:15 2020","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.81999999999999995,"15min":0.81399999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.999,"5min":33.204000000000001,"15min":38.572000000000003},"min":{"1min":0.63500000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.8049999999999999,"5min":767.24300000000005,"15min":767.24300000000005},"last":0.67100000000000004}]},{"osd":1,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":30.774000000000001,"5min":7.2549999999999999,"15min":3.4510000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":754.12800000000004,"5min":754.12800000000004,"15min":754.12800000000004},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.91400000000000003,"15min":0.91100000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4430000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.80300000000000005}]},{"osd":3,"last update":"Wed Dec 30 08:59:29 2020","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.0629999999999999,"15min":1.0900000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.476,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.747,"15min":0.747},"min":{"1min":0.55500000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.89100000000000001}]},{"osd":4,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":7.5339999999999998,"15min":3.2320000000000002},"min":{"1min":0.67200000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6200000000000001,"5min":742.45799999999997,"15min":742.45799999999997},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.2110000000000001,"15min":1.2070000000000001},"min":{"1min":0.81699999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5449999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.1970000000000001}]},{"osd":5,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":2.4820000000000002,"15min":1.631},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.464,"5min":152.709,"15min":152.709},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.1919999999999999,"15min":1.214},"min":{"1min":0.80800000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.4039999999999999,"5min":2.181,"15min":2.181},"last":0.96899999999999997}]},{"osd":6,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":4.1260000000000003,"5min":29.439,"15min":41.308},"min":{"1min":0.753,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":73.947000000000003,"5min":874.04999999999995,"15min":874.04999999999995},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.1439999999999999,"15min":1.1819999999999999},"min":{"1min":0.73399999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.508,"5min":4.0439999999999996,"15min":4.0439999999999996},"last":1.0700000000000001}]},{"osd":7,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":20.821999999999999,"5min":21.945,"15min":26.760999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":489.90300000000002,"5min":502.05900000000003,"15min":502.05900000000003},"last":1.214},{"interface":"front","average":{"1min":1.3220000000000001,"5min":23.478000000000002,"15min":33.302999999999997},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":2.0270000000000001,"5min":502.10599999999999,"15min":502.10599999999999},"last":1.2290000000000001}]}]},{"osd":3,"up_from":41,"seq":176093659166,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400584,"kb_used_data":387072,"kb_used_omap":0,"kb_used_meta":13503,"kb_avail":93971256,"statfs":{"total":96636764160,"available":96226566144,"internally_reserved":0,"allocated":396361728,"data_stored":394019686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":972,"internal_metadata":13827124},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":1,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.93000000000000005,"15min":0.94999999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.242,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.84599999999999997,"15min":0.86799999999999999},"min":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90100000000000002,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.85499999999999998}]},{"osd":2,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.83699999999999997,"15min":0.85299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1319999999999999,"5min":1.581,"15min":1.581},"last":0.628},{"interface":"front","average":{"1min":26.884,"5min":6.1639999999999997,"15min":2.71},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":469.85500000000002,"5min":469.85500000000002,"15min":469.85500000000002},"last":0.82099999999999995}]},{"osd":4,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.0760000000000001,"15min":1.0840000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.254,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.23,"15min":1.254},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.4079999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.90800000000000003}]},{"osd":5,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":25.748000000000001,"15min":29.870000000000001},"min":{"1min":0.81799999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.2010000000000001,"5min":524.471,"15min":524.471},"last":0.88},{"interface":"front","average":{"1min":47.161999999999999,"5min":10.489000000000001,"15min":4.3769999999999998},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":830.05200000000002,"5min":830.05200000000002,"15min":830.05200000000002},"last":0.94699999999999995}]},{"osd":6,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.254,"15min":1.258},"min":{"1min":1.046,"5min":1.046,"15min":1.046},"max":{"1min":1.5960000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.181},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.266,"15min":1.268},"min":{"1min":1.071,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.6259999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.204}]},{"osd":7,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.298,"15min":1.3220000000000001},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.3009999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.089},{"interface":"front","average":{"1min":1.159,"5min":1.264,"15min":1.282},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.1599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":444756,"kb_used_data":430292,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":93927084,"statfs":{"total":96636764160,"available":96181334016,"internally_reserved":0,"allocated":440619008,"data_stored":438222994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:57:13 2020","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.298},{"interface":"front","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.3360000000000001}]},{"osd":1,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.119,"15min":1.1479999999999999},"min":{"1min":0.755,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.4039999999999999,"5min":1.661,"15min":1.661},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.022,"15min":1.012},"min":{"1min":0.78600000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4019999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.361}]},{"osd":2,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.161,"15min":1.1870000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.498,"5min":1.776,"15min":1.776},"last":1.282},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0940000000000001,"15min":1.1040000000000001},"min":{"1min":0.84499999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.446,"5min":1.478,"15min":1.478},"last":1.446}]},{"osd":3,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1000000000000001,"15min":1.0920000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.2050000000000001,"15min":1.2210000000000001},"min":{"1min":0.91400000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.222}]},{"osd":5,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.96399999999999997,"15min":0.94599999999999995},"min":{"1min":0.81399999999999995,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3440000000000001,"5min":1.45,"15min":1.45},"last":0.995},{"interface":"front","average":{"1min":1.014,"5min":0.98999999999999999,"15min":0.94899999999999995},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.226,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.26}]},{"osd":6,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.1160000000000001,"15min":1.153},"min":{"1min":0.73499999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.4950000000000001,"5min":1.71,"15min":1.71},"last":1.4199999999999999},{"interface":"front","average":{"1min":18.32,"5min":4.508,"15min":2.214},"min":{"1min":0.90100000000000002,"5min":0.629,"15min":0.629},"max":{"1min":276.41800000000001,"5min":276.41800000000001,"15min":276.41800000000001},"last":1.0489999999999999}]},{"osd":7,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":3.4380000000000002,"15min":1.8959999999999999},"min":{"1min":1.022,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.385,"5min":181.79499999999999,"15min":181.79499999999999},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":5.6840000000000002,"15min":2.7799999999999998},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":1.4510000000000001,"5min":353.72500000000002,"15min":353.72500000000002},"last":1.476}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":354768,"kb_used_data":340820,"kb_used_omap":0,"kb_used_meta":13824,"kb_avail":94017072,"statfs":{"total":96636764160,"available":96273481728,"internally_reserved":0,"allocated":348999680,"data_stored":347497227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14155776},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:12 2020","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.1439999999999999,"15min":1.1599999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.4239999999999999,"5min":1.526,"15min":1.526},"last":1},{"interface":"front","average":{"1min":42.222000000000001,"5min":9.3520000000000003,"15min":3.8740000000000001},"min":{"1min":0.83799999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":821.10900000000004,"5min":821.10900000000004,"15min":821.10900000000004},"last":0.89200000000000002}]},{"osd":1,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0009999999999999,"15min":0.98499999999999999},"min":{"1min":0.79100000000000004,"5min":0.627,"15min":0.627},"max":{"1min":1.2769999999999999,"5min":1.379,"15min":1.379},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":5.5030000000000001,"15min":2.5710000000000002},"min":{"1min":0.84099999999999997,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.3300000000000001,"5min":440.44999999999999,"15min":440.44999999999999},"last":0.91800000000000004}]},{"osd":2,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":7.0289999999999999,"15min":2.9950000000000001},"min":{"1min":0.752,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.294,"5min":605.79899999999998,"15min":605.79899999999998},"last":0.94599999999999995},{"interface":"front","average":{"1min":23.163,"5min":5.524,"15min":2.5800000000000001},"min":{"1min":0.88200000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":501.64600000000002,"5min":501.64600000000002,"15min":501.64600000000002},"last":1.329}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1419999999999999,"15min":1.1339999999999999},"min":{"1min":0.90600000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.411,"5min":1.494,"15min":1.494},"last":1.282},{"interface":"front","average":{"1min":1.159,"5min":17.323,"15min":20.016999999999999},"min":{"1min":0.92700000000000005,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.393,"5min":404.60300000000001,"15min":404.60300000000001},"last":1.0800000000000001}]},{"osd":4,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":21.57,"15min":22.916},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.4079999999999999,"5min":704.28399999999999,"15min":704.28399999999999},"last":1.113},{"interface":"front","average":{"1min":0.92100000000000004,"5min":46.520000000000003,"15min":66.030000000000001},"min":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.0920000000000001,"5min":861.90599999999995,"15min":861.90599999999995},"last":1.0529999999999999}]},{"osd":6,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0700000000000001,"15min":1.0660000000000001},"min":{"1min":0.66300000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.4299999999999999,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.085,"5min":1.1020000000000001,"15min":1.121},"min":{"1min":0.77700000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4410000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.145}]},{"osd":7,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.284,"15min":1.3919999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3899999999999999,"5min":8.3629999999999995,"15min":8.3629999999999995},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":11.603999999999999,"15min":12.991},"min":{"1min":0.79200000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.6000000000000001,"5min":291.65300000000002,"15min":291.65300000000002},"last":1.3089999999999999}]}]}]}} , 2020-12-30T08:59:48.214 INFO:tasks.ceph.ceph_manager.ceph:recovered! , 2020-12-30T08:59:48.215 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2020-12-30T23:00:23.759 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 33, in nested , yield vars , File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-12-30-1123/qa/tasks/ceph.py", line 1876, in task 2020-12-30T23:00:23.760 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T09:00:18.496 DEBUG:teuthology.orchestra.run.smithi169:> mktemp , 2020-12-30T09:00:18.513 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.C79Qha9RXd , 2020-12-30T09:00:18.514 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.C79Qha9RXd 2020-12-30T23:00:23.760 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T09:00:35.626 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- , 2020-12-30T09:00:35.644 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- , 2020-12-30T09:00:35.706 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2020-12-30T23:00:23.814 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f69dd512816] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x55d303af9fe6] 2020-12-30T23:00:23.815 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.563908+0000", , 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.00019472900000000001 , 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.815 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.187571661, , 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "type_data": { , 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "delayed", 2020-12-30T23:00:23.816 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.107 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", , 2020-12-30T08:57:52.108 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.759798+0000", , 2020-12-30T08:57:52.108 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 5.1649e-05 2020-12-30T23:00:23.816 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "initiated", , 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.611817+0000", 2020-12-30T23:00:23.817 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "started", , 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.867908+0000", 2020-12-30T23:00:23.818 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "sub_op_commit_rec", , 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.879692+0000", 2020-12-30T23:00:23.818 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.189 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.190 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.190 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.819 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", 2020-12-30T23:00:23.819 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", 2020-12-30T23:00:23.820 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "reached pg", , 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: "events": [ , 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.820 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.272 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.002481568 , 2020-12-30T08:57:52.272 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.273 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.821 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.292 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 8.7658000000000007e-05 , 2020-12-30T08:57:52.293 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.293 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.822 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.87972017899999999 , 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2020-12-30T23:00:23.822 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "type_data": { , 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "started", , 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "events": [ 2020-12-30T23:00:23.823 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:28.437546+0000", , 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 6.2249999999999997e-06 , 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.823 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 600, , 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: "ops": [ , 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.824 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", , 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.755828+0000", 2020-12-30T23:00:23.825 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "header_read", 2020-12-30T23:00:23.825 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: } 2020-12-30T23:00:23.826 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.820 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4.3782000000000002e-05 , 2020-12-30T08:57:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.826 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:10.493947+0000", , 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4294967295.9999809 , 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.827 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.310989+0000", , 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.012873951999999999 , 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: } 2020-12-30T23:00:23.827 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "queued_for_pg", , 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:17.586655+0000", , 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4.7491000000000003e-05 2020-12-30T23:00:23.828 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", , 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:33.372282+0000", 2020-12-30T23:00:23.829 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:44.022360+0000", , 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "age": 8.7134268660000007, , 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.346765198, 2020-12-30T23:00:23.829 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "started", 2020-12-30T23:00:23.830 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.963 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.0046062250000000002 , 2020-12-30T08:57:52.963 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.964 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.830 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.983 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.984 INFO:teuthology.orchestra.run.smithi169.stdout:} , 2020-12-30T08:57:52.985 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-12-30T23:00:23.831 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", 2020-12-30T23:00:23.832 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.403 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0 , 2020-12-30T08:57:53.404 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.404 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.832 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: "description": "osd_op(client.4598.0:1164 4.0 4:249a18ca:::smithi04340772-260:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e34)", , 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:04.096815+0000", 2020-12-30T23:00:23.833 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:09.599534+0000", , 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.036992197999999997 , 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.833 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.470 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", , 2020-12-30T08:57:53.470 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.144872+0000", , 2020-12-30T08:57:53.471 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 1.0566e-05 2020-12-30T23:00:23.834 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "initiated", , 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.156798+0000", 2020-12-30T23:00:23.834 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.513 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:53.513 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:53.514 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.835 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.535 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.535 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.536 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "queued_for_pg", 2020-12-30T23:00:23.836 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4294967295.9999795 , 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.836 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: "description": "osd_op(client.4598.0:2898 4.0 4:1f210125:::smithi04340772-284:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e38)", , 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:17.505116+0000", 2020-12-30T23:00:23.837 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.600 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:22.235621+0000", , 2020-12-30T08:57:53.600 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.035873975000000002 , 2020-12-30T08:57:53.601 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.837 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3320 , 2020-12-30T08:57:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.027 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2020-12-30T23:00:23.838 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:23.886767+0000", , 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8839999999999997e-06 , 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:23.838 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:32.913785+0000", , 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049845687999999999 , 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:23.839 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", , 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:27.332932+0000", , 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2020-12-30T23:00:23.840 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", , 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:27.333311+0000", , 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1931999999999999e-05 2020-12-30T23:00:23.840 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", , 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:28.423645+0000", 2020-12-30T23:00:23.841 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for readable", , 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:28.440591+0000", 2020-12-30T23:00:23.841 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { , 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4598", , 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "v1:172.21.15.43:0/972417042", 2020-12-30T23:00:23.842 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2020-12-30T23:00:23.843 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 , 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2020-12-30T23:00:23.843 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { , 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", , 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2020-12-30T23:00:23.844 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.249599+0000", , 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4746000000000001e-05 , 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:23.844 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.636403+0000", , 2020-12-30T08:57:54.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0044951799999999997 , 2020-12-30T08:57:54.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:23.845 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.655240+0000", , 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047171491000000003 , 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2020-12-30T23:00:23.847 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:58:15.685 INFO:tasks.ceph.osd.0:Stopped , 2020-12-30T08:58:15.685 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 , osd.0 is already down. 2020-12-30T23:00:23.848 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:59:26.097 INFO:teuthology.orchestra.run.smithi043.stdout:mon.a: {} , 2020-12-30T08:59:26.113 INFO:teuthology.orchestra.run.smithi043.stdout:mon.b: {} , 2020-12-30T08:59:26.127 INFO:teuthology.orchestra.run.smithi043.stdout:mon.c: {} 2020-12-30T23:00:23.849 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:59:48.212 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2020-12-30T08:59:47.527120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1407295001,"num_objects":477,"num_object_clones":0,"num_object_copies":1431,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":449,"num_whiteouts":3,"num_read":3286,"num_read_kb":2359369,"num_write":3461,"num_write_kb":1206573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":773,"num_bytes_recovered":1169499301,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":887,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":44,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2967,"ondisk_log_size":2967,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2702316,"kb_used_data":2595628,"kb_used_omap":11,"kb_used_meta":105780,"kb_avail":752272404,"statfs":{"total":773094113280,"available":770326941696,"internally_reserved":0,"allocated":2657923072,"data_stored":2643000988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11269,"internal_metadata":108319739},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.529700"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":"111","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252322+0000","last_change":"2020-12-30T08:59:26.805818+0000","last_active":"2020-12-30T08:59:40.252322+0000","last_peered":"2020-12-30T08:59:40.252322+0000","last_clean":"2020-12-30T08:59:40.252322+0000","last_became_active":"2020-12-30T08:59:26.805280+0000","last_became_peered":"2020-12-30T08:59:26.805280+0000","last_unstale":"2020-12-30T08:59:40.252322+0000","last_undegraded":"2020-12-30T08:59:40.252322+0000","last_fullsized":"2020-12-30T08:59:40.252322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:04.206849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:04.206849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"57'44","reported_seq":"156","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252340+0000","last_change":"2020-12-30T08:59:09.166665+0000","last_active":"2020-12-30T08:59:40.252340+0000","last_peered":"2020-12-30T08:59:40.252340+0000","last_clean":"2020-12-30T08:59:40.252340+0000","last_became_active":"2020-12-30T08:56:39.905257+0000","last_became_peered":"2020-12-30T08:56:39.905257+0000","last_unstale":"2020-12-30T08:59:40.252340+0000","last_undegraded":"2020-12-30T08:59:40.252340+0000","last_fullsized":"2020-12-30T08:59:40.252340+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'44","last_scrub_stamp":"2020-12-30T08:59:09.166549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:09.166549+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63684286,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":42,"num_read_kb":51083,"num_write":29,"num_write_kb":4590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":319,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":25,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.578479+0000","last_change":"2020-12-30T08:58:46.630261+0000","last_active":"2020-12-30T08:59:40.578479+0000","last_peered":"2020-12-30T08:59:40.578479+0000","last_clean":"2020-12-30T08:59:40.578479+0000","last_became_active":"2020-12-30T08:56:17.453297+0000","last_became_peered":"2020-12-30T08:56:17.453297+0000","last_unstale":"2020-12-30T08:59:40.578479+0000","last_undegraded":"2020-12-30T08:59:40.578479+0000","last_fullsized":"2020-12-30T08:59:40.578479+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:46.630148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:46.630148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"57'794","reported_seq":"1885","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.596274+0000","last_change":"2020-12-30T08:59:31.962315+0000","last_active":"2020-12-30T08:59:41.596274+0000","last_peered":"2020-12-30T08:59:41.596274+0000","last_clean":"2020-12-30T08:59:41.596274+0000","last_became_active":"2020-12-30T08:59:31.961183+0000","last_became_peered":"2020-12-30T08:59:31.961183+0000","last_unstale":"2020-12-30T08:59:41.596274+0000","last_undegraded":"2020-12-30T08:59:41.596274+0000","last_fullsized":"2020-12-30T08:59:41.596274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'794","last_scrub_stamp":"2020-12-30T08:59:09.405174+0000","last_deep_scrub":"54'709","last_deep_scrub_stamp":"2020-12-30T08:57:41.784481+0000","last_clean_scrub_stamp":"2020-12-30T08:59:09.405174+0000","log_size":794,"ondisk_log_size":794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":324600875,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":956,"num_read_kb":765811,"num_write":895,"num_write_kb":309669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":381323268,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"125","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.432965+0000","last_change":"2020-12-30T08:59:26.807827+0000","last_active":"2020-12-30T08:59:40.432965+0000","last_peered":"2020-12-30T08:59:40.432965+0000","last_clean":"2020-12-30T08:59:40.432965+0000","last_became_active":"2020-12-30T08:59:26.807377+0000","last_became_peered":"2020-12-30T08:59:26.807377+0000","last_unstale":"2020-12-30T08:59:40.432965+0000","last_undegraded":"2020-12-30T08:59:40.432965+0000","last_fullsized":"2020-12-30T08:59:40.432965+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.818206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.818206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"57'598","reported_seq":"1017","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252337+0000","last_change":"2020-12-30T08:59:08.196312+0000","last_active":"2020-12-30T08:59:40.252337+0000","last_peered":"2020-12-30T08:59:40.252337+0000","last_clean":"2020-12-30T08:59:40.252337+0000","last_became_active":"2020-12-30T08:56:35.883250+0000","last_became_peered":"2020-12-30T08:56:35.883250+0000","last_unstale":"2020-12-30T08:59:40.252337+0000","last_undegraded":"2020-12-30T08:59:40.252337+0000","last_fullsized":"2020-12-30T08:59:40.252337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'598","last_scrub_stamp":"2020-12-30T08:59:08.196199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:34.853614+0000","last_clean_scrub_stamp":"2020-12-30T08:59:08.196199+0000","log_size":598,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":299758950,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":689,"num_read_kb":446464,"num_write":755,"num_write_kb":264543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.578494+0000","last_change":"2020-12-30T08:58:53.577047+0000","last_active":"2020-12-30T08:59:40.578494+0000","last_peered":"2020-12-30T08:59:40.578494+0000","last_clean":"2020-12-30T08:59:40.578494+0000","last_became_active":"2020-12-30T08:56:18.363833+0000","last_became_peered":"2020-12-30T08:56:18.363833+0000","last_unstale":"2020-12-30T08:59:40.578494+0000","last_undegraded":"2020-12-30T08:59:40.578494+0000","last_fullsized":"2020-12-30T08:59:40.578494+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:53.576887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:53.576887+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":"145","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001415+0000","last_change":"2020-12-30T08:59:29.950684+0000","last_active":"2020-12-30T08:59:41.001415+0000","last_peered":"2020-12-30T08:59:41.001415+0000","last_clean":"2020-12-30T08:59:41.001415+0000","last_became_active":"2020-12-30T08:59:29.950219+0000","last_became_peered":"2020-12-30T08:59:29.950219+0000","last_unstale":"2020-12-30T08:59:41.001415+0000","last_undegraded":"2020-12-30T08:59:41.001415+0000","last_fullsized":"2020-12-30T08:59:41.001415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":4,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'2","last_scrub_stamp":"2020-12-30T08:58:43.023209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:05.570594+0000","last_clean_scrub_stamp":"2020-12-30T08:58:43.023209+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"57'737","reported_seq":"1556","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.577945+0000","last_change":"2020-12-30T08:59:27.273198+0000","last_active":"2020-12-30T08:59:40.577945+0000","last_peered":"2020-12-30T08:59:40.577945+0000","last_clean":"2020-12-30T08:59:40.577945+0000","last_became_active":"2020-12-30T08:57:34.885212+0000","last_became_peered":"2020-12-30T08:57:34.885212+0000","last_unstale":"2020-12-30T08:59:40.577945+0000","last_undegraded":"2020-12-30T08:59:40.577945+0000","last_fullsized":"2020-12-30T08:59:40.577945+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'737","last_scrub_stamp":"2020-12-30T08:59:27.273101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:34.853614+0000","last_clean_scrub_stamp":"2020-12-30T08:59:27.273101+0000","log_size":737,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":370415644,"num_objects":124,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":0,"num_read":829,"num_read_kb":576628,"num_write":921,"num_write_kb":353530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":210025962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"112","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001472+0000","last_change":"2020-12-30T08:59:26.807466+0000","last_active":"2020-12-30T08:59:41.001472+0000","last_peered":"2020-12-30T08:59:41.001472+0000","last_clean":"2020-12-30T08:59:41.001472+0000","last_became_active":"2020-12-30T08:59:26.806602+0000","last_became_peered":"2020-12-30T08:59:26.806602+0000","last_unstale":"2020-12-30T08:59:41.001472+0000","last_undegraded":"2020-12-30T08:59:41.001472+0000","last_fullsized":"2020-12-30T08:59:41.001472+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:06.756813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:06.756813+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"76'751","reported_seq":"1682","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.000117+0000","last_change":"2020-12-30T08:59:30.952423+0000","last_active":"2020-12-30T08:59:41.000117+0000","last_peered":"2020-12-30T08:59:41.000117+0000","last_clean":"2020-12-30T08:59:41.000117+0000","last_became_active":"2020-12-30T08:59:30.951003+0000","last_became_peered":"2020-12-30T08:59:30.951003+0000","last_unstale":"2020-12-30T08:59:41.000117+0000","last_undegraded":"2020-12-30T08:59:41.000117+0000","last_fullsized":"2020-12-30T08:59:41.000117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'751","last_scrub_stamp":"2020-12-30T08:59:17.342114+0000","last_deep_scrub":"76'751","last_deep_scrub_stamp":"2020-12-30T08:59:17.342114+0000","last_clean_scrub_stamp":"2020-12-30T08:59:17.342114+0000","log_size":751,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":288680016,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":742,"num_read_kb":486335,"num_write":828,"num_write_kb":269542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":495679005,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"57'31","reported_seq":"176","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.000230+0000","last_change":"2020-12-30T08:59:30.953648+0000","last_active":"2020-12-30T08:59:41.000230+0000","last_peered":"2020-12-30T08:59:41.000230+0000","last_clean":"2020-12-30T08:59:41.000230+0000","last_became_active":"2020-12-30T08:59:30.952821+0000","last_became_peered":"2020-12-30T08:59:30.952821+0000","last_unstale":"2020-12-30T08:59:41.000230+0000","last_undegraded":"2020-12-30T08:59:41.000230+0000","last_fullsized":"2020-12-30T08:59:41.000230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'31","last_scrub_stamp":"2020-12-30T08:59:18.654169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:18.654169+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41801084,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":14,"num_read_kb":4410,"num_write":24,"num_write_kb":3616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":82471066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":293,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":14,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"115","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001437+0000","last_change":"2020-12-30T08:59:26.807130+0000","last_active":"2020-12-30T08:59:41.001437+0000","last_peered":"2020-12-30T08:59:41.001437+0000","last_clean":"2020-12-30T08:59:41.001437+0000","last_became_active":"2020-12-30T08:59:26.806409+0000","last_became_peered":"2020-12-30T08:59:26.806409+0000","last_unstale":"2020-12-30T08:59:41.001437+0000","last_undegraded":"2020-12-30T08:59:41.001437+0000","last_fullsized":"2020-12-30T08:59:41.001437+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.095702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.095702+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252325+0000","last_change":"2020-12-30T08:58:49.511387+0000","last_active":"2020-12-30T08:59:40.252325+0000","last_peered":"2020-12-30T08:59:40.252325+0000","last_clean":"2020-12-30T08:59:40.252325+0000","last_became_active":"2020-12-30T08:56:18.101527+0000","last_became_peered":"2020-12-30T08:56:18.101527+0000","last_unstale":"2020-12-30T08:59:40.252325+0000","last_undegraded":"2020-12-30T08:59:40.252325+0000","last_fullsized":"2020-12-30T08:59:40.252325+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:49.511270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:49.511270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"99","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.577925+0000","last_change":"2020-12-30T08:59:26.808227+0000","last_active":"2020-12-30T08:59:40.577925+0000","last_peered":"2020-12-30T08:59:40.577925+0000","last_clean":"2020-12-30T08:59:40.577925+0000","last_became_active":"2020-12-30T08:59:26.807467+0000","last_became_peered":"2020-12-30T08:59:26.807467+0000","last_unstale":"2020-12-30T08:59:40.577925+0000","last_undegraded":"2020-12-30T08:59:40.577925+0000","last_fullsized":"2020-12-30T08:59:40.577925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.951459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.951459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"92","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.310721+0000","last_change":"2020-12-30T08:58:34.893621+0000","last_active":"2020-12-30T08:59:40.310721+0000","last_peered":"2020-12-30T08:59:40.310721+0000","last_clean":"2020-12-30T08:59:40.310721+0000","last_became_active":"2020-12-30T08:56:17.453289+0000","last_became_peered":"2020-12-30T08:56:17.453289+0000","last_unstale":"2020-12-30T08:59:40.310721+0000","last_undegraded":"2020-12-30T08:59:40.310721+0000","last_fullsized":"2020-12-30T08:59:40.310721+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:34.893489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:34.893489+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"57'10","reported_seq":"102","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.596156+0000","last_change":"2020-12-30T08:58:50.189729+0000","last_active":"2020-12-30T08:59:41.596156+0000","last_peered":"2020-12-30T08:59:41.596156+0000","last_clean":"2020-12-30T08:59:41.596156+0000","last_became_active":"2020-12-30T08:56:40.286191+0000","last_became_peered":"2020-12-30T08:56:40.286191+0000","last_unstale":"2020-12-30T08:59:41.596156+0000","last_undegraded":"2020-12-30T08:59:41.596156+0000","last_fullsized":"2020-12-30T08:59:41.596156+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'10","last_scrub_stamp":"2020-12-30T08:58:50.189625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:58:50.189625+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18354146,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":28638,"num_write":7,"num_write_kb":1074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":275,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":5,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":4,"stat_sum":{"num_bytes":123839516,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":3,"num_read":70,"num_read_kb":84131,"num_write":60,"num_write_kb":9280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":82471066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":887,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":44,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":363405312,"data_stored":363060657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":12,"acting":12,"num_store_stats":7},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":1283455485,"num_objects":428,"num_object_clones":0,"num_object_copies":1284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":428,"num_whiteouts":0,"num_read":3216,"num_read_kb":2275238,"num_write":3399,"num_write_kb":1197284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":739,"num_bytes_recovered":1087028235,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2289242112,"data_stored":2277676566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2880,"ondisk_log_size":2880,"up":12,"acting":12,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10297,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46828,"kb_used_data":40940,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":94325012,"statfs":{"total":96636764160,"available":96588812288,"internally_reserved":0,"allocated":41922560,"data_stored":41513062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:57:16 2020","interfaces":[{"interface":"back","average":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":81.242000000000004,"5min":81.242000000000004,"15min":81.242000000000004},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1.0409999999999999}]},{"osd":1,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":19.870000000000001,"15min":28.206},"min":{"1min":0.83199999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4970000000000001,"5min":501.495,"15min":501.495},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":26.629000000000001,"15min":25.437000000000001},"min":{"1min":0.73399999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.5720000000000001,"5min":958.63099999999997,"15min":958.63099999999997},"last":1.2589999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":59.027999999999999,"5min":12.720000000000001,"15min":5.008},"min":{"1min":0.85799999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":927.80799999999999,"5min":927.80799999999999,"15min":927.80799999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":21.974,"5min":12.337999999999999,"15min":4.8849999999999998},"min":{"1min":0.98799999999999999,"5min":0.753,"15min":0.753},"max":{"1min":333.58300000000003,"5min":597.27800000000002,"15min":597.27800000000002},"last":1.093}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.202,"15min":1.216},"min":{"1min":0.81399999999999995,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.45,"5min":1.474,"15min":1.474},"last":1.2310000000000001},{"interface":"front","average":{"1min":8.3640000000000008,"5min":2.5859999999999999,"15min":1.623},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":123.247,"5min":123.247,"15min":123.247},"last":0.84299999999999997}]},{"osd":4,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":26.962,"15min":38.468000000000004},"min":{"1min":0.81499999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5609999999999999,"5min":747.37,"15min":747.37},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":1.052,"15min":1.0920000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.486,"5min":1.623,"15min":1.623},"last":1.1080000000000001}]},{"osd":5,"last update":"Wed Dec 30 08:59:24 2020","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1379999999999999,"15min":1.125},"min":{"1min":0.76500000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.6160000000000001,"5min":1.7,"15min":1.7},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.1879999999999999,"15min":1.2390000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5469999999999999,"5min":2.198,"15min":2.198},"last":1.198}]},{"osd":6,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":8.0190000000000001,"5min":27.074000000000002,"15min":37.076000000000001},"min":{"1min":0.77300000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":106.654,"5min":861.60799999999995,"15min":861.60799999999995},"last":1.177},{"interface":"front","average":{"1min":52.908999999999999,"5min":11.640000000000001,"15min":4.7830000000000004},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":778.01900000000001,"5min":778.01900000000001,"15min":778.01900000000001},"last":1.3280000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617820,"kb_used_data":597728,"kb_used_omap":0,"kb_used_meta":19968,"kb_avail":93754020,"statfs":{"total":96636764160,"available":96004116480,"internally_reserved":0,"allocated":612073472,"data_stored":608728491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20447232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:15 2020","interfaces":[{"interface":"back","average":{"1min":63.369,"5min":56.469000000000001,"15min":55.319000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":999.76300000000003,"5min":999.76300000000003,"15min":999.76300000000003},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.04,"15min":1.028},"min":{"1min":0.746,"5min":0.62,"15min":0.62},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":1.3069999999999999}]},{"osd":1,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.01,"15min":0.97699999999999998},"min":{"1min":0.79400000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.3480000000000001,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.875},{"interface":"front","average":{"1min":0.96799999999999997,"5min":36.709000000000003,"15min":52.585000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.3009999999999999,"5min":953.40800000000002,"15min":953.40800000000002},"last":1.1899999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":46.109999999999999,"5min":10.127000000000001,"15min":4.109},"min":{"1min":0.98899999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":714.93299999999999,"5min":714.93299999999999,"15min":714.93299999999999},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.206,"15min":1.1990000000000001},"min":{"1min":1.0309999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.4870000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.6060000000000001}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.151,"5min":1.139,"15min":1.137},"min":{"1min":0.94599999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.3919999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.274}]},{"osd":4,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.96199999999999997,"15min":0.97899999999999998},"min":{"1min":0.63600000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.393,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0740000000000001,"15min":1.0860000000000001},"min":{"1min":0.81299999999999994,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":1.375}]},{"osd":5,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.1020000000000001,"15min":1.123},"min":{"1min":0.77300000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.286,"5min":1.637,"15min":1.637},"last":1.3540000000000001},{"interface":"front","average":{"1min":33.957000000000001,"5min":47.049999999999997,"15min":60.192999999999998},"min":{"1min":0.81299999999999994,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":691.76199999999994,"5min":985.03399999999999,"15min":985.03399999999999},"last":1.3080000000000001}]},{"osd":7,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":21.876000000000001,"5min":5.3460000000000001,"15min":2.5979999999999999},"min":{"1min":0.86099999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":435.39499999999998,"5min":435.39499999999998,"15min":435.39499999999998},"last":1.331},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2330000000000001,"15min":1.2490000000000001},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.5740000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":247340,"kb_used_data":233900,"kb_used_omap":0,"kb_used_meta":13312,"kb_avail":94124500,"statfs":{"total":96636764160,"available":96383488000,"internally_reserved":0,"allocated":239513600,"data_stored":238202894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13631488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:13 2020","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.0569999999999999,"15min":1.0880000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2869999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77500000000000002,"15min":0.77600000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.74399999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":27.254000000000001,"15min":39.072000000000003},"min":{"1min":0.47199999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84799999999999998,"5min":882.37400000000002,"15min":882.37400000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.90400000000000003,"15min":0.92200000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.298,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed Dec 30 08:59:31 2020","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.95299999999999996,"15min":0.95699999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.29,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.86899999999999999,"15min":0.88200000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.302,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.80500000000000005}]},{"osd":4,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1339999999999999,"15min":1.1040000000000001},"min":{"1min":0.92200000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.3899999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.1599999999999999,"15min":1.169},"min":{"1min":0.73299999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.46,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.84699999999999998}]},{"osd":5,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2649999999999999,"15min":1.2829999999999999},"min":{"1min":0.89100000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.4159999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2390000000000001,"15min":1.2470000000000001},"min":{"1min":1.002,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.383,"5min":1.96,"15min":1.96},"last":1.0229999999999999}]},{"osd":6,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2390000000000001,"15min":1.2749999999999999},"min":{"1min":0.82299999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4259999999999999,"5min":4.4829999999999997,"15min":4.4829999999999997},"last":1.149},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.242,"15min":1.272},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.4159999999999999,"5min":1.734,"15min":1.734},"last":1.0069999999999999}]},{"osd":7,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":19.978999999999999,"15min":28.309000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.486,"5min":623.82100000000003,"15min":623.82100000000003},"last":828.46799999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.2250000000000001,"15min":1.2350000000000001},"min":{"1min":0.91700000000000004,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.3500000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.98699999999999999}]}]},{"osd":0,"up_from":75,"seq":322122547208,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410660,"kb_used_data":392612,"kb_used_omap":10,"kb_used_meta":17909,"kb_avail":93961180,"statfs":{"total":96636764160,"available":96216248320,"internally_reserved":0,"allocated":402034688,"data_stored":399673841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10297,"internal_metadata":18339783},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.649}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.679}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":179560,"kb_used_data":172264,"kb_used_omap":0,"kb_used_meta":7168,"kb_avail":94192280,"statfs":{"total":96636764160,"available":96452894720,"internally_reserved":0,"allocated":176398336,"data_stored":175142793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7340032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:15 2020","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.81999999999999995,"15min":0.81399999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.999,"5min":33.204000000000001,"15min":38.572000000000003},"min":{"1min":0.63500000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.8049999999999999,"5min":767.24300000000005,"15min":767.24300000000005},"last":0.67100000000000004}]},{"osd":1,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":30.774000000000001,"5min":7.2549999999999999,"15min":3.4510000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":754.12800000000004,"5min":754.12800000000004,"15min":754.12800000000004},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.91400000000000003,"15min":0.91100000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4430000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.80300000000000005}]},{"osd":3,"last update":"Wed Dec 30 08:59:29 2020","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.0629999999999999,"15min":1.0900000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.476,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.747,"15min":0.747},"min":{"1min":0.55500000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.89100000000000001}]},{"osd":4,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":7.5339999999999998,"15min":3.2320000000000002},"min":{"1min":0.67200000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6200000000000001,"5min":742.45799999999997,"15min":742.45799999999997},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.2110000000000001,"15min":1.2070000000000001},"min":{"1min":0.81699999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5449999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.1970000000000001}]},{"osd":5,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":2.4820000000000002,"15min":1.631},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.464,"5min":152.709,"15min":152.709},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.1919999999999999,"15min":1.214},"min":{"1min":0.80800000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.4039999999999999,"5min":2.181,"15min":2.181},"last":0.96899999999999997}]},{"osd":6,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":4.1260000000000003,"5min":29.439,"15min":41.308},"min":{"1min":0.753,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":73.947000000000003,"5min":874.04999999999995,"15min":874.04999999999995},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.1439999999999999,"15min":1.1819999999999999},"min":{"1min":0.73399999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.508,"5min":4.0439999999999996,"15min":4.0439999999999996},"last":1.0700000000000001}]},{"osd":7,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":20.821999999999999,"5min":21.945,"15min":26.760999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":489.90300000000002,"5min":502.05900000000003,"15min":502.05900000000003},"last":1.214},{"interface":"front","average":{"1min":1.3220000000000001,"5min":23.478000000000002,"15min":33.302999999999997},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":2.0270000000000001,"5min":502.10599999999999,"15min":502.10599999999999},"last":1.2290000000000001}]}]},{"osd":3,"up_from":41,"seq":176093659166,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400584,"kb_used_data":387072,"kb_used_omap":0,"kb_used_meta":13503,"kb_avail":93971256,"statfs":{"total":96636764160,"available":96226566144,"internally_reserved":0,"allocated":396361728,"data_stored":394019686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":972,"internal_metadata":13827124},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":1,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.93000000000000005,"15min":0.94999999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.242,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.84599999999999997,"15min":0.86799999999999999},"min":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90100000000000002,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.85499999999999998}]},{"osd":2,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.83699999999999997,"15min":0.85299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1319999999999999,"5min":1.581,"15min":1.581},"last":0.628},{"interface":"front","average":{"1min":26.884,"5min":6.1639999999999997,"15min":2.71},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":469.85500000000002,"5min":469.85500000000002,"15min":469.85500000000002},"last":0.82099999999999995}]},{"osd":4,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.0760000000000001,"15min":1.0840000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.254,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.23,"15min":1.254},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.4079999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.90800000000000003}]},{"osd":5,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":25.748000000000001,"15min":29.870000000000001},"min":{"1min":0.81799999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.2010000000000001,"5min":524.471,"15min":524.471},"last":0.88},{"interface":"front","average":{"1min":47.161999999999999,"5min":10.489000000000001,"15min":4.3769999999999998},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":830.05200000000002,"5min":830.05200000000002,"15min":830.05200000000002},"last":0.94699999999999995}]},{"osd":6,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.254,"15min":1.258},"min":{"1min":1.046,"5min":1.046,"15min":1.046},"max":{"1min":1.5960000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.181},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.266,"15min":1.268},"min":{"1min":1.071,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.6259999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.204}]},{"osd":7,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.298,"15min":1.3220000000000001},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.3009999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.089},{"interface":"front","average":{"1min":1.159,"5min":1.264,"15min":1.282},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.1599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":444756,"kb_used_data":430292,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":93927084,"statfs":{"total":96636764160,"available":96181334016,"internally_reserved":0,"allocated":440619008,"data_stored":438222994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:57:13 2020","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.298},{"interface":"front","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.3360000000000001}]},{"osd":1,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.119,"15min":1.1479999999999999},"min":{"1min":0.755,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.4039999999999999,"5min":1.661,"15min":1.661},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.022,"15min":1.012},"min":{"1min":0.78600000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4019999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.361}]},{"osd":2,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.161,"15min":1.1870000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.498,"5min":1.776,"15min":1.776},"last":1.282},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0940000000000001,"15min":1.1040000000000001},"min":{"1min":0.84499999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.446,"5min":1.478,"15min":1.478},"last":1.446}]},{"osd":3,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1000000000000001,"15min":1.0920000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.2050000000000001,"15min":1.2210000000000001},"min":{"1min":0.91400000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.222}]},{"osd":5,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.96399999999999997,"15min":0.94599999999999995},"min":{"1min":0.81399999999999995,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3440000000000001,"5min":1.45,"15min":1.45},"last":0.995},{"interface":"front","average":{"1min":1.014,"5min":0.98999999999999999,"15min":0.94899999999999995},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.226,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.26}]},{"osd":6,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.1160000000000001,"15min":1.153},"min":{"1min":0.73499999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.4950000000000001,"5min":1.71,"15min":1.71},"last":1.4199999999999999},{"interface":"front","average":{"1min":18.32,"5min":4.508,"15min":2.214},"min":{"1min":0.90100000000000002,"5min":0.629,"15min":0.629},"max":{"1min":276.41800000000001,"5min":276.41800000000001,"15min":276.41800000000001},"last":1.0489999999999999}]},{"osd":7,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":3.4380000000000002,"15min":1.8959999999999999},"min":{"1min":1.022,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.385,"5min":181.79499999999999,"15min":181.79499999999999},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":5.6840000000000002,"15min":2.7799999999999998},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":1.4510000000000001,"5min":353.72500000000002,"15min":353.72500000000002},"last":1.476}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":354768,"kb_used_data":340820,"kb_used_omap":0,"kb_used_meta":13824,"kb_avail":94017072,"statfs":{"total":96636764160,"available":96273481728,"internally_reserved":0,"allocated":348999680,"data_stored":347497227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14155776},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:12 2020","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.1439999999999999,"15min":1.1599999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.4239999999999999,"5min":1.526,"15min":1.526},"last":1},{"interface":"front","average":{"1min":42.222000000000001,"5min":9.3520000000000003,"15min":3.8740000000000001},"min":{"1min":0.83799999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":821.10900000000004,"5min":821.10900000000004,"15min":821.10900000000004},"last":0.89200000000000002}]},{"osd":1,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0009999999999999,"15min":0.98499999999999999},"min":{"1min":0.79100000000000004,"5min":0.627,"15min":0.627},"max":{"1min":1.2769999999999999,"5min":1.379,"15min":1.379},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":5.5030000000000001,"15min":2.5710000000000002},"min":{"1min":0.84099999999999997,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.3300000000000001,"5min":440.44999999999999,"15min":440.44999999999999},"last":0.91800000000000004}]},{"osd":2,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":7.0289999999999999,"15min":2.9950000000000001},"min":{"1min":0.752,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.294,"5min":605.79899999999998,"15min":605.79899999999998},"last":0.94599999999999995},{"interface":"front","average":{"1min":23.163,"5min":5.524,"15min":2.5800000000000001},"min":{"1min":0.88200000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":501.64600000000002,"5min":501.64600000000002,"15min":501.64600000000002},"last":1.329}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1419999999999999,"15min":1.1339999999999999},"min":{"1min":0.90600000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.411,"5min":1.494,"15min":1.494},"last":1.282},{"interface":"front","average":{"1min":1.159,"5min":17.323,"15min":20.016999999999999},"min":{"1min":0.92700000000000005,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.393,"5min":404.60300000000001,"15min":404.60300000000001},"last":1.0800000000000001}]},{"osd":4,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":21.57,"15min":22.916},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.4079999999999999,"5min":704.28399999999999,"15min":704.28399999999999},"last":1.113},{"interface":"front","average":{"1min":0.92100000000000004,"5min":46.520000000000003,"15min":66.030000000000001},"min":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.0920000000000001,"5min":861.90599999999995,"15min":861.90599999999995},"last":1.0529999999999999}]},{"osd":6,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0700000000000001,"15min":1.0660000000000001},"min":{"1min":0.66300000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.4299999999999999,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.085,"5min":1.1020000000000001,"15min":1.121},"min":{"1min":0.77700000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4410000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.145}]},{"osd":7,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.284,"15min":1.3919999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3899999999999999,"5min":8.3629999999999995,"15min":8.3629999999999995},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":11.603999999999999,"15min":12.991},"min":{"1min":0.79200000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.6000000000000001,"5min":291.65300000000002,"15min":291.65300000000002},"last":1.3089999999999999}]}]}]}} , 2020-12-30T08:59:48.214 INFO:tasks.ceph.ceph_manager.ceph:recovered! , 2020-12-30T08:59:48.215 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2020-12-30T23:00:23.851 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 33, in nested , yield vars , File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-12-30-1123/qa/tasks/ceph.py", line 1876, in task 2020-12-30T23:00:23.852 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T09:00:18.496 DEBUG:teuthology.orchestra.run.smithi169:> mktemp , 2020-12-30T09:00:18.513 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.C79Qha9RXd , 2020-12-30T09:00:18.514 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.C79Qha9RXd 2020-12-30T23:00:23.852 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T09:00:35.626 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- , 2020-12-30T09:00:35.644 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- , 2020-12-30T09:00:35.706 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2020-12-30T23:00:23.903 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f69dd512816] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x55d303af9fe6] 2020-12-30T23:00:23.904 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.563908+0000", , 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.00019472900000000001 , 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.905 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.187571661, , 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "type_data": { , 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "delayed", 2020-12-30T23:00:23.906 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.107 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", , 2020-12-30T08:57:52.108 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.759798+0000", , 2020-12-30T08:57:52.108 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 5.1649e-05 2020-12-30T23:00:23.906 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "initiated", , 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.611817+0000", 2020-12-30T23:00:23.907 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "started", , 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.867908+0000", 2020-12-30T23:00:23.907 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "sub_op_commit_rec", , 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.879692+0000", 2020-12-30T23:00:23.908 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.189 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.190 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.190 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.909 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", 2020-12-30T23:00:23.909 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", 2020-12-30T23:00:23.910 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "reached pg", , 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: "events": [ , 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.910 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.272 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.002481568 , 2020-12-30T08:57:52.272 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.273 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.911 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.292 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 8.7658000000000007e-05 , 2020-12-30T08:57:52.293 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.293 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.911 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.87972017899999999 , 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2020-12-30T23:00:23.912 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "type_data": { , 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "started", , 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "events": [ 2020-12-30T23:00:23.912 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:28.437546+0000", , 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 6.2249999999999997e-06 , 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.913 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 600, , 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: "ops": [ , 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.914 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", , 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.755828+0000", 2020-12-30T23:00:23.914 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "header_read", 2020-12-30T23:00:23.915 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: } 2020-12-30T23:00:23.915 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.820 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4.3782000000000002e-05 , 2020-12-30T08:57:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.916 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:10.493947+0000", , 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4294967295.9999809 , 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.916 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.310989+0000", , 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.012873951999999999 , 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: } 2020-12-30T23:00:23.917 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "queued_for_pg", , 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:17.586655+0000", , 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4.7491000000000003e-05 2020-12-30T23:00:23.918 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", , 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:33.372282+0000", 2020-12-30T23:00:23.918 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:44.022360+0000", , 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "age": 8.7134268660000007, , 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.346765198, 2020-12-30T23:00:23.919 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "started", 2020-12-30T23:00:23.919 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.963 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.0046062250000000002 , 2020-12-30T08:57:52.963 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.964 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.920 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.983 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.984 INFO:teuthology.orchestra.run.smithi169.stdout:} , 2020-12-30T08:57:52.985 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-12-30T23:00:23.920 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", 2020-12-30T23:00:23.921 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.403 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0 , 2020-12-30T08:57:53.404 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.404 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.921 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: "description": "osd_op(client.4598.0:1164 4.0 4:249a18ca:::smithi04340772-260:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e34)", , 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:04.096815+0000", 2020-12-30T23:00:23.922 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:09.599534+0000", , 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.036992197999999997 , 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.922 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.470 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", , 2020-12-30T08:57:53.470 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.144872+0000", , 2020-12-30T08:57:53.471 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 1.0566e-05 2020-12-30T23:00:23.923 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "initiated", , 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.156798+0000", 2020-12-30T23:00:23.924 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.513 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:53.513 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:53.514 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.924 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.535 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.535 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.536 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "queued_for_pg", 2020-12-30T23:00:23.925 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4294967295.9999795 , 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:23.925 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: "description": "osd_op(client.4598.0:2898 4.0 4:1f210125:::smithi04340772-284:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e38)", , 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:17.505116+0000", 2020-12-30T23:00:23.926 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.600 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:22.235621+0000", , 2020-12-30T08:57:53.600 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.035873975000000002 , 2020-12-30T08:57:53.601 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:23.926 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3320 , 2020-12-30T08:57:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.027 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2020-12-30T23:00:23.927 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:23.886767+0000", , 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8839999999999997e-06 , 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:23.927 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:32.913785+0000", , 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049845687999999999 , 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:23.928 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", , 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:27.332932+0000", , 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2020-12-30T23:00:23.928 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", , 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:27.333311+0000", , 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1931999999999999e-05 2020-12-30T23:00:23.929 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", , 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:28.423645+0000", 2020-12-30T23:00:23.930 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for readable", , 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:28.440591+0000", 2020-12-30T23:00:23.930 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { , 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4598", , 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "v1:172.21.15.43:0/972417042", 2020-12-30T23:00:23.931 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2020-12-30T23:00:23.931 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 , 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2020-12-30T23:00:23.932 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { , 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", , 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2020-12-30T23:00:23.932 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.249599+0000", , 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4746000000000001e-05 , 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:23.933 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.636403+0000", , 2020-12-30T08:57:54.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0044951799999999997 , 2020-12-30T08:57:54.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:23.934 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.655240+0000", , 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047171491000000003 , 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2020-12-30T23:00:23.935 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:58:15.685 INFO:tasks.ceph.osd.0:Stopped , 2020-12-30T08:58:15.685 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 , osd.0 is already down. 2020-12-30T23:00:23.936 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:59:26.097 INFO:teuthology.orchestra.run.smithi043.stdout:mon.a: {} , 2020-12-30T08:59:26.113 INFO:teuthology.orchestra.run.smithi043.stdout:mon.b: {} , 2020-12-30T08:59:26.127 INFO:teuthology.orchestra.run.smithi043.stdout:mon.c: {} 2020-12-30T23:00:23.938 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:59:48.212 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2020-12-30T08:59:47.527120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1407295001,"num_objects":477,"num_object_clones":0,"num_object_copies":1431,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":449,"num_whiteouts":3,"num_read":3286,"num_read_kb":2359369,"num_write":3461,"num_write_kb":1206573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":773,"num_bytes_recovered":1169499301,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":887,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":44,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2967,"ondisk_log_size":2967,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2702316,"kb_used_data":2595628,"kb_used_omap":11,"kb_used_meta":105780,"kb_avail":752272404,"statfs":{"total":773094113280,"available":770326941696,"internally_reserved":0,"allocated":2657923072,"data_stored":2643000988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11269,"internal_metadata":108319739},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.529700"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":"111","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252322+0000","last_change":"2020-12-30T08:59:26.805818+0000","last_active":"2020-12-30T08:59:40.252322+0000","last_peered":"2020-12-30T08:59:40.252322+0000","last_clean":"2020-12-30T08:59:40.252322+0000","last_became_active":"2020-12-30T08:59:26.805280+0000","last_became_peered":"2020-12-30T08:59:26.805280+0000","last_unstale":"2020-12-30T08:59:40.252322+0000","last_undegraded":"2020-12-30T08:59:40.252322+0000","last_fullsized":"2020-12-30T08:59:40.252322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:04.206849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:04.206849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"57'44","reported_seq":"156","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252340+0000","last_change":"2020-12-30T08:59:09.166665+0000","last_active":"2020-12-30T08:59:40.252340+0000","last_peered":"2020-12-30T08:59:40.252340+0000","last_clean":"2020-12-30T08:59:40.252340+0000","last_became_active":"2020-12-30T08:56:39.905257+0000","last_became_peered":"2020-12-30T08:56:39.905257+0000","last_unstale":"2020-12-30T08:59:40.252340+0000","last_undegraded":"2020-12-30T08:59:40.252340+0000","last_fullsized":"2020-12-30T08:59:40.252340+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'44","last_scrub_stamp":"2020-12-30T08:59:09.166549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:09.166549+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63684286,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":42,"num_read_kb":51083,"num_write":29,"num_write_kb":4590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":319,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":25,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.578479+0000","last_change":"2020-12-30T08:58:46.630261+0000","last_active":"2020-12-30T08:59:40.578479+0000","last_peered":"2020-12-30T08:59:40.578479+0000","last_clean":"2020-12-30T08:59:40.578479+0000","last_became_active":"2020-12-30T08:56:17.453297+0000","last_became_peered":"2020-12-30T08:56:17.453297+0000","last_unstale":"2020-12-30T08:59:40.578479+0000","last_undegraded":"2020-12-30T08:59:40.578479+0000","last_fullsized":"2020-12-30T08:59:40.578479+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:46.630148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:46.630148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"57'794","reported_seq":"1885","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.596274+0000","last_change":"2020-12-30T08:59:31.962315+0000","last_active":"2020-12-30T08:59:41.596274+0000","last_peered":"2020-12-30T08:59:41.596274+0000","last_clean":"2020-12-30T08:59:41.596274+0000","last_became_active":"2020-12-30T08:59:31.961183+0000","last_became_peered":"2020-12-30T08:59:31.961183+0000","last_unstale":"2020-12-30T08:59:41.596274+0000","last_undegraded":"2020-12-30T08:59:41.596274+0000","last_fullsized":"2020-12-30T08:59:41.596274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'794","last_scrub_stamp":"2020-12-30T08:59:09.405174+0000","last_deep_scrub":"54'709","last_deep_scrub_stamp":"2020-12-30T08:57:41.784481+0000","last_clean_scrub_stamp":"2020-12-30T08:59:09.405174+0000","log_size":794,"ondisk_log_size":794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":324600875,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":956,"num_read_kb":765811,"num_write":895,"num_write_kb":309669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":381323268,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"125","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.432965+0000","last_change":"2020-12-30T08:59:26.807827+0000","last_active":"2020-12-30T08:59:40.432965+0000","last_peered":"2020-12-30T08:59:40.432965+0000","last_clean":"2020-12-30T08:59:40.432965+0000","last_became_active":"2020-12-30T08:59:26.807377+0000","last_became_peered":"2020-12-30T08:59:26.807377+0000","last_unstale":"2020-12-30T08:59:40.432965+0000","last_undegraded":"2020-12-30T08:59:40.432965+0000","last_fullsized":"2020-12-30T08:59:40.432965+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.818206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.818206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"57'598","reported_seq":"1017","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252337+0000","last_change":"2020-12-30T08:59:08.196312+0000","last_active":"2020-12-30T08:59:40.252337+0000","last_peered":"2020-12-30T08:59:40.252337+0000","last_clean":"2020-12-30T08:59:40.252337+0000","last_became_active":"2020-12-30T08:56:35.883250+0000","last_became_peered":"2020-12-30T08:56:35.883250+0000","last_unstale":"2020-12-30T08:59:40.252337+0000","last_undegraded":"2020-12-30T08:59:40.252337+0000","last_fullsized":"2020-12-30T08:59:40.252337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'598","last_scrub_stamp":"2020-12-30T08:59:08.196199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:34.853614+0000","last_clean_scrub_stamp":"2020-12-30T08:59:08.196199+0000","log_size":598,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":299758950,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":689,"num_read_kb":446464,"num_write":755,"num_write_kb":264543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.578494+0000","last_change":"2020-12-30T08:58:53.577047+0000","last_active":"2020-12-30T08:59:40.578494+0000","last_peered":"2020-12-30T08:59:40.578494+0000","last_clean":"2020-12-30T08:59:40.578494+0000","last_became_active":"2020-12-30T08:56:18.363833+0000","last_became_peered":"2020-12-30T08:56:18.363833+0000","last_unstale":"2020-12-30T08:59:40.578494+0000","last_undegraded":"2020-12-30T08:59:40.578494+0000","last_fullsized":"2020-12-30T08:59:40.578494+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:53.576887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:53.576887+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":"145","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001415+0000","last_change":"2020-12-30T08:59:29.950684+0000","last_active":"2020-12-30T08:59:41.001415+0000","last_peered":"2020-12-30T08:59:41.001415+0000","last_clean":"2020-12-30T08:59:41.001415+0000","last_became_active":"2020-12-30T08:59:29.950219+0000","last_became_peered":"2020-12-30T08:59:29.950219+0000","last_unstale":"2020-12-30T08:59:41.001415+0000","last_undegraded":"2020-12-30T08:59:41.001415+0000","last_fullsized":"2020-12-30T08:59:41.001415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":4,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'2","last_scrub_stamp":"2020-12-30T08:58:43.023209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:05.570594+0000","last_clean_scrub_stamp":"2020-12-30T08:58:43.023209+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"57'737","reported_seq":"1556","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.577945+0000","last_change":"2020-12-30T08:59:27.273198+0000","last_active":"2020-12-30T08:59:40.577945+0000","last_peered":"2020-12-30T08:59:40.577945+0000","last_clean":"2020-12-30T08:59:40.577945+0000","last_became_active":"2020-12-30T08:57:34.885212+0000","last_became_peered":"2020-12-30T08:57:34.885212+0000","last_unstale":"2020-12-30T08:59:40.577945+0000","last_undegraded":"2020-12-30T08:59:40.577945+0000","last_fullsized":"2020-12-30T08:59:40.577945+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'737","last_scrub_stamp":"2020-12-30T08:59:27.273101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:34.853614+0000","last_clean_scrub_stamp":"2020-12-30T08:59:27.273101+0000","log_size":737,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":370415644,"num_objects":124,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":0,"num_read":829,"num_read_kb":576628,"num_write":921,"num_write_kb":353530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":210025962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"112","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001472+0000","last_change":"2020-12-30T08:59:26.807466+0000","last_active":"2020-12-30T08:59:41.001472+0000","last_peered":"2020-12-30T08:59:41.001472+0000","last_clean":"2020-12-30T08:59:41.001472+0000","last_became_active":"2020-12-30T08:59:26.806602+0000","last_became_peered":"2020-12-30T08:59:26.806602+0000","last_unstale":"2020-12-30T08:59:41.001472+0000","last_undegraded":"2020-12-30T08:59:41.001472+0000","last_fullsized":"2020-12-30T08:59:41.001472+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:06.756813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:06.756813+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"76'751","reported_seq":"1682","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.000117+0000","last_change":"2020-12-30T08:59:30.952423+0000","last_active":"2020-12-30T08:59:41.000117+0000","last_peered":"2020-12-30T08:59:41.000117+0000","last_clean":"2020-12-30T08:59:41.000117+0000","last_became_active":"2020-12-30T08:59:30.951003+0000","last_became_peered":"2020-12-30T08:59:30.951003+0000","last_unstale":"2020-12-30T08:59:41.000117+0000","last_undegraded":"2020-12-30T08:59:41.000117+0000","last_fullsized":"2020-12-30T08:59:41.000117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'751","last_scrub_stamp":"2020-12-30T08:59:17.342114+0000","last_deep_scrub":"76'751","last_deep_scrub_stamp":"2020-12-30T08:59:17.342114+0000","last_clean_scrub_stamp":"2020-12-30T08:59:17.342114+0000","log_size":751,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":288680016,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":742,"num_read_kb":486335,"num_write":828,"num_write_kb":269542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":495679005,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"57'31","reported_seq":"176","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.000230+0000","last_change":"2020-12-30T08:59:30.953648+0000","last_active":"2020-12-30T08:59:41.000230+0000","last_peered":"2020-12-30T08:59:41.000230+0000","last_clean":"2020-12-30T08:59:41.000230+0000","last_became_active":"2020-12-30T08:59:30.952821+0000","last_became_peered":"2020-12-30T08:59:30.952821+0000","last_unstale":"2020-12-30T08:59:41.000230+0000","last_undegraded":"2020-12-30T08:59:41.000230+0000","last_fullsized":"2020-12-30T08:59:41.000230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'31","last_scrub_stamp":"2020-12-30T08:59:18.654169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:18.654169+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41801084,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":14,"num_read_kb":4410,"num_write":24,"num_write_kb":3616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":82471066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":293,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":14,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"115","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001437+0000","last_change":"2020-12-30T08:59:26.807130+0000","last_active":"2020-12-30T08:59:41.001437+0000","last_peered":"2020-12-30T08:59:41.001437+0000","last_clean":"2020-12-30T08:59:41.001437+0000","last_became_active":"2020-12-30T08:59:26.806409+0000","last_became_peered":"2020-12-30T08:59:26.806409+0000","last_unstale":"2020-12-30T08:59:41.001437+0000","last_undegraded":"2020-12-30T08:59:41.001437+0000","last_fullsized":"2020-12-30T08:59:41.001437+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.095702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.095702+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252325+0000","last_change":"2020-12-30T08:58:49.511387+0000","last_active":"2020-12-30T08:59:40.252325+0000","last_peered":"2020-12-30T08:59:40.252325+0000","last_clean":"2020-12-30T08:59:40.252325+0000","last_became_active":"2020-12-30T08:56:18.101527+0000","last_became_peered":"2020-12-30T08:56:18.101527+0000","last_unstale":"2020-12-30T08:59:40.252325+0000","last_undegraded":"2020-12-30T08:59:40.252325+0000","last_fullsized":"2020-12-30T08:59:40.252325+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:49.511270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:49.511270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"99","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.577925+0000","last_change":"2020-12-30T08:59:26.808227+0000","last_active":"2020-12-30T08:59:40.577925+0000","last_peered":"2020-12-30T08:59:40.577925+0000","last_clean":"2020-12-30T08:59:40.577925+0000","last_became_active":"2020-12-30T08:59:26.807467+0000","last_became_peered":"2020-12-30T08:59:26.807467+0000","last_unstale":"2020-12-30T08:59:40.577925+0000","last_undegraded":"2020-12-30T08:59:40.577925+0000","last_fullsized":"2020-12-30T08:59:40.577925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.951459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.951459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"92","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.310721+0000","last_change":"2020-12-30T08:58:34.893621+0000","last_active":"2020-12-30T08:59:40.310721+0000","last_peered":"2020-12-30T08:59:40.310721+0000","last_clean":"2020-12-30T08:59:40.310721+0000","last_became_active":"2020-12-30T08:56:17.453289+0000","last_became_peered":"2020-12-30T08:56:17.453289+0000","last_unstale":"2020-12-30T08:59:40.310721+0000","last_undegraded":"2020-12-30T08:59:40.310721+0000","last_fullsized":"2020-12-30T08:59:40.310721+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:34.893489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:34.893489+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"57'10","reported_seq":"102","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.596156+0000","last_change":"2020-12-30T08:58:50.189729+0000","last_active":"2020-12-30T08:59:41.596156+0000","last_peered":"2020-12-30T08:59:41.596156+0000","last_clean":"2020-12-30T08:59:41.596156+0000","last_became_active":"2020-12-30T08:56:40.286191+0000","last_became_peered":"2020-12-30T08:56:40.286191+0000","last_unstale":"2020-12-30T08:59:41.596156+0000","last_undegraded":"2020-12-30T08:59:41.596156+0000","last_fullsized":"2020-12-30T08:59:41.596156+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'10","last_scrub_stamp":"2020-12-30T08:58:50.189625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:58:50.189625+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18354146,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":28638,"num_write":7,"num_write_kb":1074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":275,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":5,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":4,"stat_sum":{"num_bytes":123839516,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":3,"num_read":70,"num_read_kb":84131,"num_write":60,"num_write_kb":9280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":82471066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":887,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":44,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":363405312,"data_stored":363060657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":12,"acting":12,"num_store_stats":7},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":1283455485,"num_objects":428,"num_object_clones":0,"num_object_copies":1284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":428,"num_whiteouts":0,"num_read":3216,"num_read_kb":2275238,"num_write":3399,"num_write_kb":1197284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":739,"num_bytes_recovered":1087028235,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2289242112,"data_stored":2277676566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2880,"ondisk_log_size":2880,"up":12,"acting":12,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10297,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46828,"kb_used_data":40940,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":94325012,"statfs":{"total":96636764160,"available":96588812288,"internally_reserved":0,"allocated":41922560,"data_stored":41513062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:57:16 2020","interfaces":[{"interface":"back","average":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":81.242000000000004,"5min":81.242000000000004,"15min":81.242000000000004},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1.0409999999999999}]},{"osd":1,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":19.870000000000001,"15min":28.206},"min":{"1min":0.83199999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4970000000000001,"5min":501.495,"15min":501.495},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":26.629000000000001,"15min":25.437000000000001},"min":{"1min":0.73399999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.5720000000000001,"5min":958.63099999999997,"15min":958.63099999999997},"last":1.2589999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":59.027999999999999,"5min":12.720000000000001,"15min":5.008},"min":{"1min":0.85799999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":927.80799999999999,"5min":927.80799999999999,"15min":927.80799999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":21.974,"5min":12.337999999999999,"15min":4.8849999999999998},"min":{"1min":0.98799999999999999,"5min":0.753,"15min":0.753},"max":{"1min":333.58300000000003,"5min":597.27800000000002,"15min":597.27800000000002},"last":1.093}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.202,"15min":1.216},"min":{"1min":0.81399999999999995,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.45,"5min":1.474,"15min":1.474},"last":1.2310000000000001},{"interface":"front","average":{"1min":8.3640000000000008,"5min":2.5859999999999999,"15min":1.623},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":123.247,"5min":123.247,"15min":123.247},"last":0.84299999999999997}]},{"osd":4,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":26.962,"15min":38.468000000000004},"min":{"1min":0.81499999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5609999999999999,"5min":747.37,"15min":747.37},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":1.052,"15min":1.0920000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.486,"5min":1.623,"15min":1.623},"last":1.1080000000000001}]},{"osd":5,"last update":"Wed Dec 30 08:59:24 2020","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1379999999999999,"15min":1.125},"min":{"1min":0.76500000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.6160000000000001,"5min":1.7,"15min":1.7},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.1879999999999999,"15min":1.2390000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5469999999999999,"5min":2.198,"15min":2.198},"last":1.198}]},{"osd":6,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":8.0190000000000001,"5min":27.074000000000002,"15min":37.076000000000001},"min":{"1min":0.77300000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":106.654,"5min":861.60799999999995,"15min":861.60799999999995},"last":1.177},{"interface":"front","average":{"1min":52.908999999999999,"5min":11.640000000000001,"15min":4.7830000000000004},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":778.01900000000001,"5min":778.01900000000001,"15min":778.01900000000001},"last":1.3280000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617820,"kb_used_data":597728,"kb_used_omap":0,"kb_used_meta":19968,"kb_avail":93754020,"statfs":{"total":96636764160,"available":96004116480,"internally_reserved":0,"allocated":612073472,"data_stored":608728491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20447232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:15 2020","interfaces":[{"interface":"back","average":{"1min":63.369,"5min":56.469000000000001,"15min":55.319000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":999.76300000000003,"5min":999.76300000000003,"15min":999.76300000000003},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.04,"15min":1.028},"min":{"1min":0.746,"5min":0.62,"15min":0.62},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":1.3069999999999999}]},{"osd":1,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.01,"15min":0.97699999999999998},"min":{"1min":0.79400000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.3480000000000001,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.875},{"interface":"front","average":{"1min":0.96799999999999997,"5min":36.709000000000003,"15min":52.585000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.3009999999999999,"5min":953.40800000000002,"15min":953.40800000000002},"last":1.1899999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":46.109999999999999,"5min":10.127000000000001,"15min":4.109},"min":{"1min":0.98899999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":714.93299999999999,"5min":714.93299999999999,"15min":714.93299999999999},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.206,"15min":1.1990000000000001},"min":{"1min":1.0309999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.4870000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.6060000000000001}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.151,"5min":1.139,"15min":1.137},"min":{"1min":0.94599999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.3919999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.274}]},{"osd":4,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.96199999999999997,"15min":0.97899999999999998},"min":{"1min":0.63600000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.393,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0740000000000001,"15min":1.0860000000000001},"min":{"1min":0.81299999999999994,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":1.375}]},{"osd":5,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.1020000000000001,"15min":1.123},"min":{"1min":0.77300000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.286,"5min":1.637,"15min":1.637},"last":1.3540000000000001},{"interface":"front","average":{"1min":33.957000000000001,"5min":47.049999999999997,"15min":60.192999999999998},"min":{"1min":0.81299999999999994,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":691.76199999999994,"5min":985.03399999999999,"15min":985.03399999999999},"last":1.3080000000000001}]},{"osd":7,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":21.876000000000001,"5min":5.3460000000000001,"15min":2.5979999999999999},"min":{"1min":0.86099999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":435.39499999999998,"5min":435.39499999999998,"15min":435.39499999999998},"last":1.331},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2330000000000001,"15min":1.2490000000000001},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.5740000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":247340,"kb_used_data":233900,"kb_used_omap":0,"kb_used_meta":13312,"kb_avail":94124500,"statfs":{"total":96636764160,"available":96383488000,"internally_reserved":0,"allocated":239513600,"data_stored":238202894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13631488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:13 2020","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.0569999999999999,"15min":1.0880000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2869999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77500000000000002,"15min":0.77600000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.74399999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":27.254000000000001,"15min":39.072000000000003},"min":{"1min":0.47199999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84799999999999998,"5min":882.37400000000002,"15min":882.37400000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.90400000000000003,"15min":0.92200000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.298,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed Dec 30 08:59:31 2020","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.95299999999999996,"15min":0.95699999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.29,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.86899999999999999,"15min":0.88200000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.302,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.80500000000000005}]},{"osd":4,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1339999999999999,"15min":1.1040000000000001},"min":{"1min":0.92200000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.3899999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.1599999999999999,"15min":1.169},"min":{"1min":0.73299999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.46,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.84699999999999998}]},{"osd":5,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2649999999999999,"15min":1.2829999999999999},"min":{"1min":0.89100000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.4159999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2390000000000001,"15min":1.2470000000000001},"min":{"1min":1.002,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.383,"5min":1.96,"15min":1.96},"last":1.0229999999999999}]},{"osd":6,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2390000000000001,"15min":1.2749999999999999},"min":{"1min":0.82299999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4259999999999999,"5min":4.4829999999999997,"15min":4.4829999999999997},"last":1.149},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.242,"15min":1.272},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.4159999999999999,"5min":1.734,"15min":1.734},"last":1.0069999999999999}]},{"osd":7,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":19.978999999999999,"15min":28.309000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.486,"5min":623.82100000000003,"15min":623.82100000000003},"last":828.46799999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.2250000000000001,"15min":1.2350000000000001},"min":{"1min":0.91700000000000004,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.3500000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.98699999999999999}]}]},{"osd":0,"up_from":75,"seq":322122547208,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410660,"kb_used_data":392612,"kb_used_omap":10,"kb_used_meta":17909,"kb_avail":93961180,"statfs":{"total":96636764160,"available":96216248320,"internally_reserved":0,"allocated":402034688,"data_stored":399673841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10297,"internal_metadata":18339783},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.649}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.679}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":179560,"kb_used_data":172264,"kb_used_omap":0,"kb_used_meta":7168,"kb_avail":94192280,"statfs":{"total":96636764160,"available":96452894720,"internally_reserved":0,"allocated":176398336,"data_stored":175142793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7340032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:15 2020","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.81999999999999995,"15min":0.81399999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.999,"5min":33.204000000000001,"15min":38.572000000000003},"min":{"1min":0.63500000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.8049999999999999,"5min":767.24300000000005,"15min":767.24300000000005},"last":0.67100000000000004}]},{"osd":1,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":30.774000000000001,"5min":7.2549999999999999,"15min":3.4510000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":754.12800000000004,"5min":754.12800000000004,"15min":754.12800000000004},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.91400000000000003,"15min":0.91100000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4430000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.80300000000000005}]},{"osd":3,"last update":"Wed Dec 30 08:59:29 2020","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.0629999999999999,"15min":1.0900000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.476,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.747,"15min":0.747},"min":{"1min":0.55500000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.89100000000000001}]},{"osd":4,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":7.5339999999999998,"15min":3.2320000000000002},"min":{"1min":0.67200000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6200000000000001,"5min":742.45799999999997,"15min":742.45799999999997},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.2110000000000001,"15min":1.2070000000000001},"min":{"1min":0.81699999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5449999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.1970000000000001}]},{"osd":5,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":2.4820000000000002,"15min":1.631},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.464,"5min":152.709,"15min":152.709},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.1919999999999999,"15min":1.214},"min":{"1min":0.80800000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.4039999999999999,"5min":2.181,"15min":2.181},"last":0.96899999999999997}]},{"osd":6,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":4.1260000000000003,"5min":29.439,"15min":41.308},"min":{"1min":0.753,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":73.947000000000003,"5min":874.04999999999995,"15min":874.04999999999995},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.1439999999999999,"15min":1.1819999999999999},"min":{"1min":0.73399999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.508,"5min":4.0439999999999996,"15min":4.0439999999999996},"last":1.0700000000000001}]},{"osd":7,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":20.821999999999999,"5min":21.945,"15min":26.760999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":489.90300000000002,"5min":502.05900000000003,"15min":502.05900000000003},"last":1.214},{"interface":"front","average":{"1min":1.3220000000000001,"5min":23.478000000000002,"15min":33.302999999999997},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":2.0270000000000001,"5min":502.10599999999999,"15min":502.10599999999999},"last":1.2290000000000001}]}]},{"osd":3,"up_from":41,"seq":176093659166,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400584,"kb_used_data":387072,"kb_used_omap":0,"kb_used_meta":13503,"kb_avail":93971256,"statfs":{"total":96636764160,"available":96226566144,"internally_reserved":0,"allocated":396361728,"data_stored":394019686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":972,"internal_metadata":13827124},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":1,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.93000000000000005,"15min":0.94999999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.242,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.84599999999999997,"15min":0.86799999999999999},"min":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90100000000000002,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.85499999999999998}]},{"osd":2,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.83699999999999997,"15min":0.85299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1319999999999999,"5min":1.581,"15min":1.581},"last":0.628},{"interface":"front","average":{"1min":26.884,"5min":6.1639999999999997,"15min":2.71},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":469.85500000000002,"5min":469.85500000000002,"15min":469.85500000000002},"last":0.82099999999999995}]},{"osd":4,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.0760000000000001,"15min":1.0840000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.254,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.23,"15min":1.254},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.4079999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.90800000000000003}]},{"osd":5,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":25.748000000000001,"15min":29.870000000000001},"min":{"1min":0.81799999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.2010000000000001,"5min":524.471,"15min":524.471},"last":0.88},{"interface":"front","average":{"1min":47.161999999999999,"5min":10.489000000000001,"15min":4.3769999999999998},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":830.05200000000002,"5min":830.05200000000002,"15min":830.05200000000002},"last":0.94699999999999995}]},{"osd":6,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.254,"15min":1.258},"min":{"1min":1.046,"5min":1.046,"15min":1.046},"max":{"1min":1.5960000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.181},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.266,"15min":1.268},"min":{"1min":1.071,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.6259999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.204}]},{"osd":7,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.298,"15min":1.3220000000000001},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.3009999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.089},{"interface":"front","average":{"1min":1.159,"5min":1.264,"15min":1.282},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.1599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":444756,"kb_used_data":430292,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":93927084,"statfs":{"total":96636764160,"available":96181334016,"internally_reserved":0,"allocated":440619008,"data_stored":438222994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:57:13 2020","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.298},{"interface":"front","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.3360000000000001}]},{"osd":1,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.119,"15min":1.1479999999999999},"min":{"1min":0.755,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.4039999999999999,"5min":1.661,"15min":1.661},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.022,"15min":1.012},"min":{"1min":0.78600000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4019999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.361}]},{"osd":2,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.161,"15min":1.1870000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.498,"5min":1.776,"15min":1.776},"last":1.282},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0940000000000001,"15min":1.1040000000000001},"min":{"1min":0.84499999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.446,"5min":1.478,"15min":1.478},"last":1.446}]},{"osd":3,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1000000000000001,"15min":1.0920000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.2050000000000001,"15min":1.2210000000000001},"min":{"1min":0.91400000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.222}]},{"osd":5,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.96399999999999997,"15min":0.94599999999999995},"min":{"1min":0.81399999999999995,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3440000000000001,"5min":1.45,"15min":1.45},"last":0.995},{"interface":"front","average":{"1min":1.014,"5min":0.98999999999999999,"15min":0.94899999999999995},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.226,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.26}]},{"osd":6,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.1160000000000001,"15min":1.153},"min":{"1min":0.73499999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.4950000000000001,"5min":1.71,"15min":1.71},"last":1.4199999999999999},{"interface":"front","average":{"1min":18.32,"5min":4.508,"15min":2.214},"min":{"1min":0.90100000000000002,"5min":0.629,"15min":0.629},"max":{"1min":276.41800000000001,"5min":276.41800000000001,"15min":276.41800000000001},"last":1.0489999999999999}]},{"osd":7,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":3.4380000000000002,"15min":1.8959999999999999},"min":{"1min":1.022,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.385,"5min":181.79499999999999,"15min":181.79499999999999},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":5.6840000000000002,"15min":2.7799999999999998},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":1.4510000000000001,"5min":353.72500000000002,"15min":353.72500000000002},"last":1.476}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":354768,"kb_used_data":340820,"kb_used_omap":0,"kb_used_meta":13824,"kb_avail":94017072,"statfs":{"total":96636764160,"available":96273481728,"internally_reserved":0,"allocated":348999680,"data_stored":347497227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14155776},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:12 2020","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.1439999999999999,"15min":1.1599999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.4239999999999999,"5min":1.526,"15min":1.526},"last":1},{"interface":"front","average":{"1min":42.222000000000001,"5min":9.3520000000000003,"15min":3.8740000000000001},"min":{"1min":0.83799999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":821.10900000000004,"5min":821.10900000000004,"15min":821.10900000000004},"last":0.89200000000000002}]},{"osd":1,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0009999999999999,"15min":0.98499999999999999},"min":{"1min":0.79100000000000004,"5min":0.627,"15min":0.627},"max":{"1min":1.2769999999999999,"5min":1.379,"15min":1.379},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":5.5030000000000001,"15min":2.5710000000000002},"min":{"1min":0.84099999999999997,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.3300000000000001,"5min":440.44999999999999,"15min":440.44999999999999},"last":0.91800000000000004}]},{"osd":2,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":7.0289999999999999,"15min":2.9950000000000001},"min":{"1min":0.752,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.294,"5min":605.79899999999998,"15min":605.79899999999998},"last":0.94599999999999995},{"interface":"front","average":{"1min":23.163,"5min":5.524,"15min":2.5800000000000001},"min":{"1min":0.88200000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":501.64600000000002,"5min":501.64600000000002,"15min":501.64600000000002},"last":1.329}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1419999999999999,"15min":1.1339999999999999},"min":{"1min":0.90600000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.411,"5min":1.494,"15min":1.494},"last":1.282},{"interface":"front","average":{"1min":1.159,"5min":17.323,"15min":20.016999999999999},"min":{"1min":0.92700000000000005,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.393,"5min":404.60300000000001,"15min":404.60300000000001},"last":1.0800000000000001}]},{"osd":4,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":21.57,"15min":22.916},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.4079999999999999,"5min":704.28399999999999,"15min":704.28399999999999},"last":1.113},{"interface":"front","average":{"1min":0.92100000000000004,"5min":46.520000000000003,"15min":66.030000000000001},"min":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.0920000000000001,"5min":861.90599999999995,"15min":861.90599999999995},"last":1.0529999999999999}]},{"osd":6,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0700000000000001,"15min":1.0660000000000001},"min":{"1min":0.66300000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.4299999999999999,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.085,"5min":1.1020000000000001,"15min":1.121},"min":{"1min":0.77700000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4410000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.145}]},{"osd":7,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.284,"15min":1.3919999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3899999999999999,"5min":8.3629999999999995,"15min":8.3629999999999995},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":11.603999999999999,"15min":12.991},"min":{"1min":0.79200000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.6000000000000001,"5min":291.65300000000002,"15min":291.65300000000002},"last":1.3089999999999999}]}]}]}} , 2020-12-30T08:59:48.214 INFO:tasks.ceph.ceph_manager.ceph:recovered! , 2020-12-30T08:59:48.215 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2020-12-30T23:00:23.940 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 33, in nested , yield vars , File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-12-30-1123/qa/tasks/ceph.py", line 1876, in task 2020-12-30T23:00:23.940 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T09:00:18.496 DEBUG:teuthology.orchestra.run.smithi169:> mktemp , 2020-12-30T09:00:18.513 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.C79Qha9RXd , 2020-12-30T09:00:18.514 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.C79Qha9RXd 2020-12-30T23:00:23.941 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T09:00:35.626 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- , 2020-12-30T09:00:35.644 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- , 2020-12-30T09:00:35.706 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2020-12-30T23:00:24.005 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f69dd512816] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x55d303af9fe6] 2020-12-30T23:00:24.006 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.563908+0000", , 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.00019472900000000001 , 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.007 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.187571661, , 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "type_data": { , 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "delayed", 2020-12-30T23:00:24.008 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.107 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", , 2020-12-30T08:57:52.108 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.759798+0000", , 2020-12-30T08:57:52.108 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 5.1649e-05 2020-12-30T23:00:24.008 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "initiated", , 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.611817+0000", 2020-12-30T23:00:24.009 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "started", , 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.867908+0000", 2020-12-30T23:00:24.010 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "sub_op_commit_rec", , 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.879692+0000", 2020-12-30T23:00:24.010 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.189 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.190 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.190 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.011 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", 2020-12-30T23:00:24.011 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", 2020-12-30T23:00:24.012 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "reached pg", , 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: "events": [ , 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.012 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.272 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.002481568 , 2020-12-30T08:57:52.272 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.273 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.013 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.292 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 8.7658000000000007e-05 , 2020-12-30T08:57:52.293 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.293 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.013 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.87972017899999999 , 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2020-12-30T23:00:24.014 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "type_data": { , 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "started", , 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "events": [ 2020-12-30T23:00:24.015 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:28.437546+0000", , 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 6.2249999999999997e-06 , 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.015 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 600, , 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: "ops": [ , 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.016 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", , 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.755828+0000", 2020-12-30T23:00:24.016 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "header_read", 2020-12-30T23:00:24.017 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: } 2020-12-30T23:00:24.018 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.820 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4.3782000000000002e-05 , 2020-12-30T08:57:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.018 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:10.493947+0000", , 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4294967295.9999809 , 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.019 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.310989+0000", , 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.012873951999999999 , 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: } 2020-12-30T23:00:24.019 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "queued_for_pg", , 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:17.586655+0000", , 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4.7491000000000003e-05 2020-12-30T23:00:24.020 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", , 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:33.372282+0000", 2020-12-30T23:00:24.020 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:44.022360+0000", , 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "age": 8.7134268660000007, , 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.346765198, 2020-12-30T23:00:24.021 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "started", 2020-12-30T23:00:24.021 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.963 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.0046062250000000002 , 2020-12-30T08:57:52.963 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.964 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.022 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.983 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.984 INFO:teuthology.orchestra.run.smithi169.stdout:} , 2020-12-30T08:57:52.985 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-12-30T23:00:24.023 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", 2020-12-30T23:00:24.023 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.403 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0 , 2020-12-30T08:57:53.404 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.404 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.024 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: "description": "osd_op(client.4598.0:1164 4.0 4:249a18ca:::smithi04340772-260:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e34)", , 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:04.096815+0000", 2020-12-30T23:00:24.024 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:09.599534+0000", , 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.036992197999999997 , 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.025 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.470 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", , 2020-12-30T08:57:53.470 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.144872+0000", , 2020-12-30T08:57:53.471 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 1.0566e-05 2020-12-30T23:00:24.025 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "initiated", , 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.156798+0000", 2020-12-30T23:00:24.026 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.513 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:53.513 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:53.514 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.027 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.535 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.535 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.536 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "queued_for_pg", 2020-12-30T23:00:24.027 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4294967295.9999795 , 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.028 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: "description": "osd_op(client.4598.0:2898 4.0 4:1f210125:::smithi04340772-284:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e38)", , 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:17.505116+0000", 2020-12-30T23:00:24.028 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.600 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:22.235621+0000", , 2020-12-30T08:57:53.600 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.035873975000000002 , 2020-12-30T08:57:53.601 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.029 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3320 , 2020-12-30T08:57:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.027 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2020-12-30T23:00:24.029 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:23.886767+0000", , 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8839999999999997e-06 , 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:24.030 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:32.913785+0000", , 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049845687999999999 , 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:24.031 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", , 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:27.332932+0000", , 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2020-12-30T23:00:24.031 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", , 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:27.333311+0000", , 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1931999999999999e-05 2020-12-30T23:00:24.032 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", , 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:28.423645+0000", 2020-12-30T23:00:24.032 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for readable", , 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:28.440591+0000", 2020-12-30T23:00:24.033 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { , 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4598", , 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "v1:172.21.15.43:0/972417042", 2020-12-30T23:00:24.033 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2020-12-30T23:00:24.034 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 , 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2020-12-30T23:00:24.035 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { , 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", , 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2020-12-30T23:00:24.035 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.249599+0000", , 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4746000000000001e-05 , 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:24.036 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.636403+0000", , 2020-12-30T08:57:54.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0044951799999999997 , 2020-12-30T08:57:54.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:24.037 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.655240+0000", , 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047171491000000003 , 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2020-12-30T23:00:24.038 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:58:15.685 INFO:tasks.ceph.osd.0:Stopped , 2020-12-30T08:58:15.685 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 , osd.0 is already down. 2020-12-30T23:00:24.039 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:59:26.097 INFO:teuthology.orchestra.run.smithi043.stdout:mon.a: {} , 2020-12-30T08:59:26.113 INFO:teuthology.orchestra.run.smithi043.stdout:mon.b: {} , 2020-12-30T08:59:26.127 INFO:teuthology.orchestra.run.smithi043.stdout:mon.c: {} 2020-12-30T23:00:24.040 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:59:48.212 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2020-12-30T08:59:47.527120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1407295001,"num_objects":477,"num_object_clones":0,"num_object_copies":1431,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":449,"num_whiteouts":3,"num_read":3286,"num_read_kb":2359369,"num_write":3461,"num_write_kb":1206573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":773,"num_bytes_recovered":1169499301,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":887,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":44,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2967,"ondisk_log_size":2967,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2702316,"kb_used_data":2595628,"kb_used_omap":11,"kb_used_meta":105780,"kb_avail":752272404,"statfs":{"total":773094113280,"available":770326941696,"internally_reserved":0,"allocated":2657923072,"data_stored":2643000988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11269,"internal_metadata":108319739},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.529700"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":"111","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252322+0000","last_change":"2020-12-30T08:59:26.805818+0000","last_active":"2020-12-30T08:59:40.252322+0000","last_peered":"2020-12-30T08:59:40.252322+0000","last_clean":"2020-12-30T08:59:40.252322+0000","last_became_active":"2020-12-30T08:59:26.805280+0000","last_became_peered":"2020-12-30T08:59:26.805280+0000","last_unstale":"2020-12-30T08:59:40.252322+0000","last_undegraded":"2020-12-30T08:59:40.252322+0000","last_fullsized":"2020-12-30T08:59:40.252322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:04.206849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:04.206849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"57'44","reported_seq":"156","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252340+0000","last_change":"2020-12-30T08:59:09.166665+0000","last_active":"2020-12-30T08:59:40.252340+0000","last_peered":"2020-12-30T08:59:40.252340+0000","last_clean":"2020-12-30T08:59:40.252340+0000","last_became_active":"2020-12-30T08:56:39.905257+0000","last_became_peered":"2020-12-30T08:56:39.905257+0000","last_unstale":"2020-12-30T08:59:40.252340+0000","last_undegraded":"2020-12-30T08:59:40.252340+0000","last_fullsized":"2020-12-30T08:59:40.252340+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'44","last_scrub_stamp":"2020-12-30T08:59:09.166549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:09.166549+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63684286,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":42,"num_read_kb":51083,"num_write":29,"num_write_kb":4590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":319,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":25,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.578479+0000","last_change":"2020-12-30T08:58:46.630261+0000","last_active":"2020-12-30T08:59:40.578479+0000","last_peered":"2020-12-30T08:59:40.578479+0000","last_clean":"2020-12-30T08:59:40.578479+0000","last_became_active":"2020-12-30T08:56:17.453297+0000","last_became_peered":"2020-12-30T08:56:17.453297+0000","last_unstale":"2020-12-30T08:59:40.578479+0000","last_undegraded":"2020-12-30T08:59:40.578479+0000","last_fullsized":"2020-12-30T08:59:40.578479+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:46.630148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:46.630148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"57'794","reported_seq":"1885","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.596274+0000","last_change":"2020-12-30T08:59:31.962315+0000","last_active":"2020-12-30T08:59:41.596274+0000","last_peered":"2020-12-30T08:59:41.596274+0000","last_clean":"2020-12-30T08:59:41.596274+0000","last_became_active":"2020-12-30T08:59:31.961183+0000","last_became_peered":"2020-12-30T08:59:31.961183+0000","last_unstale":"2020-12-30T08:59:41.596274+0000","last_undegraded":"2020-12-30T08:59:41.596274+0000","last_fullsized":"2020-12-30T08:59:41.596274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'794","last_scrub_stamp":"2020-12-30T08:59:09.405174+0000","last_deep_scrub":"54'709","last_deep_scrub_stamp":"2020-12-30T08:57:41.784481+0000","last_clean_scrub_stamp":"2020-12-30T08:59:09.405174+0000","log_size":794,"ondisk_log_size":794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":324600875,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":956,"num_read_kb":765811,"num_write":895,"num_write_kb":309669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":381323268,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"125","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.432965+0000","last_change":"2020-12-30T08:59:26.807827+0000","last_active":"2020-12-30T08:59:40.432965+0000","last_peered":"2020-12-30T08:59:40.432965+0000","last_clean":"2020-12-30T08:59:40.432965+0000","last_became_active":"2020-12-30T08:59:26.807377+0000","last_became_peered":"2020-12-30T08:59:26.807377+0000","last_unstale":"2020-12-30T08:59:40.432965+0000","last_undegraded":"2020-12-30T08:59:40.432965+0000","last_fullsized":"2020-12-30T08:59:40.432965+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.818206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.818206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"57'598","reported_seq":"1017","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252337+0000","last_change":"2020-12-30T08:59:08.196312+0000","last_active":"2020-12-30T08:59:40.252337+0000","last_peered":"2020-12-30T08:59:40.252337+0000","last_clean":"2020-12-30T08:59:40.252337+0000","last_became_active":"2020-12-30T08:56:35.883250+0000","last_became_peered":"2020-12-30T08:56:35.883250+0000","last_unstale":"2020-12-30T08:59:40.252337+0000","last_undegraded":"2020-12-30T08:59:40.252337+0000","last_fullsized":"2020-12-30T08:59:40.252337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'598","last_scrub_stamp":"2020-12-30T08:59:08.196199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:34.853614+0000","last_clean_scrub_stamp":"2020-12-30T08:59:08.196199+0000","log_size":598,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":299758950,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":689,"num_read_kb":446464,"num_write":755,"num_write_kb":264543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.578494+0000","last_change":"2020-12-30T08:58:53.577047+0000","last_active":"2020-12-30T08:59:40.578494+0000","last_peered":"2020-12-30T08:59:40.578494+0000","last_clean":"2020-12-30T08:59:40.578494+0000","last_became_active":"2020-12-30T08:56:18.363833+0000","last_became_peered":"2020-12-30T08:56:18.363833+0000","last_unstale":"2020-12-30T08:59:40.578494+0000","last_undegraded":"2020-12-30T08:59:40.578494+0000","last_fullsized":"2020-12-30T08:59:40.578494+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:53.576887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:53.576887+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":"145","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001415+0000","last_change":"2020-12-30T08:59:29.950684+0000","last_active":"2020-12-30T08:59:41.001415+0000","last_peered":"2020-12-30T08:59:41.001415+0000","last_clean":"2020-12-30T08:59:41.001415+0000","last_became_active":"2020-12-30T08:59:29.950219+0000","last_became_peered":"2020-12-30T08:59:29.950219+0000","last_unstale":"2020-12-30T08:59:41.001415+0000","last_undegraded":"2020-12-30T08:59:41.001415+0000","last_fullsized":"2020-12-30T08:59:41.001415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":4,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'2","last_scrub_stamp":"2020-12-30T08:58:43.023209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:05.570594+0000","last_clean_scrub_stamp":"2020-12-30T08:58:43.023209+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"57'737","reported_seq":"1556","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.577945+0000","last_change":"2020-12-30T08:59:27.273198+0000","last_active":"2020-12-30T08:59:40.577945+0000","last_peered":"2020-12-30T08:59:40.577945+0000","last_clean":"2020-12-30T08:59:40.577945+0000","last_became_active":"2020-12-30T08:57:34.885212+0000","last_became_peered":"2020-12-30T08:57:34.885212+0000","last_unstale":"2020-12-30T08:59:40.577945+0000","last_undegraded":"2020-12-30T08:59:40.577945+0000","last_fullsized":"2020-12-30T08:59:40.577945+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'737","last_scrub_stamp":"2020-12-30T08:59:27.273101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:34.853614+0000","last_clean_scrub_stamp":"2020-12-30T08:59:27.273101+0000","log_size":737,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":370415644,"num_objects":124,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":0,"num_read":829,"num_read_kb":576628,"num_write":921,"num_write_kb":353530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":210025962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"112","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001472+0000","last_change":"2020-12-30T08:59:26.807466+0000","last_active":"2020-12-30T08:59:41.001472+0000","last_peered":"2020-12-30T08:59:41.001472+0000","last_clean":"2020-12-30T08:59:41.001472+0000","last_became_active":"2020-12-30T08:59:26.806602+0000","last_became_peered":"2020-12-30T08:59:26.806602+0000","last_unstale":"2020-12-30T08:59:41.001472+0000","last_undegraded":"2020-12-30T08:59:41.001472+0000","last_fullsized":"2020-12-30T08:59:41.001472+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:06.756813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:06.756813+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"76'751","reported_seq":"1682","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.000117+0000","last_change":"2020-12-30T08:59:30.952423+0000","last_active":"2020-12-30T08:59:41.000117+0000","last_peered":"2020-12-30T08:59:41.000117+0000","last_clean":"2020-12-30T08:59:41.000117+0000","last_became_active":"2020-12-30T08:59:30.951003+0000","last_became_peered":"2020-12-30T08:59:30.951003+0000","last_unstale":"2020-12-30T08:59:41.000117+0000","last_undegraded":"2020-12-30T08:59:41.000117+0000","last_fullsized":"2020-12-30T08:59:41.000117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'751","last_scrub_stamp":"2020-12-30T08:59:17.342114+0000","last_deep_scrub":"76'751","last_deep_scrub_stamp":"2020-12-30T08:59:17.342114+0000","last_clean_scrub_stamp":"2020-12-30T08:59:17.342114+0000","log_size":751,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":288680016,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":742,"num_read_kb":486335,"num_write":828,"num_write_kb":269542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":495679005,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"57'31","reported_seq":"176","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.000230+0000","last_change":"2020-12-30T08:59:30.953648+0000","last_active":"2020-12-30T08:59:41.000230+0000","last_peered":"2020-12-30T08:59:41.000230+0000","last_clean":"2020-12-30T08:59:41.000230+0000","last_became_active":"2020-12-30T08:59:30.952821+0000","last_became_peered":"2020-12-30T08:59:30.952821+0000","last_unstale":"2020-12-30T08:59:41.000230+0000","last_undegraded":"2020-12-30T08:59:41.000230+0000","last_fullsized":"2020-12-30T08:59:41.000230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'31","last_scrub_stamp":"2020-12-30T08:59:18.654169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:18.654169+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41801084,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":14,"num_read_kb":4410,"num_write":24,"num_write_kb":3616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":82471066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":293,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":14,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"115","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001437+0000","last_change":"2020-12-30T08:59:26.807130+0000","last_active":"2020-12-30T08:59:41.001437+0000","last_peered":"2020-12-30T08:59:41.001437+0000","last_clean":"2020-12-30T08:59:41.001437+0000","last_became_active":"2020-12-30T08:59:26.806409+0000","last_became_peered":"2020-12-30T08:59:26.806409+0000","last_unstale":"2020-12-30T08:59:41.001437+0000","last_undegraded":"2020-12-30T08:59:41.001437+0000","last_fullsized":"2020-12-30T08:59:41.001437+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.095702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.095702+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252325+0000","last_change":"2020-12-30T08:58:49.511387+0000","last_active":"2020-12-30T08:59:40.252325+0000","last_peered":"2020-12-30T08:59:40.252325+0000","last_clean":"2020-12-30T08:59:40.252325+0000","last_became_active":"2020-12-30T08:56:18.101527+0000","last_became_peered":"2020-12-30T08:56:18.101527+0000","last_unstale":"2020-12-30T08:59:40.252325+0000","last_undegraded":"2020-12-30T08:59:40.252325+0000","last_fullsized":"2020-12-30T08:59:40.252325+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:49.511270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:49.511270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"99","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.577925+0000","last_change":"2020-12-30T08:59:26.808227+0000","last_active":"2020-12-30T08:59:40.577925+0000","last_peered":"2020-12-30T08:59:40.577925+0000","last_clean":"2020-12-30T08:59:40.577925+0000","last_became_active":"2020-12-30T08:59:26.807467+0000","last_became_peered":"2020-12-30T08:59:26.807467+0000","last_unstale":"2020-12-30T08:59:40.577925+0000","last_undegraded":"2020-12-30T08:59:40.577925+0000","last_fullsized":"2020-12-30T08:59:40.577925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.951459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.951459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"92","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.310721+0000","last_change":"2020-12-30T08:58:34.893621+0000","last_active":"2020-12-30T08:59:40.310721+0000","last_peered":"2020-12-30T08:59:40.310721+0000","last_clean":"2020-12-30T08:59:40.310721+0000","last_became_active":"2020-12-30T08:56:17.453289+0000","last_became_peered":"2020-12-30T08:56:17.453289+0000","last_unstale":"2020-12-30T08:59:40.310721+0000","last_undegraded":"2020-12-30T08:59:40.310721+0000","last_fullsized":"2020-12-30T08:59:40.310721+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:34.893489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:34.893489+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"57'10","reported_seq":"102","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.596156+0000","last_change":"2020-12-30T08:58:50.189729+0000","last_active":"2020-12-30T08:59:41.596156+0000","last_peered":"2020-12-30T08:59:41.596156+0000","last_clean":"2020-12-30T08:59:41.596156+0000","last_became_active":"2020-12-30T08:56:40.286191+0000","last_became_peered":"2020-12-30T08:56:40.286191+0000","last_unstale":"2020-12-30T08:59:41.596156+0000","last_undegraded":"2020-12-30T08:59:41.596156+0000","last_fullsized":"2020-12-30T08:59:41.596156+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'10","last_scrub_stamp":"2020-12-30T08:58:50.189625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:58:50.189625+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18354146,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":28638,"num_write":7,"num_write_kb":1074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":275,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":5,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":4,"stat_sum":{"num_bytes":123839516,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":3,"num_read":70,"num_read_kb":84131,"num_write":60,"num_write_kb":9280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":82471066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":887,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":44,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":363405312,"data_stored":363060657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":12,"acting":12,"num_store_stats":7},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":1283455485,"num_objects":428,"num_object_clones":0,"num_object_copies":1284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":428,"num_whiteouts":0,"num_read":3216,"num_read_kb":2275238,"num_write":3399,"num_write_kb":1197284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":739,"num_bytes_recovered":1087028235,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2289242112,"data_stored":2277676566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2880,"ondisk_log_size":2880,"up":12,"acting":12,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10297,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46828,"kb_used_data":40940,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":94325012,"statfs":{"total":96636764160,"available":96588812288,"internally_reserved":0,"allocated":41922560,"data_stored":41513062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:57:16 2020","interfaces":[{"interface":"back","average":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":81.242000000000004,"5min":81.242000000000004,"15min":81.242000000000004},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1.0409999999999999}]},{"osd":1,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":19.870000000000001,"15min":28.206},"min":{"1min":0.83199999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4970000000000001,"5min":501.495,"15min":501.495},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":26.629000000000001,"15min":25.437000000000001},"min":{"1min":0.73399999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.5720000000000001,"5min":958.63099999999997,"15min":958.63099999999997},"last":1.2589999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":59.027999999999999,"5min":12.720000000000001,"15min":5.008},"min":{"1min":0.85799999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":927.80799999999999,"5min":927.80799999999999,"15min":927.80799999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":21.974,"5min":12.337999999999999,"15min":4.8849999999999998},"min":{"1min":0.98799999999999999,"5min":0.753,"15min":0.753},"max":{"1min":333.58300000000003,"5min":597.27800000000002,"15min":597.27800000000002},"last":1.093}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.202,"15min":1.216},"min":{"1min":0.81399999999999995,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.45,"5min":1.474,"15min":1.474},"last":1.2310000000000001},{"interface":"front","average":{"1min":8.3640000000000008,"5min":2.5859999999999999,"15min":1.623},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":123.247,"5min":123.247,"15min":123.247},"last":0.84299999999999997}]},{"osd":4,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":26.962,"15min":38.468000000000004},"min":{"1min":0.81499999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5609999999999999,"5min":747.37,"15min":747.37},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":1.052,"15min":1.0920000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.486,"5min":1.623,"15min":1.623},"last":1.1080000000000001}]},{"osd":5,"last update":"Wed Dec 30 08:59:24 2020","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1379999999999999,"15min":1.125},"min":{"1min":0.76500000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.6160000000000001,"5min":1.7,"15min":1.7},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.1879999999999999,"15min":1.2390000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5469999999999999,"5min":2.198,"15min":2.198},"last":1.198}]},{"osd":6,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":8.0190000000000001,"5min":27.074000000000002,"15min":37.076000000000001},"min":{"1min":0.77300000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":106.654,"5min":861.60799999999995,"15min":861.60799999999995},"last":1.177},{"interface":"front","average":{"1min":52.908999999999999,"5min":11.640000000000001,"15min":4.7830000000000004},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":778.01900000000001,"5min":778.01900000000001,"15min":778.01900000000001},"last":1.3280000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617820,"kb_used_data":597728,"kb_used_omap":0,"kb_used_meta":19968,"kb_avail":93754020,"statfs":{"total":96636764160,"available":96004116480,"internally_reserved":0,"allocated":612073472,"data_stored":608728491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20447232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:15 2020","interfaces":[{"interface":"back","average":{"1min":63.369,"5min":56.469000000000001,"15min":55.319000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":999.76300000000003,"5min":999.76300000000003,"15min":999.76300000000003},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.04,"15min":1.028},"min":{"1min":0.746,"5min":0.62,"15min":0.62},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":1.3069999999999999}]},{"osd":1,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.01,"15min":0.97699999999999998},"min":{"1min":0.79400000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.3480000000000001,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.875},{"interface":"front","average":{"1min":0.96799999999999997,"5min":36.709000000000003,"15min":52.585000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.3009999999999999,"5min":953.40800000000002,"15min":953.40800000000002},"last":1.1899999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":46.109999999999999,"5min":10.127000000000001,"15min":4.109},"min":{"1min":0.98899999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":714.93299999999999,"5min":714.93299999999999,"15min":714.93299999999999},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.206,"15min":1.1990000000000001},"min":{"1min":1.0309999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.4870000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.6060000000000001}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.151,"5min":1.139,"15min":1.137},"min":{"1min":0.94599999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.3919999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.274}]},{"osd":4,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.96199999999999997,"15min":0.97899999999999998},"min":{"1min":0.63600000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.393,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0740000000000001,"15min":1.0860000000000001},"min":{"1min":0.81299999999999994,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":1.375}]},{"osd":5,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.1020000000000001,"15min":1.123},"min":{"1min":0.77300000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.286,"5min":1.637,"15min":1.637},"last":1.3540000000000001},{"interface":"front","average":{"1min":33.957000000000001,"5min":47.049999999999997,"15min":60.192999999999998},"min":{"1min":0.81299999999999994,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":691.76199999999994,"5min":985.03399999999999,"15min":985.03399999999999},"last":1.3080000000000001}]},{"osd":7,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":21.876000000000001,"5min":5.3460000000000001,"15min":2.5979999999999999},"min":{"1min":0.86099999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":435.39499999999998,"5min":435.39499999999998,"15min":435.39499999999998},"last":1.331},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2330000000000001,"15min":1.2490000000000001},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.5740000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":247340,"kb_used_data":233900,"kb_used_omap":0,"kb_used_meta":13312,"kb_avail":94124500,"statfs":{"total":96636764160,"available":96383488000,"internally_reserved":0,"allocated":239513600,"data_stored":238202894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13631488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:13 2020","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.0569999999999999,"15min":1.0880000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2869999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77500000000000002,"15min":0.77600000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.74399999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":27.254000000000001,"15min":39.072000000000003},"min":{"1min":0.47199999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84799999999999998,"5min":882.37400000000002,"15min":882.37400000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.90400000000000003,"15min":0.92200000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.298,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed Dec 30 08:59:31 2020","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.95299999999999996,"15min":0.95699999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.29,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.86899999999999999,"15min":0.88200000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.302,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.80500000000000005}]},{"osd":4,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1339999999999999,"15min":1.1040000000000001},"min":{"1min":0.92200000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.3899999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.1599999999999999,"15min":1.169},"min":{"1min":0.73299999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.46,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.84699999999999998}]},{"osd":5,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2649999999999999,"15min":1.2829999999999999},"min":{"1min":0.89100000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.4159999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2390000000000001,"15min":1.2470000000000001},"min":{"1min":1.002,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.383,"5min":1.96,"15min":1.96},"last":1.0229999999999999}]},{"osd":6,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2390000000000001,"15min":1.2749999999999999},"min":{"1min":0.82299999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4259999999999999,"5min":4.4829999999999997,"15min":4.4829999999999997},"last":1.149},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.242,"15min":1.272},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.4159999999999999,"5min":1.734,"15min":1.734},"last":1.0069999999999999}]},{"osd":7,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":19.978999999999999,"15min":28.309000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.486,"5min":623.82100000000003,"15min":623.82100000000003},"last":828.46799999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.2250000000000001,"15min":1.2350000000000001},"min":{"1min":0.91700000000000004,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.3500000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.98699999999999999}]}]},{"osd":0,"up_from":75,"seq":322122547208,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410660,"kb_used_data":392612,"kb_used_omap":10,"kb_used_meta":17909,"kb_avail":93961180,"statfs":{"total":96636764160,"available":96216248320,"internally_reserved":0,"allocated":402034688,"data_stored":399673841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10297,"internal_metadata":18339783},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.649}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.679}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":179560,"kb_used_data":172264,"kb_used_omap":0,"kb_used_meta":7168,"kb_avail":94192280,"statfs":{"total":96636764160,"available":96452894720,"internally_reserved":0,"allocated":176398336,"data_stored":175142793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7340032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:15 2020","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.81999999999999995,"15min":0.81399999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.999,"5min":33.204000000000001,"15min":38.572000000000003},"min":{"1min":0.63500000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.8049999999999999,"5min":767.24300000000005,"15min":767.24300000000005},"last":0.67100000000000004}]},{"osd":1,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":30.774000000000001,"5min":7.2549999999999999,"15min":3.4510000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":754.12800000000004,"5min":754.12800000000004,"15min":754.12800000000004},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.91400000000000003,"15min":0.91100000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4430000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.80300000000000005}]},{"osd":3,"last update":"Wed Dec 30 08:59:29 2020","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.0629999999999999,"15min":1.0900000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.476,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.747,"15min":0.747},"min":{"1min":0.55500000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.89100000000000001}]},{"osd":4,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":7.5339999999999998,"15min":3.2320000000000002},"min":{"1min":0.67200000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6200000000000001,"5min":742.45799999999997,"15min":742.45799999999997},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.2110000000000001,"15min":1.2070000000000001},"min":{"1min":0.81699999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5449999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.1970000000000001}]},{"osd":5,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":2.4820000000000002,"15min":1.631},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.464,"5min":152.709,"15min":152.709},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.1919999999999999,"15min":1.214},"min":{"1min":0.80800000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.4039999999999999,"5min":2.181,"15min":2.181},"last":0.96899999999999997}]},{"osd":6,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":4.1260000000000003,"5min":29.439,"15min":41.308},"min":{"1min":0.753,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":73.947000000000003,"5min":874.04999999999995,"15min":874.04999999999995},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.1439999999999999,"15min":1.1819999999999999},"min":{"1min":0.73399999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.508,"5min":4.0439999999999996,"15min":4.0439999999999996},"last":1.0700000000000001}]},{"osd":7,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":20.821999999999999,"5min":21.945,"15min":26.760999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":489.90300000000002,"5min":502.05900000000003,"15min":502.05900000000003},"last":1.214},{"interface":"front","average":{"1min":1.3220000000000001,"5min":23.478000000000002,"15min":33.302999999999997},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":2.0270000000000001,"5min":502.10599999999999,"15min":502.10599999999999},"last":1.2290000000000001}]}]},{"osd":3,"up_from":41,"seq":176093659166,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400584,"kb_used_data":387072,"kb_used_omap":0,"kb_used_meta":13503,"kb_avail":93971256,"statfs":{"total":96636764160,"available":96226566144,"internally_reserved":0,"allocated":396361728,"data_stored":394019686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":972,"internal_metadata":13827124},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":1,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.93000000000000005,"15min":0.94999999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.242,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.84599999999999997,"15min":0.86799999999999999},"min":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90100000000000002,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.85499999999999998}]},{"osd":2,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.83699999999999997,"15min":0.85299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1319999999999999,"5min":1.581,"15min":1.581},"last":0.628},{"interface":"front","average":{"1min":26.884,"5min":6.1639999999999997,"15min":2.71},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":469.85500000000002,"5min":469.85500000000002,"15min":469.85500000000002},"last":0.82099999999999995}]},{"osd":4,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.0760000000000001,"15min":1.0840000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.254,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.23,"15min":1.254},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.4079999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.90800000000000003}]},{"osd":5,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":25.748000000000001,"15min":29.870000000000001},"min":{"1min":0.81799999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.2010000000000001,"5min":524.471,"15min":524.471},"last":0.88},{"interface":"front","average":{"1min":47.161999999999999,"5min":10.489000000000001,"15min":4.3769999999999998},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":830.05200000000002,"5min":830.05200000000002,"15min":830.05200000000002},"last":0.94699999999999995}]},{"osd":6,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.254,"15min":1.258},"min":{"1min":1.046,"5min":1.046,"15min":1.046},"max":{"1min":1.5960000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.181},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.266,"15min":1.268},"min":{"1min":1.071,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.6259999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.204}]},{"osd":7,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.298,"15min":1.3220000000000001},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.3009999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.089},{"interface":"front","average":{"1min":1.159,"5min":1.264,"15min":1.282},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.1599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":444756,"kb_used_data":430292,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":93927084,"statfs":{"total":96636764160,"available":96181334016,"internally_reserved":0,"allocated":440619008,"data_stored":438222994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:57:13 2020","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.298},{"interface":"front","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.3360000000000001}]},{"osd":1,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.119,"15min":1.1479999999999999},"min":{"1min":0.755,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.4039999999999999,"5min":1.661,"15min":1.661},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.022,"15min":1.012},"min":{"1min":0.78600000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4019999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.361}]},{"osd":2,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.161,"15min":1.1870000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.498,"5min":1.776,"15min":1.776},"last":1.282},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0940000000000001,"15min":1.1040000000000001},"min":{"1min":0.84499999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.446,"5min":1.478,"15min":1.478},"last":1.446}]},{"osd":3,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1000000000000001,"15min":1.0920000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.2050000000000001,"15min":1.2210000000000001},"min":{"1min":0.91400000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.222}]},{"osd":5,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.96399999999999997,"15min":0.94599999999999995},"min":{"1min":0.81399999999999995,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3440000000000001,"5min":1.45,"15min":1.45},"last":0.995},{"interface":"front","average":{"1min":1.014,"5min":0.98999999999999999,"15min":0.94899999999999995},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.226,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.26}]},{"osd":6,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.1160000000000001,"15min":1.153},"min":{"1min":0.73499999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.4950000000000001,"5min":1.71,"15min":1.71},"last":1.4199999999999999},{"interface":"front","average":{"1min":18.32,"5min":4.508,"15min":2.214},"min":{"1min":0.90100000000000002,"5min":0.629,"15min":0.629},"max":{"1min":276.41800000000001,"5min":276.41800000000001,"15min":276.41800000000001},"last":1.0489999999999999}]},{"osd":7,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":3.4380000000000002,"15min":1.8959999999999999},"min":{"1min":1.022,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.385,"5min":181.79499999999999,"15min":181.79499999999999},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":5.6840000000000002,"15min":2.7799999999999998},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":1.4510000000000001,"5min":353.72500000000002,"15min":353.72500000000002},"last":1.476}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":354768,"kb_used_data":340820,"kb_used_omap":0,"kb_used_meta":13824,"kb_avail":94017072,"statfs":{"total":96636764160,"available":96273481728,"internally_reserved":0,"allocated":348999680,"data_stored":347497227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14155776},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:12 2020","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.1439999999999999,"15min":1.1599999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.4239999999999999,"5min":1.526,"15min":1.526},"last":1},{"interface":"front","average":{"1min":42.222000000000001,"5min":9.3520000000000003,"15min":3.8740000000000001},"min":{"1min":0.83799999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":821.10900000000004,"5min":821.10900000000004,"15min":821.10900000000004},"last":0.89200000000000002}]},{"osd":1,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0009999999999999,"15min":0.98499999999999999},"min":{"1min":0.79100000000000004,"5min":0.627,"15min":0.627},"max":{"1min":1.2769999999999999,"5min":1.379,"15min":1.379},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":5.5030000000000001,"15min":2.5710000000000002},"min":{"1min":0.84099999999999997,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.3300000000000001,"5min":440.44999999999999,"15min":440.44999999999999},"last":0.91800000000000004}]},{"osd":2,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":7.0289999999999999,"15min":2.9950000000000001},"min":{"1min":0.752,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.294,"5min":605.79899999999998,"15min":605.79899999999998},"last":0.94599999999999995},{"interface":"front","average":{"1min":23.163,"5min":5.524,"15min":2.5800000000000001},"min":{"1min":0.88200000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":501.64600000000002,"5min":501.64600000000002,"15min":501.64600000000002},"last":1.329}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1419999999999999,"15min":1.1339999999999999},"min":{"1min":0.90600000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.411,"5min":1.494,"15min":1.494},"last":1.282},{"interface":"front","average":{"1min":1.159,"5min":17.323,"15min":20.016999999999999},"min":{"1min":0.92700000000000005,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.393,"5min":404.60300000000001,"15min":404.60300000000001},"last":1.0800000000000001}]},{"osd":4,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":21.57,"15min":22.916},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.4079999999999999,"5min":704.28399999999999,"15min":704.28399999999999},"last":1.113},{"interface":"front","average":{"1min":0.92100000000000004,"5min":46.520000000000003,"15min":66.030000000000001},"min":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.0920000000000001,"5min":861.90599999999995,"15min":861.90599999999995},"last":1.0529999999999999}]},{"osd":6,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0700000000000001,"15min":1.0660000000000001},"min":{"1min":0.66300000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.4299999999999999,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.085,"5min":1.1020000000000001,"15min":1.121},"min":{"1min":0.77700000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4410000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.145}]},{"osd":7,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.284,"15min":1.3919999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3899999999999999,"5min":8.3629999999999995,"15min":8.3629999999999995},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":11.603999999999999,"15min":12.991},"min":{"1min":0.79200000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.6000000000000001,"5min":291.65300000000002,"15min":291.65300000000002},"last":1.3089999999999999}]}]}]}} , 2020-12-30T08:59:48.214 INFO:tasks.ceph.ceph_manager.ceph:recovered! , 2020-12-30T08:59:48.215 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2020-12-30T23:00:24.042 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 33, in nested , yield vars , File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-12-30-1123/qa/tasks/ceph.py", line 1876, in task 2020-12-30T23:00:24.043 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T09:00:18.496 DEBUG:teuthology.orchestra.run.smithi169:> mktemp , 2020-12-30T09:00:18.513 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.C79Qha9RXd , 2020-12-30T09:00:18.514 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.C79Qha9RXd 2020-12-30T23:00:24.044 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T09:00:35.626 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- , 2020-12-30T09:00:35.644 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- , 2020-12-30T09:00:35.706 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2020-12-30T23:00:24.096 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f69dd512816] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x55d303af9fe6] 2020-12-30T23:00:24.097 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.563908+0000", , 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.00019472900000000001 , 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.098 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.187571661, , 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "type_data": { , 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "delayed", 2020-12-30T23:00:24.099 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.107 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", , 2020-12-30T08:57:52.108 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.759798+0000", , 2020-12-30T08:57:52.108 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 5.1649e-05 2020-12-30T23:00:24.099 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "initiated", , 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.611817+0000", 2020-12-30T23:00:24.100 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "started", , 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.867908+0000", 2020-12-30T23:00:24.100 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "sub_op_commit_rec", , 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.879692+0000", 2020-12-30T23:00:24.101 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.189 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.190 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.190 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.102 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", 2020-12-30T23:00:24.102 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", 2020-12-30T23:00:24.103 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "reached pg", , 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: "events": [ , 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.103 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.272 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.002481568 , 2020-12-30T08:57:52.272 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.273 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.104 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.292 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 8.7658000000000007e-05 , 2020-12-30T08:57:52.293 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.293 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.104 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.87972017899999999 , 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2020-12-30T23:00:24.105 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "type_data": { , 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "started", , 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "events": [ 2020-12-30T23:00:24.106 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:28.437546+0000", , 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 6.2249999999999997e-06 , 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.106 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 600, , 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: "ops": [ , 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.107 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", , 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.755828+0000", 2020-12-30T23:00:24.107 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "header_read", 2020-12-30T23:00:24.108 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: } 2020-12-30T23:00:24.108 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.820 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4.3782000000000002e-05 , 2020-12-30T08:57:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.109 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:10.493947+0000", , 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4294967295.9999809 , 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.110 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.310989+0000", , 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.012873951999999999 , 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: } 2020-12-30T23:00:24.110 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "queued_for_pg", , 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:17.586655+0000", , 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4.7491000000000003e-05 2020-12-30T23:00:24.111 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", , 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:33.372282+0000", 2020-12-30T23:00:24.111 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:44.022360+0000", , 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "age": 8.7134268660000007, , 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.346765198, 2020-12-30T23:00:24.112 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "started", 2020-12-30T23:00:24.112 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.963 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.0046062250000000002 , 2020-12-30T08:57:52.963 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.964 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.113 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.983 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.984 INFO:teuthology.orchestra.run.smithi169.stdout:} , 2020-12-30T08:57:52.985 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-12-30T23:00:24.114 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", 2020-12-30T23:00:24.114 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.403 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0 , 2020-12-30T08:57:53.404 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.404 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.115 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: "description": "osd_op(client.4598.0:1164 4.0 4:249a18ca:::smithi04340772-260:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e34)", , 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:04.096815+0000", 2020-12-30T23:00:24.115 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:09.599534+0000", , 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.036992197999999997 , 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.116 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.470 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", , 2020-12-30T08:57:53.470 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.144872+0000", , 2020-12-30T08:57:53.471 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 1.0566e-05 2020-12-30T23:00:24.116 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "initiated", , 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.156798+0000", 2020-12-30T23:00:24.117 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.513 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:53.513 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:53.514 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.118 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.535 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.535 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.536 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "queued_for_pg", 2020-12-30T23:00:24.118 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4294967295.9999795 , 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.119 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: "description": "osd_op(client.4598.0:2898 4.0 4:1f210125:::smithi04340772-284:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e38)", , 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:17.505116+0000", 2020-12-30T23:00:24.120 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.600 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:22.235621+0000", , 2020-12-30T08:57:53.600 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.035873975000000002 , 2020-12-30T08:57:53.601 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.120 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3320 , 2020-12-30T08:57:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.027 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2020-12-30T23:00:24.121 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:23.886767+0000", , 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8839999999999997e-06 , 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:24.122 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:32.913785+0000", , 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049845687999999999 , 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:24.122 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", , 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:27.332932+0000", , 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2020-12-30T23:00:24.123 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", , 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:27.333311+0000", , 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1931999999999999e-05 2020-12-30T23:00:24.123 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", , 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:28.423645+0000", 2020-12-30T23:00:24.124 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for readable", , 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:28.440591+0000", 2020-12-30T23:00:24.124 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { , 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4598", , 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "v1:172.21.15.43:0/972417042", 2020-12-30T23:00:24.125 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2020-12-30T23:00:24.126 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 , 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2020-12-30T23:00:24.126 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { , 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", , 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2020-12-30T23:00:24.127 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.249599+0000", , 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4746000000000001e-05 , 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:24.127 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.636403+0000", , 2020-12-30T08:57:54.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0044951799999999997 , 2020-12-30T08:57:54.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:24.128 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.655240+0000", , 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047171491000000003 , 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2020-12-30T23:00:24.129 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:58:15.685 INFO:tasks.ceph.osd.0:Stopped , 2020-12-30T08:58:15.685 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 , osd.0 is already down. 2020-12-30T23:00:24.130 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:59:26.097 INFO:teuthology.orchestra.run.smithi043.stdout:mon.a: {} , 2020-12-30T08:59:26.113 INFO:teuthology.orchestra.run.smithi043.stdout:mon.b: {} , 2020-12-30T08:59:26.127 INFO:teuthology.orchestra.run.smithi043.stdout:mon.c: {} 2020-12-30T23:00:24.132 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:59:48.212 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2020-12-30T08:59:47.527120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1407295001,"num_objects":477,"num_object_clones":0,"num_object_copies":1431,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":449,"num_whiteouts":3,"num_read":3286,"num_read_kb":2359369,"num_write":3461,"num_write_kb":1206573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":773,"num_bytes_recovered":1169499301,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":887,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":44,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2967,"ondisk_log_size":2967,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2702316,"kb_used_data":2595628,"kb_used_omap":11,"kb_used_meta":105780,"kb_avail":752272404,"statfs":{"total":773094113280,"available":770326941696,"internally_reserved":0,"allocated":2657923072,"data_stored":2643000988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11269,"internal_metadata":108319739},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.529700"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":"111","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252322+0000","last_change":"2020-12-30T08:59:26.805818+0000","last_active":"2020-12-30T08:59:40.252322+0000","last_peered":"2020-12-30T08:59:40.252322+0000","last_clean":"2020-12-30T08:59:40.252322+0000","last_became_active":"2020-12-30T08:59:26.805280+0000","last_became_peered":"2020-12-30T08:59:26.805280+0000","last_unstale":"2020-12-30T08:59:40.252322+0000","last_undegraded":"2020-12-30T08:59:40.252322+0000","last_fullsized":"2020-12-30T08:59:40.252322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:04.206849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:04.206849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"57'44","reported_seq":"156","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252340+0000","last_change":"2020-12-30T08:59:09.166665+0000","last_active":"2020-12-30T08:59:40.252340+0000","last_peered":"2020-12-30T08:59:40.252340+0000","last_clean":"2020-12-30T08:59:40.252340+0000","last_became_active":"2020-12-30T08:56:39.905257+0000","last_became_peered":"2020-12-30T08:56:39.905257+0000","last_unstale":"2020-12-30T08:59:40.252340+0000","last_undegraded":"2020-12-30T08:59:40.252340+0000","last_fullsized":"2020-12-30T08:59:40.252340+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'44","last_scrub_stamp":"2020-12-30T08:59:09.166549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:09.166549+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63684286,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":42,"num_read_kb":51083,"num_write":29,"num_write_kb":4590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":319,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":25,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.578479+0000","last_change":"2020-12-30T08:58:46.630261+0000","last_active":"2020-12-30T08:59:40.578479+0000","last_peered":"2020-12-30T08:59:40.578479+0000","last_clean":"2020-12-30T08:59:40.578479+0000","last_became_active":"2020-12-30T08:56:17.453297+0000","last_became_peered":"2020-12-30T08:56:17.453297+0000","last_unstale":"2020-12-30T08:59:40.578479+0000","last_undegraded":"2020-12-30T08:59:40.578479+0000","last_fullsized":"2020-12-30T08:59:40.578479+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:46.630148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:46.630148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"57'794","reported_seq":"1885","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.596274+0000","last_change":"2020-12-30T08:59:31.962315+0000","last_active":"2020-12-30T08:59:41.596274+0000","last_peered":"2020-12-30T08:59:41.596274+0000","last_clean":"2020-12-30T08:59:41.596274+0000","last_became_active":"2020-12-30T08:59:31.961183+0000","last_became_peered":"2020-12-30T08:59:31.961183+0000","last_unstale":"2020-12-30T08:59:41.596274+0000","last_undegraded":"2020-12-30T08:59:41.596274+0000","last_fullsized":"2020-12-30T08:59:41.596274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'794","last_scrub_stamp":"2020-12-30T08:59:09.405174+0000","last_deep_scrub":"54'709","last_deep_scrub_stamp":"2020-12-30T08:57:41.784481+0000","last_clean_scrub_stamp":"2020-12-30T08:59:09.405174+0000","log_size":794,"ondisk_log_size":794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":324600875,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":956,"num_read_kb":765811,"num_write":895,"num_write_kb":309669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":381323268,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"125","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.432965+0000","last_change":"2020-12-30T08:59:26.807827+0000","last_active":"2020-12-30T08:59:40.432965+0000","last_peered":"2020-12-30T08:59:40.432965+0000","last_clean":"2020-12-30T08:59:40.432965+0000","last_became_active":"2020-12-30T08:59:26.807377+0000","last_became_peered":"2020-12-30T08:59:26.807377+0000","last_unstale":"2020-12-30T08:59:40.432965+0000","last_undegraded":"2020-12-30T08:59:40.432965+0000","last_fullsized":"2020-12-30T08:59:40.432965+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.818206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.818206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"57'598","reported_seq":"1017","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252337+0000","last_change":"2020-12-30T08:59:08.196312+0000","last_active":"2020-12-30T08:59:40.252337+0000","last_peered":"2020-12-30T08:59:40.252337+0000","last_clean":"2020-12-30T08:59:40.252337+0000","last_became_active":"2020-12-30T08:56:35.883250+0000","last_became_peered":"2020-12-30T08:56:35.883250+0000","last_unstale":"2020-12-30T08:59:40.252337+0000","last_undegraded":"2020-12-30T08:59:40.252337+0000","last_fullsized":"2020-12-30T08:59:40.252337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'598","last_scrub_stamp":"2020-12-30T08:59:08.196199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:34.853614+0000","last_clean_scrub_stamp":"2020-12-30T08:59:08.196199+0000","log_size":598,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":299758950,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":689,"num_read_kb":446464,"num_write":755,"num_write_kb":264543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.578494+0000","last_change":"2020-12-30T08:58:53.577047+0000","last_active":"2020-12-30T08:59:40.578494+0000","last_peered":"2020-12-30T08:59:40.578494+0000","last_clean":"2020-12-30T08:59:40.578494+0000","last_became_active":"2020-12-30T08:56:18.363833+0000","last_became_peered":"2020-12-30T08:56:18.363833+0000","last_unstale":"2020-12-30T08:59:40.578494+0000","last_undegraded":"2020-12-30T08:59:40.578494+0000","last_fullsized":"2020-12-30T08:59:40.578494+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:53.576887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:53.576887+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":"145","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001415+0000","last_change":"2020-12-30T08:59:29.950684+0000","last_active":"2020-12-30T08:59:41.001415+0000","last_peered":"2020-12-30T08:59:41.001415+0000","last_clean":"2020-12-30T08:59:41.001415+0000","last_became_active":"2020-12-30T08:59:29.950219+0000","last_became_peered":"2020-12-30T08:59:29.950219+0000","last_unstale":"2020-12-30T08:59:41.001415+0000","last_undegraded":"2020-12-30T08:59:41.001415+0000","last_fullsized":"2020-12-30T08:59:41.001415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":4,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'2","last_scrub_stamp":"2020-12-30T08:58:43.023209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:05.570594+0000","last_clean_scrub_stamp":"2020-12-30T08:58:43.023209+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"57'737","reported_seq":"1556","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.577945+0000","last_change":"2020-12-30T08:59:27.273198+0000","last_active":"2020-12-30T08:59:40.577945+0000","last_peered":"2020-12-30T08:59:40.577945+0000","last_clean":"2020-12-30T08:59:40.577945+0000","last_became_active":"2020-12-30T08:57:34.885212+0000","last_became_peered":"2020-12-30T08:57:34.885212+0000","last_unstale":"2020-12-30T08:59:40.577945+0000","last_undegraded":"2020-12-30T08:59:40.577945+0000","last_fullsized":"2020-12-30T08:59:40.577945+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'737","last_scrub_stamp":"2020-12-30T08:59:27.273101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:34.853614+0000","last_clean_scrub_stamp":"2020-12-30T08:59:27.273101+0000","log_size":737,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":370415644,"num_objects":124,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":0,"num_read":829,"num_read_kb":576628,"num_write":921,"num_write_kb":353530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":210025962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"112","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001472+0000","last_change":"2020-12-30T08:59:26.807466+0000","last_active":"2020-12-30T08:59:41.001472+0000","last_peered":"2020-12-30T08:59:41.001472+0000","last_clean":"2020-12-30T08:59:41.001472+0000","last_became_active":"2020-12-30T08:59:26.806602+0000","last_became_peered":"2020-12-30T08:59:26.806602+0000","last_unstale":"2020-12-30T08:59:41.001472+0000","last_undegraded":"2020-12-30T08:59:41.001472+0000","last_fullsized":"2020-12-30T08:59:41.001472+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:06.756813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:06.756813+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"76'751","reported_seq":"1682","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.000117+0000","last_change":"2020-12-30T08:59:30.952423+0000","last_active":"2020-12-30T08:59:41.000117+0000","last_peered":"2020-12-30T08:59:41.000117+0000","last_clean":"2020-12-30T08:59:41.000117+0000","last_became_active":"2020-12-30T08:59:30.951003+0000","last_became_peered":"2020-12-30T08:59:30.951003+0000","last_unstale":"2020-12-30T08:59:41.000117+0000","last_undegraded":"2020-12-30T08:59:41.000117+0000","last_fullsized":"2020-12-30T08:59:41.000117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'751","last_scrub_stamp":"2020-12-30T08:59:17.342114+0000","last_deep_scrub":"76'751","last_deep_scrub_stamp":"2020-12-30T08:59:17.342114+0000","last_clean_scrub_stamp":"2020-12-30T08:59:17.342114+0000","log_size":751,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":288680016,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":742,"num_read_kb":486335,"num_write":828,"num_write_kb":269542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":495679005,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"57'31","reported_seq":"176","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.000230+0000","last_change":"2020-12-30T08:59:30.953648+0000","last_active":"2020-12-30T08:59:41.000230+0000","last_peered":"2020-12-30T08:59:41.000230+0000","last_clean":"2020-12-30T08:59:41.000230+0000","last_became_active":"2020-12-30T08:59:30.952821+0000","last_became_peered":"2020-12-30T08:59:30.952821+0000","last_unstale":"2020-12-30T08:59:41.000230+0000","last_undegraded":"2020-12-30T08:59:41.000230+0000","last_fullsized":"2020-12-30T08:59:41.000230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'31","last_scrub_stamp":"2020-12-30T08:59:18.654169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:18.654169+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41801084,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":14,"num_read_kb":4410,"num_write":24,"num_write_kb":3616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":82471066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":293,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":14,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"115","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001437+0000","last_change":"2020-12-30T08:59:26.807130+0000","last_active":"2020-12-30T08:59:41.001437+0000","last_peered":"2020-12-30T08:59:41.001437+0000","last_clean":"2020-12-30T08:59:41.001437+0000","last_became_active":"2020-12-30T08:59:26.806409+0000","last_became_peered":"2020-12-30T08:59:26.806409+0000","last_unstale":"2020-12-30T08:59:41.001437+0000","last_undegraded":"2020-12-30T08:59:41.001437+0000","last_fullsized":"2020-12-30T08:59:41.001437+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.095702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.095702+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252325+0000","last_change":"2020-12-30T08:58:49.511387+0000","last_active":"2020-12-30T08:59:40.252325+0000","last_peered":"2020-12-30T08:59:40.252325+0000","last_clean":"2020-12-30T08:59:40.252325+0000","last_became_active":"2020-12-30T08:56:18.101527+0000","last_became_peered":"2020-12-30T08:56:18.101527+0000","last_unstale":"2020-12-30T08:59:40.252325+0000","last_undegraded":"2020-12-30T08:59:40.252325+0000","last_fullsized":"2020-12-30T08:59:40.252325+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:49.511270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:49.511270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"99","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.577925+0000","last_change":"2020-12-30T08:59:26.808227+0000","last_active":"2020-12-30T08:59:40.577925+0000","last_peered":"2020-12-30T08:59:40.577925+0000","last_clean":"2020-12-30T08:59:40.577925+0000","last_became_active":"2020-12-30T08:59:26.807467+0000","last_became_peered":"2020-12-30T08:59:26.807467+0000","last_unstale":"2020-12-30T08:59:40.577925+0000","last_undegraded":"2020-12-30T08:59:40.577925+0000","last_fullsized":"2020-12-30T08:59:40.577925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.951459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.951459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"92","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.310721+0000","last_change":"2020-12-30T08:58:34.893621+0000","last_active":"2020-12-30T08:59:40.310721+0000","last_peered":"2020-12-30T08:59:40.310721+0000","last_clean":"2020-12-30T08:59:40.310721+0000","last_became_active":"2020-12-30T08:56:17.453289+0000","last_became_peered":"2020-12-30T08:56:17.453289+0000","last_unstale":"2020-12-30T08:59:40.310721+0000","last_undegraded":"2020-12-30T08:59:40.310721+0000","last_fullsized":"2020-12-30T08:59:40.310721+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:34.893489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:34.893489+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"57'10","reported_seq":"102","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.596156+0000","last_change":"2020-12-30T08:58:50.189729+0000","last_active":"2020-12-30T08:59:41.596156+0000","last_peered":"2020-12-30T08:59:41.596156+0000","last_clean":"2020-12-30T08:59:41.596156+0000","last_became_active":"2020-12-30T08:56:40.286191+0000","last_became_peered":"2020-12-30T08:56:40.286191+0000","last_unstale":"2020-12-30T08:59:41.596156+0000","last_undegraded":"2020-12-30T08:59:41.596156+0000","last_fullsized":"2020-12-30T08:59:41.596156+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'10","last_scrub_stamp":"2020-12-30T08:58:50.189625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:58:50.189625+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18354146,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":28638,"num_write":7,"num_write_kb":1074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":275,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":5,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":4,"stat_sum":{"num_bytes":123839516,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":3,"num_read":70,"num_read_kb":84131,"num_write":60,"num_write_kb":9280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":82471066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":887,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":44,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":363405312,"data_stored":363060657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":12,"acting":12,"num_store_stats":7},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":1283455485,"num_objects":428,"num_object_clones":0,"num_object_copies":1284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":428,"num_whiteouts":0,"num_read":3216,"num_read_kb":2275238,"num_write":3399,"num_write_kb":1197284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":739,"num_bytes_recovered":1087028235,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2289242112,"data_stored":2277676566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2880,"ondisk_log_size":2880,"up":12,"acting":12,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10297,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46828,"kb_used_data":40940,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":94325012,"statfs":{"total":96636764160,"available":96588812288,"internally_reserved":0,"allocated":41922560,"data_stored":41513062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:57:16 2020","interfaces":[{"interface":"back","average":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":81.242000000000004,"5min":81.242000000000004,"15min":81.242000000000004},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1.0409999999999999}]},{"osd":1,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":19.870000000000001,"15min":28.206},"min":{"1min":0.83199999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4970000000000001,"5min":501.495,"15min":501.495},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":26.629000000000001,"15min":25.437000000000001},"min":{"1min":0.73399999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.5720000000000001,"5min":958.63099999999997,"15min":958.63099999999997},"last":1.2589999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":59.027999999999999,"5min":12.720000000000001,"15min":5.008},"min":{"1min":0.85799999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":927.80799999999999,"5min":927.80799999999999,"15min":927.80799999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":21.974,"5min":12.337999999999999,"15min":4.8849999999999998},"min":{"1min":0.98799999999999999,"5min":0.753,"15min":0.753},"max":{"1min":333.58300000000003,"5min":597.27800000000002,"15min":597.27800000000002},"last":1.093}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.202,"15min":1.216},"min":{"1min":0.81399999999999995,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.45,"5min":1.474,"15min":1.474},"last":1.2310000000000001},{"interface":"front","average":{"1min":8.3640000000000008,"5min":2.5859999999999999,"15min":1.623},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":123.247,"5min":123.247,"15min":123.247},"last":0.84299999999999997}]},{"osd":4,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":26.962,"15min":38.468000000000004},"min":{"1min":0.81499999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5609999999999999,"5min":747.37,"15min":747.37},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":1.052,"15min":1.0920000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.486,"5min":1.623,"15min":1.623},"last":1.1080000000000001}]},{"osd":5,"last update":"Wed Dec 30 08:59:24 2020","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1379999999999999,"15min":1.125},"min":{"1min":0.76500000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.6160000000000001,"5min":1.7,"15min":1.7},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.1879999999999999,"15min":1.2390000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5469999999999999,"5min":2.198,"15min":2.198},"last":1.198}]},{"osd":6,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":8.0190000000000001,"5min":27.074000000000002,"15min":37.076000000000001},"min":{"1min":0.77300000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":106.654,"5min":861.60799999999995,"15min":861.60799999999995},"last":1.177},{"interface":"front","average":{"1min":52.908999999999999,"5min":11.640000000000001,"15min":4.7830000000000004},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":778.01900000000001,"5min":778.01900000000001,"15min":778.01900000000001},"last":1.3280000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617820,"kb_used_data":597728,"kb_used_omap":0,"kb_used_meta":19968,"kb_avail":93754020,"statfs":{"total":96636764160,"available":96004116480,"internally_reserved":0,"allocated":612073472,"data_stored":608728491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20447232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:15 2020","interfaces":[{"interface":"back","average":{"1min":63.369,"5min":56.469000000000001,"15min":55.319000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":999.76300000000003,"5min":999.76300000000003,"15min":999.76300000000003},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.04,"15min":1.028},"min":{"1min":0.746,"5min":0.62,"15min":0.62},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":1.3069999999999999}]},{"osd":1,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.01,"15min":0.97699999999999998},"min":{"1min":0.79400000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.3480000000000001,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.875},{"interface":"front","average":{"1min":0.96799999999999997,"5min":36.709000000000003,"15min":52.585000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.3009999999999999,"5min":953.40800000000002,"15min":953.40800000000002},"last":1.1899999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":46.109999999999999,"5min":10.127000000000001,"15min":4.109},"min":{"1min":0.98899999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":714.93299999999999,"5min":714.93299999999999,"15min":714.93299999999999},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.206,"15min":1.1990000000000001},"min":{"1min":1.0309999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.4870000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.6060000000000001}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.151,"5min":1.139,"15min":1.137},"min":{"1min":0.94599999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.3919999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.274}]},{"osd":4,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.96199999999999997,"15min":0.97899999999999998},"min":{"1min":0.63600000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.393,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0740000000000001,"15min":1.0860000000000001},"min":{"1min":0.81299999999999994,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":1.375}]},{"osd":5,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.1020000000000001,"15min":1.123},"min":{"1min":0.77300000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.286,"5min":1.637,"15min":1.637},"last":1.3540000000000001},{"interface":"front","average":{"1min":33.957000000000001,"5min":47.049999999999997,"15min":60.192999999999998},"min":{"1min":0.81299999999999994,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":691.76199999999994,"5min":985.03399999999999,"15min":985.03399999999999},"last":1.3080000000000001}]},{"osd":7,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":21.876000000000001,"5min":5.3460000000000001,"15min":2.5979999999999999},"min":{"1min":0.86099999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":435.39499999999998,"5min":435.39499999999998,"15min":435.39499999999998},"last":1.331},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2330000000000001,"15min":1.2490000000000001},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.5740000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":247340,"kb_used_data":233900,"kb_used_omap":0,"kb_used_meta":13312,"kb_avail":94124500,"statfs":{"total":96636764160,"available":96383488000,"internally_reserved":0,"allocated":239513600,"data_stored":238202894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13631488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:13 2020","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.0569999999999999,"15min":1.0880000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2869999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77500000000000002,"15min":0.77600000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.74399999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":27.254000000000001,"15min":39.072000000000003},"min":{"1min":0.47199999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84799999999999998,"5min":882.37400000000002,"15min":882.37400000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.90400000000000003,"15min":0.92200000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.298,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed Dec 30 08:59:31 2020","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.95299999999999996,"15min":0.95699999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.29,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.86899999999999999,"15min":0.88200000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.302,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.80500000000000005}]},{"osd":4,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1339999999999999,"15min":1.1040000000000001},"min":{"1min":0.92200000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.3899999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.1599999999999999,"15min":1.169},"min":{"1min":0.73299999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.46,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.84699999999999998}]},{"osd":5,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2649999999999999,"15min":1.2829999999999999},"min":{"1min":0.89100000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.4159999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2390000000000001,"15min":1.2470000000000001},"min":{"1min":1.002,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.383,"5min":1.96,"15min":1.96},"last":1.0229999999999999}]},{"osd":6,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2390000000000001,"15min":1.2749999999999999},"min":{"1min":0.82299999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4259999999999999,"5min":4.4829999999999997,"15min":4.4829999999999997},"last":1.149},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.242,"15min":1.272},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.4159999999999999,"5min":1.734,"15min":1.734},"last":1.0069999999999999}]},{"osd":7,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":19.978999999999999,"15min":28.309000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.486,"5min":623.82100000000003,"15min":623.82100000000003},"last":828.46799999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.2250000000000001,"15min":1.2350000000000001},"min":{"1min":0.91700000000000004,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.3500000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.98699999999999999}]}]},{"osd":0,"up_from":75,"seq":322122547208,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410660,"kb_used_data":392612,"kb_used_omap":10,"kb_used_meta":17909,"kb_avail":93961180,"statfs":{"total":96636764160,"available":96216248320,"internally_reserved":0,"allocated":402034688,"data_stored":399673841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10297,"internal_metadata":18339783},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.649}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.679}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":179560,"kb_used_data":172264,"kb_used_omap":0,"kb_used_meta":7168,"kb_avail":94192280,"statfs":{"total":96636764160,"available":96452894720,"internally_reserved":0,"allocated":176398336,"data_stored":175142793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7340032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:15 2020","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.81999999999999995,"15min":0.81399999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.999,"5min":33.204000000000001,"15min":38.572000000000003},"min":{"1min":0.63500000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.8049999999999999,"5min":767.24300000000005,"15min":767.24300000000005},"last":0.67100000000000004}]},{"osd":1,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":30.774000000000001,"5min":7.2549999999999999,"15min":3.4510000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":754.12800000000004,"5min":754.12800000000004,"15min":754.12800000000004},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.91400000000000003,"15min":0.91100000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4430000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.80300000000000005}]},{"osd":3,"last update":"Wed Dec 30 08:59:29 2020","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.0629999999999999,"15min":1.0900000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.476,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.747,"15min":0.747},"min":{"1min":0.55500000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.89100000000000001}]},{"osd":4,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":7.5339999999999998,"15min":3.2320000000000002},"min":{"1min":0.67200000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6200000000000001,"5min":742.45799999999997,"15min":742.45799999999997},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.2110000000000001,"15min":1.2070000000000001},"min":{"1min":0.81699999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5449999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.1970000000000001}]},{"osd":5,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":2.4820000000000002,"15min":1.631},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.464,"5min":152.709,"15min":152.709},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.1919999999999999,"15min":1.214},"min":{"1min":0.80800000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.4039999999999999,"5min":2.181,"15min":2.181},"last":0.96899999999999997}]},{"osd":6,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":4.1260000000000003,"5min":29.439,"15min":41.308},"min":{"1min":0.753,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":73.947000000000003,"5min":874.04999999999995,"15min":874.04999999999995},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.1439999999999999,"15min":1.1819999999999999},"min":{"1min":0.73399999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.508,"5min":4.0439999999999996,"15min":4.0439999999999996},"last":1.0700000000000001}]},{"osd":7,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":20.821999999999999,"5min":21.945,"15min":26.760999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":489.90300000000002,"5min":502.05900000000003,"15min":502.05900000000003},"last":1.214},{"interface":"front","average":{"1min":1.3220000000000001,"5min":23.478000000000002,"15min":33.302999999999997},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":2.0270000000000001,"5min":502.10599999999999,"15min":502.10599999999999},"last":1.2290000000000001}]}]},{"osd":3,"up_from":41,"seq":176093659166,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400584,"kb_used_data":387072,"kb_used_omap":0,"kb_used_meta":13503,"kb_avail":93971256,"statfs":{"total":96636764160,"available":96226566144,"internally_reserved":0,"allocated":396361728,"data_stored":394019686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":972,"internal_metadata":13827124},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":1,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.93000000000000005,"15min":0.94999999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.242,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.84599999999999997,"15min":0.86799999999999999},"min":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90100000000000002,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.85499999999999998}]},{"osd":2,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.83699999999999997,"15min":0.85299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1319999999999999,"5min":1.581,"15min":1.581},"last":0.628},{"interface":"front","average":{"1min":26.884,"5min":6.1639999999999997,"15min":2.71},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":469.85500000000002,"5min":469.85500000000002,"15min":469.85500000000002},"last":0.82099999999999995}]},{"osd":4,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.0760000000000001,"15min":1.0840000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.254,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.23,"15min":1.254},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.4079999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.90800000000000003}]},{"osd":5,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":25.748000000000001,"15min":29.870000000000001},"min":{"1min":0.81799999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.2010000000000001,"5min":524.471,"15min":524.471},"last":0.88},{"interface":"front","average":{"1min":47.161999999999999,"5min":10.489000000000001,"15min":4.3769999999999998},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":830.05200000000002,"5min":830.05200000000002,"15min":830.05200000000002},"last":0.94699999999999995}]},{"osd":6,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.254,"15min":1.258},"min":{"1min":1.046,"5min":1.046,"15min":1.046},"max":{"1min":1.5960000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.181},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.266,"15min":1.268},"min":{"1min":1.071,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.6259999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.204}]},{"osd":7,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.298,"15min":1.3220000000000001},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.3009999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.089},{"interface":"front","average":{"1min":1.159,"5min":1.264,"15min":1.282},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.1599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":444756,"kb_used_data":430292,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":93927084,"statfs":{"total":96636764160,"available":96181334016,"internally_reserved":0,"allocated":440619008,"data_stored":438222994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:57:13 2020","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.298},{"interface":"front","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.3360000000000001}]},{"osd":1,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.119,"15min":1.1479999999999999},"min":{"1min":0.755,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.4039999999999999,"5min":1.661,"15min":1.661},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.022,"15min":1.012},"min":{"1min":0.78600000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4019999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.361}]},{"osd":2,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.161,"15min":1.1870000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.498,"5min":1.776,"15min":1.776},"last":1.282},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0940000000000001,"15min":1.1040000000000001},"min":{"1min":0.84499999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.446,"5min":1.478,"15min":1.478},"last":1.446}]},{"osd":3,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1000000000000001,"15min":1.0920000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.2050000000000001,"15min":1.2210000000000001},"min":{"1min":0.91400000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.222}]},{"osd":5,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.96399999999999997,"15min":0.94599999999999995},"min":{"1min":0.81399999999999995,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3440000000000001,"5min":1.45,"15min":1.45},"last":0.995},{"interface":"front","average":{"1min":1.014,"5min":0.98999999999999999,"15min":0.94899999999999995},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.226,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.26}]},{"osd":6,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.1160000000000001,"15min":1.153},"min":{"1min":0.73499999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.4950000000000001,"5min":1.71,"15min":1.71},"last":1.4199999999999999},{"interface":"front","average":{"1min":18.32,"5min":4.508,"15min":2.214},"min":{"1min":0.90100000000000002,"5min":0.629,"15min":0.629},"max":{"1min":276.41800000000001,"5min":276.41800000000001,"15min":276.41800000000001},"last":1.0489999999999999}]},{"osd":7,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":3.4380000000000002,"15min":1.8959999999999999},"min":{"1min":1.022,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.385,"5min":181.79499999999999,"15min":181.79499999999999},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":5.6840000000000002,"15min":2.7799999999999998},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":1.4510000000000001,"5min":353.72500000000002,"15min":353.72500000000002},"last":1.476}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":354768,"kb_used_data":340820,"kb_used_omap":0,"kb_used_meta":13824,"kb_avail":94017072,"statfs":{"total":96636764160,"available":96273481728,"internally_reserved":0,"allocated":348999680,"data_stored":347497227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14155776},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:12 2020","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.1439999999999999,"15min":1.1599999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.4239999999999999,"5min":1.526,"15min":1.526},"last":1},{"interface":"front","average":{"1min":42.222000000000001,"5min":9.3520000000000003,"15min":3.8740000000000001},"min":{"1min":0.83799999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":821.10900000000004,"5min":821.10900000000004,"15min":821.10900000000004},"last":0.89200000000000002}]},{"osd":1,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0009999999999999,"15min":0.98499999999999999},"min":{"1min":0.79100000000000004,"5min":0.627,"15min":0.627},"max":{"1min":1.2769999999999999,"5min":1.379,"15min":1.379},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":5.5030000000000001,"15min":2.5710000000000002},"min":{"1min":0.84099999999999997,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.3300000000000001,"5min":440.44999999999999,"15min":440.44999999999999},"last":0.91800000000000004}]},{"osd":2,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":7.0289999999999999,"15min":2.9950000000000001},"min":{"1min":0.752,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.294,"5min":605.79899999999998,"15min":605.79899999999998},"last":0.94599999999999995},{"interface":"front","average":{"1min":23.163,"5min":5.524,"15min":2.5800000000000001},"min":{"1min":0.88200000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":501.64600000000002,"5min":501.64600000000002,"15min":501.64600000000002},"last":1.329}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1419999999999999,"15min":1.1339999999999999},"min":{"1min":0.90600000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.411,"5min":1.494,"15min":1.494},"last":1.282},{"interface":"front","average":{"1min":1.159,"5min":17.323,"15min":20.016999999999999},"min":{"1min":0.92700000000000005,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.393,"5min":404.60300000000001,"15min":404.60300000000001},"last":1.0800000000000001}]},{"osd":4,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":21.57,"15min":22.916},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.4079999999999999,"5min":704.28399999999999,"15min":704.28399999999999},"last":1.113},{"interface":"front","average":{"1min":0.92100000000000004,"5min":46.520000000000003,"15min":66.030000000000001},"min":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.0920000000000001,"5min":861.90599999999995,"15min":861.90599999999995},"last":1.0529999999999999}]},{"osd":6,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0700000000000001,"15min":1.0660000000000001},"min":{"1min":0.66300000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.4299999999999999,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.085,"5min":1.1020000000000001,"15min":1.121},"min":{"1min":0.77700000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4410000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.145}]},{"osd":7,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.284,"15min":1.3919999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3899999999999999,"5min":8.3629999999999995,"15min":8.3629999999999995},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":11.603999999999999,"15min":12.991},"min":{"1min":0.79200000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.6000000000000001,"5min":291.65300000000002,"15min":291.65300000000002},"last":1.3089999999999999}]}]}]}} , 2020-12-30T08:59:48.214 INFO:tasks.ceph.ceph_manager.ceph:recovered! , 2020-12-30T08:59:48.215 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2020-12-30T23:00:24.134 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 33, in nested , yield vars , File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-12-30-1123/qa/tasks/ceph.py", line 1876, in task 2020-12-30T23:00:24.135 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T09:00:18.496 DEBUG:teuthology.orchestra.run.smithi169:> mktemp , 2020-12-30T09:00:18.513 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.C79Qha9RXd , 2020-12-30T09:00:18.514 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.C79Qha9RXd 2020-12-30T23:00:24.135 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T09:00:35.626 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- , 2020-12-30T09:00:35.644 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- , 2020-12-30T09:00:35.706 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2020-12-30T23:00:24.195 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f69dd512816] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x55d303af9fe6] 2020-12-30T23:00:24.196 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.563908+0000", , 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.00019472900000000001 , 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.197 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.187571661, , 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "type_data": { , 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "delayed", 2020-12-30T23:00:24.197 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.107 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", , 2020-12-30T08:57:52.108 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.759798+0000", , 2020-12-30T08:57:52.108 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 5.1649e-05 2020-12-30T23:00:24.198 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "initiated", , 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.611817+0000", 2020-12-30T23:00:24.198 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "started", , 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.867908+0000", 2020-12-30T23:00:24.199 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "sub_op_commit_rec", , 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.879692+0000", 2020-12-30T23:00:24.200 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.189 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.190 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.190 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.200 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", 2020-12-30T23:00:24.201 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", 2020-12-30T23:00:24.201 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "reached pg", , 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: "events": [ , 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.202 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.272 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.002481568 , 2020-12-30T08:57:52.272 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.273 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.202 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.292 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 8.7658000000000007e-05 , 2020-12-30T08:57:52.293 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.293 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.203 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.87972017899999999 , 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2020-12-30T23:00:24.204 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "type_data": { , 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "started", , 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "events": [ 2020-12-30T23:00:24.204 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:28.437546+0000", , 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 6.2249999999999997e-06 , 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.205 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 600, , 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: "ops": [ , 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.205 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", , 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.755828+0000", 2020-12-30T23:00:24.206 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "header_read", 2020-12-30T23:00:24.206 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: } 2020-12-30T23:00:24.207 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.820 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4.3782000000000002e-05 , 2020-12-30T08:57:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.208 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:10.493947+0000", , 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4294967295.9999809 , 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.208 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.310989+0000", , 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.012873951999999999 , 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: } 2020-12-30T23:00:24.209 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "queued_for_pg", , 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:17.586655+0000", , 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4.7491000000000003e-05 2020-12-30T23:00:24.209 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", , 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:33.372282+0000", 2020-12-30T23:00:24.210 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:44.022360+0000", , 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "age": 8.7134268660000007, , 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.346765198, 2020-12-30T23:00:24.210 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "started", 2020-12-30T23:00:24.211 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.963 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.0046062250000000002 , 2020-12-30T08:57:52.963 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.964 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.212 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.983 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.984 INFO:teuthology.orchestra.run.smithi169.stdout:} , 2020-12-30T08:57:52.985 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-12-30T23:00:24.212 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", 2020-12-30T23:00:24.213 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.403 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0 , 2020-12-30T08:57:53.404 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.404 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.213 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: "description": "osd_op(client.4598.0:1164 4.0 4:249a18ca:::smithi04340772-260:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e34)", , 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:04.096815+0000", 2020-12-30T23:00:24.214 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:09.599534+0000", , 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.036992197999999997 , 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.214 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.470 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", , 2020-12-30T08:57:53.470 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.144872+0000", , 2020-12-30T08:57:53.471 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 1.0566e-05 2020-12-30T23:00:24.215 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "initiated", , 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.156798+0000", 2020-12-30T23:00:24.216 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.513 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:53.513 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:53.514 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.216 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.535 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.535 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.536 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "queued_for_pg", 2020-12-30T23:00:24.217 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4294967295.9999795 , 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.217 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: "description": "osd_op(client.4598.0:2898 4.0 4:1f210125:::smithi04340772-284:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e38)", , 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:17.505116+0000", 2020-12-30T23:00:24.218 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.600 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:22.235621+0000", , 2020-12-30T08:57:53.600 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.035873975000000002 , 2020-12-30T08:57:53.601 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.218 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3320 , 2020-12-30T08:57:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.027 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2020-12-30T23:00:24.219 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:23.886767+0000", , 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8839999999999997e-06 , 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:24.219 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:32.913785+0000", , 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049845687999999999 , 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:24.220 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", , 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:27.332932+0000", , 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2020-12-30T23:00:24.221 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", , 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:27.333311+0000", , 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1931999999999999e-05 2020-12-30T23:00:24.221 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", , 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:28.423645+0000", 2020-12-30T23:00:24.222 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for readable", , 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:28.440591+0000", 2020-12-30T23:00:24.222 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { , 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4598", , 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "v1:172.21.15.43:0/972417042", 2020-12-30T23:00:24.223 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2020-12-30T23:00:24.223 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 , 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2020-12-30T23:00:24.224 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { , 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", , 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2020-12-30T23:00:24.225 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.249599+0000", , 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4746000000000001e-05 , 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:24.225 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.636403+0000", , 2020-12-30T08:57:54.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0044951799999999997 , 2020-12-30T08:57:54.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:24.226 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.655240+0000", , 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047171491000000003 , 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2020-12-30T23:00:24.227 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:58:15.685 INFO:tasks.ceph.osd.0:Stopped , 2020-12-30T08:58:15.685 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 , osd.0 is already down. 2020-12-30T23:00:24.228 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:59:26.097 INFO:teuthology.orchestra.run.smithi043.stdout:mon.a: {} , 2020-12-30T08:59:26.113 INFO:teuthology.orchestra.run.smithi043.stdout:mon.b: {} , 2020-12-30T08:59:26.127 INFO:teuthology.orchestra.run.smithi043.stdout:mon.c: {} 2020-12-30T23:00:24.230 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:59:48.212 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2020-12-30T08:59:47.527120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1407295001,"num_objects":477,"num_object_clones":0,"num_object_copies":1431,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":449,"num_whiteouts":3,"num_read":3286,"num_read_kb":2359369,"num_write":3461,"num_write_kb":1206573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":773,"num_bytes_recovered":1169499301,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":887,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":44,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2967,"ondisk_log_size":2967,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2702316,"kb_used_data":2595628,"kb_used_omap":11,"kb_used_meta":105780,"kb_avail":752272404,"statfs":{"total":773094113280,"available":770326941696,"internally_reserved":0,"allocated":2657923072,"data_stored":2643000988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11269,"internal_metadata":108319739},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.529700"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":"111","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252322+0000","last_change":"2020-12-30T08:59:26.805818+0000","last_active":"2020-12-30T08:59:40.252322+0000","last_peered":"2020-12-30T08:59:40.252322+0000","last_clean":"2020-12-30T08:59:40.252322+0000","last_became_active":"2020-12-30T08:59:26.805280+0000","last_became_peered":"2020-12-30T08:59:26.805280+0000","last_unstale":"2020-12-30T08:59:40.252322+0000","last_undegraded":"2020-12-30T08:59:40.252322+0000","last_fullsized":"2020-12-30T08:59:40.252322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:04.206849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:04.206849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"57'44","reported_seq":"156","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252340+0000","last_change":"2020-12-30T08:59:09.166665+0000","last_active":"2020-12-30T08:59:40.252340+0000","last_peered":"2020-12-30T08:59:40.252340+0000","last_clean":"2020-12-30T08:59:40.252340+0000","last_became_active":"2020-12-30T08:56:39.905257+0000","last_became_peered":"2020-12-30T08:56:39.905257+0000","last_unstale":"2020-12-30T08:59:40.252340+0000","last_undegraded":"2020-12-30T08:59:40.252340+0000","last_fullsized":"2020-12-30T08:59:40.252340+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'44","last_scrub_stamp":"2020-12-30T08:59:09.166549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:09.166549+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63684286,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":42,"num_read_kb":51083,"num_write":29,"num_write_kb":4590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":319,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":25,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.578479+0000","last_change":"2020-12-30T08:58:46.630261+0000","last_active":"2020-12-30T08:59:40.578479+0000","last_peered":"2020-12-30T08:59:40.578479+0000","last_clean":"2020-12-30T08:59:40.578479+0000","last_became_active":"2020-12-30T08:56:17.453297+0000","last_became_peered":"2020-12-30T08:56:17.453297+0000","last_unstale":"2020-12-30T08:59:40.578479+0000","last_undegraded":"2020-12-30T08:59:40.578479+0000","last_fullsized":"2020-12-30T08:59:40.578479+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:46.630148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:46.630148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"57'794","reported_seq":"1885","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.596274+0000","last_change":"2020-12-30T08:59:31.962315+0000","last_active":"2020-12-30T08:59:41.596274+0000","last_peered":"2020-12-30T08:59:41.596274+0000","last_clean":"2020-12-30T08:59:41.596274+0000","last_became_active":"2020-12-30T08:59:31.961183+0000","last_became_peered":"2020-12-30T08:59:31.961183+0000","last_unstale":"2020-12-30T08:59:41.596274+0000","last_undegraded":"2020-12-30T08:59:41.596274+0000","last_fullsized":"2020-12-30T08:59:41.596274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'794","last_scrub_stamp":"2020-12-30T08:59:09.405174+0000","last_deep_scrub":"54'709","last_deep_scrub_stamp":"2020-12-30T08:57:41.784481+0000","last_clean_scrub_stamp":"2020-12-30T08:59:09.405174+0000","log_size":794,"ondisk_log_size":794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":324600875,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":956,"num_read_kb":765811,"num_write":895,"num_write_kb":309669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":381323268,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"125","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.432965+0000","last_change":"2020-12-30T08:59:26.807827+0000","last_active":"2020-12-30T08:59:40.432965+0000","last_peered":"2020-12-30T08:59:40.432965+0000","last_clean":"2020-12-30T08:59:40.432965+0000","last_became_active":"2020-12-30T08:59:26.807377+0000","last_became_peered":"2020-12-30T08:59:26.807377+0000","last_unstale":"2020-12-30T08:59:40.432965+0000","last_undegraded":"2020-12-30T08:59:40.432965+0000","last_fullsized":"2020-12-30T08:59:40.432965+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.818206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.818206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"57'598","reported_seq":"1017","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252337+0000","last_change":"2020-12-30T08:59:08.196312+0000","last_active":"2020-12-30T08:59:40.252337+0000","last_peered":"2020-12-30T08:59:40.252337+0000","last_clean":"2020-12-30T08:59:40.252337+0000","last_became_active":"2020-12-30T08:56:35.883250+0000","last_became_peered":"2020-12-30T08:56:35.883250+0000","last_unstale":"2020-12-30T08:59:40.252337+0000","last_undegraded":"2020-12-30T08:59:40.252337+0000","last_fullsized":"2020-12-30T08:59:40.252337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'598","last_scrub_stamp":"2020-12-30T08:59:08.196199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:34.853614+0000","last_clean_scrub_stamp":"2020-12-30T08:59:08.196199+0000","log_size":598,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":299758950,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":689,"num_read_kb":446464,"num_write":755,"num_write_kb":264543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.578494+0000","last_change":"2020-12-30T08:58:53.577047+0000","last_active":"2020-12-30T08:59:40.578494+0000","last_peered":"2020-12-30T08:59:40.578494+0000","last_clean":"2020-12-30T08:59:40.578494+0000","last_became_active":"2020-12-30T08:56:18.363833+0000","last_became_peered":"2020-12-30T08:56:18.363833+0000","last_unstale":"2020-12-30T08:59:40.578494+0000","last_undegraded":"2020-12-30T08:59:40.578494+0000","last_fullsized":"2020-12-30T08:59:40.578494+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:53.576887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:53.576887+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":"145","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001415+0000","last_change":"2020-12-30T08:59:29.950684+0000","last_active":"2020-12-30T08:59:41.001415+0000","last_peered":"2020-12-30T08:59:41.001415+0000","last_clean":"2020-12-30T08:59:41.001415+0000","last_became_active":"2020-12-30T08:59:29.950219+0000","last_became_peered":"2020-12-30T08:59:29.950219+0000","last_unstale":"2020-12-30T08:59:41.001415+0000","last_undegraded":"2020-12-30T08:59:41.001415+0000","last_fullsized":"2020-12-30T08:59:41.001415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":4,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'2","last_scrub_stamp":"2020-12-30T08:58:43.023209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:05.570594+0000","last_clean_scrub_stamp":"2020-12-30T08:58:43.023209+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"57'737","reported_seq":"1556","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.577945+0000","last_change":"2020-12-30T08:59:27.273198+0000","last_active":"2020-12-30T08:59:40.577945+0000","last_peered":"2020-12-30T08:59:40.577945+0000","last_clean":"2020-12-30T08:59:40.577945+0000","last_became_active":"2020-12-30T08:57:34.885212+0000","last_became_peered":"2020-12-30T08:57:34.885212+0000","last_unstale":"2020-12-30T08:59:40.577945+0000","last_undegraded":"2020-12-30T08:59:40.577945+0000","last_fullsized":"2020-12-30T08:59:40.577945+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'737","last_scrub_stamp":"2020-12-30T08:59:27.273101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:34.853614+0000","last_clean_scrub_stamp":"2020-12-30T08:59:27.273101+0000","log_size":737,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":370415644,"num_objects":124,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":0,"num_read":829,"num_read_kb":576628,"num_write":921,"num_write_kb":353530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":210025962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"112","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001472+0000","last_change":"2020-12-30T08:59:26.807466+0000","last_active":"2020-12-30T08:59:41.001472+0000","last_peered":"2020-12-30T08:59:41.001472+0000","last_clean":"2020-12-30T08:59:41.001472+0000","last_became_active":"2020-12-30T08:59:26.806602+0000","last_became_peered":"2020-12-30T08:59:26.806602+0000","last_unstale":"2020-12-30T08:59:41.001472+0000","last_undegraded":"2020-12-30T08:59:41.001472+0000","last_fullsized":"2020-12-30T08:59:41.001472+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:06.756813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:06.756813+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"76'751","reported_seq":"1682","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.000117+0000","last_change":"2020-12-30T08:59:30.952423+0000","last_active":"2020-12-30T08:59:41.000117+0000","last_peered":"2020-12-30T08:59:41.000117+0000","last_clean":"2020-12-30T08:59:41.000117+0000","last_became_active":"2020-12-30T08:59:30.951003+0000","last_became_peered":"2020-12-30T08:59:30.951003+0000","last_unstale":"2020-12-30T08:59:41.000117+0000","last_undegraded":"2020-12-30T08:59:41.000117+0000","last_fullsized":"2020-12-30T08:59:41.000117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'751","last_scrub_stamp":"2020-12-30T08:59:17.342114+0000","last_deep_scrub":"76'751","last_deep_scrub_stamp":"2020-12-30T08:59:17.342114+0000","last_clean_scrub_stamp":"2020-12-30T08:59:17.342114+0000","log_size":751,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":288680016,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":742,"num_read_kb":486335,"num_write":828,"num_write_kb":269542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":495679005,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"57'31","reported_seq":"176","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.000230+0000","last_change":"2020-12-30T08:59:30.953648+0000","last_active":"2020-12-30T08:59:41.000230+0000","last_peered":"2020-12-30T08:59:41.000230+0000","last_clean":"2020-12-30T08:59:41.000230+0000","last_became_active":"2020-12-30T08:59:30.952821+0000","last_became_peered":"2020-12-30T08:59:30.952821+0000","last_unstale":"2020-12-30T08:59:41.000230+0000","last_undegraded":"2020-12-30T08:59:41.000230+0000","last_fullsized":"2020-12-30T08:59:41.000230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'31","last_scrub_stamp":"2020-12-30T08:59:18.654169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:18.654169+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41801084,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":14,"num_read_kb":4410,"num_write":24,"num_write_kb":3616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":82471066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":293,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":14,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"115","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001437+0000","last_change":"2020-12-30T08:59:26.807130+0000","last_active":"2020-12-30T08:59:41.001437+0000","last_peered":"2020-12-30T08:59:41.001437+0000","last_clean":"2020-12-30T08:59:41.001437+0000","last_became_active":"2020-12-30T08:59:26.806409+0000","last_became_peered":"2020-12-30T08:59:26.806409+0000","last_unstale":"2020-12-30T08:59:41.001437+0000","last_undegraded":"2020-12-30T08:59:41.001437+0000","last_fullsized":"2020-12-30T08:59:41.001437+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.095702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.095702+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252325+0000","last_change":"2020-12-30T08:58:49.511387+0000","last_active":"2020-12-30T08:59:40.252325+0000","last_peered":"2020-12-30T08:59:40.252325+0000","last_clean":"2020-12-30T08:59:40.252325+0000","last_became_active":"2020-12-30T08:56:18.101527+0000","last_became_peered":"2020-12-30T08:56:18.101527+0000","last_unstale":"2020-12-30T08:59:40.252325+0000","last_undegraded":"2020-12-30T08:59:40.252325+0000","last_fullsized":"2020-12-30T08:59:40.252325+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:49.511270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:49.511270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"99","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.577925+0000","last_change":"2020-12-30T08:59:26.808227+0000","last_active":"2020-12-30T08:59:40.577925+0000","last_peered":"2020-12-30T08:59:40.577925+0000","last_clean":"2020-12-30T08:59:40.577925+0000","last_became_active":"2020-12-30T08:59:26.807467+0000","last_became_peered":"2020-12-30T08:59:26.807467+0000","last_unstale":"2020-12-30T08:59:40.577925+0000","last_undegraded":"2020-12-30T08:59:40.577925+0000","last_fullsized":"2020-12-30T08:59:40.577925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.951459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.951459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"92","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.310721+0000","last_change":"2020-12-30T08:58:34.893621+0000","last_active":"2020-12-30T08:59:40.310721+0000","last_peered":"2020-12-30T08:59:40.310721+0000","last_clean":"2020-12-30T08:59:40.310721+0000","last_became_active":"2020-12-30T08:56:17.453289+0000","last_became_peered":"2020-12-30T08:56:17.453289+0000","last_unstale":"2020-12-30T08:59:40.310721+0000","last_undegraded":"2020-12-30T08:59:40.310721+0000","last_fullsized":"2020-12-30T08:59:40.310721+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:34.893489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:34.893489+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"57'10","reported_seq":"102","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.596156+0000","last_change":"2020-12-30T08:58:50.189729+0000","last_active":"2020-12-30T08:59:41.596156+0000","last_peered":"2020-12-30T08:59:41.596156+0000","last_clean":"2020-12-30T08:59:41.596156+0000","last_became_active":"2020-12-30T08:56:40.286191+0000","last_became_peered":"2020-12-30T08:56:40.286191+0000","last_unstale":"2020-12-30T08:59:41.596156+0000","last_undegraded":"2020-12-30T08:59:41.596156+0000","last_fullsized":"2020-12-30T08:59:41.596156+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'10","last_scrub_stamp":"2020-12-30T08:58:50.189625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:58:50.189625+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18354146,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":28638,"num_write":7,"num_write_kb":1074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":275,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":5,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":4,"stat_sum":{"num_bytes":123839516,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":3,"num_read":70,"num_read_kb":84131,"num_write":60,"num_write_kb":9280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":82471066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":887,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":44,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":363405312,"data_stored":363060657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":12,"acting":12,"num_store_stats":7},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":1283455485,"num_objects":428,"num_object_clones":0,"num_object_copies":1284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":428,"num_whiteouts":0,"num_read":3216,"num_read_kb":2275238,"num_write":3399,"num_write_kb":1197284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":739,"num_bytes_recovered":1087028235,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2289242112,"data_stored":2277676566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2880,"ondisk_log_size":2880,"up":12,"acting":12,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10297,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46828,"kb_used_data":40940,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":94325012,"statfs":{"total":96636764160,"available":96588812288,"internally_reserved":0,"allocated":41922560,"data_stored":41513062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:57:16 2020","interfaces":[{"interface":"back","average":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":81.242000000000004,"5min":81.242000000000004,"15min":81.242000000000004},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1.0409999999999999}]},{"osd":1,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":19.870000000000001,"15min":28.206},"min":{"1min":0.83199999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4970000000000001,"5min":501.495,"15min":501.495},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":26.629000000000001,"15min":25.437000000000001},"min":{"1min":0.73399999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.5720000000000001,"5min":958.63099999999997,"15min":958.63099999999997},"last":1.2589999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":59.027999999999999,"5min":12.720000000000001,"15min":5.008},"min":{"1min":0.85799999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":927.80799999999999,"5min":927.80799999999999,"15min":927.80799999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":21.974,"5min":12.337999999999999,"15min":4.8849999999999998},"min":{"1min":0.98799999999999999,"5min":0.753,"15min":0.753},"max":{"1min":333.58300000000003,"5min":597.27800000000002,"15min":597.27800000000002},"last":1.093}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.202,"15min":1.216},"min":{"1min":0.81399999999999995,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.45,"5min":1.474,"15min":1.474},"last":1.2310000000000001},{"interface":"front","average":{"1min":8.3640000000000008,"5min":2.5859999999999999,"15min":1.623},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":123.247,"5min":123.247,"15min":123.247},"last":0.84299999999999997}]},{"osd":4,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":26.962,"15min":38.468000000000004},"min":{"1min":0.81499999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5609999999999999,"5min":747.37,"15min":747.37},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":1.052,"15min":1.0920000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.486,"5min":1.623,"15min":1.623},"last":1.1080000000000001}]},{"osd":5,"last update":"Wed Dec 30 08:59:24 2020","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1379999999999999,"15min":1.125},"min":{"1min":0.76500000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.6160000000000001,"5min":1.7,"15min":1.7},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.1879999999999999,"15min":1.2390000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5469999999999999,"5min":2.198,"15min":2.198},"last":1.198}]},{"osd":6,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":8.0190000000000001,"5min":27.074000000000002,"15min":37.076000000000001},"min":{"1min":0.77300000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":106.654,"5min":861.60799999999995,"15min":861.60799999999995},"last":1.177},{"interface":"front","average":{"1min":52.908999999999999,"5min":11.640000000000001,"15min":4.7830000000000004},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":778.01900000000001,"5min":778.01900000000001,"15min":778.01900000000001},"last":1.3280000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617820,"kb_used_data":597728,"kb_used_omap":0,"kb_used_meta":19968,"kb_avail":93754020,"statfs":{"total":96636764160,"available":96004116480,"internally_reserved":0,"allocated":612073472,"data_stored":608728491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20447232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:15 2020","interfaces":[{"interface":"back","average":{"1min":63.369,"5min":56.469000000000001,"15min":55.319000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":999.76300000000003,"5min":999.76300000000003,"15min":999.76300000000003},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.04,"15min":1.028},"min":{"1min":0.746,"5min":0.62,"15min":0.62},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":1.3069999999999999}]},{"osd":1,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.01,"15min":0.97699999999999998},"min":{"1min":0.79400000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.3480000000000001,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.875},{"interface":"front","average":{"1min":0.96799999999999997,"5min":36.709000000000003,"15min":52.585000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.3009999999999999,"5min":953.40800000000002,"15min":953.40800000000002},"last":1.1899999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":46.109999999999999,"5min":10.127000000000001,"15min":4.109},"min":{"1min":0.98899999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":714.93299999999999,"5min":714.93299999999999,"15min":714.93299999999999},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.206,"15min":1.1990000000000001},"min":{"1min":1.0309999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.4870000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.6060000000000001}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.151,"5min":1.139,"15min":1.137},"min":{"1min":0.94599999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.3919999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.274}]},{"osd":4,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.96199999999999997,"15min":0.97899999999999998},"min":{"1min":0.63600000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.393,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0740000000000001,"15min":1.0860000000000001},"min":{"1min":0.81299999999999994,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":1.375}]},{"osd":5,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.1020000000000001,"15min":1.123},"min":{"1min":0.77300000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.286,"5min":1.637,"15min":1.637},"last":1.3540000000000001},{"interface":"front","average":{"1min":33.957000000000001,"5min":47.049999999999997,"15min":60.192999999999998},"min":{"1min":0.81299999999999994,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":691.76199999999994,"5min":985.03399999999999,"15min":985.03399999999999},"last":1.3080000000000001}]},{"osd":7,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":21.876000000000001,"5min":5.3460000000000001,"15min":2.5979999999999999},"min":{"1min":0.86099999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":435.39499999999998,"5min":435.39499999999998,"15min":435.39499999999998},"last":1.331},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2330000000000001,"15min":1.2490000000000001},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.5740000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":247340,"kb_used_data":233900,"kb_used_omap":0,"kb_used_meta":13312,"kb_avail":94124500,"statfs":{"total":96636764160,"available":96383488000,"internally_reserved":0,"allocated":239513600,"data_stored":238202894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13631488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:13 2020","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.0569999999999999,"15min":1.0880000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2869999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77500000000000002,"15min":0.77600000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.74399999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":27.254000000000001,"15min":39.072000000000003},"min":{"1min":0.47199999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84799999999999998,"5min":882.37400000000002,"15min":882.37400000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.90400000000000003,"15min":0.92200000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.298,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed Dec 30 08:59:31 2020","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.95299999999999996,"15min":0.95699999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.29,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.86899999999999999,"15min":0.88200000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.302,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.80500000000000005}]},{"osd":4,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1339999999999999,"15min":1.1040000000000001},"min":{"1min":0.92200000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.3899999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.1599999999999999,"15min":1.169},"min":{"1min":0.73299999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.46,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.84699999999999998}]},{"osd":5,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2649999999999999,"15min":1.2829999999999999},"min":{"1min":0.89100000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.4159999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2390000000000001,"15min":1.2470000000000001},"min":{"1min":1.002,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.383,"5min":1.96,"15min":1.96},"last":1.0229999999999999}]},{"osd":6,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2390000000000001,"15min":1.2749999999999999},"min":{"1min":0.82299999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4259999999999999,"5min":4.4829999999999997,"15min":4.4829999999999997},"last":1.149},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.242,"15min":1.272},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.4159999999999999,"5min":1.734,"15min":1.734},"last":1.0069999999999999}]},{"osd":7,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":19.978999999999999,"15min":28.309000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.486,"5min":623.82100000000003,"15min":623.82100000000003},"last":828.46799999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.2250000000000001,"15min":1.2350000000000001},"min":{"1min":0.91700000000000004,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.3500000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.98699999999999999}]}]},{"osd":0,"up_from":75,"seq":322122547208,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410660,"kb_used_data":392612,"kb_used_omap":10,"kb_used_meta":17909,"kb_avail":93961180,"statfs":{"total":96636764160,"available":96216248320,"internally_reserved":0,"allocated":402034688,"data_stored":399673841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10297,"internal_metadata":18339783},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.649}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.679}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":179560,"kb_used_data":172264,"kb_used_omap":0,"kb_used_meta":7168,"kb_avail":94192280,"statfs":{"total":96636764160,"available":96452894720,"internally_reserved":0,"allocated":176398336,"data_stored":175142793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7340032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:15 2020","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.81999999999999995,"15min":0.81399999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.999,"5min":33.204000000000001,"15min":38.572000000000003},"min":{"1min":0.63500000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.8049999999999999,"5min":767.24300000000005,"15min":767.24300000000005},"last":0.67100000000000004}]},{"osd":1,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":30.774000000000001,"5min":7.2549999999999999,"15min":3.4510000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":754.12800000000004,"5min":754.12800000000004,"15min":754.12800000000004},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.91400000000000003,"15min":0.91100000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4430000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.80300000000000005}]},{"osd":3,"last update":"Wed Dec 30 08:59:29 2020","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.0629999999999999,"15min":1.0900000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.476,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.747,"15min":0.747},"min":{"1min":0.55500000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.89100000000000001}]},{"osd":4,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":7.5339999999999998,"15min":3.2320000000000002},"min":{"1min":0.67200000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6200000000000001,"5min":742.45799999999997,"15min":742.45799999999997},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.2110000000000001,"15min":1.2070000000000001},"min":{"1min":0.81699999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5449999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.1970000000000001}]},{"osd":5,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":2.4820000000000002,"15min":1.631},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.464,"5min":152.709,"15min":152.709},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.1919999999999999,"15min":1.214},"min":{"1min":0.80800000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.4039999999999999,"5min":2.181,"15min":2.181},"last":0.96899999999999997}]},{"osd":6,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":4.1260000000000003,"5min":29.439,"15min":41.308},"min":{"1min":0.753,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":73.947000000000003,"5min":874.04999999999995,"15min":874.04999999999995},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.1439999999999999,"15min":1.1819999999999999},"min":{"1min":0.73399999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.508,"5min":4.0439999999999996,"15min":4.0439999999999996},"last":1.0700000000000001}]},{"osd":7,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":20.821999999999999,"5min":21.945,"15min":26.760999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":489.90300000000002,"5min":502.05900000000003,"15min":502.05900000000003},"last":1.214},{"interface":"front","average":{"1min":1.3220000000000001,"5min":23.478000000000002,"15min":33.302999999999997},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":2.0270000000000001,"5min":502.10599999999999,"15min":502.10599999999999},"last":1.2290000000000001}]}]},{"osd":3,"up_from":41,"seq":176093659166,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400584,"kb_used_data":387072,"kb_used_omap":0,"kb_used_meta":13503,"kb_avail":93971256,"statfs":{"total":96636764160,"available":96226566144,"internally_reserved":0,"allocated":396361728,"data_stored":394019686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":972,"internal_metadata":13827124},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":1,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.93000000000000005,"15min":0.94999999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.242,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.84599999999999997,"15min":0.86799999999999999},"min":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90100000000000002,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.85499999999999998}]},{"osd":2,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.83699999999999997,"15min":0.85299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1319999999999999,"5min":1.581,"15min":1.581},"last":0.628},{"interface":"front","average":{"1min":26.884,"5min":6.1639999999999997,"15min":2.71},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":469.85500000000002,"5min":469.85500000000002,"15min":469.85500000000002},"last":0.82099999999999995}]},{"osd":4,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.0760000000000001,"15min":1.0840000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.254,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.23,"15min":1.254},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.4079999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.90800000000000003}]},{"osd":5,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":25.748000000000001,"15min":29.870000000000001},"min":{"1min":0.81799999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.2010000000000001,"5min":524.471,"15min":524.471},"last":0.88},{"interface":"front","average":{"1min":47.161999999999999,"5min":10.489000000000001,"15min":4.3769999999999998},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":830.05200000000002,"5min":830.05200000000002,"15min":830.05200000000002},"last":0.94699999999999995}]},{"osd":6,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.254,"15min":1.258},"min":{"1min":1.046,"5min":1.046,"15min":1.046},"max":{"1min":1.5960000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.181},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.266,"15min":1.268},"min":{"1min":1.071,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.6259999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.204}]},{"osd":7,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.298,"15min":1.3220000000000001},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.3009999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.089},{"interface":"front","average":{"1min":1.159,"5min":1.264,"15min":1.282},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.1599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":444756,"kb_used_data":430292,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":93927084,"statfs":{"total":96636764160,"available":96181334016,"internally_reserved":0,"allocated":440619008,"data_stored":438222994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:57:13 2020","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.298},{"interface":"front","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.3360000000000001}]},{"osd":1,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.119,"15min":1.1479999999999999},"min":{"1min":0.755,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.4039999999999999,"5min":1.661,"15min":1.661},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.022,"15min":1.012},"min":{"1min":0.78600000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4019999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.361}]},{"osd":2,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.161,"15min":1.1870000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.498,"5min":1.776,"15min":1.776},"last":1.282},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0940000000000001,"15min":1.1040000000000001},"min":{"1min":0.84499999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.446,"5min":1.478,"15min":1.478},"last":1.446}]},{"osd":3,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1000000000000001,"15min":1.0920000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.2050000000000001,"15min":1.2210000000000001},"min":{"1min":0.91400000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.222}]},{"osd":5,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.96399999999999997,"15min":0.94599999999999995},"min":{"1min":0.81399999999999995,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3440000000000001,"5min":1.45,"15min":1.45},"last":0.995},{"interface":"front","average":{"1min":1.014,"5min":0.98999999999999999,"15min":0.94899999999999995},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.226,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.26}]},{"osd":6,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.1160000000000001,"15min":1.153},"min":{"1min":0.73499999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.4950000000000001,"5min":1.71,"15min":1.71},"last":1.4199999999999999},{"interface":"front","average":{"1min":18.32,"5min":4.508,"15min":2.214},"min":{"1min":0.90100000000000002,"5min":0.629,"15min":0.629},"max":{"1min":276.41800000000001,"5min":276.41800000000001,"15min":276.41800000000001},"last":1.0489999999999999}]},{"osd":7,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":3.4380000000000002,"15min":1.8959999999999999},"min":{"1min":1.022,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.385,"5min":181.79499999999999,"15min":181.79499999999999},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":5.6840000000000002,"15min":2.7799999999999998},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":1.4510000000000001,"5min":353.72500000000002,"15min":353.72500000000002},"last":1.476}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":354768,"kb_used_data":340820,"kb_used_omap":0,"kb_used_meta":13824,"kb_avail":94017072,"statfs":{"total":96636764160,"available":96273481728,"internally_reserved":0,"allocated":348999680,"data_stored":347497227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14155776},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:12 2020","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.1439999999999999,"15min":1.1599999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.4239999999999999,"5min":1.526,"15min":1.526},"last":1},{"interface":"front","average":{"1min":42.222000000000001,"5min":9.3520000000000003,"15min":3.8740000000000001},"min":{"1min":0.83799999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":821.10900000000004,"5min":821.10900000000004,"15min":821.10900000000004},"last":0.89200000000000002}]},{"osd":1,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0009999999999999,"15min":0.98499999999999999},"min":{"1min":0.79100000000000004,"5min":0.627,"15min":0.627},"max":{"1min":1.2769999999999999,"5min":1.379,"15min":1.379},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":5.5030000000000001,"15min":2.5710000000000002},"min":{"1min":0.84099999999999997,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.3300000000000001,"5min":440.44999999999999,"15min":440.44999999999999},"last":0.91800000000000004}]},{"osd":2,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":7.0289999999999999,"15min":2.9950000000000001},"min":{"1min":0.752,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.294,"5min":605.79899999999998,"15min":605.79899999999998},"last":0.94599999999999995},{"interface":"front","average":{"1min":23.163,"5min":5.524,"15min":2.5800000000000001},"min":{"1min":0.88200000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":501.64600000000002,"5min":501.64600000000002,"15min":501.64600000000002},"last":1.329}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1419999999999999,"15min":1.1339999999999999},"min":{"1min":0.90600000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.411,"5min":1.494,"15min":1.494},"last":1.282},{"interface":"front","average":{"1min":1.159,"5min":17.323,"15min":20.016999999999999},"min":{"1min":0.92700000000000005,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.393,"5min":404.60300000000001,"15min":404.60300000000001},"last":1.0800000000000001}]},{"osd":4,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":21.57,"15min":22.916},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.4079999999999999,"5min":704.28399999999999,"15min":704.28399999999999},"last":1.113},{"interface":"front","average":{"1min":0.92100000000000004,"5min":46.520000000000003,"15min":66.030000000000001},"min":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.0920000000000001,"5min":861.90599999999995,"15min":861.90599999999995},"last":1.0529999999999999}]},{"osd":6,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0700000000000001,"15min":1.0660000000000001},"min":{"1min":0.66300000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.4299999999999999,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.085,"5min":1.1020000000000001,"15min":1.121},"min":{"1min":0.77700000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4410000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.145}]},{"osd":7,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.284,"15min":1.3919999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3899999999999999,"5min":8.3629999999999995,"15min":8.3629999999999995},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":11.603999999999999,"15min":12.991},"min":{"1min":0.79200000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.6000000000000001,"5min":291.65300000000002,"15min":291.65300000000002},"last":1.3089999999999999}]}]}]}} , 2020-12-30T08:59:48.214 INFO:tasks.ceph.ceph_manager.ceph:recovered! , 2020-12-30T08:59:48.215 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2020-12-30T23:00:24.232 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 33, in nested , yield vars , File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-12-30-1123/qa/tasks/ceph.py", line 1876, in task 2020-12-30T23:00:24.232 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T09:00:18.496 DEBUG:teuthology.orchestra.run.smithi169:> mktemp , 2020-12-30T09:00:18.513 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.C79Qha9RXd , 2020-12-30T09:00:18.514 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.C79Qha9RXd 2020-12-30T23:00:24.233 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T09:00:35.626 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- , 2020-12-30T09:00:35.644 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- , 2020-12-30T09:00:35.706 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2020-12-30T23:00:24.287 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f69dd512816] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x55d303af9fe6] 2020-12-30T23:00:24.288 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.563908+0000", , 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.00019472900000000001 , 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.288 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.187571661, , 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "type_data": { , 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "delayed", 2020-12-30T23:00:24.289 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.107 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", , 2020-12-30T08:57:52.108 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.759798+0000", , 2020-12-30T08:57:52.108 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 5.1649e-05 2020-12-30T23:00:24.290 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "initiated", , 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.611817+0000", 2020-12-30T23:00:24.290 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "started", , 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.867908+0000", 2020-12-30T23:00:24.291 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "sub_op_commit_rec", , 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.879692+0000", 2020-12-30T23:00:24.291 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.189 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.190 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.190 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.292 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", 2020-12-30T23:00:24.293 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", 2020-12-30T23:00:24.293 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "reached pg", , 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: "events": [ , 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.294 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.272 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.002481568 , 2020-12-30T08:57:52.272 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.273 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.294 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.292 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 8.7658000000000007e-05 , 2020-12-30T08:57:52.293 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.293 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.295 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.87972017899999999 , 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2020-12-30T23:00:24.295 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "type_data": { , 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "started", , 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "events": [ 2020-12-30T23:00:24.296 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:28.437546+0000", , 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 6.2249999999999997e-06 , 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.297 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 600, , 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: "ops": [ , 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.297 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", , 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.755828+0000", 2020-12-30T23:00:24.298 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "header_read", 2020-12-30T23:00:24.298 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: } 2020-12-30T23:00:24.299 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.820 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4.3782000000000002e-05 , 2020-12-30T08:57:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.299 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:10.493947+0000", , 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4294967295.9999809 , 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.300 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.310989+0000", , 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.012873951999999999 , 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: } 2020-12-30T23:00:24.301 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "queued_for_pg", , 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:17.586655+0000", , 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4.7491000000000003e-05 2020-12-30T23:00:24.301 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", , 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:33.372282+0000", 2020-12-30T23:00:24.302 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:44.022360+0000", , 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "age": 8.7134268660000007, , 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.346765198, 2020-12-30T23:00:24.303 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "started", 2020-12-30T23:00:24.303 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.963 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.0046062250000000002 , 2020-12-30T08:57:52.963 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.964 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.304 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.983 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.984 INFO:teuthology.orchestra.run.smithi169.stdout:} , 2020-12-30T08:57:52.985 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-12-30T23:00:24.304 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", 2020-12-30T23:00:24.305 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.403 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0 , 2020-12-30T08:57:53.404 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.404 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.306 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: "description": "osd_op(client.4598.0:1164 4.0 4:249a18ca:::smithi04340772-260:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e34)", , 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:04.096815+0000", 2020-12-30T23:00:24.306 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:09.599534+0000", , 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.036992197999999997 , 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.307 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.470 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", , 2020-12-30T08:57:53.470 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.144872+0000", , 2020-12-30T08:57:53.471 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 1.0566e-05 2020-12-30T23:00:24.307 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "initiated", , 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.156798+0000", 2020-12-30T23:00:24.308 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.513 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:53.513 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:53.514 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.308 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.535 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.535 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.536 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "queued_for_pg", 2020-12-30T23:00:24.309 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4294967295.9999795 , 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.310 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: "description": "osd_op(client.4598.0:2898 4.0 4:1f210125:::smithi04340772-284:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e38)", , 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:17.505116+0000", 2020-12-30T23:00:24.311 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.600 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:22.235621+0000", , 2020-12-30T08:57:53.600 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.035873975000000002 , 2020-12-30T08:57:53.601 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.311 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3320 , 2020-12-30T08:57:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.027 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2020-12-30T23:00:24.312 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:23.886767+0000", , 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8839999999999997e-06 , 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:24.312 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:32.913785+0000", , 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049845687999999999 , 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:24.313 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", , 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:27.332932+0000", , 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2020-12-30T23:00:24.313 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", , 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:27.333311+0000", , 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1931999999999999e-05 2020-12-30T23:00:24.314 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", , 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:28.423645+0000", 2020-12-30T23:00:24.315 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for readable", , 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:28.440591+0000", 2020-12-30T23:00:24.315 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { , 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4598", , 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "v1:172.21.15.43:0/972417042", 2020-12-30T23:00:24.316 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2020-12-30T23:00:24.316 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 , 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2020-12-30T23:00:24.317 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { , 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", , 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2020-12-30T23:00:24.318 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.249599+0000", , 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4746000000000001e-05 , 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:24.318 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.636403+0000", , 2020-12-30T08:57:54.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0044951799999999997 , 2020-12-30T08:57:54.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:24.319 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.655240+0000", , 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047171491000000003 , 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2020-12-30T23:00:24.320 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:58:15.685 INFO:tasks.ceph.osd.0:Stopped , 2020-12-30T08:58:15.685 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 , osd.0 is already down. 2020-12-30T23:00:24.321 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:59:26.097 INFO:teuthology.orchestra.run.smithi043.stdout:mon.a: {} , 2020-12-30T08:59:26.113 INFO:teuthology.orchestra.run.smithi043.stdout:mon.b: {} , 2020-12-30T08:59:26.127 INFO:teuthology.orchestra.run.smithi043.stdout:mon.c: {} 2020-12-30T23:00:24.323 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:59:48.212 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2020-12-30T08:59:47.527120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1407295001,"num_objects":477,"num_object_clones":0,"num_object_copies":1431,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":449,"num_whiteouts":3,"num_read":3286,"num_read_kb":2359369,"num_write":3461,"num_write_kb":1206573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":773,"num_bytes_recovered":1169499301,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":887,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":44,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2967,"ondisk_log_size":2967,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2702316,"kb_used_data":2595628,"kb_used_omap":11,"kb_used_meta":105780,"kb_avail":752272404,"statfs":{"total":773094113280,"available":770326941696,"internally_reserved":0,"allocated":2657923072,"data_stored":2643000988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11269,"internal_metadata":108319739},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.529700"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":"111","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252322+0000","last_change":"2020-12-30T08:59:26.805818+0000","last_active":"2020-12-30T08:59:40.252322+0000","last_peered":"2020-12-30T08:59:40.252322+0000","last_clean":"2020-12-30T08:59:40.252322+0000","last_became_active":"2020-12-30T08:59:26.805280+0000","last_became_peered":"2020-12-30T08:59:26.805280+0000","last_unstale":"2020-12-30T08:59:40.252322+0000","last_undegraded":"2020-12-30T08:59:40.252322+0000","last_fullsized":"2020-12-30T08:59:40.252322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:04.206849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:04.206849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"57'44","reported_seq":"156","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252340+0000","last_change":"2020-12-30T08:59:09.166665+0000","last_active":"2020-12-30T08:59:40.252340+0000","last_peered":"2020-12-30T08:59:40.252340+0000","last_clean":"2020-12-30T08:59:40.252340+0000","last_became_active":"2020-12-30T08:56:39.905257+0000","last_became_peered":"2020-12-30T08:56:39.905257+0000","last_unstale":"2020-12-30T08:59:40.252340+0000","last_undegraded":"2020-12-30T08:59:40.252340+0000","last_fullsized":"2020-12-30T08:59:40.252340+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'44","last_scrub_stamp":"2020-12-30T08:59:09.166549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:09.166549+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63684286,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":42,"num_read_kb":51083,"num_write":29,"num_write_kb":4590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":319,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":25,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.578479+0000","last_change":"2020-12-30T08:58:46.630261+0000","last_active":"2020-12-30T08:59:40.578479+0000","last_peered":"2020-12-30T08:59:40.578479+0000","last_clean":"2020-12-30T08:59:40.578479+0000","last_became_active":"2020-12-30T08:56:17.453297+0000","last_became_peered":"2020-12-30T08:56:17.453297+0000","last_unstale":"2020-12-30T08:59:40.578479+0000","last_undegraded":"2020-12-30T08:59:40.578479+0000","last_fullsized":"2020-12-30T08:59:40.578479+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:46.630148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:46.630148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"57'794","reported_seq":"1885","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.596274+0000","last_change":"2020-12-30T08:59:31.962315+0000","last_active":"2020-12-30T08:59:41.596274+0000","last_peered":"2020-12-30T08:59:41.596274+0000","last_clean":"2020-12-30T08:59:41.596274+0000","last_became_active":"2020-12-30T08:59:31.961183+0000","last_became_peered":"2020-12-30T08:59:31.961183+0000","last_unstale":"2020-12-30T08:59:41.596274+0000","last_undegraded":"2020-12-30T08:59:41.596274+0000","last_fullsized":"2020-12-30T08:59:41.596274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'794","last_scrub_stamp":"2020-12-30T08:59:09.405174+0000","last_deep_scrub":"54'709","last_deep_scrub_stamp":"2020-12-30T08:57:41.784481+0000","last_clean_scrub_stamp":"2020-12-30T08:59:09.405174+0000","log_size":794,"ondisk_log_size":794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":324600875,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":956,"num_read_kb":765811,"num_write":895,"num_write_kb":309669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":381323268,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"125","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.432965+0000","last_change":"2020-12-30T08:59:26.807827+0000","last_active":"2020-12-30T08:59:40.432965+0000","last_peered":"2020-12-30T08:59:40.432965+0000","last_clean":"2020-12-30T08:59:40.432965+0000","last_became_active":"2020-12-30T08:59:26.807377+0000","last_became_peered":"2020-12-30T08:59:26.807377+0000","last_unstale":"2020-12-30T08:59:40.432965+0000","last_undegraded":"2020-12-30T08:59:40.432965+0000","last_fullsized":"2020-12-30T08:59:40.432965+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.818206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.818206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"57'598","reported_seq":"1017","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252337+0000","last_change":"2020-12-30T08:59:08.196312+0000","last_active":"2020-12-30T08:59:40.252337+0000","last_peered":"2020-12-30T08:59:40.252337+0000","last_clean":"2020-12-30T08:59:40.252337+0000","last_became_active":"2020-12-30T08:56:35.883250+0000","last_became_peered":"2020-12-30T08:56:35.883250+0000","last_unstale":"2020-12-30T08:59:40.252337+0000","last_undegraded":"2020-12-30T08:59:40.252337+0000","last_fullsized":"2020-12-30T08:59:40.252337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'598","last_scrub_stamp":"2020-12-30T08:59:08.196199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:34.853614+0000","last_clean_scrub_stamp":"2020-12-30T08:59:08.196199+0000","log_size":598,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":299758950,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":689,"num_read_kb":446464,"num_write":755,"num_write_kb":264543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.578494+0000","last_change":"2020-12-30T08:58:53.577047+0000","last_active":"2020-12-30T08:59:40.578494+0000","last_peered":"2020-12-30T08:59:40.578494+0000","last_clean":"2020-12-30T08:59:40.578494+0000","last_became_active":"2020-12-30T08:56:18.363833+0000","last_became_peered":"2020-12-30T08:56:18.363833+0000","last_unstale":"2020-12-30T08:59:40.578494+0000","last_undegraded":"2020-12-30T08:59:40.578494+0000","last_fullsized":"2020-12-30T08:59:40.578494+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:53.576887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:53.576887+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":"145","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001415+0000","last_change":"2020-12-30T08:59:29.950684+0000","last_active":"2020-12-30T08:59:41.001415+0000","last_peered":"2020-12-30T08:59:41.001415+0000","last_clean":"2020-12-30T08:59:41.001415+0000","last_became_active":"2020-12-30T08:59:29.950219+0000","last_became_peered":"2020-12-30T08:59:29.950219+0000","last_unstale":"2020-12-30T08:59:41.001415+0000","last_undegraded":"2020-12-30T08:59:41.001415+0000","last_fullsized":"2020-12-30T08:59:41.001415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":4,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'2","last_scrub_stamp":"2020-12-30T08:58:43.023209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:05.570594+0000","last_clean_scrub_stamp":"2020-12-30T08:58:43.023209+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"57'737","reported_seq":"1556","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.577945+0000","last_change":"2020-12-30T08:59:27.273198+0000","last_active":"2020-12-30T08:59:40.577945+0000","last_peered":"2020-12-30T08:59:40.577945+0000","last_clean":"2020-12-30T08:59:40.577945+0000","last_became_active":"2020-12-30T08:57:34.885212+0000","last_became_peered":"2020-12-30T08:57:34.885212+0000","last_unstale":"2020-12-30T08:59:40.577945+0000","last_undegraded":"2020-12-30T08:59:40.577945+0000","last_fullsized":"2020-12-30T08:59:40.577945+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'737","last_scrub_stamp":"2020-12-30T08:59:27.273101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:34.853614+0000","last_clean_scrub_stamp":"2020-12-30T08:59:27.273101+0000","log_size":737,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":370415644,"num_objects":124,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":0,"num_read":829,"num_read_kb":576628,"num_write":921,"num_write_kb":353530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":210025962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"112","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001472+0000","last_change":"2020-12-30T08:59:26.807466+0000","last_active":"2020-12-30T08:59:41.001472+0000","last_peered":"2020-12-30T08:59:41.001472+0000","last_clean":"2020-12-30T08:59:41.001472+0000","last_became_active":"2020-12-30T08:59:26.806602+0000","last_became_peered":"2020-12-30T08:59:26.806602+0000","last_unstale":"2020-12-30T08:59:41.001472+0000","last_undegraded":"2020-12-30T08:59:41.001472+0000","last_fullsized":"2020-12-30T08:59:41.001472+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:06.756813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:06.756813+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"76'751","reported_seq":"1682","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.000117+0000","last_change":"2020-12-30T08:59:30.952423+0000","last_active":"2020-12-30T08:59:41.000117+0000","last_peered":"2020-12-30T08:59:41.000117+0000","last_clean":"2020-12-30T08:59:41.000117+0000","last_became_active":"2020-12-30T08:59:30.951003+0000","last_became_peered":"2020-12-30T08:59:30.951003+0000","last_unstale":"2020-12-30T08:59:41.000117+0000","last_undegraded":"2020-12-30T08:59:41.000117+0000","last_fullsized":"2020-12-30T08:59:41.000117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'751","last_scrub_stamp":"2020-12-30T08:59:17.342114+0000","last_deep_scrub":"76'751","last_deep_scrub_stamp":"2020-12-30T08:59:17.342114+0000","last_clean_scrub_stamp":"2020-12-30T08:59:17.342114+0000","log_size":751,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":288680016,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":742,"num_read_kb":486335,"num_write":828,"num_write_kb":269542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":495679005,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"57'31","reported_seq":"176","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.000230+0000","last_change":"2020-12-30T08:59:30.953648+0000","last_active":"2020-12-30T08:59:41.000230+0000","last_peered":"2020-12-30T08:59:41.000230+0000","last_clean":"2020-12-30T08:59:41.000230+0000","last_became_active":"2020-12-30T08:59:30.952821+0000","last_became_peered":"2020-12-30T08:59:30.952821+0000","last_unstale":"2020-12-30T08:59:41.000230+0000","last_undegraded":"2020-12-30T08:59:41.000230+0000","last_fullsized":"2020-12-30T08:59:41.000230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'31","last_scrub_stamp":"2020-12-30T08:59:18.654169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:18.654169+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41801084,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":14,"num_read_kb":4410,"num_write":24,"num_write_kb":3616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":82471066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":293,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":14,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"115","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001437+0000","last_change":"2020-12-30T08:59:26.807130+0000","last_active":"2020-12-30T08:59:41.001437+0000","last_peered":"2020-12-30T08:59:41.001437+0000","last_clean":"2020-12-30T08:59:41.001437+0000","last_became_active":"2020-12-30T08:59:26.806409+0000","last_became_peered":"2020-12-30T08:59:26.806409+0000","last_unstale":"2020-12-30T08:59:41.001437+0000","last_undegraded":"2020-12-30T08:59:41.001437+0000","last_fullsized":"2020-12-30T08:59:41.001437+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.095702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.095702+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252325+0000","last_change":"2020-12-30T08:58:49.511387+0000","last_active":"2020-12-30T08:59:40.252325+0000","last_peered":"2020-12-30T08:59:40.252325+0000","last_clean":"2020-12-30T08:59:40.252325+0000","last_became_active":"2020-12-30T08:56:18.101527+0000","last_became_peered":"2020-12-30T08:56:18.101527+0000","last_unstale":"2020-12-30T08:59:40.252325+0000","last_undegraded":"2020-12-30T08:59:40.252325+0000","last_fullsized":"2020-12-30T08:59:40.252325+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:49.511270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:49.511270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"99","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.577925+0000","last_change":"2020-12-30T08:59:26.808227+0000","last_active":"2020-12-30T08:59:40.577925+0000","last_peered":"2020-12-30T08:59:40.577925+0000","last_clean":"2020-12-30T08:59:40.577925+0000","last_became_active":"2020-12-30T08:59:26.807467+0000","last_became_peered":"2020-12-30T08:59:26.807467+0000","last_unstale":"2020-12-30T08:59:40.577925+0000","last_undegraded":"2020-12-30T08:59:40.577925+0000","last_fullsized":"2020-12-30T08:59:40.577925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.951459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.951459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"92","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.310721+0000","last_change":"2020-12-30T08:58:34.893621+0000","last_active":"2020-12-30T08:59:40.310721+0000","last_peered":"2020-12-30T08:59:40.310721+0000","last_clean":"2020-12-30T08:59:40.310721+0000","last_became_active":"2020-12-30T08:56:17.453289+0000","last_became_peered":"2020-12-30T08:56:17.453289+0000","last_unstale":"2020-12-30T08:59:40.310721+0000","last_undegraded":"2020-12-30T08:59:40.310721+0000","last_fullsized":"2020-12-30T08:59:40.310721+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:34.893489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:34.893489+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"57'10","reported_seq":"102","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.596156+0000","last_change":"2020-12-30T08:58:50.189729+0000","last_active":"2020-12-30T08:59:41.596156+0000","last_peered":"2020-12-30T08:59:41.596156+0000","last_clean":"2020-12-30T08:59:41.596156+0000","last_became_active":"2020-12-30T08:56:40.286191+0000","last_became_peered":"2020-12-30T08:56:40.286191+0000","last_unstale":"2020-12-30T08:59:41.596156+0000","last_undegraded":"2020-12-30T08:59:41.596156+0000","last_fullsized":"2020-12-30T08:59:41.596156+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'10","last_scrub_stamp":"2020-12-30T08:58:50.189625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:58:50.189625+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18354146,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":28638,"num_write":7,"num_write_kb":1074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":275,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":5,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":4,"stat_sum":{"num_bytes":123839516,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":3,"num_read":70,"num_read_kb":84131,"num_write":60,"num_write_kb":9280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":82471066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":887,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":44,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":363405312,"data_stored":363060657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":12,"acting":12,"num_store_stats":7},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":1283455485,"num_objects":428,"num_object_clones":0,"num_object_copies":1284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":428,"num_whiteouts":0,"num_read":3216,"num_read_kb":2275238,"num_write":3399,"num_write_kb":1197284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":739,"num_bytes_recovered":1087028235,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2289242112,"data_stored":2277676566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2880,"ondisk_log_size":2880,"up":12,"acting":12,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10297,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46828,"kb_used_data":40940,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":94325012,"statfs":{"total":96636764160,"available":96588812288,"internally_reserved":0,"allocated":41922560,"data_stored":41513062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:57:16 2020","interfaces":[{"interface":"back","average":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":81.242000000000004,"5min":81.242000000000004,"15min":81.242000000000004},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1.0409999999999999}]},{"osd":1,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":19.870000000000001,"15min":28.206},"min":{"1min":0.83199999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4970000000000001,"5min":501.495,"15min":501.495},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":26.629000000000001,"15min":25.437000000000001},"min":{"1min":0.73399999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.5720000000000001,"5min":958.63099999999997,"15min":958.63099999999997},"last":1.2589999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":59.027999999999999,"5min":12.720000000000001,"15min":5.008},"min":{"1min":0.85799999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":927.80799999999999,"5min":927.80799999999999,"15min":927.80799999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":21.974,"5min":12.337999999999999,"15min":4.8849999999999998},"min":{"1min":0.98799999999999999,"5min":0.753,"15min":0.753},"max":{"1min":333.58300000000003,"5min":597.27800000000002,"15min":597.27800000000002},"last":1.093}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.202,"15min":1.216},"min":{"1min":0.81399999999999995,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.45,"5min":1.474,"15min":1.474},"last":1.2310000000000001},{"interface":"front","average":{"1min":8.3640000000000008,"5min":2.5859999999999999,"15min":1.623},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":123.247,"5min":123.247,"15min":123.247},"last":0.84299999999999997}]},{"osd":4,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":26.962,"15min":38.468000000000004},"min":{"1min":0.81499999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5609999999999999,"5min":747.37,"15min":747.37},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":1.052,"15min":1.0920000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.486,"5min":1.623,"15min":1.623},"last":1.1080000000000001}]},{"osd":5,"last update":"Wed Dec 30 08:59:24 2020","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1379999999999999,"15min":1.125},"min":{"1min":0.76500000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.6160000000000001,"5min":1.7,"15min":1.7},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.1879999999999999,"15min":1.2390000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5469999999999999,"5min":2.198,"15min":2.198},"last":1.198}]},{"osd":6,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":8.0190000000000001,"5min":27.074000000000002,"15min":37.076000000000001},"min":{"1min":0.77300000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":106.654,"5min":861.60799999999995,"15min":861.60799999999995},"last":1.177},{"interface":"front","average":{"1min":52.908999999999999,"5min":11.640000000000001,"15min":4.7830000000000004},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":778.01900000000001,"5min":778.01900000000001,"15min":778.01900000000001},"last":1.3280000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617820,"kb_used_data":597728,"kb_used_omap":0,"kb_used_meta":19968,"kb_avail":93754020,"statfs":{"total":96636764160,"available":96004116480,"internally_reserved":0,"allocated":612073472,"data_stored":608728491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20447232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:15 2020","interfaces":[{"interface":"back","average":{"1min":63.369,"5min":56.469000000000001,"15min":55.319000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":999.76300000000003,"5min":999.76300000000003,"15min":999.76300000000003},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.04,"15min":1.028},"min":{"1min":0.746,"5min":0.62,"15min":0.62},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":1.3069999999999999}]},{"osd":1,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.01,"15min":0.97699999999999998},"min":{"1min":0.79400000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.3480000000000001,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.875},{"interface":"front","average":{"1min":0.96799999999999997,"5min":36.709000000000003,"15min":52.585000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.3009999999999999,"5min":953.40800000000002,"15min":953.40800000000002},"last":1.1899999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":46.109999999999999,"5min":10.127000000000001,"15min":4.109},"min":{"1min":0.98899999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":714.93299999999999,"5min":714.93299999999999,"15min":714.93299999999999},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.206,"15min":1.1990000000000001},"min":{"1min":1.0309999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.4870000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.6060000000000001}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.151,"5min":1.139,"15min":1.137},"min":{"1min":0.94599999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.3919999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.274}]},{"osd":4,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.96199999999999997,"15min":0.97899999999999998},"min":{"1min":0.63600000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.393,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0740000000000001,"15min":1.0860000000000001},"min":{"1min":0.81299999999999994,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":1.375}]},{"osd":5,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.1020000000000001,"15min":1.123},"min":{"1min":0.77300000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.286,"5min":1.637,"15min":1.637},"last":1.3540000000000001},{"interface":"front","average":{"1min":33.957000000000001,"5min":47.049999999999997,"15min":60.192999999999998},"min":{"1min":0.81299999999999994,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":691.76199999999994,"5min":985.03399999999999,"15min":985.03399999999999},"last":1.3080000000000001}]},{"osd":7,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":21.876000000000001,"5min":5.3460000000000001,"15min":2.5979999999999999},"min":{"1min":0.86099999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":435.39499999999998,"5min":435.39499999999998,"15min":435.39499999999998},"last":1.331},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2330000000000001,"15min":1.2490000000000001},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.5740000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":247340,"kb_used_data":233900,"kb_used_omap":0,"kb_used_meta":13312,"kb_avail":94124500,"statfs":{"total":96636764160,"available":96383488000,"internally_reserved":0,"allocated":239513600,"data_stored":238202894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13631488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:13 2020","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.0569999999999999,"15min":1.0880000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2869999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77500000000000002,"15min":0.77600000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.74399999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":27.254000000000001,"15min":39.072000000000003},"min":{"1min":0.47199999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84799999999999998,"5min":882.37400000000002,"15min":882.37400000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.90400000000000003,"15min":0.92200000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.298,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed Dec 30 08:59:31 2020","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.95299999999999996,"15min":0.95699999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.29,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.86899999999999999,"15min":0.88200000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.302,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.80500000000000005}]},{"osd":4,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1339999999999999,"15min":1.1040000000000001},"min":{"1min":0.92200000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.3899999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.1599999999999999,"15min":1.169},"min":{"1min":0.73299999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.46,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.84699999999999998}]},{"osd":5,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2649999999999999,"15min":1.2829999999999999},"min":{"1min":0.89100000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.4159999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2390000000000001,"15min":1.2470000000000001},"min":{"1min":1.002,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.383,"5min":1.96,"15min":1.96},"last":1.0229999999999999}]},{"osd":6,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2390000000000001,"15min":1.2749999999999999},"min":{"1min":0.82299999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4259999999999999,"5min":4.4829999999999997,"15min":4.4829999999999997},"last":1.149},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.242,"15min":1.272},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.4159999999999999,"5min":1.734,"15min":1.734},"last":1.0069999999999999}]},{"osd":7,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":19.978999999999999,"15min":28.309000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.486,"5min":623.82100000000003,"15min":623.82100000000003},"last":828.46799999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.2250000000000001,"15min":1.2350000000000001},"min":{"1min":0.91700000000000004,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.3500000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.98699999999999999}]}]},{"osd":0,"up_from":75,"seq":322122547208,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410660,"kb_used_data":392612,"kb_used_omap":10,"kb_used_meta":17909,"kb_avail":93961180,"statfs":{"total":96636764160,"available":96216248320,"internally_reserved":0,"allocated":402034688,"data_stored":399673841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10297,"internal_metadata":18339783},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.649}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.679}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":179560,"kb_used_data":172264,"kb_used_omap":0,"kb_used_meta":7168,"kb_avail":94192280,"statfs":{"total":96636764160,"available":96452894720,"internally_reserved":0,"allocated":176398336,"data_stored":175142793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7340032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:15 2020","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.81999999999999995,"15min":0.81399999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.999,"5min":33.204000000000001,"15min":38.572000000000003},"min":{"1min":0.63500000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.8049999999999999,"5min":767.24300000000005,"15min":767.24300000000005},"last":0.67100000000000004}]},{"osd":1,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":30.774000000000001,"5min":7.2549999999999999,"15min":3.4510000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":754.12800000000004,"5min":754.12800000000004,"15min":754.12800000000004},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.91400000000000003,"15min":0.91100000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4430000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.80300000000000005}]},{"osd":3,"last update":"Wed Dec 30 08:59:29 2020","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.0629999999999999,"15min":1.0900000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.476,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.747,"15min":0.747},"min":{"1min":0.55500000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.89100000000000001}]},{"osd":4,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":7.5339999999999998,"15min":3.2320000000000002},"min":{"1min":0.67200000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6200000000000001,"5min":742.45799999999997,"15min":742.45799999999997},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.2110000000000001,"15min":1.2070000000000001},"min":{"1min":0.81699999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5449999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.1970000000000001}]},{"osd":5,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":2.4820000000000002,"15min":1.631},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.464,"5min":152.709,"15min":152.709},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.1919999999999999,"15min":1.214},"min":{"1min":0.80800000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.4039999999999999,"5min":2.181,"15min":2.181},"last":0.96899999999999997}]},{"osd":6,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":4.1260000000000003,"5min":29.439,"15min":41.308},"min":{"1min":0.753,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":73.947000000000003,"5min":874.04999999999995,"15min":874.04999999999995},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.1439999999999999,"15min":1.1819999999999999},"min":{"1min":0.73399999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.508,"5min":4.0439999999999996,"15min":4.0439999999999996},"last":1.0700000000000001}]},{"osd":7,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":20.821999999999999,"5min":21.945,"15min":26.760999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":489.90300000000002,"5min":502.05900000000003,"15min":502.05900000000003},"last":1.214},{"interface":"front","average":{"1min":1.3220000000000001,"5min":23.478000000000002,"15min":33.302999999999997},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":2.0270000000000001,"5min":502.10599999999999,"15min":502.10599999999999},"last":1.2290000000000001}]}]},{"osd":3,"up_from":41,"seq":176093659166,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400584,"kb_used_data":387072,"kb_used_omap":0,"kb_used_meta":13503,"kb_avail":93971256,"statfs":{"total":96636764160,"available":96226566144,"internally_reserved":0,"allocated":396361728,"data_stored":394019686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":972,"internal_metadata":13827124},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":1,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.93000000000000005,"15min":0.94999999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.242,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.84599999999999997,"15min":0.86799999999999999},"min":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90100000000000002,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.85499999999999998}]},{"osd":2,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.83699999999999997,"15min":0.85299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1319999999999999,"5min":1.581,"15min":1.581},"last":0.628},{"interface":"front","average":{"1min":26.884,"5min":6.1639999999999997,"15min":2.71},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":469.85500000000002,"5min":469.85500000000002,"15min":469.85500000000002},"last":0.82099999999999995}]},{"osd":4,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.0760000000000001,"15min":1.0840000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.254,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.23,"15min":1.254},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.4079999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.90800000000000003}]},{"osd":5,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":25.748000000000001,"15min":29.870000000000001},"min":{"1min":0.81799999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.2010000000000001,"5min":524.471,"15min":524.471},"last":0.88},{"interface":"front","average":{"1min":47.161999999999999,"5min":10.489000000000001,"15min":4.3769999999999998},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":830.05200000000002,"5min":830.05200000000002,"15min":830.05200000000002},"last":0.94699999999999995}]},{"osd":6,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.254,"15min":1.258},"min":{"1min":1.046,"5min":1.046,"15min":1.046},"max":{"1min":1.5960000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.181},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.266,"15min":1.268},"min":{"1min":1.071,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.6259999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.204}]},{"osd":7,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.298,"15min":1.3220000000000001},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.3009999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.089},{"interface":"front","average":{"1min":1.159,"5min":1.264,"15min":1.282},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.1599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":444756,"kb_used_data":430292,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":93927084,"statfs":{"total":96636764160,"available":96181334016,"internally_reserved":0,"allocated":440619008,"data_stored":438222994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:57:13 2020","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.298},{"interface":"front","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.3360000000000001}]},{"osd":1,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.119,"15min":1.1479999999999999},"min":{"1min":0.755,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.4039999999999999,"5min":1.661,"15min":1.661},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.022,"15min":1.012},"min":{"1min":0.78600000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4019999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.361}]},{"osd":2,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.161,"15min":1.1870000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.498,"5min":1.776,"15min":1.776},"last":1.282},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0940000000000001,"15min":1.1040000000000001},"min":{"1min":0.84499999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.446,"5min":1.478,"15min":1.478},"last":1.446}]},{"osd":3,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1000000000000001,"15min":1.0920000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.2050000000000001,"15min":1.2210000000000001},"min":{"1min":0.91400000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.222}]},{"osd":5,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.96399999999999997,"15min":0.94599999999999995},"min":{"1min":0.81399999999999995,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3440000000000001,"5min":1.45,"15min":1.45},"last":0.995},{"interface":"front","average":{"1min":1.014,"5min":0.98999999999999999,"15min":0.94899999999999995},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.226,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.26}]},{"osd":6,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.1160000000000001,"15min":1.153},"min":{"1min":0.73499999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.4950000000000001,"5min":1.71,"15min":1.71},"last":1.4199999999999999},{"interface":"front","average":{"1min":18.32,"5min":4.508,"15min":2.214},"min":{"1min":0.90100000000000002,"5min":0.629,"15min":0.629},"max":{"1min":276.41800000000001,"5min":276.41800000000001,"15min":276.41800000000001},"last":1.0489999999999999}]},{"osd":7,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":3.4380000000000002,"15min":1.8959999999999999},"min":{"1min":1.022,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.385,"5min":181.79499999999999,"15min":181.79499999999999},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":5.6840000000000002,"15min":2.7799999999999998},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":1.4510000000000001,"5min":353.72500000000002,"15min":353.72500000000002},"last":1.476}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":354768,"kb_used_data":340820,"kb_used_omap":0,"kb_used_meta":13824,"kb_avail":94017072,"statfs":{"total":96636764160,"available":96273481728,"internally_reserved":0,"allocated":348999680,"data_stored":347497227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14155776},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:12 2020","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.1439999999999999,"15min":1.1599999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.4239999999999999,"5min":1.526,"15min":1.526},"last":1},{"interface":"front","average":{"1min":42.222000000000001,"5min":9.3520000000000003,"15min":3.8740000000000001},"min":{"1min":0.83799999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":821.10900000000004,"5min":821.10900000000004,"15min":821.10900000000004},"last":0.89200000000000002}]},{"osd":1,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0009999999999999,"15min":0.98499999999999999},"min":{"1min":0.79100000000000004,"5min":0.627,"15min":0.627},"max":{"1min":1.2769999999999999,"5min":1.379,"15min":1.379},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":5.5030000000000001,"15min":2.5710000000000002},"min":{"1min":0.84099999999999997,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.3300000000000001,"5min":440.44999999999999,"15min":440.44999999999999},"last":0.91800000000000004}]},{"osd":2,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":7.0289999999999999,"15min":2.9950000000000001},"min":{"1min":0.752,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.294,"5min":605.79899999999998,"15min":605.79899999999998},"last":0.94599999999999995},{"interface":"front","average":{"1min":23.163,"5min":5.524,"15min":2.5800000000000001},"min":{"1min":0.88200000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":501.64600000000002,"5min":501.64600000000002,"15min":501.64600000000002},"last":1.329}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1419999999999999,"15min":1.1339999999999999},"min":{"1min":0.90600000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.411,"5min":1.494,"15min":1.494},"last":1.282},{"interface":"front","average":{"1min":1.159,"5min":17.323,"15min":20.016999999999999},"min":{"1min":0.92700000000000005,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.393,"5min":404.60300000000001,"15min":404.60300000000001},"last":1.0800000000000001}]},{"osd":4,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":21.57,"15min":22.916},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.4079999999999999,"5min":704.28399999999999,"15min":704.28399999999999},"last":1.113},{"interface":"front","average":{"1min":0.92100000000000004,"5min":46.520000000000003,"15min":66.030000000000001},"min":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.0920000000000001,"5min":861.90599999999995,"15min":861.90599999999995},"last":1.0529999999999999}]},{"osd":6,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0700000000000001,"15min":1.0660000000000001},"min":{"1min":0.66300000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.4299999999999999,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.085,"5min":1.1020000000000001,"15min":1.121},"min":{"1min":0.77700000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4410000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.145}]},{"osd":7,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.284,"15min":1.3919999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3899999999999999,"5min":8.3629999999999995,"15min":8.3629999999999995},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":11.603999999999999,"15min":12.991},"min":{"1min":0.79200000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.6000000000000001,"5min":291.65300000000002,"15min":291.65300000000002},"last":1.3089999999999999}]}]}]}} , 2020-12-30T08:59:48.214 INFO:tasks.ceph.ceph_manager.ceph:recovered! , 2020-12-30T08:59:48.215 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2020-12-30T23:00:24.325 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 33, in nested , yield vars , File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-12-30-1123/qa/tasks/ceph.py", line 1876, in task 2020-12-30T23:00:24.326 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T09:00:18.496 DEBUG:teuthology.orchestra.run.smithi169:> mktemp , 2020-12-30T09:00:18.513 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.C79Qha9RXd , 2020-12-30T09:00:18.514 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.C79Qha9RXd 2020-12-30T23:00:24.326 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T09:00:35.626 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- , 2020-12-30T09:00:35.644 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- , 2020-12-30T09:00:35.706 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2020-12-30T23:00:24.380 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.0.0-8653-gd21180a1 (d21180a1e12f36e61aced96f8de2276edb1d0150) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f69dd512816] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x55d303af9fe6] 2020-12-30T23:00:24.381 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.563908+0000", , 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.00019472900000000001 , 2020-12-30T08:57:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.382 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.187571661, , 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "type_data": { , 2020-12-30T08:57:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "delayed", 2020-12-30T23:00:24.382 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.107 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", , 2020-12-30T08:57:52.108 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.759798+0000", , 2020-12-30T08:57:52.108 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 5.1649e-05 2020-12-30T23:00:24.383 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "initiated", , 2020-12-30T08:57:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.611817+0000", 2020-12-30T23:00:24.384 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "started", , 2020-12-30T08:57:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.867908+0000", 2020-12-30T23:00:24.384 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "sub_op_commit_rec", , 2020-12-30T08:57:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.879692+0000", 2020-12-30T23:00:24.385 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.189 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.190 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.190 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.385 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.210 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", 2020-12-30T23:00:24.386 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.232 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", 2020-12-30T23:00:24.387 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "reached pg", , 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: "events": [ , 2020-12-30T08:57:52.252 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.387 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.272 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.002481568 , 2020-12-30T08:57:52.272 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.273 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.388 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.292 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 8.7658000000000007e-05 , 2020-12-30T08:57:52.293 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.293 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.388 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.87972017899999999 , 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.313 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2020-12-30T23:00:24.389 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "type_data": { , 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "flag_point": "started", , 2020-12-30T08:57:52.333 INFO:teuthology.orchestra.run.smithi169.stdout: "events": [ 2020-12-30T23:00:24.389 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:28.437546+0000", , 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 6.2249999999999997e-06 , 2020-12-30T08:57:52.353 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.390 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 600, , 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: "ops": [ , 2020-12-30T08:57:52.738 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.391 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "reached_pg", , 2020-12-30T08:57:52.759 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:03.755828+0000", 2020-12-30T23:00:24.391 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.780 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "header_read", 2020-12-30T23:00:24.392 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: } 2020-12-30T23:00:24.392 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.820 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4.3782000000000002e-05 , 2020-12-30T08:57:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.393 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:10.493947+0000", , 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4294967295.9999809 , 2020-12-30T08:57:52.841 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.394 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.310989+0000", , 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.012873951999999999 , 2020-12-30T08:57:52.862 INFO:teuthology.orchestra.run.smithi169.stdout: } 2020-12-30T23:00:24.394 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "queued_for_pg", , 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:17.586655+0000", , 2020-12-30T08:57:52.883 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4.7491000000000003e-05 2020-12-30T23:00:24.395 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", , 2020-12-30T08:57:52.903 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:33.372282+0000", 2020-12-30T23:00:24.395 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:44.022360+0000", , 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "age": 8.7134268660000007, , 2020-12-30T08:57:52.923 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.346765198, 2020-12-30T23:00:24.396 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:52.944 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "started", 2020-12-30T23:00:24.396 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.963 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.0046062250000000002 , 2020-12-30T08:57:52.963 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:52.964 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.397 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:52.983 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:52.984 INFO:teuthology.orchestra.run.smithi169.stdout:} , 2020-12-30T08:57:52.985 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-12-30T23:00:24.397 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", 2020-12-30T23:00:24.398 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.403 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0 , 2020-12-30T08:57:53.404 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.404 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.399 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: "description": "osd_op(client.4598.0:1164 4.0 4:249a18ca:::smithi04340772-260:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e34)", , 2020-12-30T08:57:53.426 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:04.096815+0000", 2020-12-30T23:00:24.399 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:09.599534+0000", , 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.036992197999999997 , 2020-12-30T08:57:53.448 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.400 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.470 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "all_read", , 2020-12-30T08:57:53.470 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.144872+0000", , 2020-12-30T08:57:53.471 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 1.0566e-05 2020-12-30T23:00:24.400 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "initiated", , 2020-12-30T08:57:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:16.156798+0000", 2020-12-30T23:00:24.401 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.513 INFO:teuthology.orchestra.run.smithi169.stdout: ] , 2020-12-30T08:57:53.513 INFO:teuthology.orchestra.run.smithi169.stdout: } , 2020-12-30T08:57:53.514 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.402 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.535 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.535 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.536 INFO:teuthology.orchestra.run.smithi169.stdout: "event": "queued_for_pg", 2020-12-30T23:00:24.402 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 4294967295.9999795 , 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: }, , 2020-12-30T08:57:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: { 2020-12-30T23:00:24.403 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: { , 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: "description": "osd_op(client.4598.0:2898 4.0 4:1f210125:::smithi04340772-284:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e38)", , 2020-12-30T08:57:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: "initiated_at": "2020-12-30T08:57:17.505116+0000", 2020-12-30T23:00:24.403 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:53.600 INFO:teuthology.orchestra.run.smithi169.stdout: "time": "2020-12-30T08:57:22.235621+0000", , 2020-12-30T08:57:53.600 INFO:teuthology.orchestra.run.smithi169.stdout: "duration": 0.035873975000000002 , 2020-12-30T08:57:53.601 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2020-12-30T23:00:24.404 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3320 , 2020-12-30T08:57:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.027 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2020-12-30T23:00:24.404 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:23.886767+0000", , 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8839999999999997e-06 , 2020-12-30T08:57:54.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:24.405 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:32.913785+0000", , 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049845687999999999 , 2020-12-30T08:57:54.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:24.406 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", , 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:27.332932+0000", , 2020-12-30T08:57:54.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2020-12-30T23:00:24.406 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", , 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:27.333311+0000", , 2020-12-30T08:57:54.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1931999999999999e-05 2020-12-30T23:00:24.407 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", , 2020-12-30T08:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:28.423645+0000", 2020-12-30T23:00:24.408 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for readable", , 2020-12-30T08:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:28.440591+0000", 2020-12-30T23:00:24.408 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { , 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4598", , 2020-12-30T08:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "v1:172.21.15.43:0/972417042", 2020-12-30T23:00:24.409 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: { , 2020-12-30T08:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2020-12-30T23:00:24.409 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 , 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, , 2020-12-30T08:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2020-12-30T23:00:24.410 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { , 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", , 2020-12-30T08:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2020-12-30T23:00:24.411 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.249599+0000", , 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4746000000000001e-05 , 2020-12-30T08:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:24.411 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.636403+0000", , 2020-12-30T08:57:54.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0044951799999999997 , 2020-12-30T08:57:54.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2020-12-30T23:00:24.412 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2020-12-30T08:57:47.655240+0000", , 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047171491000000003 , 2020-12-30T08:57:54.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2020-12-30T23:00:24.413 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:58:15.685 INFO:tasks.ceph.osd.0:Stopped , 2020-12-30T08:58:15.685 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 , osd.0 is already down. 2020-12-30T23:00:24.415 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:59:26.097 INFO:teuthology.orchestra.run.smithi043.stdout:mon.a: {} , 2020-12-30T08:59:26.113 INFO:teuthology.orchestra.run.smithi043.stdout:mon.b: {} , 2020-12-30T08:59:26.127 INFO:teuthology.orchestra.run.smithi043.stdout:mon.c: {} 2020-12-30T23:00:24.416 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T08:59:48.212 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2020-12-30T08:59:47.527120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1407295001,"num_objects":477,"num_object_clones":0,"num_object_copies":1431,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":449,"num_whiteouts":3,"num_read":3286,"num_read_kb":2359369,"num_write":3461,"num_write_kb":1206573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":773,"num_bytes_recovered":1169499301,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":887,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":44,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2967,"ondisk_log_size":2967,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2702316,"kb_used_data":2595628,"kb_used_omap":11,"kb_used_meta":105780,"kb_avail":752272404,"statfs":{"total":773094113280,"available":770326941696,"internally_reserved":0,"allocated":2657923072,"data_stored":2643000988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11269,"internal_metadata":108319739},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.529700"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":"111","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252322+0000","last_change":"2020-12-30T08:59:26.805818+0000","last_active":"2020-12-30T08:59:40.252322+0000","last_peered":"2020-12-30T08:59:40.252322+0000","last_clean":"2020-12-30T08:59:40.252322+0000","last_became_active":"2020-12-30T08:59:26.805280+0000","last_became_peered":"2020-12-30T08:59:26.805280+0000","last_unstale":"2020-12-30T08:59:40.252322+0000","last_undegraded":"2020-12-30T08:59:40.252322+0000","last_fullsized":"2020-12-30T08:59:40.252322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:04.206849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:04.206849+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"57'44","reported_seq":"156","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252340+0000","last_change":"2020-12-30T08:59:09.166665+0000","last_active":"2020-12-30T08:59:40.252340+0000","last_peered":"2020-12-30T08:59:40.252340+0000","last_clean":"2020-12-30T08:59:40.252340+0000","last_became_active":"2020-12-30T08:56:39.905257+0000","last_became_peered":"2020-12-30T08:56:39.905257+0000","last_unstale":"2020-12-30T08:59:40.252340+0000","last_undegraded":"2020-12-30T08:59:40.252340+0000","last_fullsized":"2020-12-30T08:59:40.252340+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'44","last_scrub_stamp":"2020-12-30T08:59:09.166549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:09.166549+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63684286,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":42,"num_read_kb":51083,"num_write":29,"num_write_kb":4590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":319,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":25,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.578479+0000","last_change":"2020-12-30T08:58:46.630261+0000","last_active":"2020-12-30T08:59:40.578479+0000","last_peered":"2020-12-30T08:59:40.578479+0000","last_clean":"2020-12-30T08:59:40.578479+0000","last_became_active":"2020-12-30T08:56:17.453297+0000","last_became_peered":"2020-12-30T08:56:17.453297+0000","last_unstale":"2020-12-30T08:59:40.578479+0000","last_undegraded":"2020-12-30T08:59:40.578479+0000","last_fullsized":"2020-12-30T08:59:40.578479+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:46.630148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:46.630148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"57'794","reported_seq":"1885","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.596274+0000","last_change":"2020-12-30T08:59:31.962315+0000","last_active":"2020-12-30T08:59:41.596274+0000","last_peered":"2020-12-30T08:59:41.596274+0000","last_clean":"2020-12-30T08:59:41.596274+0000","last_became_active":"2020-12-30T08:59:31.961183+0000","last_became_peered":"2020-12-30T08:59:31.961183+0000","last_unstale":"2020-12-30T08:59:41.596274+0000","last_undegraded":"2020-12-30T08:59:41.596274+0000","last_fullsized":"2020-12-30T08:59:41.596274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'794","last_scrub_stamp":"2020-12-30T08:59:09.405174+0000","last_deep_scrub":"54'709","last_deep_scrub_stamp":"2020-12-30T08:57:41.784481+0000","last_clean_scrub_stamp":"2020-12-30T08:59:09.405174+0000","log_size":794,"ondisk_log_size":794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":324600875,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":956,"num_read_kb":765811,"num_write":895,"num_write_kb":309669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":381323268,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"125","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.432965+0000","last_change":"2020-12-30T08:59:26.807827+0000","last_active":"2020-12-30T08:59:40.432965+0000","last_peered":"2020-12-30T08:59:40.432965+0000","last_clean":"2020-12-30T08:59:40.432965+0000","last_became_active":"2020-12-30T08:59:26.807377+0000","last_became_peered":"2020-12-30T08:59:26.807377+0000","last_unstale":"2020-12-30T08:59:40.432965+0000","last_undegraded":"2020-12-30T08:59:40.432965+0000","last_fullsized":"2020-12-30T08:59:40.432965+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.818206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.818206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"57'598","reported_seq":"1017","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252337+0000","last_change":"2020-12-30T08:59:08.196312+0000","last_active":"2020-12-30T08:59:40.252337+0000","last_peered":"2020-12-30T08:59:40.252337+0000","last_clean":"2020-12-30T08:59:40.252337+0000","last_became_active":"2020-12-30T08:56:35.883250+0000","last_became_peered":"2020-12-30T08:56:35.883250+0000","last_unstale":"2020-12-30T08:59:40.252337+0000","last_undegraded":"2020-12-30T08:59:40.252337+0000","last_fullsized":"2020-12-30T08:59:40.252337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'598","last_scrub_stamp":"2020-12-30T08:59:08.196199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:34.853614+0000","last_clean_scrub_stamp":"2020-12-30T08:59:08.196199+0000","log_size":598,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":299758950,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":689,"num_read_kb":446464,"num_write":755,"num_write_kb":264543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.578494+0000","last_change":"2020-12-30T08:58:53.577047+0000","last_active":"2020-12-30T08:59:40.578494+0000","last_peered":"2020-12-30T08:59:40.578494+0000","last_clean":"2020-12-30T08:59:40.578494+0000","last_became_active":"2020-12-30T08:56:18.363833+0000","last_became_peered":"2020-12-30T08:56:18.363833+0000","last_unstale":"2020-12-30T08:59:40.578494+0000","last_undegraded":"2020-12-30T08:59:40.578494+0000","last_fullsized":"2020-12-30T08:59:40.578494+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:53.576887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:53.576887+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":"145","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001415+0000","last_change":"2020-12-30T08:59:29.950684+0000","last_active":"2020-12-30T08:59:41.001415+0000","last_peered":"2020-12-30T08:59:41.001415+0000","last_clean":"2020-12-30T08:59:41.001415+0000","last_became_active":"2020-12-30T08:59:29.950219+0000","last_became_peered":"2020-12-30T08:59:29.950219+0000","last_unstale":"2020-12-30T08:59:41.001415+0000","last_undegraded":"2020-12-30T08:59:41.001415+0000","last_fullsized":"2020-12-30T08:59:41.001415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":4,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'2","last_scrub_stamp":"2020-12-30T08:58:43.023209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:05.570594+0000","last_clean_scrub_stamp":"2020-12-30T08:58:43.023209+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"57'737","reported_seq":"1556","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.577945+0000","last_change":"2020-12-30T08:59:27.273198+0000","last_active":"2020-12-30T08:59:40.577945+0000","last_peered":"2020-12-30T08:59:40.577945+0000","last_clean":"2020-12-30T08:59:40.577945+0000","last_became_active":"2020-12-30T08:57:34.885212+0000","last_became_peered":"2020-12-30T08:57:34.885212+0000","last_unstale":"2020-12-30T08:59:40.577945+0000","last_undegraded":"2020-12-30T08:59:40.577945+0000","last_fullsized":"2020-12-30T08:59:40.577945+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'737","last_scrub_stamp":"2020-12-30T08:59:27.273101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:34.853614+0000","last_clean_scrub_stamp":"2020-12-30T08:59:27.273101+0000","log_size":737,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":370415644,"num_objects":124,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":0,"num_read":829,"num_read_kb":576628,"num_write":921,"num_write_kb":353530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":210025962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"112","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001472+0000","last_change":"2020-12-30T08:59:26.807466+0000","last_active":"2020-12-30T08:59:41.001472+0000","last_peered":"2020-12-30T08:59:41.001472+0000","last_clean":"2020-12-30T08:59:41.001472+0000","last_became_active":"2020-12-30T08:59:26.806602+0000","last_became_peered":"2020-12-30T08:59:26.806602+0000","last_unstale":"2020-12-30T08:59:41.001472+0000","last_undegraded":"2020-12-30T08:59:41.001472+0000","last_fullsized":"2020-12-30T08:59:41.001472+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:06.756813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:06.756813+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"76'751","reported_seq":"1682","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.000117+0000","last_change":"2020-12-30T08:59:30.952423+0000","last_active":"2020-12-30T08:59:41.000117+0000","last_peered":"2020-12-30T08:59:41.000117+0000","last_clean":"2020-12-30T08:59:41.000117+0000","last_became_active":"2020-12-30T08:59:30.951003+0000","last_became_peered":"2020-12-30T08:59:30.951003+0000","last_unstale":"2020-12-30T08:59:41.000117+0000","last_undegraded":"2020-12-30T08:59:41.000117+0000","last_fullsized":"2020-12-30T08:59:41.000117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'751","last_scrub_stamp":"2020-12-30T08:59:17.342114+0000","last_deep_scrub":"76'751","last_deep_scrub_stamp":"2020-12-30T08:59:17.342114+0000","last_clean_scrub_stamp":"2020-12-30T08:59:17.342114+0000","log_size":751,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":288680016,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":742,"num_read_kb":486335,"num_write":828,"num_write_kb":269542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":495679005,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"57'31","reported_seq":"176","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.000230+0000","last_change":"2020-12-30T08:59:30.953648+0000","last_active":"2020-12-30T08:59:41.000230+0000","last_peered":"2020-12-30T08:59:41.000230+0000","last_clean":"2020-12-30T08:59:41.000230+0000","last_became_active":"2020-12-30T08:59:30.952821+0000","last_became_peered":"2020-12-30T08:59:30.952821+0000","last_unstale":"2020-12-30T08:59:41.000230+0000","last_undegraded":"2020-12-30T08:59:41.000230+0000","last_fullsized":"2020-12-30T08:59:41.000230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'31","last_scrub_stamp":"2020-12-30T08:59:18.654169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:18.654169+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41801084,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":14,"num_read_kb":4410,"num_write":24,"num_write_kb":3616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":82471066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":293,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":14,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"115","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.001437+0000","last_change":"2020-12-30T08:59:26.807130+0000","last_active":"2020-12-30T08:59:41.001437+0000","last_peered":"2020-12-30T08:59:41.001437+0000","last_clean":"2020-12-30T08:59:41.001437+0000","last_became_active":"2020-12-30T08:59:26.806409+0000","last_became_peered":"2020-12-30T08:59:26.806409+0000","last_unstale":"2020-12-30T08:59:41.001437+0000","last_undegraded":"2020-12-30T08:59:41.001437+0000","last_fullsized":"2020-12-30T08:59:41.001437+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.095702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.095702+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"90","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.252325+0000","last_change":"2020-12-30T08:58:49.511387+0000","last_active":"2020-12-30T08:59:40.252325+0000","last_peered":"2020-12-30T08:59:40.252325+0000","last_clean":"2020-12-30T08:59:40.252325+0000","last_became_active":"2020-12-30T08:56:18.101527+0000","last_became_peered":"2020-12-30T08:56:18.101527+0000","last_unstale":"2020-12-30T08:59:40.252325+0000","last_undegraded":"2020-12-30T08:59:40.252325+0000","last_fullsized":"2020-12-30T08:59:40.252325+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:49.511270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:49.511270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"99","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.577925+0000","last_change":"2020-12-30T08:59:26.808227+0000","last_active":"2020-12-30T08:59:40.577925+0000","last_peered":"2020-12-30T08:59:40.577925+0000","last_clean":"2020-12-30T08:59:40.577925+0000","last_became_active":"2020-12-30T08:59:26.807467+0000","last_became_peered":"2020-12-30T08:59:26.807467+0000","last_unstale":"2020-12-30T08:59:40.577925+0000","last_undegraded":"2020-12-30T08:59:40.577925+0000","last_fullsized":"2020-12-30T08:59:40.577925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:59:03.951459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:59:03.951459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"92","reported_epoch":"93","state":"active+clean","last_fresh":"2020-12-30T08:59:40.310721+0000","last_change":"2020-12-30T08:58:34.893621+0000","last_active":"2020-12-30T08:59:40.310721+0000","last_peered":"2020-12-30T08:59:40.310721+0000","last_clean":"2020-12-30T08:59:40.310721+0000","last_became_active":"2020-12-30T08:56:17.453289+0000","last_became_peered":"2020-12-30T08:56:17.453289+0000","last_unstale":"2020-12-30T08:59:40.310721+0000","last_undegraded":"2020-12-30T08:59:40.310721+0000","last_fullsized":"2020-12-30T08:59:40.310721+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-12-30T08:58:34.893489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:16.435655+0000","last_clean_scrub_stamp":"2020-12-30T08:58:34.893489+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"57'10","reported_seq":"102","reported_epoch":"94","state":"active+clean","last_fresh":"2020-12-30T08:59:41.596156+0000","last_change":"2020-12-30T08:58:50.189729+0000","last_active":"2020-12-30T08:59:41.596156+0000","last_peered":"2020-12-30T08:59:41.596156+0000","last_clean":"2020-12-30T08:59:41.596156+0000","last_became_active":"2020-12-30T08:56:40.286191+0000","last_became_peered":"2020-12-30T08:56:40.286191+0000","last_unstale":"2020-12-30T08:59:41.596156+0000","last_undegraded":"2020-12-30T08:59:41.596156+0000","last_fullsized":"2020-12-30T08:59:41.596156+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'10","last_scrub_stamp":"2020-12-30T08:58:50.189625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-12-30T08:56:38.870267+0000","last_clean_scrub_stamp":"2020-12-30T08:58:50.189625+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18354146,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":28638,"num_write":7,"num_write_kb":1074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":275,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":5,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":4,"stat_sum":{"num_bytes":123839516,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":3,"num_read":70,"num_read_kb":84131,"num_write":60,"num_write_kb":9280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":82471066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":887,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":44,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":363405312,"data_stored":363060657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":12,"acting":12,"num_store_stats":7},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":1283455485,"num_objects":428,"num_object_clones":0,"num_object_copies":1284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":428,"num_whiteouts":0,"num_read":3216,"num_read_kb":2275238,"num_write":3399,"num_write_kb":1197284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":739,"num_bytes_recovered":1087028235,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2289242112,"data_stored":2277676566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2880,"ondisk_log_size":2880,"up":12,"acting":12,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10297,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46828,"kb_used_data":40940,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":94325012,"statfs":{"total":96636764160,"available":96588812288,"internally_reserved":0,"allocated":41922560,"data_stored":41513062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:57:16 2020","interfaces":[{"interface":"back","average":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":81.242000000000004,"5min":81.242000000000004,"15min":81.242000000000004},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1.0409999999999999}]},{"osd":1,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":19.870000000000001,"15min":28.206},"min":{"1min":0.83199999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4970000000000001,"5min":501.495,"15min":501.495},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":26.629000000000001,"15min":25.437000000000001},"min":{"1min":0.73399999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.5720000000000001,"5min":958.63099999999997,"15min":958.63099999999997},"last":1.2589999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":59.027999999999999,"5min":12.720000000000001,"15min":5.008},"min":{"1min":0.85799999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":927.80799999999999,"5min":927.80799999999999,"15min":927.80799999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":21.974,"5min":12.337999999999999,"15min":4.8849999999999998},"min":{"1min":0.98799999999999999,"5min":0.753,"15min":0.753},"max":{"1min":333.58300000000003,"5min":597.27800000000002,"15min":597.27800000000002},"last":1.093}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.202,"15min":1.216},"min":{"1min":0.81399999999999995,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.45,"5min":1.474,"15min":1.474},"last":1.2310000000000001},{"interface":"front","average":{"1min":8.3640000000000008,"5min":2.5859999999999999,"15min":1.623},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":123.247,"5min":123.247,"15min":123.247},"last":0.84299999999999997}]},{"osd":4,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":26.962,"15min":38.468000000000004},"min":{"1min":0.81499999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5609999999999999,"5min":747.37,"15min":747.37},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":1.052,"15min":1.0920000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.486,"5min":1.623,"15min":1.623},"last":1.1080000000000001}]},{"osd":5,"last update":"Wed Dec 30 08:59:24 2020","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1379999999999999,"15min":1.125},"min":{"1min":0.76500000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.6160000000000001,"5min":1.7,"15min":1.7},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.1879999999999999,"15min":1.2390000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5469999999999999,"5min":2.198,"15min":2.198},"last":1.198}]},{"osd":6,"last update":"Wed Dec 30 08:59:19 2020","interfaces":[{"interface":"back","average":{"1min":8.0190000000000001,"5min":27.074000000000002,"15min":37.076000000000001},"min":{"1min":0.77300000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":106.654,"5min":861.60799999999995,"15min":861.60799999999995},"last":1.177},{"interface":"front","average":{"1min":52.908999999999999,"5min":11.640000000000001,"15min":4.7830000000000004},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":778.01900000000001,"5min":778.01900000000001,"15min":778.01900000000001},"last":1.3280000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617820,"kb_used_data":597728,"kb_used_omap":0,"kb_used_meta":19968,"kb_avail":93754020,"statfs":{"total":96636764160,"available":96004116480,"internally_reserved":0,"allocated":612073472,"data_stored":608728491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":20447232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:15 2020","interfaces":[{"interface":"back","average":{"1min":63.369,"5min":56.469000000000001,"15min":55.319000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":999.76300000000003,"5min":999.76300000000003,"15min":999.76300000000003},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.04,"15min":1.028},"min":{"1min":0.746,"5min":0.62,"15min":0.62},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":1.3069999999999999}]},{"osd":1,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.01,"15min":0.97699999999999998},"min":{"1min":0.79400000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.3480000000000001,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.875},{"interface":"front","average":{"1min":0.96799999999999997,"5min":36.709000000000003,"15min":52.585000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.3009999999999999,"5min":953.40800000000002,"15min":953.40800000000002},"last":1.1899999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":46.109999999999999,"5min":10.127000000000001,"15min":4.109},"min":{"1min":0.98899999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":714.93299999999999,"5min":714.93299999999999,"15min":714.93299999999999},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.206,"15min":1.1990000000000001},"min":{"1min":1.0309999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.4870000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.6060000000000001}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.151,"5min":1.139,"15min":1.137},"min":{"1min":0.94599999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.3919999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.274}]},{"osd":4,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.96199999999999997,"15min":0.97899999999999998},"min":{"1min":0.63600000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.393,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0740000000000001,"15min":1.0860000000000001},"min":{"1min":0.81299999999999994,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":1.375}]},{"osd":5,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.1020000000000001,"15min":1.123},"min":{"1min":0.77300000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.286,"5min":1.637,"15min":1.637},"last":1.3540000000000001},{"interface":"front","average":{"1min":33.957000000000001,"5min":47.049999999999997,"15min":60.192999999999998},"min":{"1min":0.81299999999999994,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":691.76199999999994,"5min":985.03399999999999,"15min":985.03399999999999},"last":1.3080000000000001}]},{"osd":7,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":21.876000000000001,"5min":5.3460000000000001,"15min":2.5979999999999999},"min":{"1min":0.86099999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":435.39499999999998,"5min":435.39499999999998,"15min":435.39499999999998},"last":1.331},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2330000000000001,"15min":1.2490000000000001},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.5740000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":247340,"kb_used_data":233900,"kb_used_omap":0,"kb_used_meta":13312,"kb_avail":94124500,"statfs":{"total":96636764160,"available":96383488000,"internally_reserved":0,"allocated":239513600,"data_stored":238202894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13631488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:13 2020","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.0569999999999999,"15min":1.0880000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2869999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77500000000000002,"15min":0.77600000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.74399999999999999}]},{"osd":2,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":27.254000000000001,"15min":39.072000000000003},"min":{"1min":0.47199999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84799999999999998,"5min":882.37400000000002,"15min":882.37400000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.90400000000000003,"15min":0.92200000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.298,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed Dec 30 08:59:31 2020","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.95299999999999996,"15min":0.95699999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.29,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.86899999999999999,"15min":0.88200000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.302,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.80500000000000005}]},{"osd":4,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1339999999999999,"15min":1.1040000000000001},"min":{"1min":0.92200000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.3899999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.1599999999999999,"15min":1.169},"min":{"1min":0.73299999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.46,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.84699999999999998}]},{"osd":5,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2649999999999999,"15min":1.2829999999999999},"min":{"1min":0.89100000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.4159999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2390000000000001,"15min":1.2470000000000001},"min":{"1min":1.002,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.383,"5min":1.96,"15min":1.96},"last":1.0229999999999999}]},{"osd":6,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2390000000000001,"15min":1.2749999999999999},"min":{"1min":0.82299999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4259999999999999,"5min":4.4829999999999997,"15min":4.4829999999999997},"last":1.149},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.242,"15min":1.272},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.4159999999999999,"5min":1.734,"15min":1.734},"last":1.0069999999999999}]},{"osd":7,"last update":"Wed Dec 30 08:59:13 2020","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":19.978999999999999,"15min":28.309000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.486,"5min":623.82100000000003,"15min":623.82100000000003},"last":828.46799999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.2250000000000001,"15min":1.2350000000000001},"min":{"1min":0.91700000000000004,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.3500000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.98699999999999999}]}]},{"osd":0,"up_from":75,"seq":322122547208,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410660,"kb_used_data":392612,"kb_used_omap":10,"kb_used_meta":17909,"kb_avail":93961180,"statfs":{"total":96636764160,"available":96216248320,"internally_reserved":0,"allocated":402034688,"data_stored":399673841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10297,"internal_metadata":18339783},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.649}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.679}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":179560,"kb_used_data":172264,"kb_used_omap":0,"kb_used_meta":7168,"kb_avail":94192280,"statfs":{"total":96636764160,"available":96452894720,"internally_reserved":0,"allocated":176398336,"data_stored":175142793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7340032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:15 2020","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.81999999999999995,"15min":0.81399999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.999,"5min":33.204000000000001,"15min":38.572000000000003},"min":{"1min":0.63500000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.8049999999999999,"5min":767.24300000000005,"15min":767.24300000000005},"last":0.67100000000000004}]},{"osd":1,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":30.774000000000001,"5min":7.2549999999999999,"15min":3.4510000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":754.12800000000004,"5min":754.12800000000004,"15min":754.12800000000004},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.91400000000000003,"15min":0.91100000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4430000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.80300000000000005}]},{"osd":3,"last update":"Wed Dec 30 08:59:29 2020","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.0629999999999999,"15min":1.0900000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.476,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.747,"15min":0.747},"min":{"1min":0.55500000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.89100000000000001}]},{"osd":4,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":7.5339999999999998,"15min":3.2320000000000002},"min":{"1min":0.67200000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6200000000000001,"5min":742.45799999999997,"15min":742.45799999999997},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.2110000000000001,"15min":1.2070000000000001},"min":{"1min":0.81699999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5449999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.1970000000000001}]},{"osd":5,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":2.4820000000000002,"15min":1.631},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.464,"5min":152.709,"15min":152.709},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.1919999999999999,"15min":1.214},"min":{"1min":0.80800000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.4039999999999999,"5min":2.181,"15min":2.181},"last":0.96899999999999997}]},{"osd":6,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":4.1260000000000003,"5min":29.439,"15min":41.308},"min":{"1min":0.753,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":73.947000000000003,"5min":874.04999999999995,"15min":874.04999999999995},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.1439999999999999,"15min":1.1819999999999999},"min":{"1min":0.73399999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.508,"5min":4.0439999999999996,"15min":4.0439999999999996},"last":1.0700000000000001}]},{"osd":7,"last update":"Wed Dec 30 08:59:16 2020","interfaces":[{"interface":"back","average":{"1min":20.821999999999999,"5min":21.945,"15min":26.760999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":489.90300000000002,"5min":502.05900000000003,"15min":502.05900000000003},"last":1.214},{"interface":"front","average":{"1min":1.3220000000000001,"5min":23.478000000000002,"15min":33.302999999999997},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":2.0270000000000001,"5min":502.10599999999999,"15min":502.10599999999999},"last":1.2290000000000001}]}]},{"osd":3,"up_from":41,"seq":176093659166,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400584,"kb_used_data":387072,"kb_used_omap":0,"kb_used_meta":13503,"kb_avail":93971256,"statfs":{"total":96636764160,"available":96226566144,"internally_reserved":0,"allocated":396361728,"data_stored":394019686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":972,"internal_metadata":13827124},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":1,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.93000000000000005,"15min":0.94999999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.242,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.84599999999999997,"15min":0.86799999999999999},"min":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90100000000000002,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.85499999999999998}]},{"osd":2,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.83699999999999997,"15min":0.85299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1319999999999999,"5min":1.581,"15min":1.581},"last":0.628},{"interface":"front","average":{"1min":26.884,"5min":6.1639999999999997,"15min":2.71},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":469.85500000000002,"5min":469.85500000000002,"15min":469.85500000000002},"last":0.82099999999999995}]},{"osd":4,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.0760000000000001,"15min":1.0840000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.254,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.23,"15min":1.254},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.4079999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.90800000000000003}]},{"osd":5,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":25.748000000000001,"15min":29.870000000000001},"min":{"1min":0.81799999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.2010000000000001,"5min":524.471,"15min":524.471},"last":0.88},{"interface":"front","average":{"1min":47.161999999999999,"5min":10.489000000000001,"15min":4.3769999999999998},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":830.05200000000002,"5min":830.05200000000002,"15min":830.05200000000002},"last":0.94699999999999995}]},{"osd":6,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.254,"15min":1.258},"min":{"1min":1.046,"5min":1.046,"15min":1.046},"max":{"1min":1.5960000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.181},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.266,"15min":1.268},"min":{"1min":1.071,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.6259999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.204}]},{"osd":7,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.298,"15min":1.3220000000000001},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.3009999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.089},{"interface":"front","average":{"1min":1.159,"5min":1.264,"15min":1.282},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.1599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":444756,"kb_used_data":430292,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":93927084,"statfs":{"total":96636764160,"available":96181334016,"internally_reserved":0,"allocated":440619008,"data_stored":438222994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:57:13 2020","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.298},{"interface":"front","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.3360000000000001}]},{"osd":1,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.119,"15min":1.1479999999999999},"min":{"1min":0.755,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.4039999999999999,"5min":1.661,"15min":1.661},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.022,"15min":1.012},"min":{"1min":0.78600000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4019999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.361}]},{"osd":2,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.161,"15min":1.1870000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.498,"5min":1.776,"15min":1.776},"last":1.282},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0940000000000001,"15min":1.1040000000000001},"min":{"1min":0.84499999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.446,"5min":1.478,"15min":1.478},"last":1.446}]},{"osd":3,"last update":"Wed Dec 30 08:59:28 2020","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1000000000000001,"15min":1.0920000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.2050000000000001,"15min":1.2210000000000001},"min":{"1min":0.91400000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.222}]},{"osd":5,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.96399999999999997,"15min":0.94599999999999995},"min":{"1min":0.81399999999999995,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3440000000000001,"5min":1.45,"15min":1.45},"last":0.995},{"interface":"front","average":{"1min":1.014,"5min":0.98999999999999999,"15min":0.94899999999999995},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.226,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.26}]},{"osd":6,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.1160000000000001,"15min":1.153},"min":{"1min":0.73499999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.4950000000000001,"5min":1.71,"15min":1.71},"last":1.4199999999999999},{"interface":"front","average":{"1min":18.32,"5min":4.508,"15min":2.214},"min":{"1min":0.90100000000000002,"5min":0.629,"15min":0.629},"max":{"1min":276.41800000000001,"5min":276.41800000000001,"15min":276.41800000000001},"last":1.0489999999999999}]},{"osd":7,"last update":"Wed Dec 30 08:59:17 2020","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":3.4380000000000002,"15min":1.8959999999999999},"min":{"1min":1.022,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.385,"5min":181.79499999999999,"15min":181.79499999999999},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":5.6840000000000002,"15min":2.7799999999999998},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":1.4510000000000001,"5min":353.72500000000002,"15min":353.72500000000002},"last":1.476}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":354768,"kb_used_data":340820,"kb_used_omap":0,"kb_used_meta":13824,"kb_avail":94017072,"statfs":{"total":96636764160,"available":96273481728,"internally_reserved":0,"allocated":348999680,"data_stored":347497227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14155776},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 30 08:58:12 2020","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.1439999999999999,"15min":1.1599999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.4239999999999999,"5min":1.526,"15min":1.526},"last":1},{"interface":"front","average":{"1min":42.222000000000001,"5min":9.3520000000000003,"15min":3.8740000000000001},"min":{"1min":0.83799999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":821.10900000000004,"5min":821.10900000000004,"15min":821.10900000000004},"last":0.89200000000000002}]},{"osd":1,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0009999999999999,"15min":0.98499999999999999},"min":{"1min":0.79100000000000004,"5min":0.627,"15min":0.627},"max":{"1min":1.2769999999999999,"5min":1.379,"15min":1.379},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":5.5030000000000001,"15min":2.5710000000000002},"min":{"1min":0.84099999999999997,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.3300000000000001,"5min":440.44999999999999,"15min":440.44999999999999},"last":0.91800000000000004}]},{"osd":2,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":7.0289999999999999,"15min":2.9950000000000001},"min":{"1min":0.752,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.294,"5min":605.79899999999998,"15min":605.79899999999998},"last":0.94599999999999995},{"interface":"front","average":{"1min":23.163,"5min":5.524,"15min":2.5800000000000001},"min":{"1min":0.88200000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":501.64600000000002,"5min":501.64600000000002,"15min":501.64600000000002},"last":1.329}]},{"osd":3,"last update":"Wed Dec 30 08:59:25 2020","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1419999999999999,"15min":1.1339999999999999},"min":{"1min":0.90600000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.411,"5min":1.494,"15min":1.494},"last":1.282},{"interface":"front","average":{"1min":1.159,"5min":17.323,"15min":20.016999999999999},"min":{"1min":0.92700000000000005,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.393,"5min":404.60300000000001,"15min":404.60300000000001},"last":1.0800000000000001}]},{"osd":4,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":21.57,"15min":22.916},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.4079999999999999,"5min":704.28399999999999,"15min":704.28399999999999},"last":1.113},{"interface":"front","average":{"1min":0.92100000000000004,"5min":46.520000000000003,"15min":66.030000000000001},"min":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.0920000000000001,"5min":861.90599999999995,"15min":861.90599999999995},"last":1.0529999999999999}]},{"osd":6,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0700000000000001,"15min":1.0660000000000001},"min":{"1min":0.66300000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.4299999999999999,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.085,"5min":1.1020000000000001,"15min":1.121},"min":{"1min":0.77700000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4410000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.145}]},{"osd":7,"last update":"Wed Dec 30 08:59:14 2020","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.284,"15min":1.3919999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3899999999999999,"5min":8.3629999999999995,"15min":8.3629999999999995},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":11.603999999999999,"15min":12.991},"min":{"1min":0.79200000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.6000000000000001,"5min":291.65300000000002,"15min":291.65300000000002},"last":1.3089999999999999}]}]}]}} , 2020-12-30T08:59:48.214 INFO:tasks.ceph.ceph_manager.ceph:recovered! , 2020-12-30T08:59:48.215 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2020-12-30T23:00:24.418 WARNING:scrape:Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 33, in nested , yield vars , File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-12-30-1123/qa/tasks/ceph.py", line 1876, in task 2020-12-30T23:00:24.419 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T09:00:18.496 DEBUG:teuthology.orchestra.run.smithi169:> mktemp , 2020-12-30T09:00:18.513 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.C79Qha9RXd , 2020-12-30T09:00:18.514 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.C79Qha9RXd 2020-12-30T23:00:24.419 WARNING:scrape:Ignoring malparsed backtrace: 2020-12-30T09:00:35.626 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- , 2020-12-30T09:00:35.644 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- , 2020-12-30T09:00:35.706 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2020-12-30T23:00:24.430 INFO:scrape:Found 13 distinct failure reasons 2020-12-30T23:00:24.432 INFO:scrape:Failure: Command failed (workunit test scrub/osd-scrub-snaps.sh) on smithi205 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21180a1e12f36e61aced96f8de2276edb1d0150 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh' 2020-12-30T23:00:24.432 INFO:scrape:2 jobs: ['5747740', '5747518'] 2020-12-30T23:00:24.433 INFO:scrape:suites intersection: ['workloads/scrub}'] 2020-12-30T23:00:24.433 INFO:scrape:suites union: ['rados/standalone/{mon_election/classic', 'rados/standalone/{mon_election/connectivity', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{ubuntu_latest}', 'workloads/scrub}'] 2020-12-30T23:00:24.434 INFO:scrape: 2020-12-30T23:00:24.434 INFO:scrape:Dead: b'2020-12-30T20:31:43.937 DEBUG:teuthology.exit:Finished running handlers' 2020-12-30T23:00:24.435 INFO:scrape:21 jobs: ['5747519', '5747690', '5747444', '5747456', '5747669', '5747739', '5747619', '5747621', '5747689', '5747742', '5747470', '5747393', '5747596', '5747500', '5747572', '5747509', '5747692', '5747356', '5747642', '5747668', '5747573'] 2020-12-30T23:00:24.435 INFO:scrape:suites intersection: [] 2020-12-30T23:00:24.436 INFO:scrape:suites union: ['1-install/nautilus', '1-pg-log-overrides/normal_pg_log', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', '2-recovery-overrides/{more-active-recovery}', '2-recovery-overrides/{more-async-partial-recovery}', '2-recovery-overrides/{more-async-recovery}', '2-recovery-overrides/{more-partial-recovery}', '2-repo_digest/defaut', '2-repo_digest/repo_digest', '3-start-upgrade', '4-wait', 'backoff/normal', 'backoff/peering', 'backoff/peering_and_degraded', 'ceph', 'clusters/{fixed-2', 'clusters/{openstack', 'crc-failures/bad_map_crc_failure', 'crc-failures/default', 'd-balancer/crush-compat', 'd-balancer/on', 'd-thrash/default/{default', 'distro$/{ubuntu_18.04_podman}', 'distro$/{ubuntu_18.04}', 'fixed-2', 'mon_election/classic', 'mon_election/connectivity', 'mon_election/connectivity}', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/osd-delay', 'msgr/async-v1only', 'msgr/async-v2only', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size', 'rados/cephadm/upgrade/{1-start', 'rados/objectstore/{backends/objectstore', 'rados/singleton-nomsgr/{all/lazy_omap_stats_output', 'rados/singleton-nomsgr/{all/msgr', 'rados/thrash-erasure-code-isa/{arch/x86_64', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'rados/verify/{centos_latest', 'recovery-overrides/{more-partial-recovery}', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{centos_8}}', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{ubuntu_latest}', 'supported-random-distro$/{ubuntu_latest}}', 'tasks/rados_api_tests', 'thrashers/careful', 'thrashers/mapgap', 'thrashers/morepggrow', 'thrashers/pggrow', 'thrashosds-health', 'thrashosds-health}', 'three-plus-one}', 'validater/valgrind}', 'workloads/cache-pool-snaps-readproxy}', 'workloads/cache-snaps}', 'workloads/cache}', 'workloads/ec-rados-plugin=isa-k=2-m=1}', 'workloads/rados_api_tests}', 'workloads/radosbench-high-concurrency}', 'workloads/rbd_cls}', 'workloads/set-chunks-read}', 'workloads/small-objects}', 'workloads/snaps-few-objects-localized}', 'workloads/snaps-few-objects}', 'workloads/test_rbd_api}'] 2020-12-30T23:00:24.436 INFO:scrape: 2020-12-30T23:00:24.437 INFO:scrape:Failure: Command failed on smithi062 with status 134: "/bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.2 -b 65536 --object-size 65536 -p unique_pool_0 bench 90 write'" 2020-12-30T23:00:24.437 INFO:scrape:2 jobs: ['5747491', '5747571'] 2020-12-30T23:00:24.437 INFO:scrape:suites intersection: ['mon_election/classic', 'rados', 'thrashosds-health'] 2020-12-30T23:00:24.438 INFO:scrape:suites union: ['1-install/nautilus-v2only', 'backoff/peering_and_degraded', 'ceph', 'cluster/{12-osds', 'clusters/{openstack', 'd-balancer/on', 'distro$/{ubuntu_18.04}', 'mon_election/classic', 'msgr-failures/fastclose', 'msgr-failures/few', 'objectstore/bluestore-hybrid', 'openstack}', 'rados', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size', 'rados/thrash-erasure-code-big/{ceph', 'recovery-overrides/{more-async-recovery}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/mapgap', 'thrashers/pggrow', 'thrashosds-health', 'three-plus-one}', 'workloads/ec-rados-plugin=jerasure-k=4-m=2}', 'workloads/radosbench}'] 2020-12-30T23:00:24.438 INFO:scrape: 2020-12-30T23:00:24.439 INFO:scrape:Failure: reached maximum tries (500) after waiting for 3000 seconds 2020-12-30T23:00:24.439 INFO:scrape:7 jobs: ['5747714', '5747398', '5747349', '5747457', '5747681', '5747728', '5747574'] 2020-12-30T23:00:24.440 INFO:scrape:suites intersection: [] 2020-12-30T23:00:24.440 INFO:scrape:suites union: ['1-install/octopus', '2-repo_digest/defaut', '2-repo_digest/repo_digest', '3-start-upgrade', '4-wait', 'backoff/normal', 'ceph', 'clusters/{openstack', 'd-balancer/on', 'distro$/{centos_8.0}', 'distro$/{centos_latest}', 'distro$/{rhel_8.0}', 'distro$/{rhel_latest}', 'distro$/{ubuntu_18.04}', 'fixed-2', 'mon_election/classic', 'mon_election/classic}', 'mon_election/connectivity}', 'msgr-failures/osd-delay', 'rados', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size', 'rados/cephadm/upgrade/{1-start', 'thrashers/morepggrow', 'thrashosds-health', 'three-plus-one}', 'workloads/radosbench}'] 2020-12-30T23:00:24.441 INFO:scrape: 2020-12-30T23:00:24.441 INFO:scrape:Failure: Command failed (workunit test cephadm/test_cephadm.sh) on smithi137 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21180a1e12f36e61aced96f8de2276edb1d0150 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' 2020-12-30T23:00:24.442 INFO:scrape:2 jobs: ['5747768', '5747614'] 2020-12-30T23:00:24.442 INFO:scrape:suites intersection: ['rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_cephadm}'] 2020-12-30T23:00:24.442 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_cephadm}'] 2020-12-30T23:00:24.443 INFO:scrape: 2020-12-30T23:00:24.443 INFO:scrape:Valgrind: mon (UninitCondition) 2020-12-30T23:00:24.444 INFO:scrape:7 jobs: ['5747482', '5747395', '5747556', '5747658', '5747405', '5747700', '5747744'] 2020-12-30T23:00:24.444 INFO:scrape:suites intersection: [] 2020-12-30T23:00:24.445 INFO:scrape:suites union: ['2-inject-leak/mon', '2-inject-leak/none', '2-inject-leak/osd', 'centos_latest}', 'ceph', 'clusters/{fixed-2', 'd-thrash/default/{default', 'mon_election/classic', 'msgr-failures/few', 'msgr/async', 'msgr/async-v2only', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/valgrind-leaks/{1-start', 'rados/verify/{centos_latest', 'tasks/mon_recovery', 'tasks/rados_api_tests', 'tasks/rados_cls_all', 'thrashosds-health}', 'validater/valgrind}'] 2020-12-30T23:00:24.445 INFO:scrape: 2020-12-30T23:00:24.445 INFO:scrape:Failure: "2020-12-30T10:42:38.663611+0000 mon.b (mon.0) 6319 : cluster [WRN] Health check failed: Telemetry requires re-opt-in (TELEMETRY_CHANGED)" in cluster log 2020-12-30T23:00:24.446 INFO:scrape:2 jobs: ['5747626', '5747540'] 2020-12-30T23:00:24.446 INFO:scrape:suites intersection: [] 2020-12-30T23:00:24.447 INFO:scrape:suites union: ['1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{more-async-partial-recovery}', 'backoff/normal', 'ceph', 'clusters/{fixed-2', 'crc-failures/default', 'd-balancer/crush-compat', 'debug/mgr', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/osd-delay', 'msgr/async-v1only', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/dashboard/{clusters/{2-node-mgr}', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{rhel_8}', 'tasks/dashboard}', 'thrashers/mapgap', 'thrashosds-health', 'workloads/pool-snaps-few-objects}'] 2020-12-30T23:00:24.447 INFO:scrape: 2020-12-30T23:00:24.448 INFO:scrape:Failure: Command crashed: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool create base 4'" 2020-12-30T23:00:24.448 INFO:scrape:1 jobs: ['5747370'] 2020-12-30T23:00:24.448 INFO:scrape:suites: ['1-install/octopus', 'backoff/peering_and_degraded', 'ceph', 'clusters/{openstack', 'd-balancer/on', 'distro$/{ubuntu_18.04}', 'mon_election/classic', 'msgr-failures/few', 'rados', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size', 'thrashers/none', 'thrashosds-health', 'three-plus-one}', 'workloads/cache-snaps}'] 2020-12-30T23:00:24.449 INFO:scrape: 2020-12-30T23:00:24.449 INFO:scrape:Failure: Command failed (workunit test crush/crush-choose-args.sh) on smithi070 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21180a1e12f36e61aced96f8de2276edb1d0150 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh' 2020-12-30T23:00:24.450 INFO:scrape:2 jobs: ['5747554', '5747775'] 2020-12-30T23:00:24.450 INFO:scrape:suites intersection: ['supported-random-distro$/{ubuntu_latest}', 'workloads/crush}'] 2020-12-30T23:00:24.450 INFO:scrape:suites union: ['rados/standalone/{mon_election/classic', 'rados/standalone/{mon_election/connectivity', 'supported-random-distro$/{ubuntu_latest}', 'workloads/crush}'] 2020-12-30T23:00:24.451 INFO:scrape: 2020-12-30T23:00:24.451 INFO:scrape:Failure: Command failed (workunit test cephadm/test_adoption.sh) on smithi107 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21180a1e12f36e61aced96f8de2276edb1d0150 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh' 2020-12-30T23:00:24.452 INFO:scrape:1 jobs: ['5747406'] 2020-12-30T23:00:24.452 INFO:scrape:suites: ['mon_election/connectivity', 'rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_adoption}'] 2020-12-30T23:00:24.452 INFO:scrape: 2020-12-30T23:00:24.453 INFO:scrape:Failure: Command crashed: 'CEPH_CLIENT_ID=2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op copy_from 50 --op write_excl 50 --pool unique_pool_0' 2020-12-30T23:00:24.453 INFO:scrape:4 jobs: ['5747539', '5747438', '5747514', '5747517'] 2020-12-30T23:00:24.454 INFO:scrape:suites intersection: ['rados'] 2020-12-30T23:00:24.454 INFO:scrape:suites union: ['1-install/octopus', '1-pg-log-overrides/normal_pg_log', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{more-async-partial-recovery}', '2-recovery-overrides/{more-async-recovery}', 'backoff/normal', 'backoff/peering', 'backoff/peering_and_degraded', 'ceph', 'clusters/{fixed-2', 'clusters/{openstack', 'crc-failures/bad_map_crc_failure', 'crc-failures/default', 'd-balancer/crush-compat', 'd-balancer/on', 'distro$/{ubuntu_18.04}', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/many', 'msgr-failures/osd-delay', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'openstack}', 'rados', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size', 'rados/singleton/{all/thrash_cache_writeback_proxy_none', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{centos_8}}', 'supported-random-distro$/{rhel_8}', 'thrashers/default', 'thrashers/mapgap', 'thrashosds-health', 'three-plus-one}', 'workloads/cache-agent-big}', 'workloads/cache-agent-small}', 'workloads/snaps-few-objects}'] 2020-12-30T23:00:24.455 INFO:scrape: 2020-12-30T23:00:24.455 INFO:scrape:Dead: b'2020-12-30T20:13:46.563 INFO:teuthology.orchestra.run.smithi200.stdout: ]' 2020-12-30T23:00:24.455 INFO:scrape:1 jobs: ['5747478'] 2020-12-30T23:00:24.456 INFO:scrape:suites: ['ceph', 'clusters/{fixed-2', 'mon_election/connectivity', 'msgr-failures/fastclose', 'objectstore/filestore-xfs', 'openstack}', 'rados', 'rados/thrash-erasure-code-isa/{arch/x86_64', 'recovery-overrides/{more-active-recovery}', 'supported-random-distro$/{rhel_8}', 'thrashers/mapgap', 'thrashosds-health', 'workloads/ec-rados-plugin=isa-k=2-m=1}'] 2020-12-30T23:00:24.456 INFO:scrape: 2020-12-30T23:00:24.457 INFO:scrape:Failure: Command failed on smithi062 with status 1: 'sudo docker kill -s 1 ceph-b372b510-4a73-11eb-8f8d-001a4aab830c-osd.8' 2020-12-30T23:00:24.457 INFO:scrape:1 jobs: ['5747430'] 2020-12-30T23:00:24.457 INFO:scrape:suites: ['1-install/mimic-v1only', 'backoff/peering_and_degraded', 'ceph', 'clusters/{openstack', 'd-balancer/crush-compat', 'distro$/{ubuntu_18.04}', 'mon_election/connectivity', 'msgr-failures/few', 'rados', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size', 'thrashers/mapgap', 'thrashosds-health', 'three-plus-one}', 'workloads/snaps-few-objects}'] 2020-12-30T23:00:24.458 INFO:scrape: 2020-12-30T23:00:24.474 INFO:teuthology.results:Sending results to tchaikov@gmail.com: Test Run: kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/ failed: 31 dead: 22 running: 0 waiting: 0 queued: 0 passed: 393 Fail ================================================================= [5747349] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{centos_8.0} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 00:18:34 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747349/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747349/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fed6ab8e292f4ad093d9138effd8a9af reached maximum tries (180) after waiting for 180 seconds [5747370] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/none thrashosds-health workloads/cache-snaps} ----------------------------------------------------------------- time: 00:27:42 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747370/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747370/ sentry: https://sentry.ceph.com/organizations/ceph/?query=50010b3c8f2c4133a469c005a95bb01d Command crashed: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool create base 4'" [5747395] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados tasks/mon_recovery validater/valgrind} ----------------------------------------------------------------- time: 01:10:09 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747395/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747395/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a61f67aeabe34894a95a315d18acd4db saw valgrind issues [5747398] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{centos_latest} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 00:17:47 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747398/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747398/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0f66cb70a647466784dd40f1b5db06c6 reached maximum tries (180) after waiting for 180 seconds [5747405] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} ----------------------------------------------------------------- time: 01:08:30 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747405/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747405/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d7f3d294417a4ef781557d3647b7a33b Command failed on smithi105 with status 1: 'sudo ceph --cluster ceph osd crush tunables default' [5747406] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/connectivity task/test_adoption} ----------------------------------------------------------------- time: 00:07:46 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747406/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747406/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6a46ece8d4044ab48b9996144af76e24 Command failed (workunit test cephadm/test_adoption.sh) on smithi107 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21180a1e12f36e61aced96f8de2276edb1d0150 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh' [5747430] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects} ----------------------------------------------------------------- time: 00:34:37 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747430/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747430/ Command failed on smithi062 with status 1: 'sudo docker kill -s 1 ceph-b372b510-4a73-11eb-8f8d-001a4aab830c-osd.8' [5747438] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:16:13 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747438/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747438/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 400000 --objects 10000 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op delete 50 --op copy_from 50 --op write_excl 50 --pool base' [5747457] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{rhel_latest} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 00:16:12 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747457/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747457/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4b6f07da187046b0b41408791ba0f2d5 reached maximum tries (180) after waiting for 180 seconds [5747482] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados tasks/rados_cls_all validater/valgrind} ----------------------------------------------------------------- time: 01:08:37 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747482/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747482/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a427668d16744c96bf68805f2e4017ae saw valgrind issues [5747491] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 00:37:22 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747491/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747491/ sentry: https://sentry.ceph.com/organizations/ceph/?query=39faebf41bd44cc4906087fef68e6305 Command failed on smithi062 with status 134: "/bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to- stderr --name client.2 -b 65536 --object-size 65536 -p unique_pool_0 bench 90 write'" [5747514] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-agent-big} ----------------------------------------------------------------- time: 00:12:22 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747514/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747514/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 10000 --objects 6600 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 1200 --op read 100 --op write 50 --op delete 50 --op copy_from 50 --op write_excl 50 --pool base' [5747517] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-agent-small} ----------------------------------------------------------------- time: 00:13:53 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747517/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747517/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 4000 --objects 500 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op delete 50 --op copy_from 50 --op write_excl 50 --pool base' [5747518] rados/standalone/{mon_election/connectivity supported-random-distro$/{centos_8} workloads/scrub} ----------------------------------------------------------------- time: 01:16:18 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747518/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747518/ sentry: https://sentry.ceph.com/organizations/ceph/?query=87a6270efc054672b7093a151a066ab6 Command failed (workunit test scrub/osd-scrub-snaps.sh) on smithi060 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21180a1e12f36e61aced96f8de2276edb1d0150 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub- snaps.sh' [5747539] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/snaps-few-objects} ----------------------------------------------------------------- time: 00:39:24 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747539/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747539/ Command crashed: 'CEPH_CLIENT_ID=2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op copy_from 50 --op write_excl 50 --pool unique_pool_0' [5747540] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/pool-snaps-few-objects} ----------------------------------------------------------------- time: 00:29:02 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747540/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747540/ "2020-12-30T09:20:53.104526+0000 mon.a (mon.0) 3029 : cluster [WRN] Health check failed: 1 daemons have recently crashed (RECENT_CRASH)" in cluster log [5747554] rados/standalone/{mon_election/classic supported-random-distro$/{ubuntu_latest} workloads/crush} ----------------------------------------------------------------- time: 00:05:54 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747554/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747554/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7753b5571bb742cba41179d3533b4661 Command failed (workunit test crush/crush-choose-args.sh) on smithi070 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21180a1e12f36e61aced96f8de2276edb1d0150 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose- args.sh' [5747556] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} ----------------------------------------------------------------- time: 01:07:55 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747556/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747556/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ed53ae9c71f545a5a7c143dab5d0dfc1 Command failed on smithi055 with status 1: 'sudo ceph --cluster ceph osd crush tunables default' [5747571] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} ----------------------------------------------------------------- time: 05:02:52 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747571/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747571/ Command failed on smithi159 with status 124: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' [5747574] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{rhel_8.0} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 00:19:55 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747574/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747574/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e40b6fb6be9d4c18ba895a4c9075ec9d reached maximum tries (180) after waiting for 180 seconds [5747614] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:07:23 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747614/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747614/ sentry: https://sentry.ceph.com/organizations/ceph/?query=081b954c4a9a4388bfaf699e4725a14c Command failed (workunit test cephadm/test_cephadm.sh) on smithi043 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21180a1e12f36e61aced96f8de2276edb1d0150 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [5747626] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zlib supported-random-distro$/{rhel_8} tasks/dashboard} ----------------------------------------------------------------- time: 01:13:05 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747626/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747626/ "2020-12-30T10:42:38.663611+0000 mon.b (mon.0) 6319 : cluster [WRN] Health check failed: Telemetry requires re-opt-in (TELEMETRY_CHANGED)" in cluster log [5747658] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/mon_recovery validater/valgrind} ----------------------------------------------------------------- time: 01:08:32 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747658/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747658/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9a22a5f76a1248ac9e0baed87facb699 saw valgrind issues [5747681] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{rhel_8.0} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 00:19:29 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747681/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747681/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a3979c938c4b48bfbb9157a94e2f2340 reached maximum tries (180) after waiting for 180 seconds [5747700] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} ----------------------------------------------------------------- time: 01:08:21 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747700/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747700/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f98fb796097b46e4ad8b06e6928060ce saw valgrind issues [5747714] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 01:13:14 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747714/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747714/ sentry: https://sentry.ceph.com/organizations/ceph/?query=30f1ffc5fa4642e6bc8afb3a21ae261f reached maximum tries (500) after waiting for 3000 seconds [5747728] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{centos_8.0} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 00:18:51 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747728/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747728/ sentry: https://sentry.ceph.com/organizations/ceph/?query=70ceb901cf4c4c288e1d5dfc68e61256 reached maximum tries (180) after waiting for 180 seconds [5747740] rados/standalone/{mon_election/classic supported-random-distro$/{ubuntu_latest} workloads/scrub} ----------------------------------------------------------------- time: 01:12:54 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747740/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747740/ sentry: https://sentry.ceph.com/organizations/ceph/?query=caf1a8c16eb648aca0ff984cbf894ca3 Command failed (workunit test scrub/osd-scrub-snaps.sh) on smithi205 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21180a1e12f36e61aced96f8de2276edb1d0150 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub- snaps.sh' [5747744] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} ----------------------------------------------------------------- time: 01:08:49 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747744/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747744/ sentry: https://sentry.ceph.com/organizations/ceph/?query=705600d07a724a3bba6a4f55d3905ac4 saw valgrind issues [5747768] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/classic task/test_cephadm} ----------------------------------------------------------------- time: 00:07:23 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747768/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747768/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f8825dc4531e415abcfc2598cc80bb63 Command failed (workunit test cephadm/test_cephadm.sh) on smithi137 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21180a1e12f36e61aced96f8de2276edb1d0150 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [5747775] rados/standalone/{mon_election/connectivity supported-random-distro$/{ubuntu_latest} workloads/crush} ----------------------------------------------------------------- time: 00:05:57 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747775/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747775/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d1b65075d92149e19a778c504be4d203 Command failed (workunit test crush/crush-choose-args.sh) on smithi093 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21180a1e12f36e61aced96f8de2276edb1d0150 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose- args.sh' Dead ================================================================= [5747356] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747356/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747356/ [5747393] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/small-objects} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747393/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747393/ [5747444] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747444/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747444/ [5747456] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/radosbench-high-concurrency} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747456/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747456/ [5747470] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747470/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747470/ [5747478] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747478/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747478/ [5747500] rados/objectstore/{backends/objectstore supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747500/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747500/ [5747509] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{ubuntu_18.04} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747509/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747509/ [5747519] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/rbd_cls} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747519/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747519/ [5747572] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados tasks/rados_api_tests validater/valgrind} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747572/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747572/ [5747573] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/set-chunks-read} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747573/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747573/ [5747596] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-localized} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747596/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747596/ [5747619] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps-readproxy} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747619/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747619/ [5747621] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{ubuntu_18.04_podman} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747621/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747621/ [5747642] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/rados_api_tests} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747642/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747642/ [5747668] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/set-chunks-read} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747668/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747668/ [5747669] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747669/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747669/ [5747689] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747689/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747689/ [5747690] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/cache-snaps} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747690/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747690/ [5747692] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747692/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747692/ [5747739] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747739/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747739/ [5747742] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/radosbench-high-concurrency} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747742/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747742/ Pass ================================================================= [5747343] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:10:26 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747343/ [5747344] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-snaps} time: 00:26:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747344/ [5747345] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:11:55 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747345/ [5747346] rados/cephadm/smoke-roleless/{distro/centos_8.0 start} time: 00:12:57 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747346/ [5747347] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:20 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747347/ [5747348] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/cache} time: 00:09:50 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747348/ [5747350] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:11:49 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747350/ [5747351] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados tasks/rados_cls_all validater/lockdep} time: 00:17:17 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747351/ [5747352] rados/singleton/{all/recovery-preemption mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:17:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747352/ [5747353] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/pool-snaps-few-objects} time: 00:22:05 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747353/ [5747354] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8} tasks/dashboard} time: 01:13:25 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747354/ [5747355] rados/objectstore/{backends/fusestore supported-random-distro$/{ubuntu_latest}} time: 00:05:03 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747355/ [5747357] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:09:29 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747357/ [5747358] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/rados_api_tests} time: 00:41:18 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747358/ [5747359] rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:31:47 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747359/ [5747360] rados/standalone/{mon_election/classic supported-random-distro$/{centos_8} workloads/erasure-code} time: 00:39:23 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747360/ [5747361] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/radosbench-high-concurrency} time: 01:19:38 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747361/ [5747362] rados/cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_orch_cli} time: 00:18:50 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747362/ [5747363] rados/singleton/{all/resolve_stuck_peering mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:10:21 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747363/ [5747364] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/radosbench} time: 01:46:01 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747364/ [5747365] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/rados_stress_watch} time: 00:19:02 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747365/ [5747366] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:10:23 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747366/ [5747367] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/connectivity start} time: 00:13:16 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747367/ [5747368] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/redirect} time: 00:13:08 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747368/ [5747369] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:08:06 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747369/ [5747371] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/redirect_promote_tests} time: 00:13:01 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747371/ [5747372] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/classic start} time: 00:15:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747372/ [5747373] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/sync-many workloads/pool-create-delete} time: 00:17:26 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747373/ [5747374] rados/singleton/{all/test-crash mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:10:35 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747374/ [5747375] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:34:22 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747375/ [5747376] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:11 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747376/ [5747377] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/redirect_set_object} time: 00:13:33 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747377/ [5747378] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:08:54 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747378/ [5747379] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:04:50 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747379/ [5747380] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-hybrid supported-random-distro$/{rhel_8} tasks/prometheus} time: 00:12:52 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747380/ [5747381] rados/cephadm/smoke-roleless/{distro/centos_latest start} time: 00:12:21 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747381/ [5747382] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/set-chunks-read} time: 00:17:06 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747382/ [5747383] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-small-objects-overwrites} time: 00:22:55 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747383/ [5747384] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:10:54 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747384/ [5747385] rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:20:44 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747385/ [5747386] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/small-objects-balanced} time: 00:24:40 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747386/ [5747387] rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:18:12 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747387/ [5747388] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:38 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747388/ [5747389] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/small-objects-localized} time: 00:22:41 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747389/ [5747390] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/connectivity start} time: 00:13:53 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747390/ [5747391] rados/standalone/{mon_election/connectivity supported-random-distro$/{ubuntu_latest} workloads/mgr} time: 00:08:41 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747391/ [5747392] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:09:28 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747392/ [5747394] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:29:20 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747394/ [5747396] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/radosbench} time: 01:33:06 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747396/ [5747397] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-balanced} time: 00:32:22 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747397/ [5747399] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:10:11 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747399/ [5747400] rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 01:18:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747400/ [5747401] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/snaps-few-objects-localized} time: 00:22:55 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747401/ [5747402] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/rados_striper} time: 00:06:15 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747402/ [5747403] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:11:05 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747403/ [5747404] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{rhel_8}} time: 00:13:50 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747404/ [5747407] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/snaps-few-objects} time: 00:25:53 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747407/ [5747408] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:05:56 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747408/ [5747409] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:12:07 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747409/ [5747410] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/write_fadvise_dontneed} time: 00:15:11 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747410/ [5747411] rados/cephadm/smoke-roleless/{distro/rhel_8.0 start} time: 00:14:17 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747411/ [5747412] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:32:16 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747412/ [5747413] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/admin_socket_objecter_requests} time: 00:31:28 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747413/ [5747414] rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:22:03 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747414/ [5747415] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:56 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747415/ [5747416] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-agent-big} time: 00:42:33 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747416/ [5747417] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_5925} time: 00:07:43 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747417/ [5747418] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:27:45 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747418/ [5747419] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:10:41 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747419/ [5747420] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/rbd_cls} time: 00:19:08 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747420/ [5747421] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-agent-small} time: 00:14:57 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747421/ [5747422] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} tasks/workunits} time: 00:06:21 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747422/ [5747423] rados/standalone/{mon_election/classic supported-random-distro$/{centos_8} workloads/misc} time: 00:28:58 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747423/ [5747424] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/connectivity start} time: 00:12:19 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747424/ [5747425] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:20:48 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747425/ [5747426] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:16:19 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747426/ [5747427] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:12:33 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747427/ [5747428] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8} tasks/dashboard} time: 01:16:38 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747428/ [5747429] rados/multimon/{clusters/6 mon_election/classic msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:08:13 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747429/ [5747431] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps} time: 00:26:47 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747431/ [5747432] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:05:52 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747432/ [5747433] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:52 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747433/ [5747434] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/classic start} time: 00:11:56 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747434/ [5747435] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-snaps-balanced} time: 00:22:31 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747435/ [5747436] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:12:29 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747436/ [5747437] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados tasks/rados_api_tests validater/lockdep} time: 00:23:49 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747437/ [5747439] rados/cephadm/smoke-roleless/{distro/rhel_latest start} time: 00:11:42 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747439/ [5747440] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-snaps} time: 00:24:12 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747440/ [5747441] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_big} time: 00:33:18 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747441/ [5747442] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:31:57 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747442/ [5747443] rados/cephadm/workunits/{distro/centos_latest mon_election/classic task/test_cephadm} time: 00:14:44 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747443/ [5747445] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:06:20 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747445/ [5747446] rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:32:31 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747446/ [5747447] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:26:12 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747447/ [5747448] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/few rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:24:44 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747448/ [5747449] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:06:01 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747449/ [5747450] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-hybrid supported-random-distro$/{centos_8} tasks/dashboard} time: 01:20:25 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747450/ [5747451] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{rhel_8}} time: 00:24:37 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747451/ [5747452] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/rados_api_tests} time: 00:24:27 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747452/ [5747453] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/connectivity start} time: 00:13:52 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747453/ [5747454] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:10:06 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747454/ [5747455] rados/standalone/{mon_election/connectivity supported-random-distro$/{centos_8} workloads/mon} time: 00:42:31 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747455/ [5747458] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:12 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747458/ [5747459] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:09:53 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747459/ [5747460] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/radosbench} time: 01:39:37 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747460/ [5747461] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} time: 00:18:02 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747461/ [5747462] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_api_tests} time: 00:28:31 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747462/ [5747463] rados/singleton/{all/admin-socket mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:08:51 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747463/ [5747464] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:23:47 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747464/ [5747465] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/redirect} time: 00:13:36 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747465/ [5747466] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-stupid supported-random-distro$/{rhel_8} tasks/crash} time: 00:12:08 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747466/ [5747467] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/classic start} time: 00:12:54 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747467/ [5747468] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/redirect_promote_tests} time: 00:11:42 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747468/ [5747469] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04 start} time: 00:11:03 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747469/ [5747471] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/redirect_set_object} time: 00:13:42 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747471/ [5747472] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/few msgr/async no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:09:06 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747472/ [5747473] rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_python} time: 00:24:40 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747473/ [5747474] rados/singleton/{all/deduptool mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:07:22 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747474/ [5747475] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_mix} time: 00:23:06 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747475/ [5747476] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:16:07 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747476/ [5747477] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/set-chunks-read} time: 00:10:44 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747477/ [5747479] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/cache-snaps} time: 00:26:13 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747479/ [5747480] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/small-objects-balanced} time: 00:18:57 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747480/ [5747481] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:29:08 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747481/ [5747483] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:09:31 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747483/ [5747484] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/connectivity task/test_cephadm_repos} time: 00:04:03 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747484/ [5747485] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/small-objects-localized} time: 00:23:18 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747485/ [5747486] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:17:34 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747486/ [5747487] rados/singleton/{all/divergent_priors mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:10:18 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747487/ [5747488] rados/standalone/{mon_election/classic supported-random-distro$/{ubuntu_latest} workloads/osd} time: 03:06:52 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747488/ [5747489] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/connectivity start} time: 00:15:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747489/ [5747490] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/small-objects} time: 00:22:20 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747490/ [5747492] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-balanced} time: 00:29:12 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747492/ [5747493] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/classic start} time: 00:12:37 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747493/ [5747494] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-localized} time: 00:31:52 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747494/ [5747495] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:19:02 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747495/ [5747496] rados/singleton/{all/divergent_priors2 mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:10:45 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747496/ [5747497] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:38 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747497/ [5747498] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start} time: 00:22:52 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747498/ [5747499] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:17:17 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747499/ [5747501] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/snaps-few-objects} time: 00:20:23 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747501/ [5747502] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/many workloads/rados_mon_osdmap_prune} time: 00:53:40 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747502/ [5747503] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-radosbench} time: 02:25:04 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747503/ [5747504] rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:35:19 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747504/ [5747505] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:15:59 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747505/ [5747506] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/filestore-xfs supported-random-distro$/{ubuntu_latest} tasks/failover} time: 00:10:19 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747506/ [5747507] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:09:11 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747507/ [5747508] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/admin_socket_objecter_requests} time: 00:46:43 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747508/ [5747510] rados/singleton/{all/dump-stuck mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:10:40 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747510/ [5747511] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:10:01 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747511/ [5747512] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:29:24 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747512/ [5747513] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_mostlyread} time: 00:29:55 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747513/ [5747515] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} time: 00:12:21 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747515/ [5747516] rados/multimon/{clusters/21 mon_election/classic msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:21:31 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747516/ [5747520] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:30:29 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747520/ [5747521] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:22:37 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747521/ [5747522] rados/cephadm/workunits/{distro/centos_latest mon_election/classic task/test_orch_cli} time: 00:19:19 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747522/ [5747523] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:51:14 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747523/ [5747524] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:29:44 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747524/ [5747525] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:09:04 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747525/ [5747526] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados tasks/mon_recovery validater/lockdep} time: 00:12:35 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747526/ [5747527] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/filestore-xfs supported-random-distro$/{centos_8} tasks/dashboard} time: 01:17:30 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747527/ [5747528] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-pool-snaps} time: 00:22:27 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747528/ [5747529] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/classic start} time: 00:13:36 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747529/ [5747530] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache-snaps-balanced} time: 00:26:46 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747530/ [5747531] rados/cephadm/smoke-roleless/{distro/ubuntu_latest start} time: 00:10:54 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747531/ [5747532] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache-snaps} time: 00:24:03 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747532/ [5747533] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:04:57 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747533/ [5747534] rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:20:59 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747534/ [5747535] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:08:59 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747535/ [5747536] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:06:09 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747536/ [5747537] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache} time: 00:10:42 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747537/ [5747538] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:56 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747538/ [5747541] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/connectivity start} time: 00:14:10 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747541/ [5747542] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/rados_api_tests} time: 00:35:07 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747542/ [5747543] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/rados_mon_workunits} time: 00:26:02 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747543/ [5747544] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:35:38 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747544/ [5747545] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-balanced} time: 00:19:31 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747545/ [5747546] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/insights} time: 00:15:41 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747546/ [5747547] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/test_rbd_api} time: 00:18:46 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747547/ [5747548] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/readwrite} time: 00:12:54 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747548/ [5747549] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/dashboard} time: 01:12:52 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747549/ [5747550] rados/objectstore/{backends/alloc-hint supported-random-distro$/{centos_8}} time: 00:09:45 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747550/ [5747551] rados/rest/{mgr-restful supported-random-distro$/{rhel_8}} time: 00:12:05 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747551/ [5747552] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:31:28 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747552/ [5747553] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:15:28 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747553/ [5747555] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/classic thrashosds-health ubuntu_latest} time: 05:15:11 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747555/ [5747557] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/radosbench-high-concurrency} time: 00:54:49 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747557/ [5747558] rados/cephadm/dashboard/{distro/centos_latest task/test_e2e} time: 00:18:31 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747558/ [5747559] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/radosbench} time: 02:27:12 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747559/ [5747560] rados/cephadm/orchestrator_cli/{2-node-mgr orchestrator_cli supported-random-distro$/{rhel_8}} time: 00:15:19 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747560/ [5747561] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/redirect} time: 00:13:56 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747561/ [5747562] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:47:58 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747562/ [5747563] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:08:39 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747563/ [5747564] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/classic start} time: 00:13:18 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747564/ [5747565] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:07:09 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747565/ [5747566] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/redirect_promote_tests} time: 00:14:28 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747566/ [5747567] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:12 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747567/ [5747568] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:08:03 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747568/ [5747569] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/redirect_set_object} time: 00:16:43 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747569/ [5747570] rados/cephadm/smoke-roleless/{distro/centos_8.0 start} time: 00:13:34 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747570/ [5747575] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:09:10 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747575/ [5747576] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/fastclose rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:22:31 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747576/ [5747577] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:11:53 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747577/ [5747578] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/small-objects-balanced} time: 00:19:37 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747578/ [5747579] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:06:18 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747579/ [5747580] rados/cephadm/workunits/{distro/centos_latest mon_election/classic task/test_adoption} time: 00:09:12 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747580/ [5747581] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/small-objects-localized} time: 00:18:30 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747581/ [5747582] rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:32:28 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747582/ [5747583] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:04 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747583/ [5747584] rados/standalone/{mon_election/connectivity supported-random-distro$/{centos_8} workloads/erasure-code} time: 00:37:53 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747584/ [5747585] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/small-objects} time: 00:23:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747585/ [5747586] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:12:38 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747586/ [5747587] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/connectivity start} time: 00:13:09 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747587/ [5747588] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/repair_test} time: 00:13:50 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747588/ [5747589] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/snaps-few-objects-balanced} time: 00:25:21 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747589/ [5747590] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:10:13 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747590/ [5747591] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:30:07 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747591/ [5747592] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/snaps-few-objects} time: 00:27:38 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747592/ [5747593] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-small-objects-fast-read} time: 00:22:07 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747593/ [5747594] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} tasks/module_selftest} time: 00:26:53 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747594/ [5747595] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:31:38 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747595/ [5747597] rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_python} time: 00:22:11 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747597/ [5747598] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} time: 00:31:11 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747598/ [5747599] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:13:02 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747599/ [5747600] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/classic start} time: 00:14:47 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747600/ [5747601] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-snappy supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:14:01 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747601/ [5747602] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{centos_8}} time: 00:12:50 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747602/ [5747603] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/write_fadvise_dontneed} time: 00:13:59 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747603/ [5747604] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:08:53 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747604/ [5747605] rados/cephadm/smoke-roleless/{distro/centos_latest start} time: 00:12:38 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747605/ [5747606] rados/multimon/{clusters/6 mon_election/classic msgr-failures/many msgr/async no_pools objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:08:24 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747606/ [5747607] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/admin_socket_objecter_requests} time: 00:28:56 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747607/ [5747608] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/radosbench} time: 01:01:04 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747608/ [5747609] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:46 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747609/ [5747610] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:07:34 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747610/ [5747611] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-agent-big} time: 00:27:56 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747611/ [5747612] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:08:18 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747612/ [5747613] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/rados_cls_all validater/lockdep} time: 00:17:21 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747613/ [5747615] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/cache-agent-small} time: 00:16:28 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747615/ [5747616] rados/standalone/{mon_election/classic supported-random-distro$/{centos_8} workloads/mgr} time: 00:12:17 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747616/ [5747617] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:09:15 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747617/ [5747618] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/connectivity start} time: 00:12:36 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747618/ [5747620] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:11:28 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747620/ [5747622] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:10:42 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747622/ [5747623] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-pool-snaps} time: 00:24:22 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747623/ [5747624] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:48 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747624/ [5747625] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/scrub_test} time: 00:08:34 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747625/ [5747627] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache-snaps-balanced} time: 00:16:21 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747627/ [5747628] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/rbd_cls} time: 00:20:48 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747628/ [5747629] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:09:11 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747629/ [5747630] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache-snaps} time: 00:26:44 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747630/ [5747631] rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:34:05 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747631/ [5747632] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/sync workloads/pool-create-delete} time: 00:18:14 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747632/ [5747633] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:15:38 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747633/ [5747634] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:21:15 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747634/ [5747635] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache} time: 00:10:05 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747635/ [5747636] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8} tasks/progress} time: 00:24:01 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747636/ [5747637] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:12:08 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747637/ [5747638] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:27:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747638/ [5747639] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/pool-snaps-few-objects} time: 00:31:36 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747639/ [5747640] rados/cephadm/smoke-roleless/{distro/rhel_8.0 start} time: 00:14:32 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747640/ [5747641] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:11:17 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747641/ [5747643] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/connectivity start} time: 00:12:36 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747643/ [5747644] rados/standalone/{mon_election/connectivity supported-random-distro$/{rhel_8} workloads/misc} time: 00:29:11 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747644/ [5747645] rados/singleton/{all/mon-config mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:10:11 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747645/ [5747646] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/radosbench-high-concurrency} time: 00:41:47 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747646/ [5747647] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:08:27 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747647/ [5747648] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/snaps-few-objects} time: 00:34:52 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747648/ [5747649] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/few msgr/async-v1only no_pools objectstore/filestore-xfs rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:13:24 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747649/ [5747650] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:11:42 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747650/ [5747651] rados/objectstore/{backends/filejournal supported-random-distro$/{rhel_8}} time: 00:08:35 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747651/ [5747652] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/radosbench} time: 02:16:54 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747652/ [5747653] rados/cephadm/workunits/{distro/centos_latest mon_election/classic task/test_cephadm_repos} time: 00:06:41 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747653/ [5747654] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:56 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747654/ [5747655] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/redirect} time: 00:13:48 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747655/ [5747656] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:05:54 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747656/ [5747657] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:31:58 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747657/ [5747659] rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:24:48 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747659/ [5747660] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:15:33 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747660/ [5747661] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/redirect_promote_tests} time: 00:15:29 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747661/ [5747662] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/classic start} time: 00:12:01 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747662/ [5747663] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests} time: 00:24:38 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747663/ [5747664] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:28:44 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747664/ [5747665] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/redirect_set_object} time: 00:17:08 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747665/ [5747666] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:15 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747666/ [5747667] rados/cephadm/smoke-roleless/{distro/rhel_latest start} time: 00:21:17 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747667/ [5747670] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/test_rbd_api} time: 00:20:45 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747670/ [5747671] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:19:07 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747671/ [5747672] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/small-objects-balanced} time: 00:19:53 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747672/ [5747673] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:12:12 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747673/ [5747674] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/connectivity start} time: 00:14:45 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747674/ [5747675] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/cosbench_64K_read_write} time: 00:23:47 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747675/ [5747676] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/small-objects-localized} time: 00:20:15 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747676/ [5747677] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_5925} time: 00:10:57 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747677/ [5747678] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects} time: 00:21:18 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747678/ [5747679] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8} tasks/prometheus} time: 00:12:03 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747679/ [5747680] rados/standalone/{mon_election/classic supported-random-distro$/{centos_8} workloads/mon} time: 00:46:41 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747680/ [5747682] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/small-objects} time: 00:27:09 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747682/ [5747683] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:09:49 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747683/ [5747684] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:32:03 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747684/ [5747685] rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:33:16 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747685/ [5747686] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/snaps-few-objects-balanced} time: 00:28:59 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747686/ [5747687] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/snaps-few-objects-localized} time: 00:30:03 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747687/ [5747688] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/connectivity task/test_orch_cli} time: 00:17:07 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747688/ [5747691] rados/multimon/{clusters/21 mon_election/classic msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:25:35 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747691/ [5747693] rados/singleton/{all/peer mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:08:58 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747693/ [5747694] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:16:16 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747694/ [5747695] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:06 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747695/ [5747696] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/classic start} time: 00:13:05 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747696/ [5747697] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/rados_cls_all} time: 00:17:22 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747697/ [5747698] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8} tasks/dashboard} time: 01:13:07 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747698/ [5747699] rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{centos_8}} time: 02:17:04 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747699/ [5747701] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:10:46 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747701/ [5747702] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados tasks/rados_api_tests validater/lockdep} time: 00:23:07 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747702/ [5747703] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/admin_socket_objecter_requests} time: 00:22:20 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747703/ [5747704] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:31:05 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747704/ [5747705] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04 start} time: 00:10:51 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747705/ [5747706] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/cosbench_64K_write} time: 00:21:02 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747706/ [5747707] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache-agent-big} time: 00:23:03 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747707/ [5747708] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:26 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747708/ [5747709] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/connectivity start} time: 00:15:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747709/ [5747710] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:11:55 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747710/ [5747711] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:18:32 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747711/ [5747712] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-agent-small} time: 00:12:49 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747712/ [5747713] rados/standalone/{mon_election/connectivity supported-random-distro$/{ubuntu_latest} workloads/osd} time: 03:09:36 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747713/ [5747715] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:19:57 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747715/ [5747716] rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:21:21 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747716/ [5747717] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps} time: 00:24:55 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747717/ [5747718] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} tasks/workunits} time: 00:06:15 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747718/ [5747719] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/classic start} time: 00:12:00 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747719/ [5747720] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/many workloads/rados_api_tests} time: 00:34:47 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747720/ [5747721] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:11:42 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747721/ [5747722] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:22:20 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747722/ [5747723] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:19:45 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747723/ [5747724] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-snaps-balanced} time: 00:25:19 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747724/ [5747725] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-stupid supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:14:35 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747725/ [5747726] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start} time: 00:22:49 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747726/ [5747727] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache-snaps} time: 00:15:51 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747727/ [5747729] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache} time: 00:12:54 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747729/ [5747730] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:11:21 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747730/ [5747731] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:05:29 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747731/ [5747732] rados/cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_adoption} time: 00:08:55 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747732/ [5747733] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/pool-snaps-few-objects} time: 00:28:47 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747733/ [5747734] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:09:22 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747734/ [5747735] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/rados_python} time: 00:15:09 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747735/ [5747736] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/few msgr/async no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:04:55 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747736/ [5747737] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/rbd_cls} time: 00:14:30 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747737/ [5747738] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/rados_api_tests} time: 00:27:35 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747738/ [5747741] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} time: 00:12:06 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747741/ [5747743] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:28:43 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747743/ [5747745] rados/singleton/{all/radostool mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:13:05 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747745/ [5747746] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/radosbench} time: 01:57:21 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747746/ [5747747] rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:32:14 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747747/ [5747748] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:26:56 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747748/ [5747749] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:37 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747749/ [5747750] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/filestore-xfs supported-random-distro$/{centos_8} tasks/dashboard} time: 01:13:06 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747750/ [5747751] rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{ubuntu_latest}} time: 02:10:24 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747751/ [5747752] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/redirect} time: 00:13:10 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747752/ [5747753] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/snaps-few-objects} time: 00:36:41 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747753/ [5747754] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/redirect_promote_tests} time: 00:13:03 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747754/ [5747755] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/classic start} time: 00:12:46 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747755/ [5747756] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:31:41 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747756/ [5747757] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:11:33 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747757/ [5747758] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/redirect_set_object} time: 00:14:55 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747758/ [5747759] rados/cephadm/smoke-roleless/{distro/ubuntu_latest start} time: 00:11:04 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747759/ [5747760] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:06:34 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747760/ [5747761] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:08:15 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747761/ [5747762] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/one workloads/rados_mon_osdmap_prune} time: 00:54:09 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747762/ [5747763] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:32:48 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747763/ [5747764] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/set-chunks-read} time: 00:16:51 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747764/ [5747765] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/connectivity start} time: 00:11:52 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747765/ [5747766] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:27:28 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747766/ [5747767] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/small-objects-balanced} time: 00:23:02 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747767/ [5747769] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:10:37 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747769/ [5747770] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/small-objects-localized} time: 00:23:16 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747770/ [5747771] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} tasks/rados_stress_watch} time: 00:18:18 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747771/ [5747772] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:12:01 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747772/ [5747773] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:29:35 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747773/ [5747774] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:12:50 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747774/ [5747776] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/connectivity thrashosds-health ubuntu_latest} time: 05:11:02 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747776/ [5747777] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/test_rbd_api} time: 00:18:29 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747777/ [5747778] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/small-objects} time: 00:19:17 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747778/ [5747779] rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:22:22 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747779/ [5747780] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/snaps-few-objects-balanced} time: 00:31:54 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747780/ [5747781] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:19:18 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747781/ [5747782] rados/multimon/{clusters/6 mon_election/classic msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:04:48 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747782/ [5747783] rados/cephadm/dashboard/{distro/centos_latest task/test_e2e} time: 00:34:07 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747783/ [5747784] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/snaps-few-objects-localized} time: 00:27:48 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747784/ [5747785] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:30:17 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747785/ [5747786] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:08:15 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747786/ [5747787] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:06:47 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747787/ [5747788] rados/cephadm/orchestrator_cli/{2-node-mgr orchestrator_cli supported-random-distro$/{ubuntu_latest}} time: 00:10:26 info: http://pulpito.ceph.com/kchai-2020-12-30_06:47:01-rados-wip-kefu-testing-2020-12-30-1123-distro-basic-smithi/5747788/